We ran a controlled experiment: AI calling agents alongside a human SDR team for 90 days, on the same lead list, with the same qualification criteria. The results were surprising — not because AI won cleanly, but because of where it won, where it lost, and what that means for how you should think about your sales stack.
The Experiment Setup
The client was a B2B SaaS company with an inside sales team of four SDRs. Lead source: a mix of inbound demo requests, content downloads, and paid lead lists. We split leads randomly — 50% to the human team, 50% to our AI calling agent. Same qualification script, same CRM, same booking link. The AI used a voice model with custom persona, trained on the client's product and ICP documentation.
We tracked six metrics: contact rate (calls answered), conversation duration, qualification rate (leads moved to pipeline), meeting booking rate, no-show rate on booked meetings, and cost per qualified lead. The 90-day run included 4,200 total leads — 2,100 per group.
Where the AI Agent Won
Volume and consistency. The AI agent made 847 calls on the first day. The human team made 312. Over 90 days, the AI contacted every lead within 4 minutes of assignment — 24/7, including weekends. The human team averaged a 6-hour first contact delay during business hours, and leads that came in Friday afternoon often waited until Monday. Speed-to-lead matters enormously in inbound — the AI's 4-minute response was a genuine structural advantage.
Cost per qualified lead: $18 for the AI versus $94 for the human team (factoring in salary, benefits, management time, and ramp). For commodity qualification — 'do you have budget, authority, need, and timeline?' — the AI was 80% cheaper with comparable output at the qualification stage.
- Contact rate: AI 71% vs. Human 58% (AI wins on speed-to-lead)
- Cost per qualified lead: AI $18 vs. Human $94
- Calls per day: AI 847 vs. Human 78 (per SDR)
- Consistency: AI followed script 100% of the time; humans varied 40%
- Weekend/after-hours coverage: AI 100%; Human 0%
Where Humans Won
Complex objection handling. When prospects pushed back with nuanced objections — 'we tried something like this before and it failed because...' or 'we're in a freeze until Q3 but I want to understand the ROI model before then' — the AI struggled to adapt. Its responses were coherent but generic, missing the opportunity to turn a soft 'no' into a future pipeline entry with a specific follow-up plan.
Relationship-qualified pipeline. The meetings booked by the human SDRs had a 34% close rate downstream. The AI-booked meetings closed at 19%. The AI was booking meetings with prospects who matched the ICP on paper but weren't genuinely engaged — they agreed to a meeting to end the call. Human SDRs picked up on subtle disengagement signals and either deepened the conversation or disqualified, resulting in higher-quality pipeline.
The AI was better at the beginning of the funnel. Humans were better at the end. The smart play is to use both where each wins.
The Right Model: AI-First, Human-Assisted
The conclusion isn't 'AI replaces SDRs' or 'AI can't do sales.' The conclusion is that AI and humans have different strengths at different stages of the qualification conversation. AI is superior for first contact, initial qualification, and high-volume outreach. Humans are superior for complex objections, relationship qualification, and anything requiring genuine improvisation.
The model we now recommend: AI agents handle all first-contact outreach and initial qualification (criteria-based pass/fail). Leads that meet basic qualification thresholds but have meaningful objections or complexity flags are escalated to a human SDR for a second call. This hybrid model delivered a 40% reduction in cost per closed deal versus either approach alone.
Key Takeaway
AI calling agents aren't a replacement for human sales — they're a redistribution of where human attention is spent. The SDRs in our experiment who worked alongside the AI became dramatically more productive: they spent zero time on first-contact outreach and 100% of their time on high-value conversations. That's not a job threat — that's a job upgrade. The teams that figure out this hybrid model early will have a cost structure and output volume their competitors can't match.
Written by
Riya Sharma
Builder at I2S — shipping AI, software, and growth systems for ambitious teams worldwide.
