2026.02.12
What actually breaks when an AI handles a real sales call
The voice agent handles Tier-1 sales and renewal calls end-to-end, including objection handling and negotiation, on Vapi. The demo version of this — agent answers, agent responds, call ends politely — takes an afternoon. The version that survives a real customer who's annoyed, in a hurry, or trying to negotiate a discount is a different, much harder problem, and almost none of the difficulty is in what the model says.
The latency budget is the real constraint
Human conversation has an expected response gap of a few hundred milliseconds. A pipeline of speech-to-text → LLM inference → text-to-speech easily blows past a second if each stage waits for the previous one to fully finish. On a sales call, a one-to-two-second dead-air gap after a customer states a price objection doesn't read as "the system is thinking" — it reads as "the system doesn't have an answer," and the customer either repeats themselves or hangs up.
The fix is architectural, not a bigger model: stream partial transcripts into the LLM before the customer finishes speaking where possible, start TTS synthesis on the first sentence of the response before the rest of the completion has finished generating, and treat total round-trip latency as a budget to be spent deliberately across the pipeline rather than an afterthought measured once at the end.
SEQUENTIAL — EACH STAGE WAITS FOR THE PREVIOUS ONE TO FINISH
STREAMED — NEXT STAGE STARTS ON PARTIAL OUTPUT
→ time flows left to right. The streamed version starts the next stage on partial output instead of waiting for the full result, so total time-to-first-audio shrinks even though each stage does the same amount of work.
Barge-in: the agent has to know when to shut up
Real customers interrupt. They talk over the agent mid-sentence to correct something, push back, or just because that's how they talk. An agent that can't detect this and keeps talking over the human doesn't just sound bad — it actively breaks the negotiation, because now the customer's actual objection (the thing that matters) got talked over and never registered. Getting barge-in right means voice activity detection has to run continuously against the live mic input, not just between the agent's turns, and the agent has to be able to cut its own TTS output mid-stream and re-plan based on what it just heard instead of finishing a now-irrelevant sentence.
Objections don't arrive pre-sorted into script branches
The naive design treats objection handling as a decision tree: if price objection, respond X; if timing objection, respond Y. Real objections on live calls are compound and ambiguous — a customer saying "this is more than I was expecting and I need to check with my team anyway" is a price objection and a stalling tactic and possibly a genuine approval-process constraint, stated in one breath. A script branch matches none of these cleanly. The agent needs to actually reason about which combination is in play and respond to the real one, which means the objection-handling layer can't be scripted responses — it has to be a model reasoning over the specific thing the customer said, grounded in real account data (via function calls back to pricing/account systems) rather than guessing at numbers, because a voice agent that hallucinates a discount it can't actually honor is worse than one that says nothing.
[FILL IN: a concrete call scenario that actually broke in production — e.g. a specific objection pattern the agent mishandled, a barge-in failure, a case where it quoted something it shouldn't have — and specifically what you changed (prompt, function-calling flow, VAD tuning, escalation rule) to fix it.]
Note
The honest failure mode to design for isn't "the agent says something dumb" — it's "the agent sounds confident while being wrong." On a sales call, confident-and-wrong closes worse than an escalation to a human, which is why the agent has an explicit bail-out path to a human rep for anything it's not grounded to answer.
What "handling" a call actually requires
End-to-end voice handling on real calls means the hard part was never the words the agent says — it's the timing of when it says them, catching when it should stop saying them, and refusing to say things it can't back with real data. Get those three right and the actual conversation design is the easy 20%.