We test voice agents for a living and that usually means that we see them fail in ways their creators never could have anticipated. Two in recent times. One agent was in charge of a settlement conversation and gave the customer a number that just wasn't right – one that had a definite meaning, but for someone who didn't write it down, it was just an embarrassment. A third agent, after just a few seconds of a call, claimed it would be a great pleasure to speak with the customer by his or her name. It had heard the name, and in the model that formed its identity was
Both of those agents entered into manual QA. Of course they did. Manual QA is 12 humans calling the person who built the thing, and a 12 human call cannot contain a p95 latency spike, a tool that silently 500s at 2am, or the one person with a thick accent, a barking dog and a 15 digit number in the account number. You can't hear most of these failures live, either. A number you've hallucinated sounds as much like a real correct number as a real correct number.
Here's the catalogue: 12 ways real traffic breaks voice agents. They don't all get equal space. Some have bit us so severely they warrant a few paragraphs; some a sentence. Not one of these is in relation to a specific stack.
When the words are wrong
Made-up facts and numbers
It's the one that keeps me up and the reason is the settlement number. The agent's figure was available to him and he quoted another one out loud with complete conviction. When a number is a promise, a confident wrong answer is not an embarrassment, in collections, lending, insurance. It's potentially binding. Explain to the customer that it is a waiver that they are eligible for and they will be expecting it.
These are not to be caught by hand, for they dwell in the questions you did not think to ask. You take tests for the policies that you have learned by rote. The model fills in answers to the blank.
What's effective is to go hunting for them: adversarial scenarios where fish are looking for details that the prompt never provided the AI (an unusual fee, a hypothetical eligibility case, a settlement on a weird case balance), and each transcript is judged by a judge who verifies each number and claim against your actual source of truth. The bar is not "did it sound right". It's "did it state something that is not covered by the knowledge base".
The identification and naming of things.The naming of things and confusion.
The name swap deserves a line of its own thanks to its clean example of the weird tail. It's rare. It's a little funny, isn't it? And you will never get it in ten friendly tests calls, as it requires a real name, timely, with a special rhythm, before the model knows who it is. It's not a "patch the name bug" lesson. It's because the odd failures happen and nothing else can happen before the customer encounters it but to execute sufficient calls such that the tail actually appears.
The back-and-forth nature of the relationship.The give and take of the relationship.
The lack of compliance and disclosures
There is no disclosure reading. No consent to record, no recording. The mini-Miranda is never called during collections time. Account information is sent out before verification of who is on the line.
This one is a good one to hide, as the disclosing event almost always occurs in the happy case. It's the interrupted call, the warm transfer, the callback that resumes mid-conversation where it quietly drops. So no need to rely on human judgment. Create a code-as-judge rule that says that disclosure is positioned before the first account detail, and apply it to all transcripts. That's not what a hundred subjective listens provides, and it's the only thing that provides it.
It can become disheveled at the ends of the turns.
The caller provides their account number in turn 3 then the agent asks for it in turn 11. Or worse, it retains a bad value and does something with it. Short test calls never arise this; the context decay is in the 8 minute call with a correction, a tangent and a hold in the middle.
Test with long, deliberately messy scenarios, in which the caller walks things back half way through the sentence ('sorry, that's four-four-two'), and see which one it was that got to your tool call. The first one or the correction.
Barge-in
The caller cuts in the agent selects one of three wrong answers. It plows through its sentence. It halts in its tracks and goes off the story. Or, the ugly one, where it turns off, but preserves the interrupted text as though spoken, in the context. You will never see this by hand, because it's a polite tester waiting his turn. There is a lot of interruption from real customers. The simulated caller requires someone with an "interruptive personality" (jazz jargon for someone who talks over you at random times and sometimes in the middle of a word) and then you observe the callers reaction, whether they were able to recover and answer the question.
Dead air
It takes four seconds for the tool call, the agent doesn't say anything, and the caller says "hello??" as a new turn and it goes off track. You wait and wait to test, for you know the bot is thinking. Your customer hears nothing after two seconds and thinks that the call dropped. Instead of using the average gap over 5 calls, measure the actual gap over hundreds of calls and ensure a filler phrase is firing when it should.
Talking over people
Related but different: the agent decides you're done when you have just paused to think, and begins speaking. It is about turn-detecting tuning, and those people you can least afford to punish slow talkers, non-native speakers and anyone reading digits off a card are the people who get punished. Of course it behaves for you, you test it fluently, quickly, in your accent, and in a quiet room. Repeat similar scenario with each persona for different cadence, scoring for overlap (when agent begins to speak and caller continues).
What it hears, what it says!
A value is corrupted by the speech layer.
Two failures and one root. A quick look at the path into the road and “fifty” becomes “fifteen”, a name is garbled, a confirmation number loses a letter and the agent clicks on the wrong item. When leaving, the correct answer is mispronounced: a date without any indication of the month, a number without the decimal point, an account number that is said before it can be typed in.
Both manage to pass a human oversight process, as well. You are speaking loud enough, and the bot doesn't hear you correctly, you subconsciously repeat yourself and smooth over the exact thing you were looking for. It's easy to hear what you expected to hear when you know the correct answer. Hence the need to test at the audio level using an accent variation, different speech rates, various background sounds and evaluate the audio. By the construction of the evaluation, if it sees only clean text, this whole category is invisible to it.
Timing and plumbing
Latency in the tail
You take a nice 800 milliseconds to respond. Your p95 is four seconds, and you lines up right on your calls that are in your one slow tool, which is typically the calls that are most important. Ten calls do not make a p95. This isn't a discipline issue, it's math. Perform multiple simultaneous calls to achieve a true distribution, divide latency by turn type and forget about the "average.Perform multiple concurrent calls for a real distribution, separate latency by turn type and don't bother with the "average". The important number to consider is the frequency with which you reach the point where callers begin talking over the bot, and what code path you're taking to get there.
The first ten items are all the ones that are covered up quietly:
The lookup times out, fails with a 500 or returns empty - the model does what models do and creates a smooth plausible sentence. The caller is informed that the appointment is scheduled. Not confirmed. Your test tools work, production's don't, sometimes, at the worst hour.
So smash them on the purpose. Test for scenarios where the backend errors, it returns nothing, or it crawls and the agent tells a lie about it (a happy ending). You should never have experienced a deliberately broken tool until you have tested it yourself – then you will know what your agent says when things go wrong. Only when they go right do you know what it says.
Broken handoffs
The call is lost when it's transferred to a human, or it connects, but no context, and the customer begins all the way from the beginning with another agent who is already annoyed. Everyone runs tests on "did the transfer connect", the easy half. No one checks the hard half: Is the summary that went with it accurate? On both, assert, and remember, that being too hasty is a failure, and being not hasty is a failure.
Voicemail and IVR
On outbound, the agent speaks its entire pitch to a voicemail greeting or waits for a man or woman and an IVR asks the agent to press 1. No one considers answering machine as a conversation and, therefore, tests against it. It remains an experiment, though. Test scenarios with voicemail and IVR trees to ensure that the agent is able to hear them and take the appropriate action: leave the message, press the key or hang up.
What is the common factor of all the numbers 12?
Align them and the same shape will always be formed. There are three reasons that manual testing may not have detected the failure: the failure was not present in the sample, the testers were not evil enough to cause the failure, or the failure was not audible to a human during the test. Reasons one and three simultaneously were the settlement number. First of all, the name swap.
There's no solution to that through harder testing. It is corrected with different types of testing: with an disagreeable caller, with a sample large enough for the rare stuff to emerge, and with a score on each call versus a memory of whether it "sounded fine. That's the entire loop, regardless of which object you use to make it. Create hostile, long noisy, weird scenarios, play them loudly! For each call have a corresponding explicit check, where possible, a deterministic check and a judged check. Adjust the tuning, the tool or the prompt. Then replay the same suite and show that the number has changed, if it hasn't then the fix was a guess.
We didn't build this as if it could not be done wrong. We constructed it so that you know your failure rate, mode by mode in advance of a customer seeking you out.
If you'll only remember one thing, here it is: Choose the top three failures from the list above that would affect a real customer the most if they occurred tomorrow and write a test for each of them. This alone is a head start over most voice AI teams that are shipping right now, and it's an afternoon project.