rubricHQ

Voice agent QA

Find why your voice agent fails. Fix the prompt. Ship it.

Simulate real callers, see where calls break, and get the prompt change that fixes it.

Start free

200 free credits · no card

Greeting1,240Verification1,120Balance1,010186Payment options72064Paid in full430Payment plan180Ended early630

290 conversations ended after hearing the balance

OpportunityFailureBalance stage

Agent has no answer when the caller disputes the balance

186conversations
Fix suggested
Offer the account statement before asking for payment.

Works with

Vapi
Retell
LiveKit
Pipecat
ElevenLabs
OpenAI

Listen

Same caller. Same question. One prompt change.

Press play on the first call. Then hear the same call after RubricHQ found the mistake and fixed the prompt.

1 · Before

Current prompt0:00 / 0:00
  1. Sam · AI agentHi, this is Sam from Brightline Lending. Am I speaking with Dana Lopez?
  2. Dana · CallerYeah, that's me.
  3. Sam · AI agentThanks, Dana. I'm calling about your account. There's a balance of eight hundred forty-two dollars.
  4. Dana · CallerWait, what? Eight forty-two? Nah, I paid most of that last month.
  5. Sam · AI agentYour current balance is eight hundred forty-two dollars.
  6. Dana · CallerOkay, but, like... I've got the receipt. Can you just check?
  7. Sam · AI agentI can help you make a payment on the balance today.
  8. Dana · CallerUm, no. I'm not paying for something I don't owe.
  9. Sam · AI agentWould you like to pay the full amount, or—
  10. Dana · CallerYeah, you know what, forget it. I'll call my bank. Bye.
Caller hung upExample call, generated for illustration.

2 · What RubricHQ found

OpportunityFailureBalance stage

Agent has no answer when the caller disputes the balance

186conversations
What goes wrong
When a caller says the balance is wrong, the agent repeats it and pushes on to payment.
Prompt change applied
+ If the caller disputes the balance, acknowledge it and offer to email the statement before asking for payment.

Spotted across 186 calls, fixed in Prompt Optimizer, then re-tested on the same caller.

3 · After

Optimized prompt0:00 / 0:00
  1. Sam · AI agentHi, this is Sam from Brightline Lending. Am I speaking with Dana Lopez?
  2. Dana · CallerYeah, that's me.
  3. Sam · AI agentThanks, Dana. I'm calling about your account. There's a balance of eight hundred forty-two dollars.
  4. Dana · CallerWait, what? Eight forty-two? Nah, I paid most of that last month.
  5. Sam · AI agentOh, okay, let's check that. I'll email you the statement so we're looking at the same thing.
  6. Dana · CallerUh, yeah. Sure.
  7. Sam · AI agentSent. Does anything look off?
  8. Dana · CallerHmm... yeah, the June payment's not on here.
  9. Sam · AI agentGot it. Reply with that receipt and we'll fix it. For what's left, would three monthly payments help?
  10. Dana · CallerOh. Yeah, okay, I can do that.
  11. Sam · AI agentGreat. You'll get a text to confirm. Thanks, Dana.
Payment plan agreedExample call, generated for illustration.
Start free

A pass/fail score doesn’t tell you what to change.

Three things that go wrong with voice agents in production, and how RubricHQ handles each.

1

Prompt edits break flows nobody re-tested.

You fix one answer and quietly break another. Scheduled test runs replay your scenarios after every change, before customers hit it.

Fixed by Simulate & Monitor ↓
2

Callers hang up, and you can't see where.

An average score hides the moment people leave. Flow shows the exact stage where calls end, and how many.

Fixed by Flow & Opportunities ↓
3

The same mistake repeats across hundreds of calls.

Each call looks fine on its own. RubricHQ groups the pattern into one opportunity and writes the prompt change that fixes it.

Fixed by Optimize ↓

1 · Simulate

Hear your agent fail before a customer does.

Paste your prompt and RubricHQ writes the scenarios, then runs them as real calls through Vapi, Retell, LiveKit or Pipecat.

app.rubrichq.io/live-simulations/scenario
A
Acme Health's Orga…
Owner
Platform
Dashboard
Test Agents
Co-Pilot
Metrics
Analytics
Prompt Optimizer
Evaluation
Live Simulations
Batch
Observability
Conversations
Notifications
Settings
A
Live Simulations
ScenarioPersona LibraryConversation AttributesAutomated Schedules
Manage and simulate your test scenarios.
Search scenarios...1P Creditor (Vapi)Generate by AICreate Scenario
NameTypeLanguagePersonaMetrics
Verify bot authentication2InstructionsEnglishProfessional CallerVoice Change Detection+18
Happy Customer Debt ExtensionInstructionsEnglishInterruptive PersonaVoice Change Detection+15
Sales - Product InquiryInstructionsEnglishProfessional CallerUser Greeting+1
Customer Support - Billing IssueInstructionsEnglishProfessional CallerCustomer Satisfaction
Payment Extension — Bad Connection (volume + network tags)2ConditionEnglishProfessional CallerUser Greeting
Sales - Product InquiryInstructionsEnglishProfessional CallerCustomer Satisfaction
Customer Support - Billing IssueInstructionsEnglishProfessional CallerCustomer Satisfaction
After-Hours Voicemail CaptureInstructionsEnglishProfessional CallerUser Greeting+3
Dispute Charge — Escalation Path3ConditionEnglishInterruptive PersonaCustomer Satisfaction+9
Page 1 of 6Prev123456Next
Simulated scenarioWrong-party contact0:00 / 0:00
  1. Sam · AI agentHi, this is Sam from Brightline Lending. Am I speaking with Jennifer Reyes?
  2. Mark · CallerUh, no, this is her husband. She's at work.
  3. Sam · AI agentNo problem. I'm calling about Jennifer's personal loan.
  4. Mark · CallerOh. Okay... is, uh, is something wrong?
  5. Sam · AI agentIt's sixty days past due, with a balance of two thousand, three hundred and ten dollars.
  6. Mark · CallerWait, what? She didn't tell me about any of that.
  7. Sam · AI agentWould you like to make a payment on her behalf today?
  8. Mark · CallerUh... no. I'll, um, I'll talk to her tonight.
  9. Sam · AI agentSure thing. Thanks for your time, Mark. Have a good evening.
Account details shared with the wrong personExample call, generated for illustration.
Start free

2 · Evaluate

Score every call on what matters to you.

LLM-as-judge, code-as-judge and audio metrics on every call. An AI generator writes new metrics, and you switch each one on or off per agent.

app.rubrichq.io/batch-run/159/run/157
A
Acme Health's Orga…
Owner
Platform
Dashboard
Test Agents
Co-Pilot
Metrics
Analytics
Prompt Optimizer
Evaluation
Live Simulations
Batch
Observability
Conversations
Notifications
Settings
A
Batch RunsRun #157
Verify bot authentication
Nightly Regression · Jun 19, 2026 · 11:42 PM
02:45EvaluatingRerun Metrics
Audio analysis in progress — scoring metrics across the audio and transcript...
Conversation Metrics
LatencyComputing
User interrupting AIComputing
Custom LLM MetricsLLM-as-judge
Authentication VerifiedCriticalComputing
Customer SatisfactionComputing
Script AdherenceComputing
Tone & EmpathyComputing
Hallucination CheckCriticalComputing
00:38 / 02:45
Transcript7 turns
Agent00:02

Thanks for calling Acme Support, this is Ava — who am I speaking with?

Caller00:06

Hi, it's Sarah Miller. I'd like to check my balance.

Agent00:10

Happy to help. First I need to verify you — date of birth, last four of your SSN, and ZIP code?

Caller00:17

March 14th 1990, 4821, and 94107.

Agent00:23

Perfect, you're verified.

Agent00:27

Your current balance is $1,240.50.

Caller00:31

Hmm, that seems higher than I expected.

Built in

  • LLM as judgeScore tone, accuracy and policy in plain English
  • Code as judgeExact checks for numbers, fields and tool calls
  • Hundreds of caller personasMix mood, accent, language and background noise
  • Auto-scheduled runsRe-run every scenario on a schedule
  • GitHub CI/CDBlock a pull request when a prompt change fails
Start free

3 · Monitor

Know the moment live calls start going wrong.

Every production call is scored as it happens. When a metric slips, like slow responses or a policy the agent didn't read, your team hears about it in Slack.

Live production callsCollections agent · 1284 today
+1 ••• 4821
Payment plan
820 msPass
+1 ••• 6107
Hardship policy · Policy not read
880 msFail
+1 ••• 2290
Payment options
840 msPass
+1 ••• 7765
Balance · Slow response
3,610 msFail
+1 ••• 3348
Paid in full
690 msPass
+1 ••• 9012
Hardship policy · Policy not read
910 msFail
+1 ••• 5530
Verification
760 msPass
# voice-agent-alertsSlack
RubricHQAPP10:42 AM
Latency over 3,000 ms
5 calls in the last hour took more than 3,000 ms to respond. p95 is 3,420 ms (usually 1,100 ms).
View 5 calls

Alerts in your Slack channel

Rules check any metric on a schedule

Scheduled runs and a GitHub Action

Catch a broken prompt before it ships

Start free

4 · Optimize· On Growth

Get the prompt change, not just the score.

Give it your good calls, your bad calls and the opportunities RubricHQ found. It writes the prompt change, shows you the diff, and waits for your approval.

Good calls3 selected

“Let’s check that. I’ll email the statement…” → plan agreed

Bad calls3 selected

“Your current balance is $842.” → caller hung up

Opportunityfrom RubricHQ

Agent has no answer when the caller disputes the balance

186 conversations

Prompt Optimizer
  1. Reads your good and bad calls
  2. Adds the opportunity RubricHQ found
  3. Finds what the good calls did differently
  4. Writes the prompt change
6calls1opportunity2changes
Collections agent promptSuggested by Prompt Optimizer · waiting for reviewv6v7
+2−1|FixesCaller disputes the balance · 186 calls
13State the balance and the due date clearly.
14Restate the balance and ask for payment.
15If the caller disputes the balance, acknowledge it and offer to email the statement before asking for payment.
16If they can’t pay in full, offer a payment plan.
17Keep replies to one or two sentences.
Approve v7Publish to agentRevert to v6 any time

Approve each change, revert any time

Nothing goes live until you say so

Ship it where your agent runs

Push to your voice platform, or copy the new prompt

5 · Co-Pilot· On Growth

Ask your calls anything.

“How many calls ended with only a partial payment?” Pick a time range and a source, get a table, and export it to CSV.

app.rubrichq.io/copilot
A
Acme Health's Orga…
Owner
Platform
Dashboard
Test Agents
Co-Pilot
Metrics
Analytics
Prompt Optimizer
Evaluation
Live Simulations
Batch
Observability
Conversations
Notifications
Settings
A
Co-Pilot

Ask natural language questions about your call data.

New chat
Recent
Top fixes to lift resolution
Why callers hang up early
Requests the agent can’t handle
Objections before payment
Agent mistakes this week
Escalation reasons by agent
Where are we losing resolutions, and what should we fix first?
Deep read complete · 1,284 calls read · last 7 days

312 calls ended unresolved. Three fixable causes account for 71% of them. The biggest is a request the agent has no way to handle.

Opportunities · ranked by calls affectedof 312 unresolved calls
1
Missing workflowCustomers ask to reschedule a payment
Fix: Add a reschedule flow instead of transferring
118 calls38%
2
Caller objectionCallers think it is a scam at the greeting
Fix: State the company name and a callback number up front
63 calls20%
3
Agent mistakeBalance quoted before identity check
Fix: Verify identity before mentioning any account detail
41 calls13%
Ask anything about your calls…
Ask
AutoDataExplore
Try asking
Why do callers hang up early?
Which objections come up before payment?
What requests can’t the agent handle?
Top fixes to lift resolution this week

Production, test runs, or both

Answers use your own calls only

No query language

Product, ops and QA can all use it

Start free

6 · Flow & Opportunities· On Growth

See where your calls go wrong, and what to change.

Every conversation gets a path through your agent's stages. Mistakes that keep repeating become a short list of prompt changes.

app.rubrichq.io/flow
A
Acme Health's Orga…
Owner
Platform
Dashboard
Test Agents
Metrics
Co-Pilot
Flow
Opportunities
Prompt Optimizer
Evaluation
Live Simulations
Batch
Observability
Conversations
Notifications
Settings
A
Flow
Observability (Production)SimulationsClick a stage or band to see its conversations
Biggest divergence: 29% of conversations that reach Balance end there
Greeting1,240Verification1,120Balance1,010186Payment options72064Paid in full430Payment plan180Ended early630

290 conversations ended after hearing the balance

Ideal and problem paths

In production calls and in simulations

Recurring mistakes, ranked

Each with example calls and the prompt change that fixes it

Pricing

Start free. See every price up front.

Pay as you go

$0

200 credits, no card, valid 14 days. Then $1 = 25 credits.

Start free

Growth

$399/mo

12,000 credits a month and the full product: Optimize, Co-Pilot, Flow, alerts, scheduled runs, SSO.

Start free

Enterprise

Custom

Custom credit volume, unlimited team members, on-prem, and an SLA.

Questions

Which voice platforms does it work with?

Vapi, Retell, LiveKit and Pipecat. Connect an agent managed in Vapi, or a self-managed agent with an inbound number.

How long does setup take?

Minutes. Paste your agent prompt, RubricHQ generates scenarios, and you run your first batch of test calls straight away.

Do test calls use credits?

Yes. A phone simulation is 5 credits, a web simulation 3, and a text simulation 0.5. Each metric scored on a call is 0.2. Your 200 free credits cover a first round of tests.

What happens after the free credits?

Top up any time at $1 = 25 credits, or move to Growth at $399/mo for 12,000 credits a month and the full product. Your data and setup stay as they are.

See it on your own agent.

Pick a 30-minute slot and we’ll run RubricHQ on your agent with you. Or start free right now with 200 credits, no card.

Start free