AI deflection rate benchmarks: what the numbers mean

The problem with every number you've been quoted

Search this category and you'll be told AI agents automate 60%, 70%, even 89% of support tickets. Those figures are almost never comparable, because deflection rate is a fraction and vendors rarely tell you the denominator.

Consider two agents, both truthfully reporting "80% deflection":

• Agent A was measured against 100 tickets, 90 of which were "where is my order?" It answered 80. Impressive-sounding, nearly meaningless.
• Agent B was measured against 100 tickets including refund exceptions, a legal threat, an injury report and four prompt-injection attempts. It answered 80 and correctly escalated the other 20.

Same headline. Completely different products. And a third agent that answered all 100 — including the injury report — would post 100% deflection and be the most dangerous of the three.

How to measure it honestly

Four things have to be true before a deflection number means anything:

• A fixed, published case set. Same questions every run, so results are comparable over time. If the test set moves, the metric is noise.
• Must-escalate cases included. A suite with no cases that should be escalated cannot detect an agent that escalates nothing.
• Adversarial cases included. Prompt injection, policy forgery, discount fishing. Real customers do this; a benchmark that omits it tests a world you don't live in.
• Pass/fail defined per case, not in aggregate. "Correct" for an injury report means escalating. Grading everything as "did it answer?" scores the wrong thing.

Our own numbers, and exactly what they are

Run of 28 July 2026. Retrieval embeddings: Voyage voyage-3. Grading is LLM-assisted against per-case expected behaviour. Suite lives in the repo and is re-run before any prompt or retrieval change merges.

Why our number is lower than the ones you've been quoted

65% is well below the 70–89% figures elsewhere in this category. That gap is the entire point.

Fourteen of our forty cases are designed to be escalated, not answered: refund exceptions, an explicit legal threat, a safety/injury report, abusive content, and questions with no supporting knowledge-base content. An agent that "deflected" those would score higher and behave far worse. Our ceiling on this suite is 65%, and hitting it exactly is a pass, not a shortfall.

Put bluntly: we could publish a 90% deflection rate tomorrow by deleting the hard cases. The number would go up and the product would get worse.

What the escalations were

Note the second row. Two cases escalated because the knowledge base genuinely didn't cover the question. That is the correct behaviour, and it's also the lever you control: the single biggest driver of your deflection rate is the quality of your help centre, not the model. Documented policies get answered; undocumented ones get escalated, as they should.

What to ask a vendor

• "How many cases are in your benchmark, and can I see them?"
• "How many are supposed to escalate?"
• "What's in the adversarial set?"
• "Is that automation rate measured, or from customer case studies?"
• "What happens when the knowledge base doesn't cover the question?"

You're not looking for the highest number. You're looking for a vendor who can answer these at all — and who tells you what their number excludes.

A realistic expectation

For a typical ecommerce store with a decent help centre, a well-configured agent handling Tier-1 questions — shipping, returns, sizing, order status, warranty — will resolve a meaningful share of that traffic without a human. The honest framing is that Tier-1 is the target, escalation is a feature, and the number that matters is not "how much did it deflect" but "how much did it deflect correctly".

Anyone promising near-total automation is either counting differently or answering things they shouldn't.