Eval-driven AI deployment — our framework for shipping with confidence

"Ship it and see what happens" is the default deployment strategy for most AI projects. The problem is that AI agents fail differently from traditional software. A bug in a web form produces an error message. A bad answer from an AI agent looks exactly like a good one — the customer has no way to tell the difference until the damage is done.

This post covers the eval framework we use for every WhatsApp AI agent we deploy — what we test, how we categorize failures, the iteration loop that takes an agent from "mostly works" to "safe to deploy," and why 85% is the threshold that separates net-positive from net-negative AI.

Why "ship and see" fails for AI

Traditional software earns trust incrementally — every correct interaction builds confidence. AI agents work in reverse. A single wrong answer in the first hour can destroy trust permanently. The customer doesn't think "oh, a bug, they'll fix it." They think "this thing doesn't know what it's talking about" and they stop using it.

We've seen this pattern enough times to build our entire process around preventing it. Every agent goes through an eval harness before it touches a real conversation. No exceptions.

What we mean by eval

Not unit tests. Not "does it respond." Each eval case is a full conversational scenario: a user message (sometimes multi-turn), the expected behavior, and acceptance criteria. A typical case looks like this:

Case #047: 8GB VPS with NVMe
---------------------------------------
Input:    "Do you have an 8GB VPS with NVMe storage?"
Expected: Agent returns the correct plan name, correct
          specs (vCPUs, RAM, storage type, bandwidth),
          and does NOT mention plans that don't match.
Persona:  Goal-oriented
Pass if:  Plan name correct, specs match catalog,
          no hallucinated plans, tone matches persona.
Fail if:  Wrong plan, invented specs, mentioned
          a plan without NVMe, or expert-level tone.

Every case has explicit pass and fail criteria. There's no "looks about right" — either the response meets the rubric or it doesn't. This precision matters because vague eval produces vague improvements. When you know exactly why something failed, you know exactly what to fix.

The 6 failure categories

Every failed eval case gets tagged with one of six categories. This taxonomy drives the fix priority — we always address the highest-frequency category first.

1. Factual error — Wrong spec, wrong availability, wrong feature. The agent says a server has 32GB RAM when the catalog says 16GB. This is the most dangerous category because the customer has no way to know the answer is wrong.
2. Hallucination — Invented a product, plan, or capability that doesn't exist in the catalog. Distinct from factual error: a factual error gets a real product wrong, a hallucination creates a product from nothing.
3. Omission — Missed a relevant option the user should have seen. The catalog has three plans that match, the agent only surfaces one. Less dangerous than factual errors but still costs conversions.
4. Tone mismatch — Expert-level response to an explorer-level question, or vice versa. Technically correct but experientially wrong. The customer feels talked down to or overwhelmed.
5. Confidence failure — Answered when it should have escalated to a human agent, or escalated when it should have answered. Both directions are costly: false confidence spreads misinformation, false uncertainty wastes human agent time.
6. Format failure — Unreadable response. Wall of text with no structure, broken formatting, truncated mid-sentence, or WhatsApp-incompatible markup. The answer might be correct but the customer can't parse it.

How we write eval cases

We start from real customer patterns. Wherever possible, we work with actual conversation transcripts — the queries that appear repeatedly, the ones that lead to sales, the ones that lead to complaints, and the ones that stump human agents. These form the core of the eval suite.

From those representative cases, we write expected outputs for each. Then we add adversarial cases designed to break things:

• Trick questions — asking about products that don't exist, or combining specs that don't appear together in any plan.
• Out-of-scope requests — "Can you book me a flight?" The agent should decline gracefully, not attempt to help.
• Multi-language mixing — Hindi-English code-switching mid-sentence, which is extremely common in Indian WhatsApp conversations.
• Ambiguous abbreviations — terms that mean different things depending on product context.
• Multi-turn context traps — the customer contradicts themselves between turns to test whether the agent anchors on stale context.

Each case gets a pass/fail rubric — not subjective "does this look good" but specific "does the response contain X, avoid Y, and match persona Z." The rubric is the contract. If the rubric is unclear, we rewrite the rubric before re-running the eval.

The iteration loop

The cycle is simple but disciplined: run all cases, score them, group failures by category, fix the highest-frequency failure category first, re-run, repeat. No fixing multiple categories at once — that obscures which change produced which improvement.

A typical iteration trajectory looks like this: the first cycle exposes retrieval problems (wrong documents being pulled), the second exposes coverage gaps (missing multi-plan comparisons), the third exposes generation issues (tone mismatches, formatting). Each cycle addresses one failure category and lifts the overall pass rate.

Each cycle takes roughly half a day. The eval run itself is fast — a hundred-plus cases complete in minutes. The slow part is diagnosis: reading every failure, deciding which category it belongs to, and figuring out the root cause. That diagnostic work is irreducibly manual.

Why 85% is the go-live floor

85% is our go-live floor, not the target. It means "the agent is safe to put in front of customers with human oversight." The distinction matters because it changes what you optimize for.

Below 85%, the failure rate is high enough that human agents spend more time correcting the AI than they save. The AI becomes a net cost — it generates work instead of absorbing it. Every wrong answer needs a human to step in, apologize, and provide the right information. At scale, that's worse than having no AI at all.

Above 85%, the AI is net-positive. It handles the majority of queries correctly and escalates the rest. Human agents focus on the complex cases the AI can't handle, which is what they should be doing anyway. The AI absorbs the repetitive load, humans handle the edge cases.

We keep iterating after launch. But the 85% floor is what gates the launch itself. Ship below that line and you're paying for an AI agent that makes your support team's job harder, not easier.

What eval reveals that testing doesn't

In our experience, systematic eval catches failure modes that manual testing consistently misses:

1. Multi-product comparison queries — when a customer asks "which is better, Plan A or Plan B?", the agent needs to retrieve both plans and compare them on the dimensions the customer cares about. Agents without specific tuning for this tend to retrieve one plan and hallucinate the other's specs. Only systematic eval across many comparison queries reveals the pattern.
2. Ambiguous terminology — the same term can mean different things in different product lines. The agent conflates these, and manual testing rarely catches it because testers unconsciously use the right context.
3. Session context decay in multi-turn conversations — beyond several messages, agents tend to drop earlier context. A customer who specified their use case in message one and asks a follow-up in message six gets a generic response that ignores the use case. This only surfaces in multi-turn eval cases that simulate realistic conversation lengths.

All three of these are the kind of failure that looks rare in a demo but happens constantly in production. The demo query is always clean, well-formed, and single-turn. Production queries are messy, ambiguous, and cumulative. Eval bridges that gap — it forces you to test the messy cases before your customers find them for you.

Eval isn't the exciting part of building AI agents. It's the part that decides whether your agent earns trust or burns it. Every hour spent writing eval cases saves many hours of post-launch firefighting. Every failure category you identify before launch is a support ticket that never gets filed.

If you're evaluating whether your data is ready for an AI agent, our data onboarding service is where most engagements start.