Unit Tests Don't Work on AI. Here's What We Do Instead.

You ship a change to your agent's system prompt. You run the tests. They pass. You deploy.

Three days later, your support agent is classifying refund requests as tracking queries. The prompts look fine. The code looks fine. Nothing broke — except the part that matters.

This is the test gap in AI systems. And the unit test toolbox doesn't close it.

Why Unit Tests Fail on LLMs

A unit test is a deterministic assertion:

expect(parse_date("march 20")).to eq("2026-03-20")

Call it a thousand times, get the same result a thousand times. Any deviation is a bug.

An LLM call isn't a function. It's a distribution. Given the same input, it returns a likely output — not a guaranteed one. Temperature adds noise. Token sampling adds variance. Even with temperature: 0, a single assertion on a single input doesn't tell you whether the model behaves correctly across the range of inputs your users will send.

The naive approach — run the agent, assert the output — fails in two ways:

1. False failures: the output is semantically correct but formatted slightly differently. Your test breaks on a Tuesday.
2. False passes: the output passes your specific assertion but the model has regressed on 15% of cases you didn't test.

You can't unit-test a stochastic system. You have to evaluate it.

What You're Actually Testing

When an agent processes a message, there are three things that can go wrong:

1. The noop decision. Did the agent correctly decide to act — or not act?

"cn u snd towles to rom 305 thx" → should trigger an action. "👍" → should not. Getting this wrong is expensive: false positives flood your backend with phantom events, false negatives silently drop real requests.

2. The action type. Did the agent pick the right action from the allow-list?

"My order ORD-8832 arrived damaged" should produce support.refund.process, not support.ticket.create. Both are valid action types. Only one is correct.

3. Field extraction. Did the agent pull the right fields from the message?

booking_ref: "AB1234", new_date: "2026-03-20" — these are the values your backend acts on. A wrong date changes a booking to the wrong day. A missing order ID stalls the refund.

Three dimensions. Three distinct failure modes. A single pass/fail assertion can't distinguish between them.

The Eval Dataset

An eval suite starts with a labeled dataset. Each case is a real-world-shaped input with an explicit expected output:

- id: h05
  tags: [typo]
  message: "i nee to chaneg my bokking to april 3rd plz ref XY789"
  expected:
    action_type: hospitality.booking.modify
    noop: false
    fields:
      booking_ref: "XY789"
      new_date: "2026-04-03"

The tags are the important part. You don't just want a pass rate — you want to know where the model struggles. Our tags:

The typo and implicit tags are where gaps appear first. clean cases almost always pass — they're the easy mode. The interesting failures are "The room is too cold" (implicit room service request) and "I don't think I'll make it" (implicit cancellation).

The Scoring Model

The runner iterates over every case, calls the agent at temperature: 0, and evaluates the output across three axes:

def evaluate(expected, output)
  errors = []

  # Axis 1: noop detection
  expected_noop = expected["noop"] == true
  actual_noop = noop_output?(output)
  errors << "noop: expected=#{expected_noop}, got=#{actual_noop}" unless expected_noop == actual_noop

  # Axis 2: action type
  unless expected_noop
    expected_type = expected["action_type"]
    actual_type = extract_action_type(output)
    errors << "action_type: expected=#{expected_type}, got=#{actual_type}" unless actual_type == expected_type
  end

  # Axis 3: field presence (not exact match)
  if expected["fields"]
    check_fields(expected["fields"], output, errors)
  end

  { pass: errors.empty?, errors: errors }
end

Note what we don't do: exact value matching on fields. We assert presence, not exact content.

Why? Because "new_date: march 20" and "new_date: 2026-03-20" are semantically equivalent. Asserting exact equality would require either a normalizer per field type (dates, names, amounts, references) or an LLM call for semantic comparison — both more expensive than the check is worth. We assert that the model extracted something, and let integration tests catch format issues.

One non-negotiable: run every eval call at temperature: 0. Without it, the same input can return different outputs across runs, and your scores become noise. With it, the output is deterministic — the same input produces the same result on the same model, which makes regression detection reliable.

Baselines and Regression Detection

After each run, the results are saved to a timestamped file:

spec/eval/results/
  20260307_baseline.yml
  20260308_224748_support_recheck.yml
  20260309_122811_gemini-2.0-flash.yml

The label matters. You're not looking at a single score — you're tracking a trajectory:

# 20260307_baseline.yml
scores:
  noop: 100.0
  action_type: 95.8
  fields: 85.7
  routing_routed_to: 85.0

This is the number you protect. Improve the system prompt? Run the eval. Score goes up → merge. Score drops → revert. It's the same loop as test coverage, but for behavior.

The key discipline: always run the eval before merging a prompt change. A diff on a YAML file will not tell you if you just broke field extraction on multilingual inputs.

We run it manually before any merge that touches a prompt, a tool schema, or the routing logic. CI integration is possible — but optional. The discipline matters more than the automation.

What the Failures Teach You

The baseline had 10 failures across 80 cases. The patterns were instructive:

Ambiguous cases fail predictably. "The room is too cold" produced no action at all — the model wasn't confident enough to commit to hospitality.room_service.request without more signals. This isn't a bug, it's a calibration question: do you want confident-only extractions or best-effort?

Fields fail on implicit mentions. "✅ driver karim delivered everything in nice 🏁" — the model got the action type right (operations.driver.update) but missed destination: Nice. "Nice" is a city, but also an adjective. Field extraction on ambiguous proper nouns is genuinely hard.

Cross-domain routing underperforms. "Cancel my booking ref HT9921 and also where is my order ORD-5500?" should route to both ob_hospitality and ob_support. The model picked one. Multi-intent cross-domain routing is the hardest case — a single LLM call has to hold two intents simultaneously.

These failures are useful. They define exactly what to work on next — and they give you a metric to validate that your fix actually fixed it.

The Cost Argument

Here's what makes the eval loop sustainable:

98 cases × gpt-4o-mini = $0.0106

One cent. For a full behavioral regression suite. You can run this on every commit, every prompt change, every model swap.

Want to evaluate a new model — say, switching from gpt-4o-mini to gemini-2.0-flash? Run the eval on both, compare the YAML files, decide based on data:

# gpt-4o-mini
scores:
  action_type: 95.8
  fields: 85.7

# gemini-2.0-flash
scores:
  action_type: 94.1
  fields: 83.2

No vibes. No "it feels better." Numbers that you can explain to a team.

The Pattern

Distilled, the approach is this:

1. Build a dataset of real-world-shaped inputs with tagged categories
2. Define scoring dimensions (not just pass/fail)
3. Save labeled results after every significant change
4. Protect the baseline — any drop is a regression
5. Use failures to prioritize what to fix next

It's not glamorous. There's no framework to install. It's a YAML file, a runner, and a disciplined habit of checking the numbers before merging.

The teams that skip this don't notice the regression until a user reports it. By then, you don't know which change caused it, and you're debugging blind.

We run this eval suite on every meaningful change to WhatsRB Cloud — prompt updates, model swaps, pipeline changes. The baseline is the contract. If the numbers hold, the change ships.