Evaluation-Driven Development: CI/CD for Systems That Don't Return the Same Answer Twice

In traditional software, you write a test, it passes or fails, and you ship with confidence. The feedback loop is tight. The contract is clear. If assertEqual(output, expected) passes, you know the system works — for that case, under those conditions.

In agent systems, the same input produces different outputs every time. The screening agent evaluates the same resume and gives you a slightly different assessment on every run. The ranking agent reorders candidates with minor variations. The evaluation agent emphasizes different strengths depending on which tokens the model samples first.

Most teams respond to this reality by... not testing. They eyeball outputs, vibe-check a handful of examples, and ship. This is how production regressions happen silently. A prompt tweak that improves one case quietly degrades fifty others — and nobody notices until a customer reports that recommendations have gone sideways.

Why Traditional Testing Fails

The core issue is deceptively simple: deterministic assertions are meaningless for non-deterministic outputs. assertEqual(output, expected) doesn't work when the output is a 500-word candidate evaluation that varies on every run. You can't snapshot the expected output and diff against it. You can't hash it. You can't do exact string matching against prose that's generated fresh each time.

This creates what I call the eval gap — the space between "we can't use traditional tests" and "so we don't test at all." Teams recognize that their existing testing infrastructure doesn't apply. But instead of building new testing infrastructure that does apply, they default to manual review.

Manual review looks like this: someone on the team runs a few examples, reads the outputs, decides they "look right," and approves the change. There's no baseline comparison. There's no statistical rigor. There's no regression detection. There's a human reading three outputs and making a gut call.

The Three-Tier Eval Architecture

Evaluation-driven development isn't one thing — it's a layered system, and each layer catches a different class of failure. We learned this the hard way by building evals bottom-up after production regressions forced our hand.

Unit Evals

Test a single agent in isolation. Does the screening agent produce a coherent evaluation for a known-good candidate? Does it identify the right skills? Does it flag the correct gaps? You're not checking for exact output — you're checking for semantic correctness against a rubric.

Unit evals are fast, cheap, and catch the most common regressions: prompt changes that break a single agent's behavior. Run hundreds of these in minutes.

Integration Evals

Test agent chains. Does the screening agent → evaluation agent → ranking agent pipeline produce consistent rankings when given the same candidate pool? Integration evals catch the failures that unit evals miss — the cases where each agent individually performs well but the composition produces drift, amplification, or incoherent handoffs.

E2E Evals

Test full workflows against the gold standard: human expert judgment. Does the complete hiring pipeline — from resume intake to final recommendation — produce results that a senior recruiter would agree with? E2E evals are expensive and slow, but they're the only layer that validates the system's actual business value.

LLM-as-a-Judge

You can't have a human review every eval output — it doesn't scale. The solution that's emerged across the industry is LLM-as-a-judge — using a separate model to evaluate your agent's outputs programmatically.

Two dominant patterns have proven reliable in production:

1. Rubric-based evaluation. Define explicit criteria — completeness, accuracy, relevance, tone — and have the judge model score the output on each dimension. The rubric converts subjective quality into structured, comparable scores.
2. Pairwise comparison. Show the judge model two outputs and ask which is better. This sidesteps the calibration problem — models are more reliable at relative comparisons than absolute scoring.

The meta-problem is real: who evaluates the evaluator? If your judge model has systematic biases or blind spots, your entire eval pipeline inherits those flaws. Calibration against human labels is non-negotiable — you need a periodically refreshed set of human-judged examples that validate the judge model's alignment.

Regression Gates

This is the key concept — the thing that transforms evals from a nice-to-have into a deployment discipline.

Every prompt change, model upgrade, or pipeline modification must pass through a regression gate before reaching production. The gate runs your eval suite against a versioned dataset and compares the results against baseline scores. If scores drop below threshold — on any tier, on any metric — the change is blocked.

The workflow looks like this:

1. Developer makes a change — new prompt, updated rubric, model swap
2. CI pipeline triggers the eval suite against the canonical dataset
3. Results are compared to the last known-good baseline
4. If all metrics hold or improve — the gate opens and the change proceeds
5. If any metric regresses beyond the defined tolerance — the gate blocks and the developer investigates

The threshold question matters. Not every metric needs to be identical to baseline — you're dealing with non-deterministic systems, so some variance is expected. The key is defining statistically meaningful regression versus normal run-to-run variance. We run each eval multiple times and compare distributions, not single scores.

Eval Datasets as Living Artifacts

Your eval dataset is not a static fixture file you write once and forget. It's a living artifact — versioned, continuously enriched, and arguably the most valuable intellectual property in your AI stack.

Three sources feed a healthy eval dataset:

• Curated golden examples. Hand-labeled by domain experts. High confidence, high quality. These are your ground truth anchors.
• Synthetic data for edge cases. Use LLMs to generate adversarial inputs — unusual resume formats, ambiguous qualifications, conflicting signals. Edge cases that are rare in production but catastrophic when mishandled.
• Production failure recycling. Every production failure, every customer-reported quality issue, every flagged output becomes a new eval case. This is the feedback loop that makes your eval suite smarter over time.

Version your eval datasets with the same rigor as your code. Tag releases. Track changes. When a new eval case is added, document why — which production failure or edge case motivated it.

Observability Meets Evals

Runtime monitoring for agent systems requires metrics that traditional APM tools don't provide. Latency, error rate, and throughput tell you whether the system is running. They don't tell you whether it's working correctly.

Agent-specific observability signals close this gap:

• Semantic drift detection. Are outputs changing character over time? If your screening agent's evaluations are gradually becoming more verbose, more conservative, or more generic — that's semantic drift, and it's invisible to traditional metrics.
• Tool selection accuracy. Is the agent choosing the right tools for the task? A sourcing agent that stops using the skills-match tool and falls back to keyword search is functionally degraded — but its error rate is zero.
• Reasoning chain analysis. Is the agent's internal reasoning coherent? Incoherent reasoning that produces correct-looking outputs is a ticking time bomb — it means the agent is getting lucky, not getting it right.

The connection to evals is direct: your runtime observability signals should trigger eval reruns when they detect anomalies. Semantic drift detected? Run the eval suite. Tool selection patterns shifted? Run the eval suite. Observability and evals are two sides of the same quality system.

The Eval-Driven Development Loop

The full loop mirrors TDD — but adapted for non-deterministic systems:

1. Write evals first. Before building or modifying an agent, define what "correct" looks like. What rubric dimensions matter? What's the acceptable score range? What edge cases must be handled?
2. Build or modify the agent. Implement the change — new prompt, updated tool, different model.
3. Run the eval suite. Execute all three tiers against the versioned dataset. Compare against baseline.
4. Iterate until green. If evals regress, investigate and fix. Don't ship until the gate opens.
5. Gate deployment. The passing eval suite is the deployment criterion — not manual review, not gut feel, not "it looks right."

The CI/CD of the AI Era

We spent decades as an industry building the discipline of automated testing, continuous integration, and deployment gates for deterministic software. That discipline is what allows teams to ship with confidence — not because they're sure nothing will break, but because they have automated systems that catch regressions before they reach users.

Agent systems need the same discipline — adapted for non-determinism, built on statistical comparison instead of exact matching, and layered from unit evals through E2E validation. The tooling is still maturing. The patterns are still being established. But the core principle is identical: if you can't measure it, you can't ship it with confidence.