AI evaluation and agent systems

How DoorDash Evaluates an AI Agent: A Practical Case Study

An AI agent can sound helpful while choosing the wrong tool, ignoring a constraint or acting on unsupported information. DoorDash’s Ask DoorDash evaluation system offers a practical lesson: assess the complete interaction, make the evidence reviewable, and build repeatable tests before changes reach customers.

What does AI agent evaluation mean?

An AI agent evaluation is a structured way to check whether an agent completes a user’s task safely and correctly. A chatbot’s reply can often be judged as a single piece of text. An agent, however, may hold a multi-turn conversation, call tools, receive data, make decisions and show results in an interface.

That means a plausible-sounding final answer is not enough evidence of success. The agent might have misunderstood the request, selected the wrong action or made a claim that its tool results do not support. A useful evaluation therefore looks at both what the user saw and what the system did to produce it.

DoorDash’s Ask DoorDash assistant is designed to help people discover restaurants or shop for groceries through conversation. Its evaluation case study focuses on how to assess the quality of those interactions, not just individual model responses.

How the DoorDash evaluation system is structured

Ask DoorDash uses a multi-agent setup. An Orchestrator receives a user request and routes it to a specialised agent, such as one for restaurant discovery. That specialist may handle the request or return control to the Orchestrator if the conversation needs a different route.

The evaluation system follows this structure, checking each part where its behaviour can be measured most directly:

  • Routing: Did the Orchestrator send the request to an appropriate agent?
  • Guardrails: Did the interaction meet shared requirements such as communication quality and trust?
  • Task capability: Did the specialist agent satisfy the user’s requirements for that particular task?

This separation makes failures easier to diagnose. If a restaurant-search task goes wrong, the team can investigate whether the request was routed incorrectly, a general rule was broken, or the restaurant agent failed a task-specific requirement.

Rubrics turn open-ended goals into checks

Many tasks have several valid outcomes. A person looking for a restaurant may care about budget, delivery time and dietary needs, but there might be more than one suitable recommendation. A good evaluation should reward acceptable paths rather than compare every response to one prewritten “correct” answer.

DoorDash uses rubrics: sets of criteria describing what success looks like. In the case study, individual criteria are graded as binary checks and combined into a session-level score. The judge first checks whether a criterion applies to the current conversation. If it does not, that item is skipped rather than counted as a failure.

Example: A restaurant-discovery rubric might check whether recommended options satisfy the user’s stated budget, delivery-time and dietary constraints. A grocery-shopping rubric might check whether requested changes were made to the shopping list.

The criteria need to be specific enough for consistent grading, but broad enough to accept different valid outcomes. They also need evidence. DoorDash records agent sessions as traces that include user inputs, model outputs, tool calls and responses, and interface elements shown to the user.

A transcript builder converts those traces into a more focused view for evaluation. It reassembles information from different steps, removes material that does not help assess quality, and selects the evidence needed for each criterion. For instance, a check about whether a claim is grounded can receive the claim alongside the relevant tool output, rather than an entire noisy session.

Simulation makes pre-launch tests repeatable

Testing against live users is not a safe way to rehearse a change, and a past conversation may not be reproducible if the underlying data has changed. DoorDash addresses this with a conversation simulator: a simulated user follows a scenario, while fixtures provide fixed responses for relevant tool calls.

A fixture is recorded test data used in place of a live response. If a scenario tests reordering groceries, for example, the agent can be given the same recorded order history each time. This keeps the conditions stable, so a different result is more likely to reflect a change in the agent rather than a change in the available data.

For offline testing, the team also describes using scenario-specific checklists. These can verify concrete outcomes such as whether a shopping list contains relevant ingredients or stays within a specified budget. Such targeted checks complement broader rubric criteria.

DoorDash reported that a typical comprehensive evaluation sweep covered 50 scenarios with eight trials each. The simulator reduced the conversation-generation step for that suite from more than six hours of manual work to about 20 minutes. The key lesson is not that every team needs the same number of scenarios; it is that repeatable test conditions make comparisons more useful.

Using an LLM judge without treating it as ground truth

Human reviewers can make nuanced judgements, but manually inspecting every long conversation and tool trace is costly. DoorDash uses a large language model (LLM) judge to grade prepared transcripts against the rubric at scale, while keeping human review involved in defining criteria, labelling calibration examples and auditing the judge.

The judge assesses criteria independently and returns a verdict with a short rationale based on the relevant evidence. The team describes refining its prompts with GEPA prompt optimisation and retaining revisions that improved agreement with human labels on a held-out set. When rubrics change—for example, when the agent gains a new capability—the judge may need fresh labels and recalibration.

This is an important distinction: an automated judge is another system that needs evaluation. It can misread evidence or produce false positives. Calibration against human-labelled examples and continued auditing help reveal those weaknesses; they do not make the judge infallible.

DoorDash also uses the same rubric and judge for simulated offline sessions and real production sessions. That gives the team a shared measurement approach: a live failure can be reproduced in a controlled test, and an offline change can be assessed against the same criteria used in production.

What the case study reports

DoorDash reports that its evaluation harness expanded quality monitoring from roughly one employee-submitted feedback item to 2,000 automatically graded sessions per day. The company says that broader signal helped it identify recurring failure patterns and contributed to an eight-point improvement in its agent-quality score before nationwide launch, with error rates cut nearly in half.

The team also describes using evaluation to validate a base-model migration. An initial quality drop turned out to reflect differences in how the system handled tool inputs and prompts, rather than a weaker model. After targeted fixes, the team reported quality parity within the evaluation’s noise and a 35% latency reduction in production without a loss in quality metrics.

These are results reported by DoorDash for its system and its own measurements. They are not universal benchmarks, and the case study does not make them a promise that another agent will achieve similar outcomes.

Lessons and limitations for AI teams

  • Measure actions as well as answers. Include the tool calls and supporting evidence needed to understand how an agent reached its result.
  • Define success around user intent. Use task-specific criteria that allow multiple valid outcomes, and skip checks that do not apply.
  • Control the test environment. Fixed scenarios and fixtures help separate product changes from changing upstream data.
  • Calibrate automated judgement. Compare the judge with human-labelled cases, examine disagreements and keep auditing the evaluation pipeline itself.
  • Connect scores to fixes. Evaluation creates value when teams use failures to improve prompts, tool handling, routing or other parts of the system.

A practical checklist for evaluating an AI agent

  1. Map the flow. List the agents, tools and user-visible outputs involved in a task.
  2. Write observable criteria. Define what success and failure look like using evidence available in the session.
  3. Mark applicability. Decide when each criterion should be graded and when it should be skipped.
  4. Build representative scenarios. Include routine requests, edge cases, ambiguous instructions and high-impact failure cases.
  5. Freeze changing inputs. Use controlled fixtures where upstream data would otherwise make test runs inconsistent.
  6. Calibrate the judge. Compare automated verdicts against human labels and review disagreements before relying on aggregate scores.
  7. Use the same measurement logic across environments. Align offline tests and production monitoring where practical, and verify that the evaluation still reflects user experience.
  8. Close the loop. Turn recurring failures into targeted changes, rerun the tests, then monitor the result in production.

The central idea in the DoorDash case study is straightforward: make agent behaviour observable, make evaluation repeatable, and treat measurement as part of the product rather than a final release check. The specific rubrics and test scenarios will vary, but a clear link between evidence, criteria and corrective action is useful across agent projects.

Keep building your AI skills

Explore practical AI learning from Baig Academy for professionals, leaders and students who want to use AI thoughtfully in their work.

Further reading

Building Ask DoorDash (Part 3): Evaluation — the original DoorDash case study.

Discovery Call · 15 Minutes

Let’s map your team’s AI fluency plan.

Tell us a little about your team. We’ll reply within one business day with initial thoughts and next steps. No pressure, no pitch.

or
Prefer to pick a time right away?Open the booking calendar and grab a 15-minute slot.
GDPR-native NDA by default Reply within one business day