AI Agent Evaluation Metrics: What to Count and What Not To

AI agent evaluation metrics say whether an agent did its job, how it got there, what that took, what it must never do, whether it would do the same tomorrow, and whether the grader can be believed. No single score covers all six. Report four numbers, each with its denominator and its interval, and treat the rest as diagnostics.

The four answer four questions: did the run reach the right end state, does it do so every time, did it break a rule, and what does a success cost? Choosing them is this post’s synthesis. The list does not come from the book this site belongs to, and it is not a published standard, so the sections below give the reasons and you can disagree with them.

Which AI agent evaluation metrics should you report first?

Report four: task success rate graded on the end state, pass^k, a policy-violation count, and cost per successful task. Each answers one question a release decision turns on, and each exposes a failure the other three cannot see. The panel is this post’s choice; the book supplies the parts.

Chapter 16 of AI Agents, Engineered asks you to “track cost, latency, and step count on the same fixed bank,” and sets the safety rule for a regression gate: “any failure on the safety cases, the pipeline stops.” Its warning about a lone rate is that “an average is exactly the wrong summary for a system whose product is its worst runs.”

Read the third row as a count with a gate at zero, and remember that zero violations in n runs is not a zero rate. If runs are independent, the 95 percent upper bound on the true rate solves (1 − p)ⁿ = 0.05, which is about 3 ÷ n. Two hundred clean runs still allow about 1.5 percent, and 40 allow about 7 (standard statistics, computed for this post).

Keep it a count all the same. A rate invites someone to average it into the other three, and that is the one thing the panel must prevent.

Worked example: the same four numbers for two agents

Two illustrative agents show how the panel changes shape while the four questions stay fixed. A support agent that issues refunds writes to a system you can query. A research agent that writes reports leaves a document and a trace, and nothing else.

The research agent has no end state to query, so its task success rate is a pass-all rate over binary checks, and part of its grader is a model. That makes the judge-agreement row of the reference table mandatory for it. Its violation rules are ones the success checks cannot see, so the third number stays independent of the first.

What does each agent metric measure, and how does it mislead?

Each of the AI agent evaluation metrics below answers one of six questions: outcome, path, cost, safety, stability, or whether the grader itself can be believed. The table has 23 rows, and the buttons narrow it to the question you came with. The six family names are this post’s grouping.

Formulas are given in words as their source states them. Where the “Defined in” column says no published origin was found, the formula is this post’s wording of common usage, and tools will differ. The “How it misleads” column is my analysis unless it names a source.

A table of metric names is the kind of generic list some practitioners warn about. Read it as vocabulary for definitions you write against your own failures; Husain and Shankar’s evals FAQ (2026) allows generic metrics only as “exploration signals to identify traces worth reviewing.”

How do the six families fit together?

Outcome says whether the job was done, and the other five families say how far to believe that. Confusion about AI agent evaluation metrics starts, in my reading, when a number from one family is taken as the answer to another family’s question.

Outcome and path

Outcome is the family the panel starts with, and the book’s rule for it is “grade the state, not the prose.” Progress metrics belong here as development signals: one benchmark scores half the share of checkpoint points earned plus half the full-completion flag (Xu et al., 2024), which is useful below 100 percent and wrong as a gate.

Path metrics explain why an outcome moved. A tool call can be scored against a reference set, checked against required and forbidden lists, or simply counted. The matchers, and what each lets through, are a subject of their own in agent trajectory evaluation; why per-step accuracy is not task success is the subject of why agent errors compound.

Cost and safety

Cost covers money, time and human attention. Chapter 15 asks for “latency at the percentiles rather than the mean, because agents produce long tails,” and calls the step-count distribution “your loop detector.”

Human intervention rate has no published definition that I could find. In one vendor’s 2026 telemetry on its own coding product, new users interrupted in about 5 percent of turns and experienced users in about 9 percent, while sessions run with full auto-approval rose from roughly 20 percent to over 40 percent (Anthropic, 2026). The rate went up as experience and auto-approval went up.

Safety numbers describe a tail, so they are counted and gated, never averaged. Debenedetti and colleagues (2024) define targeted attack success rate as “the fraction of security cases where the attacker’s goal is met,” where a security case pairs a user task with an injection task. The source reports it beside benign utility and utility under attack, since an agent that refuses everything scores a perfect zero on attack success.

Stability and the grader

Stability asks whether the number would survive a rerun. A 2026 paper that evaluated 15 models on two benchmarks reports that “outcome consistency remains low across all models,” meaning “agents that can solve a task often fail to do so consistently” (Rabanser et al.). pass@k and pass^k are the two standard summaries, and the next section defines them.

Grader metrics measure the instrument. If an LLM judge scores any row, its agreement with human labels is part of the metric’s definition; the method is in whether an LLM judge is reliable, and the judge agreement calculator does the kappa arithmetic. Intervals and sample sizes belong to how many eval examples you need.

What is pass@k, and how does pass^k differ?

pass@k is the probability that at least one of k attempts at a task succeeds; pass^k is the probability that all k succeed. Chapter 16 calls the first “the optimistic bound” and the second “the pessimistic bound,” and the gap between them the reliability envelope.

The name pass@k and its estimator come from code generation. Chen and colleagues (2021) “generate n ≥ k samples per task,” count “the number of correct samples c ≤ n which pass unit tests,” and report the mean over problems of 1 − C(n−c, k) ÷ C(n, k). Kulal and colleagues (2019) had reported the same idea as “success rate at B.”

Chen’s paper also says how to read the number: as “the best out of k samples, where the best sample is picked by an oracle with prior knowledge of the unit tests.” Without a picker, pass@k describes a product you do not have.

pass^k was introduced for agents. Yao and colleagues (2024) define it as “the chance that all k i.i.d. task trials are successful, averaged across tasks,” estimated as the mean over tasks of C(c, k) ÷ C(n, k). In that paper, one frontier model of 2024 passed about 61 percent of one domain’s tasks on a single trial, and its pass⁸ fell to about 25 percent.

One worked number

Take an illustrative rate. Suppose every attempt succeeds 90 percent of the time, independently. Then pass¹⁰ = 0.9¹⁰ = 34.9 percent, and pass@10 = 1 − (1 − 0.9)¹⁰, which the calculator below prints as “> 99.9%”. Chapter 16 calls this agent “a pass@k triumph and a pass^k catastrophe, and both descriptions are true.”

The sliders use those closed forms, which assume one shared success rate and independent attempts. Under “Use your own runs instead,” enter the attempts and passes for one task, and for any k up to n the tool switches to the two estimators above. Which number a product should quote is the whole question of pass@k versus pass^k.

The k is a product decision: the number of consecutive unattended runs that must all be right. Write it into the metric’s definition and run n ≥ k trials per task. With n = k each task scores 0 or 1, so pass^k on 40 tasks is exactly as precise as a pass rate on 40 tasks, and it needs the same interval.

How do you know whether an agent metric can be trusted?

A number can be compared with last week’s when three things are stated beside it: the trials per task, the interval, and what was counted as an infrastructure error. Without them, a change in the number has explanations other than the agent, and you cannot tell them apart.

Sampling. A suite that reports 90 percent from 45 passes on 50 tasks, each run once, has a 95 percent Wilson interval of 78.6 to 95.7 percent (standard statistics, computed for this post; the eval sample size calculator reproduces it). Chapter 16 puts the consequence plainly: “a two-point shift on a fifty-task suite is well inside the noise.”

The environment. One lab’s 2026 experiment on its own model held the model, harness and tasks fixed and changed only the resource limits of a coding benchmark’s sandbox. Infrastructure error rates fell “from 5.8% at strict enforcement to 0.5% when uncapped,” and the score rose by 6 percentage points between the strict setting and no cap at all (p < 0.01). The author’s advice is that “leaderboard differences below 3 percentage points deserve skepticism until the eval configuration is documented and matched” (Anthropic, 2026). Excluding errored runs has its own risk: if errors fall mostly on long or hard tasks, the rate is flattered, so show it with errors counted as fails beside it.

The grader. When a model does the grading, a score can rise because outputs drifted toward what the judge prefers. Pin the judge’s version in the definition, and re-measure its agreement with human labels whenever either changes.

Why does the headline number stay green while the agent breaks?

A headline success rate stays green because it grades one thing, usually the final state or the final message, and the break happened somewhere it does not look. The published measurements differ in denominator, so read each with its own.

• Successes that broke a rule. A 2026 preprint ran three frontier models on two domains of one customer-service benchmark, four trials per task, and found that “27–78% of benchmark reported successes are corrupt successes concealing violations across interaction and integrity” (Cao, Driouich and Thomas). That is a share of credited successes, with violations identified by a model judge the authors checked by hand.
• Success under policy. Levy and colleagues (2024) scored three open web agents and report that “their average CuP is less than two-thirds of their nominal completion rate.” Both rates share one denominator, all tasks.
• A grader that passes an idle agent. Zhu and colleagues (2025) found that on one tool-use benchmark “a trivial agent that returns empty responses is considered successful on intentionally impossible tasks” and “achieves a 38% success rate.”
• Wrong arguments. In a July 2026 forum post from an account named after an evaluation vendor, the author writes that “Task-success stayed green until we added argument-level scoring.” It is one anecdote.

The panel catches some of these and not others. Its policy-violation count would have moved in the first two cases while the success rate stayed put. The third is a grader fault that none of the four numbers sees, which is why the reference table carries a do-nothing baseline in the grader family.

The fourth depends on the task. Where a wrong argument leaves a wrong record, grading the end state catches it, which is why the first number is defined on state. Where the call only reads, it takes an argument-level path check.

How do you define a metric so two people compute the same number?

Write down the unit, the numerator, the denominator, the grader, the trials and the aggregation before anyone computes anything. One name hides two formulas whenever one of those is left to the tool’s default, and the defaults differ.

pass@k is the clean example. The estimator’s own authors warn against the shortcut of plugging the empirical pass@1 rate into 1 − (1 − p)^k: “we show that it is biased” (Chen et al., 2021, section 2.1). Their Appendix A gives the direction: the shortcut “results in a consistent underestimate.” Two dashboards can both say pass@10 and disagree, one using the per-task estimator and one the shortcut.

Units cause the same split. An August 2026 discussion in one open-source tracing project asks how to count an agent that retries one call and then succeeds: “1 operation, 2 attempts (operation success 100%, attempt success 50%)” or “2 operations with 50% success.” Denominators do too: a July 2026 issue in an open-source evaluation framework reports that “a run with 1 success and 9 errors” under a tolerant error setting was shown as plain accuracy 1.0. Both are reports about particular tools at a date, cited as examples of the category.

The template below has fourteen fields. Copy it once per metric on your panel.

metric:           <name, exactly as it appears on the dashboard>
question:         <the one question this number answers>
gate:             <decision it feeds: ship | roll back | investigate>; threshold: <value, or "zero">
unit:             <task | trial | turn | tool call | session>; a retry <is | is not> a new unit
numerator:        <what is counted>
denominator:      <what it is divided by>; excluded: <what, and where it is reported>
grader:           <code check | rubric | judge | human>, version <id>
                  if a judge: agreement with human labels <kappa, n labels, date>
trials_per_task:  <n>, each from a clean environment
                  errored trials: <rerun up to r times | dropped>
                  a task left with fewer than <k> scored trials is <excluded and listed | scored as a fail>
aggregation:      <mean over tasks of passed/trials | pass^k, k=<k> | pass@k, k=<k>
                  | tasks passing >= m of n | count>
uncertainty:      <interval across tasks: method and level>
                  or <count out of n runs; if zero, the 95% upper bound, about 3/n>
environment:      <agent version: code, prompt, model, tool schemas>
                  limits: <steps, time, resources>; a run that hits a limit is scored <fail | infra_error>
                  infra_error: <what counts>; reported separately, never as a fail
task_set:         <name and version, n tasks>; covers <request types>; leaves out <request types>
                  or live traffic: <sampling rule, window>
blind_spot:       <the bad run this number still passes>; caught by <companion metric>
owner:            <who>; grader last checked against read transcripts on <date>

Example: task success rate for the refund agent

Here the template is filled for the support agent’s first number. Every value is illustrative.

metric:           Task success rate (refund agent)
question:         Did the refund end in the right state?
gate:             ship; threshold: lower end of the 95% interval at or above 80%
unit:             trial; a retry inside a run is not a new unit
numerator:        trials where the ledger shows one refund, for the right order
                  and amount, and no other record changed: 180
denominator:      scored trials: 200; excluded: 4 errored executions,
                  listed under infra_error
grader:           code check against the refund ledger, version 7; no judge
trials_per_task:  5, each from a reset sandbox
                  errored trials: rerun once (4 reruns, all scored)
                  a task left with fewer than 5 scored trials is excluded and listed (none)
aggregation:      mean over the 40 tasks of passed trials / 5
                  30 tasks at 5 of 5, 10 tasks at 3 of 5:
                  (30 x 1.0 + 10 x 0.6) / 40 = 90.0%
uncertainty:      95% interval across the 40 per-task rates, mean +/- 1.96 x SE
                  SE = 0.175 / sqrt(40) = 0.028, so 84.6% to 95.4%
                  (not the pooled Wilson interval on 180 of 200: 85.1% to 93.4%)
environment:      agent build 41 (code, prompt, model, tool schemas pinned)
                  limits: 12 steps, 120 seconds; a run that hits a limit is scored fail
                  infra_error: sandbox crash, tool or model endpoint not responding, rate limit
task_set:         refunds-v3, 40 tasks from real tickets; covers single-order
                  refunds; leaves out partial refunds and chargebacks
blind_spot:       a correct refund issued without the identity check;
                  caught by the policy-violation count
owner:            <name>; grader last checked against 20 read transcripts on <date>

The uncertainty line follows the rule in how many eval examples you need: the task is the unit, so average each task’s trials into one score and compute the interval across tasks. The 40 per-task rates have a standard deviation of 0.175, so the standard error is 0.175 ÷ √40 = 0.028 and the interval is 90 ± 1.96 × 2.8 points, or 84.6 to 95.4 percent. That is a normal approximation; at 40 tasks, cross-check it with a bootstrap over tasks.

The pooled Wilson interval, 85.1 to 93.4, treats 200 trials as independent and is too narrow. Here the across-task standard error is 0.028 ÷ 0.021 = 1.3 times the pooled one, and the ratio depends on the data. Miller’s 2024 paper on error bars reports clustered standard errors “over 3X larger than naive standard errors,” but that figure is for one reading benchmark whose questions share passages (3.05 times; 1.10 and 1.88 on two others), not for repeated trials.

The second and third numbers

For pass^k, k = 5 is the product’s choice, five unattended refunds in a row, and n = k, so a task scores 1 only when every trial passed. That gives pass⁵ = 30 ÷ 40 = 75 percent, with a 95 percent Wilson interval on 30 of 40 tasks of 59.8 to 85.8 percent.

A dashboard that takes the pooled shortcut instead reports 0.9⁵ = 59 percent. That sits just under the interval’s lower end, so 40 tasks do not measure the sixteen-point gap well. What the example shows is the direction: when tasks differ in difficulty, the pooled shortcut understates pass^k.

The third number is a count: zero violations in 200 runs, which bounds the rate at about 1.5 percent only if the runs were independent. Five trials of one task are not, so the bound is optimistic.

What does a successful task cost?

Cost per successful task is the cost of all runs, failed ones included, divided by the number of runs graded successful.

The nearest published definition is a 2025 preprint on language models. It defines cost-of-pass for one problem as “the expected monetary cost to obtain one correct solution”: the expected cost of one attempt divided by the probability of a correct answer (Erol et al.). The pooled form for agent tasks, used here, is practitioner usage; I found no paper that fixes it.

Continue the refund example with illustrative units. Suppose all 204 executions, the four errored ones included, cost 5,000 units. Cost per task, counted per scored trial, is 5,000 ÷ 200 = 25 units, and cost per successful task is 5,000 ÷ 180 = 27.8.

A cheaper configuration that spends 3,000 units and passes 100 of 200 trials costs 3,000 ÷ 100 = 30 per success. The cheaper run is the dearer result, though on this few successes a gap of 27.8 against 30 is inside the noise.

Kapoor and colleagues (2024) make the research case, that “agent evaluations must be cost-controlled,” and Chapter 19 of the book has the same rule for any cost change: “The pair of numbers is the deliverable; either alone is a story.”

For the numerator, the agent cost-per-task estimator models tokens per run from the step count; it does not compute this ratio. How that bill builds up, step by step, is the larger question of what an AI agent costs to run.

Should you combine everything into one score?

Prefer the panel, and if one number is demanded, make it a pass-all rate with safety kept outside it. The disagreement among practitioners is narrower than it looks.

The case for one number is organizational. Husain and Shankar concede that “people in your organization may want a single number to track,” and offer the pass-all rate: “An example passes only if it passes every check.” Research frameworks publish aggregates too: Rabanser and colleagues (2026) compute an overall score that “uses a uniform average across pillars by default.”

The case against is that a composite moves without saying what moved. The same FAQ discourages “complicated weighted scores,” and Chapter 16 makes the point about graders: “several small graders, each owning one dimension (accuracy, format, safety), tell you what broke; one monolithic ‘was it good?’ only tells you that.”

Both sides agree on safety. Those authors “explicitly exclude safety from the overall aggregate because safety violations are inherently a tail phenomenon,” and their reason is that averaging safety with the other dimensions “obscures critical tail risks.” My position follows from that: four numbers, a gate at zero on the third, and a pass-all rate over the blocking checks if a single tile is required.

Which agent metrics are vanity metrics?

A vanity metric is a number that can improve while the product gets worse. Chapter 15 defines the plainest kind as “totals that grow impressively on a slide (requests served, tokens processed) while saying nothing about whether anyone was served well,” and gives a test: “The tile you glance at every morning should be capable of ruining your morning; if it can only ever go up, it is decoration.”

Totals that only grow (the book names the first two; the third is this post’s addition):

• Requests served rises with traffic, whatever the answers were like.
• Tokens processed rises when the agent loops, which is the opposite of improvement.
• Runs completed rises when the agent stops early and says it is done.

Flattering averages, a different fault and this post’s list:

• Average judge score on a five-point scale can rise as outputs drift toward the judge’s taste, which is my reading; Chapter 16’s own complaint is that absolute scores “drift and bunch.”
• Mean latency can fall while the slowest tenth of users wait longer.
• Deflection or containment rate rises when customers give up. One vendor’s caution about its own category (2026) defines deflection rate as “contacts handled without a human, regardless of outcome.”

What do these metrics not tell you?

None of the AI agent evaluation metrics on this page tells you whether the task set resembles your traffic, or what failure you have not named yet. The sources are also thin in places, and I would rather say where.

Coverage of reality. Every rate here is a rate on a task set, which is why a golden dataset built from real traces matters more than any formula above. An offline eval set also holds still while usage moves.

The unnamed failure. Chapter 15 says it best: “metrics tell you that quality dipped, and only the trace shows you the new failure you had no name for yet.” For an agent that only writes documents, a model judge sits under most of the panel.

Thin sources. Human intervention rate has no agreed unit, and cost per successful task has one preprint behind its per-problem form and none behind the pooled one. The four-number panel and the six families are this post’s synthesis, untested beyond the two illustrative agents above.

Dated evidence. Every percentage quoted from a study describes particular agents on particular benchmarks in the stated year. Several list no venue on their arXiv record, and the newest of those, the corrupt-success study, is unreplicated; another source is one lab’s report on its own model.

The one thing to keep

When someone asks how good the agent is, answer with four numbers and what each one hides: task success rate with its trials and its interval across tasks, pass^k with its k, the policy-violation count with the number of clean runs behind a zero, and cost per successful task. Then say which of the four moved since last time, and open the diagnostic rows only for that one.

Whichever AI agent evaluation metrics you add after that, give each a gate line in its definition. The book’s own caution applies to all of them: “a green suite is evidence, never proof.”

The full treatment, from multi-trial statistics to judges and regression gates, is in Chapter 16, “Evaluating Agents”, in the full book. The free agent evaluation guide collects the related posts and tools, and you can see the formats.