# Code review metrics: what to track when agents write the PRs

URL: https://codepanel.ai/guides/code-review-metrics

Code review metrics for agent-written PRs and model reviewers: precision, blocking recall, fix vs dispute rate, rounds to green and cost per PR.

Code review metrics are numbers that tell you whether review catches defects, how fast it does, and what it costs. When agents write and fix pull requests, keep four classic ones (time to first review, rounds per pull request, change size, escaped defects) and add five for model reviewers: finding precision, blocking recall, fix and dispute rate, rounds to green, and cost per pull request.

| Metric | What it tells you | Measured from | Main trap |
| --- | --- | --- | --- |
| Time to first review | Whether the author agent waits | PR timestamps | Counting hours when the agent needs minutes |
| Review rounds per PR | How long the loop runs | Round history | Fewer rounds because the reviewer saw less |
| Change size | Whether review can see the change | Diff stats | Counting docs and data as code |
| Escaped defects | What review missed | Issue tracker | Blaming review for what it never saw |
| Finding precision | How much of the review is true | Verdicts you trust | Letting the author agent score itself |
| Blocking recall | Real defects that stop a merge | Fixtures with known bugs | Measuring it on live pull requests |
| Fix rate and dispute rate | How findings are answered | Ratings and disputes | A 100% fix rate looks like quality |
| Rounds to green | How soon the gate clears | Round history | Pushing fixes one at a time |
| Cost per PR | What the loop spends | Spend records | Dividing by rounds, not pull requests |

## Time to first review: minutes, not days

**Definition.** Time from a pull request being ready to the first review landing, taken from GitHub timestamps.

For people this is a day-scale number. For a coding agent it is minutes, because the session that opened the pull request is usually still open and waiting. A review that arrives after the session ends is acted on by a new session with no memory of the change.

**The trap.** Measuring from "opened" counts time a draft sat idle. CodePanel holds each push for 60 seconds so a burst of commits gets one review ([triggering reviews](/docs/triggering-reviews)); that wait is deliberate and belongs in the number.

## Review rounds per PR

**Definition.** How many review rounds a pull request goes through before it merges. **Compute** the mean, the 90th percentile and the share that took five or more. CodePanel's `get_quality` tool returns exactly these under `convergence.rounds`: `mean`, `p90` and `fiveOrMore`, for pull requests merged in the window (up to the newest 60), and marks a segment `insufficientData` below 20 pull requests.

**The trap.** Fewer rounds is good only while precision holds. A reviewer that sees less, or reports less, also produces fewer rounds. Read this number next to finding precision, never alone. Also split rounds by what started them: a push, a manual `/review`, or a base-branch merge. CodePanel reports that split as `roundsByTrigger`, because a base merge is a round the author did not choose.

## Change size

**Definition.** Lines changed and files touched per pull request. **Compute** additions plus deletions from the diff, and report the median.

Size matters more for a model than for a person, because a model has a context window and a person has patience. CodePanel sends a changed file whole up to 24,000 characters and otherwise about 40 lines around each change, for up to 25 files. A 40-file pull request is partly reviewed from the diff alone. As of 17 October 2026 the median merged pull request on CodePanel's own repository changed 434.5 lines across 12 files, which is far past the sizes studied for human review ([the evidence is in best practices](/guides/code-review-best-practices)).

**The trap.** Additions plus deletions count documents, snapshots and data files. Report size with generated and lock files excluded, or the median says more about your fixtures than your changes.

## Escaped defects

**Definition.** Defects found after merge that review could have caught. Trace each production bug to the pull request that introduced it (`git blame` at the fix's parent) and ask whether the reviewer saw those lines.

It matters most and is hardest to collect: it arrives weeks late, needs a judgement per bug, and `get_quality` does not report it. Treat it as a quarterly audit.

**The trap.** A defect in code the reviewer was never shown is a context failure, not a judgement failure. Sort escaped defects by whether the lines were in the reviewer's view; the fixes differ.

## Finding precision

**Definition.** Of the findings a reviewer reports, the share that are true. **Compute** true positives divided by true positives plus false positives, over findings that have a verdict you trust.

CodePanel's `get_quality` counts only verdicts that did not come from the author agent alone. A finding counts as a true positive when a person rates it up, or when the agent disputed it and the orchestrator, rereading the code, upheld it or revised it. Counting a revised finding as right is the lenient choice: the problem was real, but the finding may have overstated it. `get_quality` reports `disputesUpheld` and `disputesRevised` apart under `truePositives`, so you can recompute precision counting only findings that were right as stated. It counts as a false positive when a person rates it `false_positive` or `not_worth_fixing`, or when the orchestrator withdrew it after a dispute. Precision comes with a 95% interval, and stays `null` until a row has 20 verdicts. It is broken out by model, role, severity and confidence, which is where it becomes useful. The [merge gate page](/docs/merge-gate) reports that of 72 low-confidence warnings that got a checkable verdict on its busiest repository in the 14 days to 23 October 2026, 66 were false positives. An overall figure would have hidden that; the `warning/low` row shows it, and the default gate no longer blocks on them.

**The trap.** Who gave the verdict. An author agent that fixes every finding has a perfect "rated up" rate whether the findings were right or not. CodePanel keeps those ratings in a separate `unverified` block and counts them toward no precision figure. If you build your own, do the same. See [reducing false positives](/guides/reducing-false-positives) for what moves this number.

## Blocking recall

**Definition.** Of the real defects in a set of changes, the share caught by a finding strong enough to block the merge. It differs from plain recall, which counts a catch at any severity: a defect reported as a suggestion is caught but does not stop anything.

**Compute** it on fixtures: pull requests where you know the defect. CodePanel's eval harness scores a defect as blocking-caught only if the matching finding is one the default merge gate blocks on. It is hard to measure on live pull requests, because it needs labels for the defects a reviewer missed, and those are rarely collected. That is what the [evaluation guide](/guides/evaluating-ai-code-reviewers) is for, and the [benchmarks](/benchmarks) apply it to single models.

The harness tracks it separately so a change that lowers noise by demoting findings shows what it costs. In CodePanel's own paired run on 3 November 2026 (50 bug fixtures, two repeats), putting a performance reviewer in the Tests & Maintainability seat changed blocking recall by 0 points, with an interval of -5 to +5.

**The trap.** An interval that wide bounds the change to about five points either way, so a three-point loss is not ruled out. Report the interval with the number.

## Fix rate against dispute rate

**Definition.** Of the findings raised, how many the author agent fixed, and how many it disputed with a reason.

CodePanel records both. An agent disputes a finding by rating it down with a reason such as `false_positive` and a note citing a file and line; the orchestrator then reads the code and answers `withdrawn`, `revised` or `upheld`. The `get_quality` `unverified` block carries `addressedByCommit` (a commit edited the finding's lines), `upByAgents` and `downByAgents`.

**Read them as a pair.** A high fix rate with almost no disputes means good findings or an agent that does as it is told. Many disputes mostly `upheld` means the agent is rebutting real problems; mostly `withdrawn` means the reviewer is noisy. The outcome is the evidence, not the rate.

**The trap.** Treating "resolved" as "right". A finding that closes after a commit touched its lines was addressed, not proved correct.

## Rounds to green

**Definition.** How many rounds before the merge gate first clears, and how often it clears and then goes red again.

`get_quality` reports `firstGreen`: the share of pull requests green after round one, the mean round at which they first went green, and how many merged without ever being green after a round. It also reports `afterGreen`: rounds that began on a green gate, and how many of them (`reBlocked`) raised a new warning or worse. The gate state is rebuilt from findings and ratings, so the red share is a lower bound.

**The trap.** Every push is a round. An agent that pushes after each fix pays for a round per push; fixing everything open and pushing once is the cheapest improvement to this number.

## Cost per PR

**Definition.** What review spends on one pull request, across all its rounds. CodePanel's `get_spend` returns this month's cost on a repository: `totalUsd`, split into `ownKeyUsd` and `creditsUsd`, with a breakdown by model and role and the failed calls included. Divide by merged pull requests for a per-PR figure. A per-PR limit caps the review calls one pull request can make, and calls whose output could not be used still count toward it ([budgets and limits](/docs/budgets-and-limits)).

The [cost guide](/guides/ai-code-review-cost) works through a full pull request, and [model pricing](/models/pricing) lists the rates.

**The trap.** Cheapest per round is not cheapest per pull request. A cheaper reviewer with lower precision costs extra rounds, and each one is paid for twice: once for the findings, once for the fixes they trigger. Compare cost per merged pull request at equal precision.

## Which three to start with

Start with finding precision, rounds to green and cost per PR: is the review true, does it converge, what does it cost. All three come from `get_quality` and `get_spend` over MCP, or `GET /quality` and `GET /spend` over HTTP ([API reference](/docs/api)). Add blocking recall when you change a model or a role. How findings are answered is in [findings](/docs/findings) and [feedback](/docs/feedback); how the loop runs is on the [AI code review](/ai-code-review) page.