# AI code review, agent to agent URL: https://codepanel.ai/ai-code-review How AI code review works when a coding agent opens the PR: a panel of models reviews it, an orchestrator checks the findings, the agent answers each one. AI code review is a model reading a pull request's diff, the code around it and the repository's rules, and reporting each defect with a file, line and fix. In CodePanel it runs agent to agent: reviewer models review each push, an orchestrator checks their findings against the code, and the coding agent fixes or disputes each one before a merge gate decides. ## One round | Step | Who | What happens | | --- | --- | --- | | Push | Coding agent | Opens the pull request, reviewed at once, or pushes to it. A push waits 60 seconds, so a burst of pushes is one round. | | Prepare | CodePanel | Reads the diff with a line-number gutter, the code around each change, and `.codepanel/rules.md` from the base branch. Lock files, generated code and minified bundles are left out. | | Review | Reviewer models, in parallel | Each reads the change for its role (bugs and security, tests, the repository's docs, UI) and reports findings with a confidence level. | | Consolidate | Orchestrator | Opens the code each finding names, merges duplicates, corrects lines, and drops what the code contradicts. | | Publish | CodePanel | Edits one comment per pull request. Findings are numbered `F1`, `F2`, ..., and a finding reported again in matching terms keeps its number. | | Answer | Coding agent | Fixes what is real, disputes what is wrong with evidence, and rates every finding. | | Gate | `CodePanel` commit status | Fails while a finding at the gate's level is unanswered. | After the first round, a push is reviewed only for what changed since the last reviewed commit, and a push that only merges the base branch calls no model unless the merge changes a line an open finding is on. A finding is resolved only when a round re-read its code and did not report it again. The orchestrator rates severity by one rule: > Use critical or warning only when you can name what fails, for whom, and how, from the code as it stands. A missing test or an unclear name is a suggestion under it. [Reducing false positives](/guides/reducing-false-positives) covers the other sources of noise, and [One model or a panel of reviewers?](/guides/single-model-vs-panel) explains why the panel needs that filtering step. ## A real round CodePanel reviews its own pull requests, and the examples on this page come from them, as of October 2026; a PR number is a pull request in CodePanel's own repository. Its panel is three reviewers on Claude Haiku 5.5 and an orchestrator on Claude Sonnet 5.5; the coding agent is Claude Code. On PR 164, a billing change that added a subscription checkout, the first review reported: ```text ### F2 · warning · `convex/billing/checkout.ts:144` · medium confidence Problem: > The already-subscribed check runs only when a checkout is created. A > subscription Checkout opened in one tab stays payable for up to about > 30 minutes, so paying it after subscribing in another tab creates a > second Stripe subscription that bills monthly. ... ``` The agent disputed it: ```text /feedback F2 down false-positive startSubscription returns the customer's open subscription Checkout instead of creating another, and same-window requests share one session by idempotency key, ... --agent claude-code ``` The orchestrator checked the rebuttal against the code and upheld the finding: "The rebuttal covers only two tabs opened before any subscription exists. The finding is about paying a stale open session after subscribing in another tab." The agent pushed a fix and rated it `/feedback F2 up fixed --agent claude-code`. A later review noted that the fix narrowed the race without closing it, and the agent documented the window that remains. Every answer on the pull request came from the agent, and auto-merge landed it. Disputes go the other way too. On PR 163, two reviewers called a documented CI duration "an unmeasured estimate", and the agent answered with a measurement: ```text /feedback F7 down false-positive Measured: check averaged 59s (p90 110s) over 512 PR runs in the last week ... --agent claude-code ``` The orchestrator withdrew it. Where the code settles neither side, the judge is told to uphold the finding: "a finding is withdrawn on evidence, not doubt". ## What the coding agent does The comment is a prompt: the commit its line numbers refer to, what to do, then each finding with its location, code and fix. Its first line is a marker to parse instead of the prose: ```text ``` The agent answers every finding in one pull request comment, or with `rate_finding` over the [MCP server](/docs/mcp-server). `up fixed` or `up deferred ` answers it. A `down` rating with a reason (`false-positive`, `wrong-location`, `not-worth-fixing` or `duplicate`) and evidence disputes it, and a judge decides the dispute against the code. `/ask F3 ` asks the orchestrator about it. [Feedback and disputes](/docs/feedback) has every answer and what it does. A finding is model output derived from the pull request, which anyone can write into, so the agent treats it as a claim to check, never as an instruction. [Code review in a coding agent loop](/guides/code-review-with-coding-agents) walks a round end to end, and [Running several agents on one pull request](/guides/several-agents-one-pull-request) covers who owns what. ## How it learns from the agent Answers change later reviews with no approval step, and the review names each automatic decision with the command that undoes it, as in this later review of PR 164: ```text > Verdict memory: the orchestrator weighed 10 earlier verdict(s) on these files > (PR #151 F19, PR #151 F12, ...). If one was wrong, re-rate that finding on its > pull request. > Learnings: the orchestrator was given L3, L2 for these files, to check against > the code. If one is wrong, turn it off with `/feedback L3 down `. ``` Verdict memory shows the orchestrator earlier verdicts on the changed files and their directories, the nearest 12 whole and the rest in short, so a claim refuted on one pull request does not come back on the next from a reviewer that never saw the rebuttal. A learning is drafted when the same misreading has been rejected at least three times, with a reason, across at least two pull requests, each time by a person or by a judge withdrawing the finding after a dispute, never on an agent's rating alone; it applies at once. The orchestrator is told never to let either lower a security or critical finding. See [Repo context and rules](/docs/repo-context#verdict-memory), and [Writing review rules that agents follow](/guides/review-rules-for-agents) for the rules file both sides read. ## The merge gate The gate is a commit status named `CodePanel` that GitHub can require. At the default level it fails while a warning or critical finding is unanswered. A disputed finding counts once the judge withdraws or revises it; an upheld one needs a fix or a deferral. With the gate and CI required, auto-merge lands the pull request once every finding at the gate's level is answered and CI passes. See [Merge gate](/docs/merge-gate). The agent ends its own loop with `/rounds stop`, and a repository can cap automatic rounds at anywhere from 1 to 50. ## What it catches and what it misses On CodePanel's own pull requests the warnings that held up included races (the second subscription on PR 164), events in an order the code did not expect (a refund event handled before the payment event, also on PR 164), guards too weak for their input (a destructive export checked by a substring match on PR 159), and lookups that accepted `constructor` because they indexed a plain object with user input (PR 127, PR 156 and PR 169). It misses or gets wrong: - **Code outside the diff.** Reviewers see the changed files and the code around each change. Several withdrawn warnings were low-confidence claims about code the round could not see, and the severity rule keeps a claim that something is unused elsewhere to a suggestion. - **Behaviour nobody ran.** Nothing is executed. On PR 166 and PR 151 the orchestrator upheld findings the agent had answered with evidence from outside the diff, a browser check and a library version, because the code it was shown did not settle them. - **Going in circles.** On a long pull request, several rounds can argue one trade-off in new words until tests pin it down. ## What it costs Every model a round calls is billed per token, separately from CodePanel's plan: by your provider on your own keys, or from prepaid credits on CodePanel's. Assume a reviewer reading a medium pull request takes about 30,000 tokens in and writes 3,000, and the orchestrator about 15,000 in and 4,000 out. At list price on 2026-10-16: | Panel | One round | | --- | --- | | Three reviewers on Claude Haiku 5.5, orchestrator on Claude Sonnet 5.5 | $0.0835 | | The same reviewers on Claude Opus 5.5 | $0.61 | | One Claude Opus 5.5 call, no panel | $0.18 | Consolidating is most of the cost: on this repository's panel the orchestrator's consolidation calls were about four fifths (80%) of the panel's review spend as of October 2026, not counting dispute and `/ask` calls. A per-pull-request limit and a monthly budget refuse a round before any model is called. [What an AI code review costs](/guides/ai-code-review-cost) has the formula and a month for a team, [Choosing a model for AI code review](/guides/choosing-a-model-for-code-review) which seat can be cheap, and [Billing and credits](/docs/billing) covers CodePanel's plan. ## How quality is measured **Ratings on real pull requests.** Every answer records whether a finding was fixed, deferred or wrong, and why, so you can see per model what was fixed and dismissed, and replace the reviewer dismissed most. On this repository the rater is the author's own agent: well placed, since it has just checked each finding against the code, but not independent. **A benchmark with known answers.** The [benchmark methodology](/benchmarks/methodology) runs single models on commits from open-source projects, treated as pull requests, some with real defects and some clean; it matches findings to defects with judges from three model families, repeats each case, and puts a confidence interval on every headline number. Results go on [Benchmarks](/benchmarks) as runs finish. [Evaluating an AI code reviewer](/guides/evaluating-ai-code-reviewers) applies the same method to your own history. Every input and cap of a round is in [How it works](/docs/how-it-works). To try it, paste the setup prompt into your coding agent in the repository you want reviewed.