# Why CodePanel reviews with a panel of models

URL: https://codepanel.ai/blog/why-a-panel-of-models

Several cheap reviewers from different model families, then one orchestrator that checks every finding against the code. Why we built review this way.

CodePanel reviews every pull request with several models instead of one. Small reviewers from different model families read the change in parallel and report everything they notice. Then one orchestrator model reads their findings next to the code, throws out what the code contradicts, settles disagreements and posts a single comment. This post explains why we built it that way, and what we got wrong along the way.

## The problem with one reviewer

Ask one model to review a diff and you get one model's blind spots. Ask it again and you get a slightly different review: some findings repeat, some appear, some vanish. That variation is a hint. Each run is a sample of what the model can notice, and one sample misses things.

Models from different families miss different things. They were trained on different data and tuned with different feedback. Run Claude, GPT and Gemini on the same change and their findings overlap less than repeat runs of any one of them. Coverage comes from that difference.

So the first idea was simple: run several. The first version of that idea was bad.

## More reviewers, more noise

Post five reviewers' findings and the author reads the same bug five times in five phrasings, mixed with each reviewer's own false alarms. Recall goes up. So does everything else. A review nobody reads finds nothing.

The fix was to split the work in two:

| Stage | Instruction | Model |
| --- | --- | --- |
| Reviewers | Report everything you notice, with a confidence level. Ground each finding in a line you can see. | Small, cheap, spread across providers |
| Orchestrator | You are the filter. Check every finding against the code. Removing a bad finding is as valuable as keeping a good one. | One mid-size model |

Reviewers are told explicitly that they are not the last word. That makes them more useful: a reviewer told to be careful self-censors and misses things. The orchestrator is told explicitly that it is the precision stage.

## The orchestrator reads the code

The early orchestrator saw only the reviewers' text. It judged findings by how convincing they sounded, and confident wrong findings sound convincing.

Now it gets the numbered diff and the surrounding code for every file a finding names. It can open line 42 and see that the null check a reviewer says is missing sits on line 39. It can see that the "removed" validation is still present in the new revision. Its prompt grows with what was reported, not with the size of the change: a quiet review is cheap to consolidate.

It does five things with that code:

1. Merges duplicates, and records which reviewers agreed.
2. Checks each finding against the line it names, and drops findings the code contradicts.
3. Corrects line numbers for real findings that point at the wrong place.
4. Treats claims about library behaviour with suspicion when the code shown does not settle them, because a reviewer's training may predate the version you use.
5. Resolves disagreements and says which reading it believes.

## Disagreement is information

When three reviewers from three families flag the same line independently, that is the strongest signal in the round, and the orchestrator ranks it accordingly.

When reviewers contradict each other, we did not want both readings posted. One reviewer says a lock is missing; another says the write is idempotent and needs none. The orchestrator is told to pick a side, explain why, weigh each reviewer's stated confidence, and judge the argument on the code. A confident reviewer can still be wrong.

What it must not do is count votes. The bugs only one family notices are the reason to have a panel. A majority rule would drop exactly those.

## Severity means something breaks

Our first real evaluation of the pipeline found that the most common warning on correct pull requests was some version of "no tests cover this". Often true. Rarely a warning: nothing fails because of it.

So the orchestrator applies one rule. Critical or warning only when it can name what fails, for whom and how, from the code as it stands. Missing tests, docs, naming and duplication are suggestions, however worthwhile. Warnings went back to meaning "this will break".

## Cheap seats, one expensive one

The panel is built so most of the money goes where judgement is needed. A medium pull request is about 30,000 input tokens per reviewer. At list price on 2026-10-16, four Claude Haiku 5.5 reviewers cost $0.018 together, and a Claude Sonnet 5.5 orchestrator reading 15,000 tokens of findings and code costs about $0.07. One Claude Opus 5.5 review of the same change costs $0.18.

You can still put a frontier model on the panel. We suggest it for one role, a deep Security pass on repositories that handle auth, payments or secrets, rather than on every seat. The [cost guide](/guides/ai-code-review-cost) has the full arithmetic.

## When things fail

A panel degrades instead of breaking. Reviewers run in parallel, so a round takes as long as the slowest one. If a provider is down or a key is out of credit, that reviewer is dropped, the others continue, and the posted comment names the reviewer that failed and why. Silent gaps were a real problem early on: a reviewer whose account ran out of credit simply stopped contributing, and nothing in the review said so.

If the orchestrator itself fails or runs out of time, the round still posts. Findings are clustered so duplicates merge and agreement is counted, and the review says the orchestrator was unavailable. That is worse than a filtered review and much better than none.

## Written for agents

The comment the orchestrator produces is read by coding agents far more often than by people, so it is one prompt rather than a page: a header with counts and the commit the line numbers refer to, instructions, then each finding once with a stable name like `F3`. The finding text is quoted and the instructions call it data to verify, because it is model output derived from a pull request anyone could have written.

Agents rate findings when they are done, with a one-line `/feedback` reply. Those ratings are how we measure whether the panel is getting better, alongside the [benchmark](/blog/how-the-benchmark-is-built).

## What we would tell anyone building this

- Diversity across model families buys more than a bigger model in one seat.
- A panel without a filter is worse than one model.
- The filter must see the code, not just the findings.
- Define severity by what breaks.
- Make failures visible in the review itself.

The [How CodePanel works](/docs/how-it-works) page has the exact limits and steps. The [single model vs panel guide](/guides/single-model-vs-panel) covers when one model is the better choice.