Best LLM for code review: benchmark results and how to choose

Of the models CodePanel has ranked, eight share first place on real bugs from open-source history in the run of 2026-11-04: Claude Opus 5.5, GPT-6.1 Sol, Gemini 4 Argon, Claude Sonnet 5.5, GPT-5.6 Sol, Gemini 3.8 Flash, GPT-6 Luna and Claude Haiku 5.5. Claude Opus 5.5 has the highest measured recall, with GPT-6.1 Sol close behind, and Haiku 5.5 and GPT-6 Luna tie them for first at a fraction of the cost: a GPT-6.1 Sol review costs 14× what a GPT-6 Luna review does.

Models ranked on real bugs

Each model reviewed alone, as CodePanel's Bugs & Security reviewer, a set of commits from open-source projects treated as pull requests. Some introduced a known bug and the rest were correct changes. Frontier models ran on their makers' own APIs and open-weight models on Cloudflare Workers AI. Recall is the share of known bugs the model reported, on the bugs every ranked model was run on; false alarms are warning-or-worse findings on the correct changes.

Three rules keep the comparison fair. A model is never judged by a judge from its own maker: its findings are decided by the other two judges, and both must agree. A review the model could not finish (it timed out, ran out of output or answered with something unreadable) counts as a miss, so the completion rate is part of the score. And a model whose calls failed for reasons outside its control, such as a billing refusal or a dropped connection, on more than one round in five is left out of the ranking rather than scored on the rounds that got through.

Cost per review is measured, not estimated from a typical call: each review's real token counts, priced at the provider's list rates with the same price table CodePanel uses for its cost ledger and spending limits. The benchmarks page adds latency, line placement and each model's completion rate, next to every priced model.

Completion moves the low end of the table. Kimi K2.7 Code finished fewer than half of its reviews, timing out or running out of output on the rest, so it scores well below its catch rate on the reviews it finished. Until 2026-11-04 the benchmark gave reviewers a 16,000-token output limit, half what CodePanel's own reviewers get. Gemini 3.8 Flash and Gemini 3.1 Pro, which spend most of their output thinking, ran out on many reviews under it, so both were run again on every bug under the 32,000-token limit. Each review from the rerun replaces one that hit the old limit; where the old runs failed on a bug more times than the rerun reviewed it, the rest still count as misses, as they do for a model not yet run again. Qwen3.8 27B is not ranked: more than a quarter of its calls were dropped by the host before it answered.

Models that share a rank are tied

The bracketed range is a 95% interval from resampling the pull requests: the range the model's recall plausibly falls in on pull requests like these. Intervals this wide mean a gap of a few points between two models is noise. So a rank is not a position in the list. It is one plus the number of models that catch measurably more bugs, judged by a test paired by pull request and corrected for the number of pairs compared. Eight models at rank 1 are tied for first, not first to eighth.

Ties also cross rank numbers. GLM-5.3 Flash and Gemini 3.1 Pro are at rank 2 only because Claude Opus 5.5 catches measurably more than they do, and GLM-5.3 is at rank 3 only because Claude Opus 5.5 and GPT-6.1 Sol do; no other model catches measurably more than any of the three. In this run, the gaps the data does support include these:

• Claude Opus 5.5 catches measurably more than every model outside the first-place group, and GPT-6.1 Sol than every one of them but GLM-5.3 Flash and Gemini 3.1 Pro.
• Gemini 4 Argon catches measurably more than every model outside it but Gemini 3.1 Pro and the two GLM-5.3 models.
• No model in the first-place group catches measurably more than another one in it.
• Gemini 3.5 Flash-Lite catches measurably less than every model but Gemma 4 26B A4B.

A larger fixture set will narrow the intervals; the methodology page explains how they are computed.

Which one to pick

For the most bugs caught: Claude Opus 5.5 or GPT-6.1 Sol. Opus 5.5 has the highest measured recall and is the only model measurably above both GLM-5.3 models; GPT-6.1 Sol is close behind, and neither is measurably above the other six models tied for first. GPT-6.1 Sol is the cheaper of the two, at $0.0401 a review against $0.1525, and raises fewer false alarms: 0.9 [0.4–1.5] per clean pull request against 1.7 [1.1–2.3].

For a reviewer seat in a panel: Claude Haiku 5.5 or GPT-6 Luna. Both are tied for first, at $0.0038 and $0.0028 a review. In a panel, a later step filters out false alarms, so the reviewer's job is coverage, and coverage is cheap. Seats from two makers make different mistakes, so the pair covers more than either alone.

For few false alarms: GPT-6 Luna raises 0.7 [0.3–1.1] per clean pull request, the fewest of the models tied for first. Gemma 4 26B A4B and Gemini 3.5 Flash-Lite raise fewer still, and have the two lowest measured recalls. Low false alarms only count next to the recall that came with them.

Watch line placement. GPT-OSS 120B put 49% of its catches on the defect's lines, while every other model put most of its catches there. A finding that names the right bug on the wrong line costs the coding agent a search, and an agent that cannot find the bug may rebut a real finding.

List prices for a typical call

A review call is mostly input: the diff, the code around it and your rules. A medium pull request is about 30,000 input tokens and 3,000 output tokens. At list price on 2026-10-16:

One Opus 5.5 call costs 40× what one Haiku 5.5 call does. These are list-price estimates for a typical call, not measured reviews, so they do not line up with the cost column above. Every model's rates are on the pricing page, and the cost guide works through a full pull request.

Where to spend more

Keep the reviewers cheap and spend on two seats, where one call does work that every finding depends on.

The step that filters findings. If several reviewers report, something has to decide which findings hold up: read the code each finding points at, merge duplicates, correct line numbers and settle disagreements. Every finding passes through it, so it is the seat where spending reaches the whole review. In CodePanel this is the orchestrator, and the starter panel puts it on a mid-size model at medium effort.

Deep passes on risky code. Authentication, payments, secrets and anything that parses untrusted input. A security pass that traces input from the request to the query is a long chain of reasoning, so if you add a costlier model, add it for that role on those repositories rather than upgrading every seat.

Rule checks, test gaps and stale docstrings are narrower: each is a check of the diff against something written down, so put your rules in the reviewer's prompt and keep those seats on cheap models.

Reasoning effort and latency

Reasoning tokens are billed at the output rate, so effort is a cost lever on top of model choice. A Sonnet 5.5 call with 30,000 input tokens and 3,000 tokens of answer costs $0.09. Add 10,000 tokens of reasoning and it costs $0.19, for the same visible output. Use low effort for rule checks and test gaps, medium for a general bug pass and for filtering findings, and high only for deep security passes.

Effort adds time, and so does the model. Reviewers that run in parallel finish when the slowest one does, so one slow seat sets the wait for every pull request. In the run above, models at the same medium effort averaged from 11s (Claude Sonnet 5.5) to 427s (Gemini 3.8 Flash) a review. If your coding agent waits on each review before its next push, put slow models only on the repositories where they earn it.

Training cutoffs and library APIs

Every model knows libraries as they were when it was trained, so it may flag a new, correct API as a mistake: a hook it has never seen, a signature that changed. These false alarms are confident and specific, which makes them expensive to dismiss. Give the reviewer your own docs and rules so it knows which version you use, and treat a finding that claims a library "does not support" something with suspicion unless the code shown settles it. CodePanel's reviewers are told to mark such findings low confidence, and the orchestrator to keep them only when the risk is real either way.

A starting configuration

The ranking measures one model alone on a fixed set of bugs. Your code is not that set, so run the panel for two weeks, look at what was dismissed and what was missed, and change one seat at a time. Evaluating an AI code reviewer shows how to build the comparison from your own history, and AI code review covers how the whole loop fits together. In CodePanel, models are grouped into cost tiers, and the Spend & budget tab shows spend by model and role, and per pull request.