AI code review leaderboard

Claude Opus 5.5 caught the most known bugs (90%). The data can't separate it from 7 other models, which share rank 1. The right choice also depends on false alarms and cost, which the leaderboard sorts by.

GPT-6.1 Sol found bugs most cheaply among models that catch at least 75% as many as the best: $0.0514 per bug caught, counting every review, clean PRs included.

GLM-5.3 Flash (Workers AI), at 61% of bugs caught. Open-weight models such as GLM, DeepSeek, Kimi, Qwen and gpt-oss run here on hosts like Cloudflare Workers AI and Groq, and the leaderboard's Open weights filter lists them. You can also self-host them.

Each of the 47 bug test PRs replays a real upstream change that contained a bug, later found and fixed in the project. A model catches it when one of its findings names the same root cause at the right place. The score is the share caught across all of its reviews, with a 95% range.

Any warning or worse on one of the 15 clean PRs, real changes that nothing later fixed or reverted. A reviewer that flags problems that aren't there costs the reader time and teaches people to ignore it, so false alarms are shown next to bugs caught rather than folded into one score.

SWE-bench and similar tests ask a model to write a fix. This benchmark asks it to read a change and spot what's wrong, the job a reviewer does, inside CodePanel's real review pipeline. It also counts false alarms and cost, which a fix-the-bug score doesn't.

List price is per token. Models differ in how many bugs they find, how much they flag that isn't wrong, and how many tokens they spend on a review, so the same rate card can give very different results. Compare any two on their head-to-head page, which says whether the gap is larger than run-to-run noise.

When the gap between two models is smaller than the run-to-run spread, the data can't say which is better, so they share a rank (shown with =). Ranks come from paired comparisons on the same PRs, corrected for comparing many pairs at once.

The headline counts only bugs introduced from 2025 on, after most training cut-offs, and the test set is kept private, with a canary string so training pipelines can exclude it. Older bugs are still run and reported separately on each model's page.

Three judge models from different providers (Claude Sonnet 5.5, GPT-6.1 Sol, Gemini 3.1 Pro) vote. Judges tend to favour their own family, so a model is never judged by its own maker: that judge's vote is left out and the other two must agree. A model from another maker needs two of the three. A judge only decides whether a finding names the known bug, never how good the review is.

From the tokens each review actually used, at each provider's standard list price (checked 2026-10-16), without batch or cache discounts. Cost per bug divides the spend on every review, clean PRs included, by the bugs caught. A model with no score shows the list price of a typical review call.

Every model runs as CodePanel's Bugs & Security reviewer with the same prompt and context, at medium reasoning effort or the nearest level the model supports. A model that takes no effort setting runs at its default.

A run starts every week, and new models are added as providers release them. A model is ranked only once it has been scored on 80% of the test PRs, never on a handful of them.

The test set and harness stay private so they stay out of training data, but every finding and judge vote behind a published run is kept with it, so any score can be re-checked or re-judged. The methodology page explains each step and its limits.

Yes. Every model listed here can sit on a CodePanel review panel, with your own provider key, alongside other models. Each model page links to setting one up.