# First benchmark results: seven open models on real bugs

URL: https://codepanel.ai/blog/open-models-benchmark-results

DeepSeek V4 Pro and GLM-5.3 Flash tied for first, and the Flash model costs a seventeenth as much per review. What we ran and what surprised us.

The first published run of the CodePanel benchmark, on 2026-10-17, ranks seven open-weight models served on Cloudflare Workers AI. DeepSeek V4 Pro and GLM-5.3 Flash tied for first on real bugs from open-source history. A DeepSeek V4 Pro review cost 17× what a GLM-5.3 Flash review did. Most of the other gaps in the table are smaller than the data can resolve.

## What we ran

Each model reviewed commits from open-source projects, treated as pull requests, on its own, as CodePanel's Bugs & Security reviewer, with the prompts, diff format and surrounding code that production uses. Some introduced a bug that a later commit fixed; the rest were correct changes, where any warning-or-worse finding is a false alarm. Three judges, one each from Anthropic, OpenAI and Google, decided which findings caught which bug. The [methodology page](/benchmarks/methodology) has the fixture rules, the judging and how the intervals are computed, and [How we built the code review benchmark](/blog/how-the-benchmark-is-built) explains why it is built that way.

The DeepSeek and GLM models ran at low reasoning effort, Qwen3.8 27B and GPT-OSS 120B at medium, and Gemma 4 26B A4B with no effort setting. Open models came first because Workers AI serves them cheaply.

## The results

| Rank | Model | Recall [95% CI] | False alarms per clean PR | Cost per review |
| --- | --- | --- | --- | --- |
| 1 | [DeepSeek V4 Pro](/models/deepseek/deepseek-v4-pro-workers-ai) | 81% [69%–92%] | 2.8 [2.2–3.4] | $0.0622 |
| 1 | [GLM-5.3 Flash](/models/zai/glm-5.3-flash-workers-ai) | 61% [48%–74%] | 1.8 [1.4–2.1] | $0.0036 |
| 2 | [DeepSeek V4 Flash](/models/deepseek/deepseek-v4-flash-workers-ai) | 59% [45%–73%] | 1.4 [1.0–1.8] | $0.0177 |
| 2 | [GLM-5.3](/models/zai/glm-5.3-workers-ai) | 57% [44%–70%] | 2.0 [1.4–2.7] | $0.0316 |
| 2 | [Qwen3.8 27B](/models/qwen/qwen3.8-27b-workers-ai) | 45% [31%–60%] | 0.8 [0.5–1.2] | $0.0309 |
| 2 | [GPT-OSS 120B](/models/openai/gpt-oss-120b-workers-ai) | 44% [31%–57%] | 2.7 [2.3–3.2] | $0.0074 |
| 5 | [Gemma 4 26B A4B](/models/google/gemma-4-26b-a4b-workers-ai) | 23% [10%–36%] | 0.3 [0.1–0.5] | $0.0046 |

Recall is the share of known bugs a model reported. Cost per review is each review's real token counts priced at Workers AI's list rates, with the same price table CodePanel uses for its cost ledger and spending limits.

A rank is one plus the number of models that catch measurably more bugs, by a paired test corrected for the number of pairs compared. Models that share a rank are tied, and ties cross rank numbers too: GLM-5.3 Flash is at rank 1 because nothing beats it measurably, not because it beats DeepSeek V4 Flash. Only two kinds of gap are real in this run. DeepSeek V4 Pro catches measurably more than every other model except GLM-5.3 Flash, and Gemma 4 26B A4B catches measurably less than DeepSeek V4 Pro, DeepSeek V4 Flash, GLM-5.3 Flash and GLM-5.3. Everything else is within the noise.

## What surprised us

**A Flash model tied for first.** We expected the cheap tier to trail. GLM-5.3 Flash shares rank 1 at $0.0036 a review, and GLM-5.3, the full model in its family, cost 8.8× as much per review with no measurable gap between the two. For a reviewer seat whose findings something else filters, that is the most useful result in the table.

**In DeepSeek's family, the larger model did pay.** DeepSeek V4 Pro caught measurably more than DeepSeek V4 Flash, the only measurable gap between two models of one family in this run. So "the Flash model is enough" was not contradicted for GLM and failed for DeepSeek. It is a result per model, not a rule.

**The quietest models had low recall.** The two quietest, Gemma 4 26B A4B and Qwen3.8 27B, have the lowest and third-lowest measured recall: Gemma catches measurably less than four models, and Qwen measurably less than DeepSeek V4 Pro. A reviewer that says little raises few false alarms, so a false-alarm figure means little without the recall beside it. Noise does not predict quality either: the two models with the most false alarms, DeepSeek V4 Pro and GPT-OSS 120B, sit near opposite ends of the recall column. Working alone, a reviewer trades one against the other, which is why CodePanel puts an orchestrator after its reviewers to check each finding against the code ([why a panel of models](/blog/why-a-panel-of-models)), and why [reducing false positives](/guides/reducing-false-positives) starts there rather than at the reviewer's prompt.

**GPT-OSS 120B found bugs on the wrong lines.** Of its catches, 40% landed on the defect's lines; every other model put at least 84% of its catches there. A finding that names the right bug on the wrong line costs the coding agent a search, and an agent that cannot find the bug may dispute a real finding.

**DeepSeek V4 Pro was slow.** DeepSeek V4 Pro averaged 366 seconds a review and Qwen3.8 27B 658 seconds, against 55 seconds for GPT-OSS 120B. Reviewers run in parallel, so the slowest seat sets the wait for every push.

## What comes next

Frontier models from Anthropic, OpenAI and Google are next, and a larger fixture set will narrow the intervals. The live ranking is on [Benchmarks](/benchmarks) and the [open-weight models page](/models/open-weight#results); the numbers in this post are from the 2026-10-17 run and will not change with later ones. [Best LLM for code review](/guides/choosing-a-model-for-code-review) turns the results into a choice per seat. To try one of these models as a reviewer, add a Cloudflare Workers AI key in CodePanel's settings ([providers and keys](/docs/providers-and-keys)), then ask your coding agent to add the model to your panel through the [MCP server](/docs/mcp-server).