C code review: which models catch the bugs
Too few test PRs with known bugs in C code have been scored to rank the models on them yet. The page fills in as the benchmark grows.
| Model | Caught here | Every bug | Difference |
|---|---|---|---|
| Claude Opus 5.5 | Too few scored test PRs here to show a result | ||
| GPT-6.1 Sol | Too few scored test PRs here to show a result | ||
| Gemini 4 Argon | Too few scored test PRs here to show a result | ||
| Claude Sonnet 5.5 | Too few scored test PRs here to show a result | ||
| GPT-5.6 Sol | Too few scored test PRs here to show a result | ||
| Gemini 3.8 Flash | Too few scored test PRs here to show a result | ||
| GPT-6 Luna | Too few scored test PRs here to show a result | ||
| Claude Haiku 5.5 | Too few scored test PRs here to show a result | ||
| Gemini 3.1 Pro | Too few scored test PRs here to show a result | ||
| GLM-5.3 Flash (Workers AI) | Too few scored test PRs here to show a result | ||
| GLM-5.3 (Workers AI) | Too few scored test PRs here to show a result | ||
| DeepSeek V4 Pro (Workers AI) | Too few scored test PRs here to show a result | ||
| DeepSeek V4 Flash (Workers AI) | Too few scored test PRs here to show a result | ||
| GPT-OSS 120B (Workers AI) | Too few scored test PRs here to show a result | ||
| Kimi K2.7 Code (Workers AI) | Too few scored test PRs here to show a result | ||
| Gemma 4 26B A4B (Workers AI) | Too few scored test PRs here to show a result | ||
| Gemini 3.5 Flash-Lite | Too few scored test PRs here to show a result | ||
The change “Event Thread could stall when not notified of new queries on existing connections that are in a bad state” in c-ares/c-ares, a change of 100 to 300 lines that landed in September 2025, introduced a resource-handling bug that a later upstream commit fixed.
Claude Opus 5.5, GPT-6.1 Sol and Claude Sonnet 5.5 caught it; Gemini 3.8 Flash, Gemini 3.1 Pro, GLM-5.3 Flash, GLM-5.3, DeepSeek V4 Pro (Workers AI) and Kimi K2.7 Code (Workers AI) caught it in part; Gemini 4 Argon, GPT-5.6 Sol, GPT-6 Luna, Claude Haiku 5.5, DeepSeek V4 Flash (Workers AI), GPT-OSS 120B (Workers AI), Gemma 4 26B A4B and Gemini 3.5 Flash-Lite missed it.