C++ code review: which models catch the bugs
Of the 17 models ranked on C++ code, 5 models tied for the most known bugs: 100%. 8 of the other 12 models have 95% intervals that overlap theirs. Gemma 4 26B A4B caught the fewest, 0%. Together they caught 57% of known bugs in C++ code, against 60% for the same 17 across the whole benchmark.
| Model | Caught here | Every bug | Difference |
|---|---|---|---|
| Claude Opus 5.5 | 100% | 90% | +10 pts |
| Gemini 4 Argon | 100% | 82% | +18 pts |
| Claude Sonnet 5.5 | 100% | 77% | +23 pts |
| GLM-5.3 Flash (Workers AI) | 100% | 61% | +39 pts |
| GLM-5.3 (Workers AI) | 100% | 61% | +39 pts |
| GPT-6.1 Sol | 50%[0%–100%] | 89% | −39 pts |
| GPT-5.6 Sol | 50%[0%–100%] | 72% | −22 pts |
| Gemini 3.8 Flash | 50%[0%–100%] | 68% | −18 pts |
| GPT-6 Luna | 50%[0%–100%] | 67% | −17 pts |
| Claude Haiku 5.5 | 50%[0%–100%] | 64% | −14 pts |
| Gemini 3.1 Pro | 50%[0%–100%] | 67% | −17 pts |
| DeepSeek V4 Flash (Workers AI) | 50%[0%–100%] | 49% | +1 pt |
| Gemini 3.5 Flash-Lite | 50%[0%–100%] | 11% | +39 pts |
| DeepSeek V4 Pro (Workers AI) | 38%[25%–50%] | 54% | −16 pts |
| GPT-OSS 120B (Workers AI) | 33%[0%–67%] | 40% | −7 pts |
| Kimi K2.7 Code (Workers AI) | 6%[0%–13%] | 38% | −32 pts |
| Gemma 4 26B A4B (Workers AI) | 0% | 26% | −26 pts |
C++ was the fifth-hardest of the 12 languages for these models.
GLM-5.3 Flash gains the most here: 100%, against 61% over every bug.
GPT-6.1 Sol drops the most: 50%, against 89% over every bug.
The change “rpc : cache and reuse compute graphs” in ggml-org/llama.cpp, a change of 100 to 300 lines that landed in November 2025, introduced a security bug that a later upstream commit fixed.
Claude Opus 5.5, Gemini 4 Argon, Claude Sonnet 5.5, GLM-5.3 Flash and GLM-5.3 caught it; DeepSeek V4 Pro (Workers AI) caught it in part; GPT-6.1 Sol, GPT-5.6 Sol, Gemini 3.8 Flash, GPT-6 Luna, Claude Haiku 5.5, Gemini 3.1 Pro, DeepSeek V4 Flash (Workers AI), GPT-OSS 120B (Workers AI), Kimi K2.7 Code (Workers AI), Gemma 4 26B A4B and Gemini 3.5 Flash-Lite missed it.