CodePanel

Swift code review: which models catch the bugs

Of the 17 models ranked on Swift code, 9 models tied for the most known bugs: 100%. 5 of the other 8 models have 95% intervals that overlap theirs. Gemma 4 26B A4B and Gemini 3.5 Flash-Lite tied for the fewest, 0%.

Updated 2026-11-04Full leaderboardMethodology

Results

Known defects caught, with 95% intervals. The tick is the model's recall over every bug.

Swift code review: which models catch the bugs
ModelCaught hereEvery bug
Claude Opus 5.5100%90%
GPT-6.1 Sol100%89%
Gemini 4 Argon100%82%
Claude Sonnet 5.5100%77%
GPT-5.6 Sol100%72%
Gemini 3.8 Flash100%68%
GPT-6 Luna100%67%
Gemini 3.1 Pro100%67%
GLM-5.3 (Workers AI)100%61%
DeepSeek V4 Pro (Workers AI)75%[50%–100%]54%
Claude Haiku 5.567%[0%–100%]64%
GLM-5.3 Flash (Workers AI)67%[33%–100%]61%
DeepSeek V4 Flash (Workers AI)67%[33%–100%]49%
Kimi K2.7 Code (Workers AI)46%[20%–67%]38%
GPT-OSS 120B (Workers AI)44%[0%–100%]40%
Gemma 4 26B A4B (Workers AI)0%26%
Gemini 3.5 Flash-Lite0%11%

Against the rest of the benchmark

On average these models caught 74% of known bugs in Swift code, more often than the 60% the same 17 caught across the whole benchmark: the easiest of the 12 languages.

GLM-5.3 gains the most here: 100%, against 61% over every bug.

Gemma 4 26B A4B drops the most: 0%, against 26% over every bug.

A test PR that split the models

The change “DatabaseQueue and DatabaseSnapshot reads can access task locals” in groue/GRDB.swift, a change of under 100 lines that landed in July 2025, introduced a concurrency bug that a later upstream commit fixed.

Claude Opus 5.5, GPT-6.1 Sol, Gemini 4 Argon, Claude Sonnet 5.5, GPT-5.6 Sol, Gemini 3.8 Flash, GPT-6 Luna, Claude Haiku 5.5, Gemini 3.1 Pro and GLM-5.3 caught it; GLM-5.3 Flash, DeepSeek V4 Pro (Workers AI), DeepSeek V4 Flash (Workers AI) and Kimi K2.7 Code (Workers AI) caught it in part; GPT-OSS 120B (Workers AI), Gemma 4 26B A4B and Gemini 3.5 Flash-Lite missed it.