Blog
Blog
Notes from building CodePanel: code review as agents working with agents, how multi-model review works, how we measure it, and what we learned. RSS feed.
- How to tell if an AI code review change actually helpsPaired runs over the same fixtures, intervals resampled over fixtures, a sign-flip test and Holm. The pipeline changes we measured and did not ship.The CodePanel team
- Recall is not enough: measuring false positives in code reviewHow CodePanel counts false alarms: clean and trap fixtures, blocking recall, and precision from verdicts the author agent cannot give itself.The CodePanel team
- Claude Sonnet 5.5 was the fastest reviewer in our benchmark, and tied for firstClaude Sonnet 5.5 tied for first in the 2026-10-17 run at 10 seconds a review, the fastest of 17 models, and cost more per review than its list price suggests.The CodePanel team
- GPT-6.1 Sol had the highest measured recall in our benchmark, at about a third of GPT-5.6 Sol's costIn the 2026-10-17 run, GPT-6.1 Sol had the highest measured recall of 17 models, with few false alarms, at about a third of GPT-5.6 Sol's cost per review.The CodePanel team
- First benchmark results: seven open models on real bugsDeepSeek V4 Pro and GLM-5.3 Flash tied for first, and the Flash model costs a seventeenth as much per review. What we ran and what surprised us.The CodePanel team
- Code review should be agents working with agentsWhy CodePanel is built for a coding agent and a panel of reviewer agents working on one pull request, and what people still decide.The CodePanel team
- How we built the code review benchmarkReal defects from open-source history, clean changes to count false alarms, a three-family judge panel and intervals on every number. How it grows.The CodePanel team
- Why CodePanel reviews with a panel of modelsSeveral cheap reviewers from different model families, then one orchestrator that checks every finding against the code. Why we built review this way.The CodePanel team