Secure code review

Secure code review is reading a source change for vulnerabilities before it merges: trace untrusted input to where it is used, check authorization on every path, and look for secrets, risky dependencies and unsafe defaults. NIST's Secure Software Development Framework lists it as practice PW.7 (SP 800-218 v1.1).

When an agent writes the change, its pipeline and docs need the same reading as the application code: a workflow runs with real credentials, and the next agent copies a docs example. CodePanel reviews its own repository, where Claude Code writes the pull requests and answers the reviews. As of 17 October 2026, Claude Code wrote 147 of the 150 pull requests merged from PR 61 to PR 213, and answered the reviews on them. For design, tests and readability see the code review checklist; this page covers security only.

What to look for in a diff

Two current lists give a reviewer a shortlist: MITRE's 2025 CWE Top 25, ranked by frequency and severity from 39,080 CVE records published between 1 June 2024 and 1 June 2025, and the OWASP Top 10:2025, the awareness list for web applications.

The OWASP column is this page's mapping by name. Ranks 5, 7 and 8 are memory-safety bugs that matter mainly in C and C++, so they are left out. The ranking shows what is reported most, not what reviewers miss.

The dependency row has evidence from generated code: across 576,000 samples from 16 models, Spracklen et al. (USENIX Security 2025) found invented package names at an average rate of at least 5.2% for commercial models and 21.7% for open-source ones. The commercial models were GPT-3.5 Turbo, GPT-4 and GPT-4 Turbo, chosen in January 2024, so current rates may differ, but checking that a package exists takes seconds.

Secure code review checklist

Copy this into a pull request template or an agent's rules file. Each line has a yes or no answer for the diff; a "no" is a finding.

Secure code review checklist

[ ] Input to sink: every value from a request, webhook, file, env var or
    model output is followed to where it is used (query, shell, path, HTML,
    redirect, deserializer) and validated or encoded for that sink.
[ ] Authorization before every read and write: the server checks who the
    caller is and that they own the record, on every new route, function
    and job. A frontend check does not count. A missing role denies.
[ ] Secrets: none in the diff, tests, fixtures, docs examples or CI logs,
    or passed as a command-line flag. A committed secret is rotated.
[ ] Dependencies: each new package exists, is the one intended, and is
    pinned. The lock file matches the manifest.
[ ] CI and workflows: tools are downloaded at a pinned version with a
    checksum, actions are pinned, and no job holds publish, deploy or
    id-token rights it does not use.
[ ] Crypto and comparisons: a vetted library, a password hash, random
    values from a cryptographic source, and signatures and tokens compared
    in constant time, never with ===.
[ ] Webhooks: the signature is checked over the raw body before any work is
    scheduled, and the job carries ids, not the payload.
[ ] Errors and logs: no stack trace, query text, provider message, token,
    key or personal data reaches the caller or the log, and no catch lets a
    failed check through.
[ ] Defaults: new flags, CORS rules, debug modes and public access start
    safe.
[ ] Tests: one proves the unauthorized caller is refused, not only that the
    owner succeeds.

NIST lists review checklists among PW.7's notional examples, and the evidence for them is mixed (the study is in the checklist guide above). Treat the list as a prompt, not proof.

Two catches in agent-written code

Both come from PR 180, where Claude Code prepared CodePanel's command-line and MCP tools for a registry listing.

The workflow that trusted "latest". A warning: the publish workflow downloaded its publishing tool from the latest release, unpinned and unchecked, then ran it in a job with id-token: write. In the finding's words: "A moved or compromised release would run with rights to publish under the org's registry namespace." Claude Code rated it fixed: the tool is now pinned and checked against a SHA-256 digest.

The private key in a command line. A docs example passed a signing key as a command-line flag, which leaks it into shell history and the process list. Claude Code disputed the finding as not worth fixing: the example was the secondary route, with the exposure explained beside it. The orchestrator upheld the finding:

"The private key exposure is a security concern, so it cannot be withdrawn as not worth fixing."

Claude Code then rewrote the page so the key-store options lead, with the flag kept only as a labelled last resort.

How CodePanel treats security findings

• Security findings get the strictest dispute rule. The orchestrator is told never to withdraw one as not worth fixing, that the agent that wrote a change wants it merged, so a rebuttal is an argument to check against the code, and to uphold when the code settles nothing either way.
• Memory is told to leave them alone. The orchestrator is told never to let verdict memory or a learning (a repository's lesson about what to skip) lower a security or critical finding.
• The starter panel already looks. Its Bugs & Security reviewer is told to look for missing auth or permission checks, injection, unvalidated input and leaked secrets. The optional Security role goes deeper (untrusted input, ownership, signatures, SSRF, unsafe deserialization) and explains how each issue could be exploited. It is added on a frontier-tier model at high effort, so it costs more per round (model prices, Reviewers).
• The gate decides what blocks. At the default level, critical and warning findings fail the CodePanel status, which holds the merge once it is a required check, and suggestions do not. To have every finding answered, suggestions included, set the level to all (Merge gate).

Scope the coding agent's token to read and reviews:write, so it can rate findings but cannot change the panel, settings or budget (Scopes, API tokens). CodePanel does not scrub secrets from the diff: patches go to the AI providers on your panel like the rest of the change, and capped patches are stored (What data goes where). Rotate any secret that was committed.

What a model reviewer cannot do

A model reviewer reads the diff and the code around it, not the running system: deployed configuration, cloud permissions, or how two services behave together. Lock files are not sent to the panel, so pair it with a dependency scanner for lock-file changes. Reviewers are told to mark claims about a library low confidence, since training data may predate your version. OWASP's Code Review Guide (v2.0, 2017) says automated tools always need human verification, because they do not understand context. Pair the panel with a static analysis tool and tests that exercise the denied path; a person decides who should be allowed to do what.

Add a Security reviewer and security rules

Adding a reviewer needs the panel:write scope, which the coding agent's everyday token should not carry: use a separate token or the dashboard's Add reviewer picker. With the CodePanel MCP server connected, give your agent:

Use CodePanel list_repos and read this repository's settings.panelId (none
or null means your default panel), then get_panel to find that panel's _id.
Use list_roles to confirm the id of the Security role, then call
add_reviewer with that panelId and role "security". Run get_panel again and
report the new reviewer's agentId, the model it was given, and whether the
panel is ready.

The CLI does the same. Without --panel it adds to your default panel:

codepanel panel                    # panels and their ids
codepanel panel roles
codepanel panel add-reviewer --panel <id> --role security

Then add security rules to .codepanel/rules.md. A ## heading made of file patterns scopes the rules under it (Repo context):

## server/**,src/api/**
- Every route and function checks the caller's identity, then ownership of the record it touches, before any read or write. Flag one that does not.
- Compare signatures and tokens in constant time. Flag === on a secret.

## .github/workflows/*
- Flag a tool downloaded without a pinned version and checksum, or a job permission the job does not use.

Rules are read from the base branch, so a pull request cannot loosen the rules it is reviewed against. Review rules for agents covers writing them, Reviewing AI-generated code the other mistakes agents repeat, and AI code review a full round. To try it, give your agent the Quickstart.