PR review: writing a review a coding agent can act on

A PR review reads a pull request's change in the context of the code around it and returns a decision plus comments the author must answer before merge. Written for a coding agent, each comment needs a stable id, a location, a severity, the problem and the fix, and the review needs one machine-readable next step.

On CodePanel's own repository, Claude Code wrote 147 of the 150 pull requests merged from PR 61 to PR 213 as of 17 October 2026, and it reads the reviews. All but one of the 1,460 /feedback lines rating a finding in PR comments carry --agent claude-code. The AI code review pillar covers the system; this page covers the review itself.

What a reviewer reads, in order

A reviewer, human or model, should read in this order:

1. The description. It states what the change claims to do, and everything after is checked against it.
2. The diff. What the change does.
3. The code around each change. The diff cannot show a null check ten lines above a hunk, or the caller that depends on the old behaviour.
4. The tests. Would one fail without the change, and does it assert behaviour or only mirror the code?
5. The project's rules. Conventions the diff can break without a bug.

CodePanel's reviewers get the title, author and description, the diff with each line numbered from the new revision (a finding copies a line number instead of computing one), the code around each change at the head commit, and .codepanel/rules.md from the base branch. The limits are in How it works. AGENTS.md and CLAUDE.md, CI results, linked issues and the author agent's handoff are off by default (repo context).

A review returns more than a verdict

GitHub gives a reviewer three review states: Comment, Approve and Request changes, which "flags feedback that the author should address before merging". For an agent that is one value on the whole review. It does not say which comments block, which commit was read, or whether an earlier comment was answered. An Approve is an LGTM in words, and the agent cannot tell what it covers.

CodePanel never approves or requests changes. The only GitHub review event it posts is COMMENT, and only for inline suggestions, which are off by default. The decision is a commit status named CodePanel that the repository can require. An agent needs five things from a comment: a stable id (F3) that its answer names and the next round matches, with a finding reported again in matching terms keeping its id; a path:line at a named commit, so the code can be found after more pushes; a severity, which says what holds the merge; the problem and the smallest fix that would answer it; and one next step. The full vocabulary is in Findings.

Anatomy of one real PR review

The comment below is on PR 192, "Eval: one adaptive in-flight pool per provider". CodePanel posts one comment per pull request and rewrites it each round:

A finding heading packs five fields: id, severity, location, lifecycle and confidence, as in ### F3 · warning · path/file.ts:42 · open since round N · low confidence. A new finding says new, and the lifecycle field is left out on a first round. Confidence is shown only when it is low or medium. One real finding from the same comment, with its code excerpt trimmed ([...]):

### F14 · suggestion · `tools/eval/providerPools.test.ts:99` · new · low confidence
Problem:
> The new assertion hard-codes 16 instead of referencing the fallback, so a change
> to SMALL_CEILING breaks this test for an unrelated reason. [...]
```ts
[...]
	expect(pools.ceilingFor("constructor")).toBe(16);
[...]
```
Fix:
> Compare against POOL_CEILINGS.google or an exported fallback constant, and add
> a case such as 'unlisted:model'.

A default finding has only Problem, a code excerpt and Fix. Trigger:, Check: and Patch: parts exist but are off by default (evidence). The agent rated this one up useful through the API and left it for the next change to that test file.

The next step is one field

An agent should not parse the summary. get_review returns payload.nextAction: one action, the finding refs it is about and a reason. The server checks the cases in this order, so only one comes back (The review loop has the detail):

gate.blocking says whether the merge gate still holds the pull request, and gate.unanswered lists the refs holding it. It is a sibling of payload in the response, and it is what the repository merges by. Do not decide from payload.verdict or blockedBy: they count every open critical and warning finding, answered or not, so a verdict can read blocked after everything is answered. See blocking finding and the MCP server reference.

A review that caught the description drifting

The description is part of what a review reads, so a review can catch it going stale. On CodePanel's PR 184, which changed how chat finds the review it answers with, the first review's F2 (a suggestion) said that a session with no stored review fell back to the pull request's newest one, possibly from another session. Claude Code dropped the fallback and rated it:

/feedback F2 up fixed dropped the by_pr fallback; a round with no stored review now gets none --agent claude-code

The next review raised F10, a low-confidence suggestion at convex/sessions/chatContext.ts:51: "A PR description that still promises the fallback would be wrong". The code was right by then, the description was not. The agent edited it and rated the finding:

/feedback F10 up fixed the PR description now states the no-fallback behaviour --agent claude-code

The body's edit history confirms it. The first description said that a round with no stored review "falls back to the PR's newest one". The merged body says such a round gets none. An agent that fixes the code and not the description leaves the old claim in place.

Answering the review

Every finding gets one answer: a /feedback line in a PR comment, or one rate_finding call. Fix it and say up fixed, or dispute it with the evidence. How disputes are judged is in agentic code review. Rate the review itself when it was notably good or noisy:

/feedback review up every finding was accurate and small --agent claude-code
/feedback review down noisy: three suggestions on links the repo's own tests already check --agent claude-code

The reason is the useful part. Syntax is in Feedback and commands.

Hand the loop to your agent

With the CodePanel MCP server connected, give your agent this after it pushes:

Call the CodePanel wait_for_review tool with repo owner/name, pr <n>, headSha set to
the output of git rev-parse HEAD, and timeoutSeconds 45. On status timeout, call it
again; on status no_review, call trigger_review. When it finishes, call get_review
for the same pull request and act on payload.nextAction, not on payload.verdict:
- fix: fix each finding in refs, push, and start again with the new head SHA.
- wait: call wait_for_review again.
- trigger: call trigger_review, then wait_for_review.
- rate: rate each ref in refs.
- stop: report nextAction.reason. Do not trigger again.
- done: stop.
For each finding, check the problem against the code first. Then call
rate_finding with repo, pr, ref, rating, reason and a one-sentence note: up fixed,
up deferred (say where it is tracked), or down false_positive (name the file and
line that contradict it). Pass agent claude-code.

The same loop from a shell:

codepanel review owner/name#<n> --wait --sha "$(git rev-parse HEAD)"
codepanel review owner/name#<n> --json | jq -r '.payload.nextAction.action'
codepanel feedback owner/name#<n> F3 up --reason fixed --note "Added the ownership check" --agent claude-code

To try it with your agent, start from the quickstart.