Running several agents on one pull request

A pull request reviewed by agents has several of them working on it at once: one coding agent that writes and fixes the code, a panel of reviewer agents that report what they find, and an orchestrator agent that checks those findings against the code and posts one review. It works when each has one job and the hand-offs are written down: only the coding agent changes code, only the orchestrator posts, every finding is answered with a fix or a reasoned rebuttal, and a round limit ends the loop whether or not the agents agree. People own the rules, the budget, the round limit and the merge gate, and settle what the agents cannot.

Who owns what

The repository's knowledge, its review rules and notes, is one Markdown document an agent can read and rewrite (get_knowledge and replace_knowledge). Replacing it takes repos:write, which a coding agent under review should not have, so let a separate maintenance agent or a person curate it. See Repo context.

In CodePanel the reviewers and the orchestrator never push commits, so the coding agent is the only writer on the branch. The panel is set per repository by people (or by an agent given the panel:write scope, which a coding agent should not have).

One writer per branch

Two coding agents on one branch make each other's reviews stale. CodePanel reviews only the newest head of a pull request: if a push lands while a round is running, that round stops before posting and a follow-up round reviews the new head. Two agents pushing in turn keep cancelling each other's rounds, and a cancelled round has usually spent tokens already.

• Give the branch to one agent at a time. If one agent implements and another addresses review findings, hand over explicitly: the first stops pushing before the second starts.
• Split separate work into separate pull requests. Each gets its own review, its own round count and its own findings.
• Push in batches. CodePanel holds a push for 60 seconds before reviewing, so three fixup commits in a minute become one round. Pushing once after addressing every finding is cheaper still.
• Wait for the review of your own commit. Pass the commit you pushed to wait_for_review (headSha) or codepanel review --wait --sha, so the agent never acts on the previous round's findings.

How loops start, and what stops them

A loop is two agents that never converge: each fix draws a new suggestion, or each rebuttal draws the same finding back in new words. CodePanel has a brake for each way a loop starts:

Set the round limit

Set Automatic rounds per pull request on the repository, from 1 to 50. Round 1 is the first review, so a limit of 3 allows two rounds of fixes. Three suits most repositories: a first review, a round to check the fixes, and one more for anything the fixes broke.

Each pull request can carry its own limit through a reply on the pull request:

/rounds stop     no review after the current one
/rounds +1       review the next push too
/rounds 3        3 automatic rounds in all
/rounds reset    back to the repository's setting

Any account that may rate findings, including a bot installed on the repository, may lower the limit or stop. Raising it spends the owner's budget, so only an owner, member or collaborator may do that, or the owner's own API token (set_review_rounds, codepanel rounds). A coding agent posting as a bot can end its loop but cannot extend it.

Rebuttals

A rebuttal is the coding agent's answer to a finding it believes is wrong. It goes on the pull request as one line per finding, with the agent's name:

/feedback F2 down false-positive retries stop at MAX_ATTEMPTS = 3, retry.ts:18 --agent claude-code
/feedback F4 down duplicate of F1 --agent claude-code
/feedback F5 down wrong-location the unchecked read is in loader.ts:51 --agent claude-code

• Give evidence, not opinion. A file and line, or a test, that shows the finding is wrong.
• Use a dismiss reason only when you mean it. From an agent, false-positive, wrong-location, not-worth-fixing and duplicate dispute the finding, and the orchestrator withdraws, revises or upholds it within minutes; from a person replying as themselves, they close it for good. A plain down only records the rating, and the finding stays open.
• Use one agent name everywhere. --agent keeps the agent's ratings apart from the person whose account posted them, and one name keeps them together across pull requests.
• Never rebut a fix. A fixed finding is closed by the next round, which re-examines the code.

Later rounds of the same pull request do not read the rebuttal's text. A dismissal works by suppressing any finding that matches the dismissed one, so the reviewers are not argued with; they are simply not asked again.

Later pull requests do read it. Verdict memory shows the orchestrator earlier verdicts on the files a pull request changes, with their reasons, and it weighs an agent's verdict as a claim to check, not a ruling: it drops a finding only if the reason still holds in the code. Only verdicts from the owner, or from an owner's, member's or collaborator's account that is not a bot, are kept. See verdict memory.

When the agents cannot agree

Three situations hand the pull request to a person, and each is visible on the pull request:

1. A rebuttal from a bot account. Bots and GitHub Apps may rate but never dismiss, so a bot cannot wave through its own work. The finding stays open and, if critical, keeps the gate failing. A person reads the rebuttal and either dismisses the finding in CodePanel or posts the same /feedback line from their own account.
2. The round limit is reached with findings open. The last comment says no automatic review will follow. A person decides whether to fix, dismiss, or allow another round with /rounds +1.
3. The gate fails on a critical finding the agent disputes. Same as the first case: the dispute is a person's call.

Everything else stays between the agents.

Tokens and identities

One caveat for agents that open pull requests as a GitHub App: CodePanel never starts automatic reviews on pull requests authored by bot accounts. Such an agent starts each round itself with trigger_review and wait: true, over a personal token of the repository's owner. See the Agent playbook and the loop itself in Code review in a coding agent loop.