Feedback and disputes

Every finding should get one answer: a fix, or a rating with a reason. The answer decides whether the finding closes, whether the merge gate still blocks, and what later reviews of the same files remember.

Rate findings

A rating says whether a finding was right. Ratings measure which models and prompts produce good findings, and a rating or dismissal with a reason is kept as a verdict that later reviews of the same file weigh (verdict memory). Rate every finding once, with the same agent name each time, so one agent's ratings are counted together.

A rate_finding call and a /feedback --agent line count the same for the merge gate, including after an upheld dispute. A token rating always counts as an agent's. The one difference is deferred: a /feedback reply from an account with write access closes the finding, since that account vouches for it, while a rate_finding call only answers it. On a warning both unblock the gate at once. On a critical the call answers nothing: an agent's deferred there counts once a person defers it (item 3 below).

Rating the same target again replaces your earlier rating. Ratings sent through the API, MCP or the CLI are listed in the review comment's Ratings section, note included, so on a public repository the note is public.

The API and MCP rate_finding accept verdict as an alias for reason, and comment as an alias for note, including in each ratings batch item. For example, { "ref": "F11", "rating": "down", "verdict": "false_positive", "comment": "The caller checks ownership." } opens the same dispute as a down rating with reason and note. If you send both spellings, they must agree. Omit unused aliases instead of sending blank strings. In batch calls, put rating details inside each item; top-level rating details are rejected. A pending dispute blocks the merge until the orchestrator decides it, for at most 10 minutes.

Who counts as a person or an agent

The same reason has a different effect depending on who gives it.

Always add --agent <name> when an agent posts through a person's account (for example with gh pr comment). Without it the reply counts as that person's own, and a rebuttal closes the finding without the orchestrator's check.

Verdict reasons

Reasons are written with hyphens in a comment (false-positive) and underscores in the API (false_positive); both forms work everywhere.

A plain down with no reason records a rating and answers nothing: the finding still blocks. Rating a dismissed finding up does not reopen it; reopen it instead.

What counts as answered

A gated finding stops blocking the merge gate when one of these is true:

1. Resolved. A later round re-examined its code and did not report it again.
2. Dismissed. By the repository owner, a person's rebuttal, or a withdrawn dispute (dismissing).
3. A verdict since it was last reported. fixed, deferred, or a person's false_positive, wrong_location, not_worth_fixing or duplicate, given after the last round that reported the finding. On a critical finding an agent's fixed or deferred does not count here: the author agent wants its change merged, so its fixed counts once a round resolves the finding, and its deferred once a person defers it.
4. A decided dispute. An agent's disputing verdict, once the orchestrator withdrew or revised the finding.

It still blocks while a dispute is pending (for at most 10 minutes), after a dispute is upheld (until it is fixed or rated deferred), and with only useful, unclear, other or no reason. A finding a later round reports again needs a new answer, unless its dispute was decided at the commit the finding now refers to. So /feedback F3 up fixed unblocks F3 until the next review; if that review still finds the problem, F3 blocks again.

The gate level decides which severities must be answered; see Merge gate.

Disputes

An agent disputes a finding by rating it down with false_positive, wrong_location, not_worth_fixing or duplicate, and a note citing the code: a file:line, a test or a doc. dismiss_finding, codepanel findings dismiss and POST /api/v1/findings/dismiss with agent set dispute it the same way (reason defaults to false_positive).

/feedback F4 down false-positive src/auth.ts:42 checks ownership first --agent claude-code

Lifecycle

A dispute starts pending. About three minutes after the first dispute on a pull request, the orchestrator reads every pending dispute there in one call: the finding, its evidence, the code at the commit it refers to, and the rebuttal. Where CodePanel has ORCHESTRATOR_TOOLS on, it can also open and search files at that commit, on your own key with a model that takes tool calls (conditions). If a review round starts first, the round decides them instead. A dispute still undecided 10 minutes after it was made counts as an answer in the merge gate until a decision lands.

Read the outcome under Disputes in the posted comment, or as dispute on the finding in get_review, GET /api/v1/review and codepanel review --json. Disputing again at the same commit gets the same answer; after a push moves the finding, you can dispute it afresh.

Who can close what

A dispute closes only what its author could have dismissed. A withdrawal from a bot account's dispute is recorded and shown, but the finding stays open, as the bot's rating alone would leave it (the bot's verdict still answers it for the gate). A dispute from a token or from a person's account with --agent closes the finding when withdrawn.

A person's own down reply with a disputing reason closes the finding at once, without a dispute. Reopening a finding clears its dispute.

Cost and limits

Deciding a dispute is a review call. It counts toward the per-PR and monthly limits, which are checked before the call but not reserved, so two calls at once can pass a limit by one small call. It runs on your own key or ChatGPT plan when one of the panel's models has one, and otherwise on CodePanel credits if you have them; a call on credits first reserves the most it could cost, and waits with "not enough CodePanel credits" when the balance cannot cover it. A dispute that cannot be decided says why it is waiting. When the call fails (an error, or an answer cut off at the model's output limit), another of the panel's models is asked at once; if that fails too, CodePanel asks again on its own, at growing intervals, for an hour after the dispute. After that the reason says so: dispute again or comment /review to have it asked. The orchestrator prefers a model from a different family than the reviewers that reported the finding.

Ask the orchestrator

When you are unsure whether a finding is right, ask before you fix or dispute it.

The orchestrator answers from the finding, its code, the posted review and your repository notes; on the pull request the answer is a reply. Each pull request takes up to 10 questions an hour. A question never starts a review.

If the answer concedes that the finding is wrong, it is recorded as a withdrawn dispute. The finding is closed, and the reply says so, only if the asker could have dismissed it: the repository owner's token, or the own account of someone with write access, not a bot's.

A question counts toward the monthly budget and the repository's cap, like chat, checked before the call but not reserved. It runs on your own key or ChatGPT plan for the orchestrator's provider, or on CodePanel credits if you have them and no key of your own there, reserving the most it could cost first.

Learnings

/remember <text> on the pull request (or add_learning, codepanel learn) records a fact reviewers should apply from then on, and /feedback L3 up or /feedback L3 down <reason> confirms or turns off learning L3. See Repo context and the syntax.