Before applying an AI code-review fix, establish whether the reported problem exists under the application’s actual requirements. Treat the comment as a claim to investigate. Then assess the patch separately: a genuine problem can still receive an unsuitable repair.
For engineering leads, this is a review workflow decision. Automatically converting every confident comment into a code change can turn review noise into unnecessary behavior changes. Automatically dismissing generated comments can also lose useful findings. The useful middle step is a short record of the requirement, the alleged failure and the evidence for acting.
This guide proposes that record and a decision table. Its order-discount example is fictional. It does not report an independent comparison of AI review products or a production trial of this workflow.
Research distinguishes criticism from correct judgement
In their 26 June 2026 paper on requirement-conformance judgement, Haolin Jin and Huaming Chen evaluated models judging code against natural-language requirements. Their paired examples included correct and defective implementations from HumanEval, MBPP and QuixBugs. Models sometimes rejected correct implementations, and prompts requesting explanations or repairs could increase those false rejections while changing the rate of accepting defective code.
The authors also explored checking original and proposed implementations through executable tests. They acknowledge that incomplete tests can miss meaningful differences and that benchmark functions do not represent the full complexity of industrial review. These findings concern the studied model snapshots and setup, not a measured error rate for every current coding assistant.
The practical implication we draw is to distinguish an explanation from corroboration. Asking for a more elaborate rationale may help expose assumptions, but additional prose does not establish that the assumption is part of the contract. Nor does a patch demonstrate that its preceding diagnosis was correct.
Write the finding without its proposed solution
Temporarily set aside the suggested code. Restate the review comment in a form another developer can check: under a named input or system condition, this implementation produces an observed result that conflicts with a specified requirement.
For example, “replace this comparison with a stricter one” is a proposed edit. “An order exactly at the discount threshold receives a discount, although the approved rule excludes equality” is a claim about behavior. The second statement exposes the requirement that needs checking.
Our proposed finding record has five fields:
- Contract: the relevant requirement, interface guarantee or repository convention, with a location and version where available.
- Condition: the input, state or sequence that supposedly triggers the problem.
- Current behavior: a trace, execution result or inspected code path, labeled according to how it was established.
- Conflict: why that behavior violates the contract or creates a material risk.
- Unknowns: missing caller behavior, configuration or policy decisions that could change the diagnosis.
This is an editorial review aid, not a requirement to reproduce every comment in production. A type mismatch can be established through a compiler diagnostic; a security concern may require a threat analysis in an isolated environment. The aim is evidence appropriate to the claim.
A fictional discount fix that changes the contract
Suppose an internal purchasing tool has an approved rule: an eligible order qualifies for a discount when its subtotal is at least 10,000 cents. Subtotals are integer cents. Eligibility is checked elsewhere before this particular comparison.
The implementation includes equality. A fictional generated review comment says that the threshold should be exclusive and suggests changing the comparison to exclude exactly 10,000 cents. The comment sounds precise, but its premise conflicts with the stated rule.
| Subtotal in cents | Expected under the approved rule | Existing inclusive comparison | Suggested exclusive comparison |
|---|---|---|---|
| 9,999 | No discount | No discount | No discount |
| 10,000 | Discount | Discount | No discount |
| 10,001 | Discount | Discount | Discount |
The equality row identifies the consequence of the proposed change. Under this fictional contract, the existing comparison is correct and the suggestion introduces a defect. Reject the suggested edit with the contract reference and the boundary case. Do not rewrite the requirement to justify the patch.
Now change the situation: suppose the only available ticket says “discount above the threshold,” while the acceptance example includes equality. That is an unresolved requirement conflict. The code reviewer should ask the rule owner which behavior is intended before declaring either comparison correct. An assistant’s interpretation cannot settle the business decision.
A third situation is different again: the comparison receives an ineligible order because a caller bypasses eligibility checks. Changing the boundary operator would not repair that path. The evidence needs to follow the caller and the actual eligibility contract, rather than remain attached to the first line mentioned by the reviewer.
These examples illustrate three review outcomes: an unsupported finding, an unresolved contract and a real problem with the wrong proposed remedy. They are reasoning examples, not model-generated observations.
Product documentation also leaves room for false findings
GitHub’s current Copilot Agents application card describes code-review hallucination as feedback about nonexistent problems or misunderstandings of the code. It presents code review as a supplement to human review. This is the vendor’s description of its product and limitations, not independent validation of the workflow proposed here.
Use that boundary when designing automation. A comment’s presence, confident wording or severity label should not alone authorize a behavioral change. The person responsible for the change needs the relevant evidence and repository context. If the tool can generate patches, keep the decision about whether to apply one separate from the decision about whether the finding is supported.
Context matters in both directions. A local function may rely on a guarantee established by its caller. Conversely, a safe-looking local function may be reachable through a caller that violates that guarantee. Ask for the relevant path or configuration, and mark it uninspected if the reviewer cannot access it.
Give each comment an explicit disposition
Google’s practitioner guidance on review comments recommends explaining reasoning and distinguishing required changes from suggestions or information. That is engineering communication guidance, not a study of AI reviewer accuracy.
Apply the distinction to generated findings before assigning work:
| Evidence status | Review disposition | Next action |
|---|---|---|
| A demonstrated conflict with the current contract | Confirmed defect | Develop and validate a targeted repair. |
| Necessary context or a requirement is missing | Unresolved question | Obtain that evidence or an owner decision. |
| The proposed change is desirable but not required | Optional improvement | Assess its value and scope separately. |
| The claimed conflict is contradicted by the contract and checked behavior | Unsupported finding | Dismiss with the reason and evidence. |
An unverified security or reliability concern is not automatically harmless. Escalate consequential uncertainty through the team’s normal review process. Equally, avoid relabeling a preference as a defect merely to get an author to make the change.
Keep the decision near the change under review. A dismissed finding with an explanation can help the next reviewer understand why the code remains as written. Repeated false findings may indicate missing repository guidance; add only guidance that reflects a confirmed rule rather than accumulating exceptions to satisfy a bot.
Validate the repair against more than the triggering case
After confirming a defect, assess the smallest suitable patch. Check that it addresses the demonstrated condition and preserves important neighboring behavior. For the fictional discount rule, a boundary change should retain the below-threshold and above-threshold cases as well as the equality decision.
Passing a limited test set does not establish that two implementations are equivalent. Look at affected callers, side effects and existing interface guarantees. When a finding concerns performance, check the relevant workload; when it concerns access control, check the authorization path. A simple output example cannot settle every class of review claim.
Our guide to checking the intended regression in AI-generated tests covers whether a test detects the mistake it claims to detect. Here, the preceding decision remains essential: the expected behavior must be justified before either a generated test or a generated fix becomes the authority.
Start with the next consequential AI review comment. Ask the author or reviewer to complete the finding record before changing code, then record its disposition. Judge the exercise by whether it improves the change decision and makes unresolved questions visible, rather than by how many generated comments are closed.