GUIDE-PR-001 · UPDATED
AI PR review starts with evidence, not generated confidence.
AI PR review is the practice of checking what an AI-generated patch actually did before a reviewer makes a merge decision: whether it applied through the intended path, what diff was realized, which failures belong to the runner, and which evidence remains incomplete.
§ 01
What is AI PR review?
AI PR review is a preflight review discipline for patches proposed by coding agents. It gives a human reviewer a structured record of the patch’s application path, realized changes, failures, and supporting evidence instead of treating an agent’s explanation or a passing-looking diff as the decision.
That distinction matters because a patch can apply cleanly yet still be out of scope, hide a regression, or be impossible to evaluate because the test runner is unhealthy. A credible review separates those observations rather than flattening them into one pass/fail label.
§ 02
Four evidence questions before a PR decision
Did the candidate apply through the intended harness?
An applicable patch is reviewable. A patch that cannot pass the intended apply gate needs repair or a clear block reason.
What code was actually realized?
Inspect a canonical diff rather than relying on fuzzy patch text or a generated rationale. The realized change is the object under review.
Is a failure runner-caused or candidate-caused?
Broken installs, unavailable services, and missing dependencies are runner-health signals. They are not automatically candidate failures.
What evidence supports the proposed behavior?
Tests, reproductions, provenance checks, and scope review help a reviewer assess risk. Each signal should retain its source and limits.
§ 03
Where an evidence layer fits
AdaptOrch sits beside the coding agent rather than replacing the IDE, prompt, model, or pull-request workflow. It receives raw, repaired, or synthesized patch candidates and records evidence through an apply gate, canonicalization, strict re-check, provenance review, runner-health classification, and a replayable evidence DAG.
The output is intended to make review conversations specific: which candidate was assessed, which gate failed, whether a failure was environmental, what was changed, and what is still unproven. See the AI-generated patch verification guide for the gate-by-gate workflow.
§ 04
Keep the decision boundary explicit
In CEK S3, AdaptOrch operates as a shadow review layer. It can report applicability, canonical diffs, runner health, provenance risk, and replayable records. It does not prove semantic correctness, mutate the baseline-selected candidate, or let an AI rationale override execution and provenance evidence.
That boundary protects reviewers from a common failure mode: treating a fluent explanation, a clean apply, or an isolated public test pass as a blanket approval. Read the complete claim boundary before using evidence output in a production review process.
§ 05
AI PR review FAQ
Q.01What should an AI PR review verify first?
Start with intended-harness applicability, then inspect the realized diff, runner health, provenance, and available reproducible evidence.
Q.02Does a cleanly applied AI patch prove it is correct?
No. Applicability is one signal; it does not replace review of semantic behavior, test evidence, scope, or risk.
Q.03How should a team treat a broken test runner?
Keep environment-caused failures separate from candidate-caused failures. A broken install or unavailable service should not automatically become a patch failure.
Q.04What does a shadow review decide?
It records evidence, risk labels, and explicit limits for a human decision. CEK S3 does not mutate selection or claim semantic correctness.
NEXT
Start with one repository and one review question.
A useful first audit is narrow: select one repository, a representative set of AI-generated candidates, and a review question such as “which failures were actually runner-caused?” A shadow report can then make the next workflow decision inspectable.