Review to refute

Reviewing is reading the agent's output against the spec's Definition of Done by trying to refute it, not by confirming the agent's self-report. The two references are not the same set. The self-report covers only what the agent ran — the compile, the suite. The Definition of Done covers every traced check A4 produced, including the ones the agent did not run and would not report. A review that takes the report as its reference can only confirm what the report already claims; a review that takes the Definition of Done as its reference can find what the report leaves out.

Three properties make a review adversarial. It checks the spec, not the report — the reference is the Definition of Done, the traced and executed checks, never the agent's “done — tests green.” It tries to refute — each check is run as an attempt to find a counterexample, not as a box to tick. It defaults to fail when uncertain — a check that cannot be proven passing is treated as failing, so ambiguity blocks acceptance rather than passing it.

The move that distinguishes a real review from a rubber stamp is the direction of the check. A confirming review reads the self-report and runs only the checks the report already proves — it stops at “tests green.” An adversarial review runs every check in the Definition of Done as an attempt to refute, including the ones the suite cannot prove until they are run live. Where the two diverge is one check: the liveness smoke. The green suite runs server: false, so a passing suite proves nothing about a running server. The adversarial review boots the node and runs the check; the confirming review assumes it.

Toggle the meter below between the two reviews, over the three real Apollo checks on the Portal's F6.7 rung. A confirming review runs the two the suite already proves and assumes the third; an adversarial review runs all three and default-fails the liveness check until it is run live. The broadcast-only-on-success rule it refutes is the one taught in /elixir/phoenix/pubsub/broadcast.

The Portal's F6.7 ship prompt assigns its verify stage to Apollo, whose standing directive is the one this dive argues, quoted verbatim from the implementation prompt:

# f6.7 — the Apollo verify stage, verbatim
APOLLO — adversarially verify (try to REFUTE; default FAIL when uncertain)

The prompt then names the liveness gate as the reason the review must run the real check, not the report. A green suite is not a running server, verbatim from the same file:

# f6.7 — the liveness gate, verbatim
mix test runs server:false (config/test.exs), so a green suite does NOT prove the dev server boots

and from the Director's ratify gate in the same file:

# f6.7 — the ratify gate, verbatim
mix test alone does NOT satisfy the liveness criterion (the endpoint runs server: false under test)

Apollo's adversarial checks on F6.7 are all run as attempts to refute. The master-invariant grep — CatalogLive and Catalog name only Portal and Portal.PubSub (no Engine, Repo, or GenServer.call); the facade boundary holds. Broadcast only on {:ok, _} — the broadcast fires only after a successful write, and is silent on a failed one; the review refutes it by checking both directions, the rule taught in /elixir/phoenix/pubsub/broadcast. The liveness two-window smoke — boot the node, curl :4000/health for a 200, and run the rung's live two-window update; the one check the green suite does not prove. A confirming review reads the self-report and stops at “tests green.” An adversarial review runs the liveness smoke and finds that the green suite ran server: false — the failure the gates alone do not catch.

Briefing fixes the plan; supervising watches the run; reviewing decides whether the work is done. The review is adversarial: it checks the spec's Definition of Done rather than the agent's self-report, it runs each check as an attempt to refute rather than a box to tick, and it defaults to fail when uncertain. On the Portal that resolves to one decisive check the gates alone miss — the liveness smoke, because the suite runs server: false. Running an agent well is a loop the human owns: the agent runs the prompt; the human owns the gate. The module's loop closes back at its hub.

Primary sources for adversarial verification and the value of a fixed contract to refute against, and where the prompt this dive grounds on and the companion Phoenix chapter are defined in the course.