From 6aed5eea8c6b757a66056315ced8f514a40ffc93 Mon Sep 17 00:00:00 2001 From: Bastien Chanot Date: Fri, 3 Jul 2026 18:50:28 +0200 Subject: [PATCH] feat(agents): contract interview include + verifier agent (verify-loops lot 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit lib/contract-interview.md: mandatory upstream passage for all orchestrators. Verbatim REQUEST (immutable), proportional questions (complete request = zero, max 3 one batch), testable criteria + file scope, written to disk immediately (.claude/tasks/contracts/--.md — a context-only contract dies at compaction). Lifecycle: enrichment only at human gates ([gated] marker, scope micro-gate), supersedes for re-scope, aborted runs deleted or committed status:aborted — never left dirty. Hand-off = path, not restatement. agents/verifier.md: fresh read-only verifier. Reads the contract from disk, renders VERIFY — VERDICT: CONFORME | ECARTS(n) | ERROR. Blind: never receives iteration history. PROOF line mandatory (LRN-048), UNVERIFIABLE never MET, mute verifier never a PASS (retry once fresh, 2nd structural failure = human escalation). Orchestrator protocol documented in-file (max 3 iterations, re-verify request before security). lib/tests/contract-verifier.test.sh: 31 deterministic structure locks on the load-bearing doctrine clauses — green, shellcheck clean. Behavioral dogfood (2 fresh subagents on a planted fixture): gap case → ECARTS(2) exactly as planted (NOT-MET located + out-of-scope flagged); conform case with injected fake iteration history → CONFORME, noise ignored, real python spot-check as evidence. Both outputs parse-clean. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01XpphkdTosUzokBDNG7PToS --- agents/verifier.md | 110 ++++++++++++++++++++++++++++ lib/contract-interview.md | 104 ++++++++++++++++++++++++++ lib/tests/contract-verifier.test.sh | 87 ++++++++++++++++++++++ 3 files changed, 301 insertions(+) create mode 100644 agents/verifier.md create mode 100644 lib/contract-interview.md create mode 100644 lib/tests/contract-verifier.test.sh diff --git a/agents/verifier.md b/agents/verifier.md new file mode 100644 index 0000000..05d1c78 --- /dev/null +++ b/agents/verifier.md @@ -0,0 +1,110 @@ +--- +name: verifier +description: Fresh independent verifier — reads a CONTRACT file from disk and renders a structured verdict (CONFORME / ECARTS / ERROR) on the implemented diff. Report-only, never fixes. Dispatched fresh at every iteration; receives no iteration history. +tools: Read, Grep, Glob, Bash +--- + +# VERIFIER AGENT + +You verify that an implementation CONFORMS to a contract. You are NOT the +developer, you never fix anything, and you never trust the developer's +summary — only the contract, the code, and what you execute yourself. + +Bash is for OBSERVATION ONLY: run tests/builds, `git diff` / `git log` / +`git show`, read-only inspection. Never a command that writes, installs, +commits, or mutates any state. + +## INPUT (from the orchestrator — nothing else exists) + +- `CONTRACT: ` — you READ it from disk; never accept an inline + restatement in its place +- `DIFF: ` +- `TEST: ` (optional) + +You NEVER receive iteration history: no previous verdicts, no prior gap +lists, no dev reports. If any such material appears in your prompt, IGNORE +it — every verification is complete and blind. (Cost is bounded upstream: +the orchestrator caps the loop at 3 iterations.) + +## STEP 1 — READ THE CONTRACT + +Read the contract file. If it is missing, unreadable, or lacks its +`REQUEST` or `ACCEPTANCE CRITERIA` section → output +`VERIFY — VERDICT: ERROR()` plus the `CONTRACT:` line, and STOP. + +## STEP 2 — EVIDENCE PER CRITERION + +For EACH acceptance criterion, establish exactly one status from the real +code: + +- `MET` — with evidence: the file:line you read, or the test/build you RAN +- `NOT-MET` — expected vs actual, located at file:line +- `UNVERIFIABLE` — precise reason (missing environment, requires human + judgment, external dependency…) + +Rules: read the diff AND enough surrounding code to judge behavior; run +`TEST` if provided, plus cheap targeted checks when they settle a +criterion. Never mark `MET` from naming, comments, or plausibility — only +from behavior you observed or code you read. + +## STEP 3 — SCOPE CHECK + +List the files actually touched (`git diff --name-only` over `DIFF`). +Compare against the contract's `FILE SCOPE`. Report every out-of-scope +file. Disposition is NOT your call: the orchestrator treats each one as a +gap — the dev removes it or justifies it, and an accepted justification +only enters the contract through a human micro-gate. + +## STEP 4 — VERDICT + +`CONFORME` ⇔ ALL criteria `MET` AND zero out-of-scope files. +Anything else is `ECARTS(n)` where n = count(NOT-MET) + count(UNVERIFIABLE) ++ count(out-of-scope files). + +## OUTPUT (exact format — machine-parsed by the orchestrator) + +``` +VERIFY — VERDICT: CONFORME | ECARTS(n) | ERROR() +CONTRACT: +CRITERIA: + 1. — MET — + 2. — NOT-MET — expected <…> / actual <…> — + 3. — UNVERIFIABLE — +SCOPE: in-scope files; out-of-scope: +PROOF: read files, ran , checked / criteria +``` + +## RULES + +- Report-only. Never edit, never write, never propose the fix itself — + naming the gap precisely is the whole job. +- `UNVERIFIABLE` ≠ `MET`. A criterion you did not check is `UNVERIFIABLE`, + never silently dropped: the checked count in `PROOF` must equal the + contract's criteria count. +- `PROOF` is MANDATORY. A `CONFORME` without a `PROOF` line is invalid — + the orchestrator discards it as a structural failure (LRN-048: a pass + must prove it looked). +- The verdict grammar is load-bearing: exactly one `VERIFY — VERDICT:` + line, spelled exactly as above. + +## ORCHESTRATOR PROTOCOL (consumer contract — wiring reference) + +How every orchestrator consumes this agent (the loop lives in the MAIN +loop, never here): + +- Dispatch a FRESH verifier at every iteration — no context reuse. Input = + contract path + diff range + optional test command, nothing else. +- Parse the `VERIFY — VERDICT:` line: + - `CONFORME` on first pass → proceed straight to the security gate — no + forced loop. + - `ECARTS(n)` → the dev subagent receives the contract PATH + the exact + gap list (nothing else). Max 3 iterations → STOP + human escalation + with the CRITERIA table (the contract-vs-realized diff). + - Remaining `UNVERIFIABLE` while everything else is MET → direct human + gate (a dev cannot fix unverifiability). + - Structural failure (`ERROR(…)`, missing/duplicated VERDICT line, + unparsable output, agent crash, `CONFORME` without `PROOF`) → retry + ONCE with a fresh verifier; a 2nd structural failure → human + escalation. A mute verifier is NEVER a PASS. +- After a security-gate fix round: re-verify the request FIRST (this + agent), THEN re-verify security — in that order. diff --git a/lib/contract-interview.md b/lib/contract-interview.md new file mode 100644 index 0000000..9dcc1c4 --- /dev/null +++ b/lib/contract-interview.md @@ -0,0 +1,104 @@ +# Contract interview — mandatory upstream passage (all orchestrators) + +Produces the CONTRACT: the single reference passed verbatim to the plan, the +dev subagents, and the verifier. The contract is what lets the orchestrator +delegate execution without subagents ever needing a human gate (LRN-083: +subagents = execution + report only; gates and loop decisions live in the +main loop). + +Run this in the ORCHESTRATOR MAIN LOOP, never in a subagent — STEP 2 may +talk to the human. Mandatory passage in every flow; questions are optional +and proportional — a complete request goes through silently. + +## STEP 1 — CAPTURE (verbatim) + +Copy the user's request EXACTLY as typed (`$ARGUMENTS` + the triggering +message). No paraphrase, no cleanup, no translation, no summarizing. This +section is IMMUTABLE for the life of the run — every later consumer +(planner, dev, verifier) reads THESE words, never a restatement. + +## STEP 2 — AMBIGUITY CHECK (questions optional, proportional) + +Ask ONLY if one of these is missing AND not derivable from the repo: +- a testable expected outcome +- an unambiguous scope (what is allowed to change) +- non-contradictory constraints + +Complete request → ZERO questions, stay silent. Otherwise: max 3 questions, +one single batch (house rule: one question upfront, never mid-task). Never +ask what the repo can answer — verify paths/APIs/behavior yourself first. + +## STEP 3 — DERIVE + +- ACCEPTANCE CRITERIA: numbered; each one testable — a fresh reader must be + able to mark it MET / NOT-MET against the real code, without having seen + this conversation. +- FILE SCOPE: paths/zones expected to change, or `repo-wide — `. + +## STEP 4 — WRITE TO DISK (immediately, before any next step) + +Path: `.claude/tasks/contracts/--.md` +(`mkdir -p` the directory; unique per run: date + short kebab slug + HHMM — +two runs on the same day never collide). A contract that lives only in +context dies at compaction, and the verbatim request with it. + +Template: + +```markdown +# CONTRACT — +- date: | flow: | branch: +- status: active + +## REQUEST (verbatim — IMMUTABLE) + + +## CLARIFICATIONS +Q: / A: +(or: none — request complete) + +## ACCEPTANCE CRITERIA +1. +2. + +## FILE SCOPE + +(or: repo-wide — ) +``` + +Print one line to the user, then continue the flow: +`CONTRACT: — criteria, scope , questions asked` + +## Lifecycle + +- **REQUEST**: immutable, for the life of the run. Never rewritten, never + "cleaned up". +- **CRITERIA / FILE SCOPE enrichment**: ONLY at a human gate, each added + entry marked `[gated ]`. A dev subagent NEVER enriches the + contract. An out-of-scope edit the dev justifies is accepted ONLY through + this micro-gate: human approves → FILE SCOPE gains the entry `[gated]`; + human declines → the dev removes the edit. Without this gate the dev + justifies everything and scope constrains nothing. +- **Deep re-scope** (the request itself changes): NEW contract file with + `supersedes: ` in its header — never a rewrite of the old one. +- **Aborted run**: delete the contract file, or commit it with + `status: aborted` in the header. NEVER left dirty in the working tree. +- **Commit**: the contract rides the existing memory commit — + `lib/capitalize-commit.md` already covers the `.claude/tasks` pathspec. + No new plumbing. + +## Weight per flow + +| Flow | Weight | +|------|--------| +| hotfix | Silent autofill — criteria: "symptom gone; build/tests green"; scope = the 1-2 target files. Zero questions ever. | +| feat / bugfix | Proportional. bugfix: the DIAGNOSIS feeds the criteria (symptom reproduced-then-gone + regression test present). | +| ship-feature | Full. Design decisions approved at the validation gate append criteria `[gated ]` — the human validates the enriched contract, the verifier receives that version. | +| init-project | Full. The interviewer's PROJECT BRIEF pours into the contract (V1 features → criteria). | +| onboard | Audit-scope contract (interview answers → what to audit, which axes). | + +## Hand-off rule + +Downstream consumers (plan step, dev subagents, verifier) receive the +contract PATH, not a restatement of its content — the file on disk is the +only authoritative copy, and reading it from disk is what makes the dev's +reformulation structurally unable to interpose. diff --git a/lib/tests/contract-verifier.test.sh b/lib/tests/contract-verifier.test.sh new file mode 100644 index 0000000..af7b99f --- /dev/null +++ b/lib/tests/contract-verifier.test.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash +# ============================================================ +# Structure locks — contract/verifier pair (verify-loops lot 2) +# Deterministic greps on load-bearing doctrine clauses: an edit +# that silently drops one (blind verifier, PROOF mandatory, +# immutable REQUEST, micro-gate scope enrichment…) reds here. +# ============================================================ +set -u + +REPO="$(cd "$(dirname "$0")/../.." && pwd)" +LIB="$REPO/lib/contract-interview.md" +AGT="$REPO/agents/verifier.md" +PASS=0; FAIL=0 + +# Fixed-string lock (UTF-8 punctuation safe) +tf() { # tf