claude/lib/tests/run-doc-behavioral.md
Bastien Chanot 09f5b28d99 feat(doc-sync): MINOR-shape oracle + fail-loud commit (strengthen MINOR gate)
(1) lib/doc-shape.sh — deterministic shape oracle re-checks each LLM-classified
MINOR patch before the silent auto-commit; a shape-suspect patch (added heading
/ oversize / new file / non-doc) escalates to the EXISTING SIGNIFICANT gate.
Genuine MINOR still auto-commits (zero friction, BDR-036 preserved). A structural
floor under the LLM call, not a semantic SIGNIFICANT-detector (BDR-040).

(2) lib/doc-commit.sh — guard the commit itself: a rejected git commit (pre-commit
hook / protected branch / signing) now fails loud with exit 5 + empty stdout,
instead of a false "committed" + the previous HEAD's hash + exit 0 that left docs
silently uncommitted (LRN-071, 3rd occurrence of the swallowed-commit pattern
after LRN-066 and LRN-068/BLK-012).

Wired doc-syncer STEP A4 (whole-set escalation: no=revert all, select=keep subset)
+ doc-commit.md (rc-5 consumer row + ACKNOWLEDGMENTS coherence).
TDD RED->GREEN: run-doc-commit.sh 32/32, run-doc-shape.sh 19/19, behavioral C/D.
shellcheck clean. Branch-guard (3) deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C6bUdvHnajCNzgVQefZowj
2026-06-29 17:41:24 +02:00

5.1 KiB
Raw Blame History

Behavioral check — doc-sync coupled, end-to-end

The deterministic suite (run-doc-commit.sh, T1T7) proves doc-commit.sh in isolation. This is the in-vivo whole-chain check: a real dev-flow shape — code commit, then doc-syncer patches public docs, then the include commits them — with dangling code AND a forbidden .claude/ path present, proving the doc commit is coupled, surgical, AND fail-closed on an upstream scope violation.

Scenario A — coupled + surgical (the happy path)

R="$(mktemp -d)"; cd "$R"
git init -q && git config user.email t@t.t && git config user.name t
mkdir -p .claude/memory docs src
printf '# Proj\n' > README.md
printf 'baseline\n' > .claude/memory/decisions.md
git add -A && git commit -qm baseline

# 1) the flow commits CODE
printf 'feature code\n' > src/feature.txt
git add -- src/feature.txt && git commit -qm "feat: the feature"

# 2) doc-syncer patches public docs (a modified README + a created docs page).
#    It would surface PATCHED_FILES, ONE PATH PER LINE:
#       README.md
#       docs/usage.md
printf '\n## New feature\nUse --export.\n' >> README.md
printf 'usage guide\n' > docs/usage.md

# 3) a code file is left dangling (must NOT be embarked)
printf 'WIP do not commit\n' > src/dangling.txt

# 4) the include passes EACH PATCHED_FILES line as a SEPARATE arg (argv, space-safe)
doc_hash="$(bash "$HOME/.claude/lib/doc-commit.sh" commit "docs: README + usage" "README.md" "docs/usage.md")"

Expected (assert)

  • Exactly TWO commits after baseline: the code commit, then the doc commit.
  • The doc commit ($doc_hash) contains ONLY README.md + docs/usage.md — never src/feature.txt (already committed) or src/dangling.txt (WIP).
  • src/dangling.txt is still untracked after the doc commit.
  • No .claude/** path in the doc commit (doc-syncer never patches it; the helper guards it regardless).

Scenario B — fail-closed guard (the upstream-anomaly path)

# A bug upstream surfaces a forbidden path in PATCHED_FILES (doc-syncer must never
# patch .claude/ — BDR-022). The include passes it through; the helper must REFUSE.
printf 'x\n' >> .claude/memory/decisions.md          # make the forbidden path dirty
printf '\n## later\n' >> README.md                   # a legit doc also changed
bash "$HOME/.claude/lib/doc-commit.sh" commit "docs: mixed" "README.md" ".claude/memory/decisions.md"
echo "rc=$?"

Expected (assert)

  • rc=4 (scope violation), NOTHING committed — README.md is NOT half-committed.
  • stderr is loud (REFUSED …) and NAMES the offender (.claude/memory/decisions.md).
  • The include treats rc 4 as an upstream BDR-022 anomaly to investigate — not a silent skip. The refusal IS the alarm.

Scenario C — fail-loud on a rejected commit (the masked-failure path)

# The gitflow pre-commit hook (or signing, or a protected branch) rejects the doc
# commit. The helper must NOT report success: no false "committed", no stale hash.
printf '#!/bin/sh\nexit 1\n' > "$R/.git/hooks/pre-commit"; chmod +x "$R/.git/hooks/pre-commit"
printf '\n## another section\n' >> README.md
out="$(bash "$HOME/.claude/lib/doc-commit.sh" commit "docs: rejected" "README.md")"
echo "rc=$? out=[$out]"

Expected (assert)

  • rc=5 (commit rejected), out EMPTY (no stale hash leaked on stdout).
  • stderr is loud (COMMIT REJECTED …) — never a false committed.
  • HEAD did NOT move; the doc stays in the working tree, uncommitted. The orchestrator must surface this and NOT proceed to FINISH as if docs landed (doc-commit.md rc 5 row).

Scenario D — MINOR-shape oracle escalates a SIGNIFICANT-in-disguise

# doc-syncer's LLM classified a drift MINOR, but the patch ADDS A SECTION HEADING —
# structurally not a factual tweak. The oracle must overrule the MINOR call.
printf '\n## Brand new feature\n\nA whole new capability.\n' >> USAGE.md   # the "MINOR" patch
bash "$HOME/.claude/lib/doc-shape.sh" check "USAGE.md"; echo "rc=$?"

Expected (assert)

  • rc=1 (exceeds the MINOR envelope), stderr names the heading reason + USAGE.md.
  • doc-syncer STEP A4 routes this to the SIGNIFICANT gate (Apply? yes/no/select) instead of the silent auto-commit — the deterministic oracle overrules the LLM (LRN-046).
  • A genuine factual one-liner (changed command, no heading, small) returns rc=0 and stays on the silent MINOR auto-commit path — zero friction (BDR-036 preserved).
  • The oracle is a STRUCTURAL floor: a small meaning-changing edit with no heading still reads MINOR (rc 0). It reduces RISK-1's gross cases, it does not eliminate RISK-1.

If Scenario A holds, the chain is coupled (docs committed in the same breath as the flow) and surgical (no dangling code embarked). If Scenario B holds, the guard is fail-closed and loud. If Scenario C holds, a rejected commit fails LOUD instead of masking as success. If Scenario D holds, a shape-suspect MINOR is escalated to the human gate instead of auto-committed. This mirrors what feat / bugfix / hotfix do at their DOC SYNC step (inline-branch commit, no FINISH), and what ship-feature / init-project do at their DOC SYNC step BEFORE FINISH (so the doc commit reaches the merge/PR).