feat(doc-sync): MINOR-shape oracle + fail-loud commit (strengthen MINOR gate)
(1) lib/doc-shape.sh — deterministic shape oracle re-checks each LLM-classified MINOR patch before the silent auto-commit; a shape-suspect patch (added heading / oversize / new file / non-doc) escalates to the EXISTING SIGNIFICANT gate. Genuine MINOR still auto-commits (zero friction, BDR-036 preserved). A structural floor under the LLM call, not a semantic SIGNIFICANT-detector (BDR-040). (2) lib/doc-commit.sh — guard the commit itself: a rejected git commit (pre-commit hook / protected branch / signing) now fails loud with exit 5 + empty stdout, instead of a false "committed" + the previous HEAD's hash + exit 0 that left docs silently uncommitted (LRN-071, 3rd occurrence of the swallowed-commit pattern after LRN-066 and LRN-068/BLK-012). Wired doc-syncer STEP A4 (whole-set escalation: no=revert all, select=keep subset) + doc-commit.md (rc-5 consumer row + ACKNOWLEDGMENTS coherence). TDD RED->GREEN: run-doc-commit.sh 32/32, run-doc-shape.sh 19/19, behavioral C/D. shellcheck clean. Branch-guard (3) deferred. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C6bUdvHnajCNzgVQefZowj
This commit is contained in:
Bastien Chanot
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
4da2b905de
commit
09f5b28d99
@@ -59,8 +59,45 @@ echo "rc=$?"
|
||||
- The include treats rc 4 as an upstream BDR-022 anomaly to investigate — not a
|
||||
silent skip. The refusal IS the alarm.
|
||||
|
||||
## Scenario C — fail-loud on a rejected commit (the masked-failure path)
|
||||
|
||||
```bash
|
||||
# The gitflow pre-commit hook (or signing, or a protected branch) rejects the doc
|
||||
# commit. The helper must NOT report success: no false "committed", no stale hash.
|
||||
printf '#!/bin/sh\nexit 1\n' > "$R/.git/hooks/pre-commit"; chmod +x "$R/.git/hooks/pre-commit"
|
||||
printf '\n## another section\n' >> README.md
|
||||
out="$(bash "$HOME/.claude/lib/doc-commit.sh" commit "docs: rejected" "README.md")"
|
||||
echo "rc=$? out=[$out]"
|
||||
```
|
||||
|
||||
### Expected (assert)
|
||||
- `rc=5` (commit rejected), `out` EMPTY (no stale hash leaked on stdout).
|
||||
- stderr is loud (`COMMIT REJECTED …`) — never a false `committed`.
|
||||
- HEAD did NOT move; the doc stays in the working tree, uncommitted. The orchestrator
|
||||
must surface this and NOT proceed to FINISH as if docs landed (doc-commit.md rc 5 row).
|
||||
|
||||
## Scenario D — MINOR-shape oracle escalates a SIGNIFICANT-in-disguise
|
||||
|
||||
```bash
|
||||
# doc-syncer's LLM classified a drift MINOR, but the patch ADDS A SECTION HEADING —
|
||||
# structurally not a factual tweak. The oracle must overrule the MINOR call.
|
||||
printf '\n## Brand new feature\n\nA whole new capability.\n' >> USAGE.md # the "MINOR" patch
|
||||
bash "$HOME/.claude/lib/doc-shape.sh" check "USAGE.md"; echo "rc=$?"
|
||||
```
|
||||
|
||||
### Expected (assert)
|
||||
- `rc=1` (exceeds the MINOR envelope), stderr names the heading reason + `USAGE.md`.
|
||||
- doc-syncer STEP A4 routes this to the SIGNIFICANT gate (`Apply? yes/no/select`) instead
|
||||
of the silent auto-commit — the deterministic oracle overrules the LLM (LRN-046).
|
||||
- A genuine factual one-liner (changed command, no heading, small) returns `rc=0` and
|
||||
stays on the silent MINOR auto-commit path — zero friction (BDR-036 preserved).
|
||||
- The oracle is a STRUCTURAL floor: a small meaning-changing edit with no heading still
|
||||
reads MINOR (rc 0). It reduces RISK-1's gross cases, it does not eliminate RISK-1.
|
||||
|
||||
If Scenario A holds, the chain is coupled (docs committed in the same breath as the
|
||||
flow) and surgical (no dangling code embarked). If Scenario B holds, the guard is
|
||||
fail-closed and loud. This mirrors what feat / bugfix / hotfix do at their DOC SYNC
|
||||
step (inline-branch commit, no FINISH), and what ship-feature / init-project do at
|
||||
their DOC SYNC step BEFORE FINISH (so the doc commit reaches the merge/PR).
|
||||
fail-closed and loud. If Scenario C holds, a rejected commit fails LOUD instead of
|
||||
masking as success. If Scenario D holds, a shape-suspect MINOR is escalated to the
|
||||
human gate instead of auto-committed. This mirrors what feat / bugfix / hotfix do at
|
||||
their DOC SYNC step (inline-branch commit, no FINISH), and what ship-feature /
|
||||
init-project do at their DOC SYNC step BEFORE FINISH (so the doc commit reaches the merge/PR).
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
# T5 idempotent — empty list / clean tree → no-op exit 0
|
||||
# T6 unsafe git state (detached HEAD) → exit 3, no commit
|
||||
# T7 path WITH A SPACE passed as one arg → committed (argv is space-safe, no separator)
|
||||
# T8 pre-commit hook REJECTS the commit → fail LOUD (exit 5), no stale hash on stdout,
|
||||
# HEAD unmoved — the script must NOT report "committed" when git commit failed
|
||||
#
|
||||
# No -e: run every test and report, even after a failure.
|
||||
set -uo pipefail
|
||||
@@ -173,6 +175,20 @@ if git -C "$R" cat-file -e "HEAD:docs/My Guide.md" 2>/dev/null; then ok "spaced
|
||||
if [ -z "$(git -C "$R" status --porcelain -- "docs/My Guide.md")" ]; then ok "spaced doc clean (embarked as ONE file, not split)"; else ko "spaced doc still dirty"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "T8 — pre-commit hook REJECTS commit → exit 5 LOUD, no stale hash, HEAD unmoved"
|
||||
R="$(new_repo)"
|
||||
printf '#!/bin/sh\nexit 1\n' >"$R/.git/hooks/pre-commit"; chmod +x "$R/.git/hooks/pre-commit"
|
||||
BEFORE="$(git -C "$R" rev-parse HEAD)"
|
||||
printf 'feature added\n' >>"$R/README.md"
|
||||
run "$R" commit "docs: T8 rejected" "README.md"
|
||||
printf ' rc=%s out=[%s]\n' "$RC" "$OUT"
|
||||
printf ' err: %s\n' "$(printf '%s' "$ERR" | head -1)"
|
||||
if [ "$RC" -eq 5 ]; then ok "rejected commit → exit 5"; else ko "expected 5, got $RC (commit failure swallowed = masking)"; fi
|
||||
if [ -z "$OUT" ]; then ok "stdout empty (no stale hash)"; else ko "stale hash leaked on failure: [$OUT]"; fi
|
||||
if printf '%s' "$ERR" | grep -qi 'REJECTED'; then ok "stderr is loud (REJECTED)"; else ko "stderr not loud (no REJECTED — likely a false 'committed')"; fi
|
||||
if [ "$(git -C "$R" rev-parse HEAD)" = "$BEFORE" ]; then ok "HEAD unmoved (nothing committed)"; else ko "HEAD moved despite hook reject"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
rm -f "$ERRFILE"
|
||||
echo ""
|
||||
printf 'RESULT: %d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deterministic tests for lib/doc-shape.sh — the MINOR-shape oracle.
|
||||
#
|
||||
# The oracle re-checks that a patch the LLM classified MINOR actually HAS minor
|
||||
# shape, on REAL git diffs (not assumed). Each case proves a verdict:
|
||||
# S1 factual one-liner (1 add / 1 del, no heading) → 0 within envelope
|
||||
# S2 adds a `## Section` heading → 1 exceeds (structural)
|
||||
# S3 +30 plain lines, no heading → 1 exceeds (size)
|
||||
# S3b +20 plain lines (== threshold) → 0 within (boundary)
|
||||
# S3c +10 lines with DOC_SHAPE_MAX_ADDED=5 (env override) → 1 exceeds (tunable)
|
||||
# S4 dead-reference removal (-2 / +0) → 0 within (small)
|
||||
# S5 new / untracked doc file → 1 exceeds (a creation)
|
||||
# S6 a code path (not a doc) → 1 exceeds (anomaly)
|
||||
# S7 clean tracked doc (no diff) → 0 within (vacuous)
|
||||
# S8 MIXED multi-path, ONE file exceeds → 1 exceeds, offender named
|
||||
# S9 usage (check with no paths) → 2
|
||||
# S10 not a git repo → 3
|
||||
#
|
||||
# No -e: run every test and report, even after a failure.
|
||||
set -uo pipefail
|
||||
|
||||
HERE="$(cd -P "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
HELPER="$HERE/../doc-shape.sh"
|
||||
ERRFILE="$(mktemp)"
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
ok() { printf ' \033[32m✓\033[0m %s\n' "$1"; PASS=$((PASS + 1)); }
|
||||
ko() { printf ' \033[31m✗\033[0m %s\n' "$1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
# Fresh throwaway repo: a few tracked docs + one code file, committed.
|
||||
new_repo() {
|
||||
local d
|
||||
d="$(mktemp -d)"
|
||||
git -C "$d" init -q
|
||||
git -C "$d" config user.email t@t.t
|
||||
git -C "$d" config user.name tester
|
||||
mkdir -p "$d/docs" "$d/src"
|
||||
printf 'run: foo\nold line A\nold line B\n' >"$d/README.md"
|
||||
printf 'usage baseline\n' >"$d/USAGE.md"
|
||||
printf 'guide baseline\n' >"$d/docs/guide.md"
|
||||
printf 'print("hi")\n' >"$d/src/app.py"
|
||||
git -C "$d" add -A
|
||||
git -C "$d" commit -qm baseline
|
||||
printf '%s' "$d"
|
||||
}
|
||||
|
||||
# Append N numbered plain lines (no heading) to a file.
|
||||
append_lines() {
|
||||
local f="$1" n="$2" i
|
||||
for ((i = 1; i <= n; i++)); do printf 'extra line %s\n' "$i" >>"$f"; done
|
||||
}
|
||||
|
||||
# run [ENV=val] <repo> <args...> → sets RC (exit), OUT (stdout), ERR (stderr).
|
||||
# stdout MUST stay empty: the exit code carries the verdict, reasons go to stderr.
|
||||
run() {
|
||||
local r="$1"; shift
|
||||
OUT="$( (cd "$r" && "$HELPER" "$@") 2>"$ERRFILE" )"; RC=$?
|
||||
ERR="$(cat "$ERRFILE")"
|
||||
}
|
||||
|
||||
echo "S1 — factual one-liner (1 add / 1 del, no heading) → within (0)"
|
||||
R="$(new_repo)"
|
||||
printf 'run: bar\nold line A\nold line B\n' >"$R/README.md" # change one line
|
||||
run "$R" check "README.md"
|
||||
printf ' rc=%s out=[%s]\n' "$RC" "$OUT"
|
||||
if [ "$RC" -eq 0 ]; then ok "factual tweak → within (0)"; else ko "expected 0, got $RC"; fi
|
||||
if [ -z "$OUT" ]; then ok "stdout empty"; else ko "stdout leaked: [$OUT]"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S2 — adds a heading → exceeds (1, structural)"
|
||||
R="$(new_repo)"
|
||||
printf '\n## New Feature\n\nDescribes the new feature.\n' >>"$R/README.md"
|
||||
run "$R" check "README.md"
|
||||
printf ' rc=%s err=%s\n' "$RC" "$(printf '%s' "$ERR" | head -1)"
|
||||
if [ "$RC" -eq 1 ]; then ok "heading → exceeds (1)"; else ko "expected 1, got $RC"; fi
|
||||
if printf '%s' "$ERR" | grep -qi 'heading'; then ok "stderr names the heading reason"; else ko "reason not named"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S3 — +30 plain lines, no heading → exceeds (1, size)"
|
||||
R="$(new_repo)"
|
||||
append_lines "$R/README.md" 30
|
||||
run "$R" check "README.md"
|
||||
printf ' rc=%s err=%s\n' "$RC" "$(printf '%s' "$ERR" | head -1)"
|
||||
if [ "$RC" -eq 1 ]; then ok "30 added → exceeds (1)"; else ko "expected 1, got $RC"; fi
|
||||
if printf '%s' "$ERR" | grep -qi 'added'; then ok "stderr names the size reason"; else ko "reason not named"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S3b — +20 plain lines (== threshold) → within (0, boundary)"
|
||||
R="$(new_repo)"
|
||||
append_lines "$R/README.md" 20
|
||||
run "$R" check "README.md"
|
||||
printf ' rc=%s\n' "$RC"
|
||||
if [ "$RC" -eq 0 ]; then ok "20 added (== MAX) → within (0)"; else ko "expected 0, got $RC"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S3c — +10 lines with DOC_SHAPE_MAX_ADDED=5 → exceeds (1, env-tunable)"
|
||||
R="$(new_repo)"
|
||||
append_lines "$R/README.md" 10
|
||||
OUT="$( (cd "$R" && DOC_SHAPE_MAX_ADDED=5 "$HELPER" check "README.md") 2>"$ERRFILE" )"; RC=$?
|
||||
printf ' rc=%s\n' "$RC"
|
||||
if [ "$RC" -eq 1 ]; then ok "override MAX_ADDED=5, 10 added → exceeds (1)"; else ko "expected 1, got $RC"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S4 — dead-reference removal (-2 / +0) → within (0)"
|
||||
R="$(new_repo)"
|
||||
printf 'run: foo\n' >"$R/README.md" # drop the two 'old line' references
|
||||
run "$R" check "README.md"
|
||||
printf ' rc=%s\n' "$RC"
|
||||
if [ "$RC" -eq 0 ]; then ok "small removal → within (0)"; else ko "expected 0, got $RC"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S5 — new / untracked doc file → exceeds (1, a creation)"
|
||||
R="$(new_repo)"
|
||||
printf 'brand new doc\n' >"$R/NEW.md" # untracked
|
||||
run "$R" check "NEW.md"
|
||||
printf ' rc=%s err=%s\n' "$RC" "$(printf '%s' "$ERR" | head -1)"
|
||||
if [ "$RC" -eq 1 ]; then ok "untracked doc → exceeds (1)"; else ko "expected 1, got $RC"; fi
|
||||
if printf '%s' "$ERR" | grep -Eqi 'untracked|new'; then ok "stderr flags the creation"; else ko "reason not named"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S6 — a code path (not a doc) → exceeds (1, anomaly)"
|
||||
R="$(new_repo)"
|
||||
printf 'print("hi")\nprint("bye")\n' >"$R/src/app.py"
|
||||
run "$R" check "src/app.py"
|
||||
printf ' rc=%s err=%s\n' "$RC" "$(printf '%s' "$ERR" | head -1)"
|
||||
if [ "$RC" -eq 1 ]; then ok "non-doc path → exceeds (1)"; else ko "expected 1, got $RC"; fi
|
||||
if printf '%s' "$ERR" | grep -qi 'doc'; then ok "stderr flags the non-doc"; else ko "reason not named"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S7 — clean tracked doc (no diff) → within (0, vacuous)"
|
||||
R="$(new_repo)"
|
||||
run "$R" check "docs/guide.md" # unmodified
|
||||
printf ' rc=%s\n' "$RC"
|
||||
if [ "$RC" -eq 0 ]; then ok "clean path → within (0)"; else ko "expected 0, got $RC"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S8 — MIXED multi-path, ONE exceeds → exceeds (1), offender named"
|
||||
R="$(new_repo)"
|
||||
printf 'extra\n' >>"$R/README.md" # small, within
|
||||
append_lines "$R/USAGE.md" 30 # big, exceeds
|
||||
run "$R" check "README.md" "USAGE.md"
|
||||
printf ' rc=%s err=%s\n' "$RC" "$(printf '%s' "$ERR" | head -1)"
|
||||
if [ "$RC" -eq 1 ]; then ok "any path exceeds → whole set exceeds (1)"; else ko "expected 1, got $RC"; fi
|
||||
if printf '%s' "$ERR" | grep -q 'USAGE.md'; then ok "stderr names the offender (USAGE.md)"; else ko "offender not named"; fi
|
||||
if printf '%s' "$ERR" | grep -q 'README.md'; then ko "README.md wrongly flagged"; else ok "within-envelope file NOT flagged"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S9 — usage (check with no paths) → 2"
|
||||
R="$(new_repo)"
|
||||
run "$R" check
|
||||
printf ' rc=%s\n' "$RC"
|
||||
if [ "$RC" -eq 2 ]; then ok "no paths → usage (2)"; else ko "expected 2, got $RC"; fi
|
||||
rm -rf "$R"
|
||||
|
||||
echo "S10 — not a git repo → 3"
|
||||
D="$(mktemp -d)" # plain dir, no git init
|
||||
run "$D" check "README.md"
|
||||
printf ' rc=%s\n' "$RC"
|
||||
if [ "$RC" -eq 3 ]; then ok "not-a-repo → 3"; else ko "expected 3, got $RC"; fi
|
||||
rm -rf "$D"
|
||||
|
||||
rm -f "$ERRFILE"
|
||||
echo ""
|
||||
printf 'RESULT: %d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
[ "$FAIL" -eq 0 ]
|
||||
Reference in New Issue
Block a user