chore(memory): BLK-034, LRN-225, mods bugfix plan and contract

This commit is contained in:
bchanot committed 2026-10-22 22:03:40 +02:00
1 parent 35d923d9d6
commit 2741f603bb
6 files changed
+153 -1

No files matched your search

+1 -1
View File
@@ -8,7 +8,7 @@
- [x] feat A audit phase: d3635b6 (dialog writes, session model opus), a7d747c (audit row, opus floor, census, citers), dae142b (docs); gates 0/1/2 green, LRN-224
- [x] feat B (feature/routing-ceiling-rule): ceiling rule CLAUDE.global.md, README doctrine, challenge citers, deploy `effort: xhigh`, routing.json `changed` cleared; ef13713 + 887cf89, gates 0/1/2 green, BDR-123; merge = user go
- [ ] feat C: cap the default keyword prompt rules (plan|design|architecture, why|review|audit|analyze, ultrathink) at the session model, effort only like the classifier; close BLK-032 after
- [ ] bugfix lib/tests/mods.test.sh:57: CLI 2.1.306 `claude plugin validate` advisory "plugin directory warning" (ICON_MISSING) read as failure; exclude that section, keep other warnings blocking
- [x] bugfix lib/tests/mods.test.sh (bugfix/mods-validate-advisory): verdict = `--strict` rc, positive control warnfix/cleanfix; 801992a, gates green, BLK-034 LRN-225; merge = user go. Residual: 3 historical contracts keep the old scrape oracle (BLK-034)
- [ ] deferred (BDR-123): remap inline-writing best-tier skills (design Build stack, writing-skills, deprecation-and-migration, /tour single-project, /audit-delta 3d, /harden STEP 3); stale citers skills/feat/SKILL.md:6, skills/code-clean/SKILL.md:36
- [ ] residuals (accepted): spawn path unprobed, 1st gate of an outage window dies once (tour/onboard/audit-delta no retry); onAutoSwitch with main on opus marks nothing if engine auto-falls a gate off Fable
@@ -0,0 +1,44 @@
# CONTRACT — mods-validate-advisory (bugfix of lib/tests/mods.test.sh)
## REQUEST (verbatim)
lib/tests/mods.test.sh:57 fails on develop: Claude Code CLI 2.1.306 `claude plugin validate` prints an advisory "plugin directory warning" section (ICON_MISSING) and the suite treats any warning as a failure. Exclude that advisory section only; every other validate warning stays blocking. Suite: make test suite=lib/tests/mods.test.sh. Pre-existing on develop (proven via git archive baseline in the previous session).
## CLARIFICATIONS
- Internal (after the challenge, 3 lenses + 1 re-challenge): the "exclusion" is the CLI's own contract, `claude plugin validate --strict`: per-file warnings → rc 1, the plugin-directory advisory never changes the rc (only under `--profile directory`, never requested). No prose scraping is left in the suite.
- Internal: the regression proof is a positive control run through the gate's own helper on two scratch manifests (warnfix rc 1, cleanfix rc 0), written by the suite under a script-level `mktemp -d`; no tracked fixture.
- Internal: three historical contracts carry the old scrape as oracle (2026-10-08-model-router-w1a-1533.md:22, 2026-10-08-model-router-floor-1835.md:22, 2026-10-09-model-router-tiers-1237.md:23); left unedited, named at CAPITALIZE.
- Plan: `.claude/tasks/plans/2026-10-22-mods-validate-advisory-2004.md` (r3).
- GATE 1 ECARTS(2) → disposition (main loop): both gaps = criterion 6 on `mods/model-router/routing.json` (`confirmed.phases.audit` written by the mod on the main loop's first `route(phase="audit")`, register.ts:2054, after GATE 0 ran). Not an executor edit, not staged with this fix; commit of that dialog write = user call. Criterion 6 holds on the committed diff.
## ACCEPTANCE CRITERIA
1. The suite is green on CLI 2.1.306 with the ICON_MISSING advisory present in validate's output.
CHECK: make test suite=lib/tests/mods.test.sh 2>&1 | grep -E '^mods: ' | sed 's/^/n=/'
EXPECT: n=mods: 6 pass, 0 fail
EVIDENCE: MET exit=0 marker-found :: n=mods: 6 pass, 0 fail
2. The validate verdict is the exit code of `--strict`, through one helper used by check_cli and the control; no `warning` scrape and no `Validation passed` scrape remain.
CHECK: f=lib/tests/mods.test.sh; a=$(grep -c 'plugin validate --strict "' $f); b=$(grep -c 'validate_strict "' $f); c=$(grep -ciE "grep -qi? *'?warning|Validation passed" $f); echo "$a/$b/$c"
EXPECT: 1/3/0
EVIDENCE: MET exit=0 marker-found :: 1/3/0
3. Positive control: the clean scratch manifest passes `--strict` and the warning one fails it, both through the helper, both reported as suite checks.
CHECK: make test suite=lib/tests/mods.test.sh 2>&1 | grep -cE '^PASS strict control: (clean fixture passes --strict|per-file warnings fail --strict)$'
EXPECT: 2
EVIDENCE: MET exit=0 marker-found :: 2
4. The control is not vacuous: with `--strict` removed from the helper the control goes red (the warning fixture passes).
CHECK: S=/tmp/claude-1000/-home-bchanot-Documents-claude/09872630-52ba-43d2-b0a9-f9fe56869e15/scratchpad/chk; mkdir -p "$S"; sed 's/validate --strict "\$1"/validate "$1"/' lib/tests/mods.test.sh > "$S/m.sh"; MODS_ROOT=$PWD bash "$S/m.sh" 2>&1 | grep -c '^FAIL strict control: per-file warnings fail --strict'
EXPECT: 1
EVIDENCE: MET exit=0 marker-found :: 1
5. The scratch dir is script-level with an EXIT trap (never inside a function), and the ko tail names the rerun command.
CHECK: f=lib/tests/mods.test.sh; a=$(grep -c "^WORK=\"\$(mktemp -d)\"; trap 'rm -rf \"\$WORK\"' EXIT$" $f); b=$(grep -c 'rerun: claude plugin validate --strict' $f); echo "$a/$b"
EXPECT: 1/1
EVIDENCE: MET exit=0 marker-found :: 1/1
6. Only lib/tests/mods.test.sh changes; shellcheck clean on it.
CHECK: a=$(git status --short | grep -v '^?? .claude/' | grep -vc '^ M lib/tests/mods.test.sh$'); b=$(shellcheck lib/tests/mods.test.sh >/dev/null 2>&1 && echo ok || echo ko); echo "$a/$b"
EXPECT: 0/ok
EVIDENCE: MET exit=0 marker-found :: 0/ok
7. The suite still fails loudly when no mod exists (never vacuous): MODS_ROOT on an empty dir.
CHECK: d=$(mktemp -d); MODS_ROOT=$d bash lib/tests/mods.test.sh 2>&1 | grep -c '^FAIL no mod found'
EXPECT: 1
EVIDENCE: MET exit=0 marker-found :: 1
## FILE SCOPE
lib/tests/mods.test.sh
@@ -0,0 +1,91 @@
# BUGFIX PLAN r3 — mods.test.sh reads the advisory plugin-directory section as a warning
## DIAGNOSIS
BUG: `make test suite=lib/tests/mods.test.sh` fails on develop with
`FAIL model-router: validate printed a warning` (pre-existing, proven on
develop's copy via git archive in the previous session).
ROOT CAUSE: Claude Code CLI 2.1.306 `claude plugin validate` prints a
"Plugin directory checks for <dir>" section (the submission catalogue's
checks) that says "Found 1 plugin directory warning:" (ICON_MISSING).
lib/tests/mods.test.sh:57 greps the WHOLE output case-insensitively for
`warning`, so an advisory the CLI itself reports as `blocking: false`
(`--json`, `directory.findings[].blocking`) fails the suite. The same grep
also matches the hooks notes that quote register.ts identifiers (`warnOnce`
today is safe, a future `logWarning` is not) and any checkout path holding
"warning": prose scraping, fail-open and fail-closed at once.
EVIDENCE: `claude plugin validate mods/model-router` rc 0; `--strict` rc 0
(per-file warnings are the only ones `--strict` turns into rc 1:
`manifest.warnings` and `contents[].warnings` are empty); `--strict
--profile directory` rc 1 (the advisory counts only under that profile).
A scratch plugin whose manifest is `{"name":"warnfix"}` gives `⚠ Found N
warnings:` (version, description, author) and `--strict` rc 1, plain rc 0.
BLAST RADIUS: the fix touches lib/tests/mods.test.sh only. doctor.sh:211
names the command in a hint. Three historical contracts carry the same
scrape as a CHECK oracle (`grep -q 'Validation passed' && ! … grep -qi
'warning'`): .claude/tasks/contracts/2026-10-08-model-router-w1a-1533.md:22,
2026-10-08-model-router-floor-1835.md:22, 2026-10-09-model-router-tiers-1237.md:23.
Re-run on CLI 2.1.306 they fail on ICON_MISSING: known-stale oracles, left
unedited (historical), named at CAPITALIZE so a re-run is not read as a
model-router regression. `advice[]` (third top-level `--json` field) is
empty on every fixture tried; whether `--strict` escalates it is unverified.
PRIOR: none bears. RELATED MEMORY: BLK-030..033 are model-router runtime /
guardrails, not the CLI validate output; no BDR constrains the suite shape.
## CHALLENGE r1 → r2 → r3
Three lenses (correctness, robustness, simplicity), 0 BLOCKER, 5 MAJOR, all
closed by this revision: the sed range on a closing marker the CLI prints
only when findings > 0 (fail-open to EOF, hides `✔ Validation passed with
warnings`) is dropped for the CLI's own contract, the `--strict` exit code;
the tautological SAMPLE_UNTERMINATED and the invented warning samples are
dropped; the blocking branch is now exercised by a real CLI run on a
fixture plugin instead of a copy of the grep. r3 (one re-challenge, 1 MAJOR):
the control now runs through the gate's own helper, a clean sibling fixture
pins the cause (only per-file warnings differ), the ko tail names the rerun
command, WORK is script-level.
## FIX PLAN (one file: lib/tests/mods.test.sh)
1. New helper, next to `bounded`:
`validate_strict() { STRICT_OUT=$(bounded claude plugin validate --strict "$1"); }`
returns the rc of `bounded` (last command), output in the script-level
`STRICT_OUT` (same pattern as LAST_OUT in lib/tests/a11y-scan.test.sh:13).
2. check_cli, validate half (lines 53-59): `validate_strict "$dir" || rc=$?`.
rc 124 → `ko "$name: validate timed out after ${CLI_TIMEOUT}s"` (unchanged);
rc ≠ 0 → `ko "$name: validate --strict rc=$rc: $(printf '%s' "$STRICT_OUT" | grep -E '^(✘|error:|⚠ Found)' | head -2 | tr '\n' ' ') (rerun: claude plugin validate --strict $dir)"`;
rc 0 → `ok "$name: validate --strict passed"`.
The `Validation passed` substring check and the `grep -qi warning` are
removed: the exit code is the verdict. The plugin-test half (lines 60-65)
is unchanged.
3. Positive control. Script-level, right after `pass=0; fail=0`:
`WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT` (top-level, as in
lib/tests/a11y-scan.test.sh:12, never inside a function). New function
`strict_discriminates`, called once after the `use_cli` block, only when
`use_cli` = 1. It writes two scratch plugins:
- `$WORK/warnfix/.claude-plugin/plugin.json` = `{"name":"warnfix"}`
(per-file warnings: version, description, author);
- `$WORK/cleanfix/.claude-plugin/plugin.json` =
`{"name":"cleanfix","version":"0.1.0","description":"d","author":{"name":"x"}}`
(no per-file warning; the directory findings README_MISSING,
LICENSE_MISSING, ICON_MISSING are printed for both).
Then, through the helper of step 1:
- `validate_strict "$WORK/cleanfix"` → `check "strict control: clean fixture passes --strict" "$rc" 0`;
- `validate_strict "$WORK/warnfix"` → `check "strict control: per-file warnings fail --strict" "$rc" 1`;
rc 124 on either → `ko "strict control: validate timed out"`.
The pair differs only in the per-file warnings, so a parse error (CLI
without `--strict`: rc 1 on both) or a scrape added back turns the
control red; the control covers the gate's own code path.
4. Header comment (lines 4-5): "passes `claude plugin validate --strict`
(per-file warnings are errors; the advisory plugin-directory section
does not change the exit code) and `claude plugin test`".
5. No new tracked file; the two manifests are written by the suite.
## EDGE CASES
- CLI without `--strict` (older than its introduction): commander rejects
the unknown option with rc 1 on every call → check_cli red with the
`error: unknown option` line in its tail, and the clean-fixture control
red too. Accepted: `cli_unavailable` already requires `claude plugin
test`, which post-dates `--strict`.
- Icon added later to mods/model-router: no effect, the directory profile
is never requested.
- The fixture's directory findings never affect rc (no `--profile`).
## RISK: low. Test-only file, no runtime behaviour.