forked from bchanot/claude
chore(memory): BLK-034, LRN-225, mods bugfix plan and contract
This commit is contained in:
1 parent
35d923d9d6
commit
2741f603bb
6 files changed
+153
-1
No files matched your search
@@ -8,7 +8,7 @@
|
||||
- [x] feat A audit phase: d3635b6 (dialog writes, session model opus), a7d747c (audit row, opus floor, census, citers), dae142b (docs); gates 0/1/2 green, LRN-224
|
||||
- [x] feat B (feature/routing-ceiling-rule): ceiling rule CLAUDE.global.md, README doctrine, challenge citers, deploy `effort: xhigh`, routing.json `changed` cleared; ef13713 + 887cf89, gates 0/1/2 green, BDR-123; merge = user go
|
||||
- [ ] feat C: cap the default keyword prompt rules (plan|design|architecture, why|review|audit|analyze, ultrathink) at the session model, effort only like the classifier; close BLK-032 after
|
||||
- [ ] bugfix lib/tests/mods.test.sh:57: CLI 2.1.306 `claude plugin validate` advisory "plugin directory warning" (ICON_MISSING) read as failure; exclude that section, keep other warnings blocking
|
||||
- [x] bugfix lib/tests/mods.test.sh (bugfix/mods-validate-advisory): verdict = `--strict` rc, positive control warnfix/cleanfix; 801992a, gates green, BLK-034 LRN-225; merge = user go. Residual: 3 historical contracts keep the old scrape oracle (BLK-034)
|
||||
- [ ] deferred (BDR-123): remap inline-writing best-tier skills (design Build stack, writing-skills, deprecation-and-migration, /tour single-project, /audit-delta 3d, /harden STEP 3); stale citers skills/feat/SKILL.md:6, skills/code-clean/SKILL.md:36
|
||||
- [ ] residuals (accepted): spawn path unprobed, 1st gate of an outage window dies once (tour/onboard/audit-delta no retry); onAutoSwitch with main on opus marks nothing if engine auto-falls a gate off Fable
|
||||
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# CONTRACT — mods-validate-advisory (bugfix of lib/tests/mods.test.sh)
|
||||
|
||||
## REQUEST (verbatim)
|
||||
lib/tests/mods.test.sh:57 fails on develop: Claude Code CLI 2.1.306 `claude plugin validate` prints an advisory "plugin directory warning" section (ICON_MISSING) and the suite treats any warning as a failure. Exclude that advisory section only; every other validate warning stays blocking. Suite: make test suite=lib/tests/mods.test.sh. Pre-existing on develop (proven via git archive baseline in the previous session).
|
||||
|
||||
## CLARIFICATIONS
|
||||
- Internal (after the challenge, 3 lenses + 1 re-challenge): the "exclusion" is the CLI's own contract, `claude plugin validate --strict`: per-file warnings → rc 1, the plugin-directory advisory never changes the rc (only under `--profile directory`, never requested). No prose scraping is left in the suite.
|
||||
- Internal: the regression proof is a positive control run through the gate's own helper on two scratch manifests (warnfix rc 1, cleanfix rc 0), written by the suite under a script-level `mktemp -d`; no tracked fixture.
|
||||
- Internal: three historical contracts carry the old scrape as oracle (2026-10-08-model-router-w1a-1533.md:22, 2026-10-08-model-router-floor-1835.md:22, 2026-10-09-model-router-tiers-1237.md:23); left unedited, named at CAPITALIZE.
|
||||
- Plan: `.claude/tasks/plans/2026-10-22-mods-validate-advisory-2004.md` (r3).
|
||||
- GATE 1 ECARTS(2) → disposition (main loop): both gaps = criterion 6 on `mods/model-router/routing.json` (`confirmed.phases.audit` written by the mod on the main loop's first `route(phase="audit")`, register.ts:2054, after GATE 0 ran). Not an executor edit, not staged with this fix; commit of that dialog write = user call. Criterion 6 holds on the committed diff.
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. The suite is green on CLI 2.1.306 with the ICON_MISSING advisory present in validate's output.
|
||||
CHECK: make test suite=lib/tests/mods.test.sh 2>&1 | grep -E '^mods: ' | sed 's/^/n=/'
|
||||
EXPECT: n=mods: 6 pass, 0 fail
|
||||
EVIDENCE: MET exit=0 marker-found :: n=mods: 6 pass, 0 fail
|
||||
2. The validate verdict is the exit code of `--strict`, through one helper used by check_cli and the control; no `warning` scrape and no `Validation passed` scrape remain.
|
||||
CHECK: f=lib/tests/mods.test.sh; a=$(grep -c 'plugin validate --strict "' $f); b=$(grep -c 'validate_strict "' $f); c=$(grep -ciE "grep -qi? *'?warning|Validation passed" $f); echo "$a/$b/$c"
|
||||
EXPECT: 1/3/0
|
||||
EVIDENCE: MET exit=0 marker-found :: 1/3/0
|
||||
3. Positive control: the clean scratch manifest passes `--strict` and the warning one fails it, both through the helper, both reported as suite checks.
|
||||
CHECK: make test suite=lib/tests/mods.test.sh 2>&1 | grep -cE '^PASS strict control: (clean fixture passes --strict|per-file warnings fail --strict)$'
|
||||
EXPECT: 2
|
||||
EVIDENCE: MET exit=0 marker-found :: 2
|
||||
4. The control is not vacuous: with `--strict` removed from the helper the control goes red (the warning fixture passes).
|
||||
CHECK: S=/tmp/claude-1000/-home-bchanot-Documents-claude/09872630-52ba-43d2-b0a9-f9fe56869e15/scratchpad/chk; mkdir -p "$S"; sed 's/validate --strict "\$1"/validate "$1"/' lib/tests/mods.test.sh > "$S/m.sh"; MODS_ROOT=$PWD bash "$S/m.sh" 2>&1 | grep -c '^FAIL strict control: per-file warnings fail --strict'
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
5. The scratch dir is script-level with an EXIT trap (never inside a function), and the ko tail names the rerun command.
|
||||
CHECK: f=lib/tests/mods.test.sh; a=$(grep -c "^WORK=\"\$(mktemp -d)\"; trap 'rm -rf \"\$WORK\"' EXIT$" $f); b=$(grep -c 'rerun: claude plugin validate --strict' $f); echo "$a/$b"
|
||||
EXPECT: 1/1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1/1
|
||||
6. Only lib/tests/mods.test.sh changes; shellcheck clean on it.
|
||||
CHECK: a=$(git status --short | grep -v '^?? .claude/' | grep -vc '^ M lib/tests/mods.test.sh$'); b=$(shellcheck lib/tests/mods.test.sh >/dev/null 2>&1 && echo ok || echo ko); echo "$a/$b"
|
||||
EXPECT: 0/ok
|
||||
EVIDENCE: MET exit=0 marker-found :: 0/ok
|
||||
7. The suite still fails loudly when no mod exists (never vacuous): MODS_ROOT on an empty dir.
|
||||
CHECK: d=$(mktemp -d); MODS_ROOT=$d bash lib/tests/mods.test.sh 2>&1 | grep -c '^FAIL no mod found'
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
|
||||
## FILE SCOPE
|
||||
lib/tests/mods.test.sh
|
||||
@@ -0,0 +1,91 @@
|
||||
# BUGFIX PLAN r3 — mods.test.sh reads the advisory plugin-directory section as a warning
|
||||
|
||||
## DIAGNOSIS
|
||||
BUG: `make test suite=lib/tests/mods.test.sh` fails on develop with
|
||||
`FAIL model-router: validate printed a warning` (pre-existing, proven on
|
||||
develop's copy via git archive in the previous session).
|
||||
ROOT CAUSE: Claude Code CLI 2.1.306 `claude plugin validate` prints a
|
||||
"Plugin directory checks for <dir>" section (the submission catalogue's
|
||||
checks) that says "Found 1 plugin directory warning:" (ICON_MISSING).
|
||||
lib/tests/mods.test.sh:57 greps the WHOLE output case-insensitively for
|
||||
`warning`, so an advisory the CLI itself reports as `blocking: false`
|
||||
(`--json`, `directory.findings[].blocking`) fails the suite. The same grep
|
||||
also matches the hooks notes that quote register.ts identifiers (`warnOnce`
|
||||
today is safe, a future `logWarning` is not) and any checkout path holding
|
||||
"warning": prose scraping, fail-open and fail-closed at once.
|
||||
EVIDENCE: `claude plugin validate mods/model-router` rc 0; `--strict` rc 0
|
||||
(per-file warnings are the only ones `--strict` turns into rc 1:
|
||||
`manifest.warnings` and `contents[].warnings` are empty); `--strict
|
||||
--profile directory` rc 1 (the advisory counts only under that profile).
|
||||
A scratch plugin whose manifest is `{"name":"warnfix"}` gives `⚠ Found N
|
||||
warnings:` (version, description, author) and `--strict` rc 1, plain rc 0.
|
||||
BLAST RADIUS: the fix touches lib/tests/mods.test.sh only. doctor.sh:211
|
||||
names the command in a hint. Three historical contracts carry the same
|
||||
scrape as a CHECK oracle (`grep -q 'Validation passed' && ! … grep -qi
|
||||
'warning'`): .claude/tasks/contracts/2026-10-08-model-router-w1a-1533.md:22,
|
||||
2026-10-08-model-router-floor-1835.md:22, 2026-10-09-model-router-tiers-1237.md:23.
|
||||
Re-run on CLI 2.1.306 they fail on ICON_MISSING: known-stale oracles, left
|
||||
unedited (historical), named at CAPITALIZE so a re-run is not read as a
|
||||
model-router regression. `advice[]` (third top-level `--json` field) is
|
||||
empty on every fixture tried; whether `--strict` escalates it is unverified.
|
||||
PRIOR: none bears. RELATED MEMORY: BLK-030..033 are model-router runtime /
|
||||
guardrails, not the CLI validate output; no BDR constrains the suite shape.
|
||||
|
||||
## CHALLENGE r1 → r2 → r3
|
||||
Three lenses (correctness, robustness, simplicity), 0 BLOCKER, 5 MAJOR, all
|
||||
closed by this revision: the sed range on a closing marker the CLI prints
|
||||
only when findings > 0 (fail-open to EOF, hides `✔ Validation passed with
|
||||
warnings`) is dropped for the CLI's own contract, the `--strict` exit code;
|
||||
the tautological SAMPLE_UNTERMINATED and the invented warning samples are
|
||||
dropped; the blocking branch is now exercised by a real CLI run on a
|
||||
fixture plugin instead of a copy of the grep. r3 (one re-challenge, 1 MAJOR):
|
||||
the control now runs through the gate's own helper, a clean sibling fixture
|
||||
pins the cause (only per-file warnings differ), the ko tail names the rerun
|
||||
command, WORK is script-level.
|
||||
|
||||
## FIX PLAN (one file: lib/tests/mods.test.sh)
|
||||
1. New helper, next to `bounded`:
|
||||
`validate_strict() { STRICT_OUT=$(bounded claude plugin validate --strict "$1"); }`
|
||||
returns the rc of `bounded` (last command), output in the script-level
|
||||
`STRICT_OUT` (same pattern as LAST_OUT in lib/tests/a11y-scan.test.sh:13).
|
||||
2. check_cli, validate half (lines 53-59): `validate_strict "$dir" || rc=$?`.
|
||||
rc 124 → `ko "$name: validate timed out after ${CLI_TIMEOUT}s"` (unchanged);
|
||||
rc ≠ 0 → `ko "$name: validate --strict rc=$rc: $(printf '%s' "$STRICT_OUT" | grep -E '^(✘|error:|⚠ Found)' | head -2 | tr '\n' ' ') (rerun: claude plugin validate --strict $dir)"`;
|
||||
rc 0 → `ok "$name: validate --strict passed"`.
|
||||
The `Validation passed` substring check and the `grep -qi warning` are
|
||||
removed: the exit code is the verdict. The plugin-test half (lines 60-65)
|
||||
is unchanged.
|
||||
3. Positive control. Script-level, right after `pass=0; fail=0`:
|
||||
`WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT` (top-level, as in
|
||||
lib/tests/a11y-scan.test.sh:12, never inside a function). New function
|
||||
`strict_discriminates`, called once after the `use_cli` block, only when
|
||||
`use_cli` = 1. It writes two scratch plugins:
|
||||
- `$WORK/warnfix/.claude-plugin/plugin.json` = `{"name":"warnfix"}`
|
||||
(per-file warnings: version, description, author);
|
||||
- `$WORK/cleanfix/.claude-plugin/plugin.json` =
|
||||
`{"name":"cleanfix","version":"0.1.0","description":"d","author":{"name":"x"}}`
|
||||
(no per-file warning; the directory findings README_MISSING,
|
||||
LICENSE_MISSING, ICON_MISSING are printed for both).
|
||||
Then, through the helper of step 1:
|
||||
- `validate_strict "$WORK/cleanfix"` → `check "strict control: clean fixture passes --strict" "$rc" 0`;
|
||||
- `validate_strict "$WORK/warnfix"` → `check "strict control: per-file warnings fail --strict" "$rc" 1`;
|
||||
rc 124 on either → `ko "strict control: validate timed out"`.
|
||||
The pair differs only in the per-file warnings, so a parse error (CLI
|
||||
without `--strict`: rc 1 on both) or a scrape added back turns the
|
||||
control red; the control covers the gate's own code path.
|
||||
4. Header comment (lines 4-5): "passes `claude plugin validate --strict`
|
||||
(per-file warnings are errors; the advisory plugin-directory section
|
||||
does not change the exit code) and `claude plugin test`".
|
||||
5. No new tracked file; the two manifests are written by the suite.
|
||||
|
||||
## EDGE CASES
|
||||
- CLI without `--strict` (older than its introduction): commander rejects
|
||||
the unknown option with rc 1 on every call → check_cli red with the
|
||||
`error: unknown option` line in its tail, and the clean-fixture control
|
||||
red too. Accepted: `cli_unavailable` already requires `claude plugin
|
||||
test`, which post-dates `--strict`.
|
||||
- Icon added later to mods/model-router: no effect, the directory profile
|
||||
is never requested.
|
||||
- The fixture's directory findings never affect rc (no `--profile`).
|
||||
|
||||
## RISK: low. Test-only file, no runtime behaviour.
|
||||
Reference in new issue
Block a user