diff --git a/.claude/memory/evals.md b/.claude/memory/evals.md index e27a3b9..fb0482e 100644 --- a/.claude/memory/evals.md +++ b/.claude/memory/evals.md @@ -207,3 +207,16 @@ rules: - **result**: 2 real STOP conditions fired and were resolved live, not hypothetically: (1) graphifyy 0.9.8's `graphify install` traced to source (`_install_claude_hook`, pipx venv `__main__.py:2033`) confirmed as a REWRITE of the config-protected `.claude/settings.json` — diff shown, user declined, binary upgraded without hook adoption; (2) gsd-pi 3.0.0 confirmed format-INCOMPATIBLE with `status-reporter.md`'s ROADMAP.md parser by generating a real test milestone in a scratch dir (ADR-013 cutover: no ROADMAP.md at all, DB-authoritative) — user chose "patch now" over rollback, parser rewired to `gsd headless query` JSON, smoke-tested both the absent-`.gsd/` and real-`.gsd/` cases before commit. gstack's local playwright patch (BDR-029) correctly identified as disposable-by-design, backed up before discard anyway (belt-and-suspenders after an auto-mode classifier denial), reapplied via the documented `gstack_bump_playwright_if_unsupported` steps — landed one minor ahead (1.61.1 vs the pre-bump 1.61.0) since upstream had moved between backup and reapply. `make test` green after every commit (90/90 gitflow + suites); `doctor.sh` 0 errors throughout. - **anomalies**: (1) mid-session the Bash tool went universally unresponsive (`true`/`echo hello` returning non-zero, no output) right after a large heredoc `git commit` — same `/tmp` exhaustion class as [[EVAL-019]]'s anomaly (1), user confirmed and cleared it; work resumed from the last confirmed git state rather than blindly retrying. (2) MCP magic's requested "reference not plaintext" (BDR-026 pattern) turned out NOT achievable as literally asked — `${VAR}` env expansion is documented for project-scope `.mcp.json` only, not the global `~/.claude.json` where magic is registered `--scope user` (verified via 2 rounds of sourced doc lookup, not assumed); user accepted the practical ceiling (regenerate via `toggle-external.sh disable/enable` to refresh the rotated key, decline the version pin). - **action**: keep. Branch unmerged (`chore/job6-deps-upgrade`, gitflow finish = separate human signal per CLAUDE.md). [[BDR-056]] captures the policy reversal this run demonstrated; [[LRN-107]] captures the secrets-copy mandate gap the report's own incident surfaced. + +## EVAL-021 — adversarial review of the 9-job series (release/1.0.0..develop) + remediation +- **Date**: 2026-07-08 +- **output**: read-only adversarial review — 11 analyzers (1/job + validator-analyzer contract) + fresh-context verifier on 6 top findings + make test. Report `.audit/review-release-1.0.0.md`: 1 BLOQUANT (A1 trailer), 5 à corriger (A2 gitleaks hook inert, A3 back-merge gap, A4 YAML, A5 geo attribution, A8 smoke-A), 5 mineurs, 10 verified false-positives; jobs 4/5/6/8 CLEAN, validator-analyzer contract SOUND. Remediation (chore/review-remediation): A1/A2/A4/A5 fixed, A8 PROVEN (both /seo+/geo AUTO items land on disk via L1 — no silent no-op), fil-rouge guard added, A3 backfilled + rtk fix ported, A6 threshold realigned. +- **method**: analyzers write findings to scratch; main loop does the inter-jobs cross-pass + memory-sequence + trailer sweep + cost check; verifier re-derives 6 findings from scratch. Sandbox gotcha logged: `git log | grep` truncates silently → used `git rev-list`. +- **anomalies**: (1) 2 sub-agent verdicts overturned — job7 CLEAN was wrong (gitleaks hook not wired, [[LRN-114]]) and the contract-agent's tool-grant "defect" was a false-positive ([[LRN-115]]). (2) A8 smoke-A root cause was undocumented in 212f9aa; reconstructed live — dispatcher classifies by batch-id (seo A/B/C, geo G1-G7), tolerant of header wording so items aren't dropped; path-b proven to land AUTO fixes on disk. (3) A7: job1/3f639b3 broke the design-hook oracle ~10h until job2/860b803 — historical; lesson = run make test before merging a branch, not only at finish. +- **action**: keep. Remediation branch unmerged (human gate). Fil-rouge guard now prevents the partial-fix class ([[LRN-113]]). + +## EVAL-022 — job9 model pins (BDR-060) were smoke-tested but never recorded as an EVAL (M5 trace) +- **Date**: 2026-07-08 +- **output**: review M5 flagged "no EVAL trace of the BDR-060 pin smoke-test." Traced: `.claude/tasks/TODO.md` job9 PART 1 GATE P1 DID record it — verifier `CONFORME`, security-auditor `BLOCK(2)`, plugin-advisor `ACTION REQUIRED`, verdict grammar intact, mode honored, no revert. The pins (verifier/security-auditor/plugin-advisor → sonnet, ea6c126/1c270e6/5ab6c21) WERE dispatch-smoked; the only gap was that the record lived in TODO, not evals.md. +- **method**: cross-read TODO PART 1 against the M5 finding; no re-run (recorded verdicts conclusive, pins unchanged since). +- **action**: keep — record backfilled here, no re-smoke required. diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index 55614c0..9b8860b 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -365,3 +365,6 @@ rules: - **Part 2** (`a5a7b54`/`6df42e4`/`c498b93`/`70fb3b4` + hardening `212f9aa`): seo/geo analyzers re-architected to fix-bundle→L1 (validator-analyzer contract), `Agent` dropped from both `tools:`; `/seo` new STEP 1.5 applies at L1 (serial by ownership, dissolves the parallel-edit race), `/geo` → dispatch+apply orchestrator, `/harden` already end-to-end path-b (untouched), `/onboard` audit-only (untouched). [[BDR-060]] version floor + [[BDR-061]] path-b doctrine. 4 real smokes green: analyzer emits bundle + edits nothing (md5 unchanged, no files created); AUTO fix LANDS on disk via L1 hotfixer with no confirmation (the exact previously-broken path — *report but zero fix* → resolved); GATED withheld pre-accord then applied post-accord (new tier, first test); /onboard writes only the report, zero source files. - **Part 3** (`87d63bf`/`af9656f`): H2 "Load and follow" idiom → **INLINE-LOAD** verb at code-cleaner + scaffolder (main-loop-BECOMES-agent, `Agent` not involved), drop unused `Agent` from code-cleaner; H1 code-cleaner→refactorer handoff now a named artifact `.claude/audits/CODE-CLEAN-SCOPE.md`. Tight scope per user (2 cited sites, no 40-site rewrite). - Branch unmerged, human gate. **Fixed** (`5a3de92`, isolated): stripped `Co-Authored-By: Claude` from `commit-changer.md` message template — it contradicted [[no-commit-attribution]] since the template's creation (the settings.json backstop caught real commits, but the template itself would keep re-seeding the trailer). Only banned trailer in the file (no Claude-Session/--trailer). FOLLOW-UP next cycle: cross with J4-16 (lib-layer lock) to verify no other agent template carries the same trailer. +- Adversarial review of the whole 9-job series (release/1.0.0..develop) → `.audit/review-release-1.0.0.md`: 1 BLOQUANT + 5 à corriger + 5 mineurs, 10 verified false-positives. 2 sub-agent verdicts overturned (job7 gitleaks hook inert [[LRN-114]], contract tool-grant FP [[LRN-115]]). Jobs 4/5/6/8 CLEAN, validator-analyzer contract SOUND. J4-16 follow-up above CLOSED: trailer twins found in bugfixer/feater/hotfixer. +- Remediation `chore/review-remediation` (unmerged, human gate): A1 trailer purge (3 templates) + whole-surface sweep; A2 gitleaks hook re-installed (`install-hook`) + negative-secret gate proven; A4 strict-YAML quote (seo/security-auditor); A5 geo own-policy (user-approved, PERMISSIVE default kept, false CLAUDE.md attribution dropped); A8 path-b PROVEN — /seo+/geo AUTO items land on disk via L1 (no silent no-op); fil-rouge `lib/tests/run-review-guards.sh` (5 guards, teeth-verified); A3 backfill LRN-098/101 + EVAL-015 + BLK-016 + PORTED rtk fix e58037c (was live-broken on develop, ~460K tokens/30d); A6 guard 280→320 + [[BDR-062]] (supersede BDR-031's 275 target). make test GREEN throughout. +- Capitalized: [[LRN-113]] partial-fix+guard (structural), [[LRN-114]] hook-drift, [[LRN-115]] analyzer report-grants (FP1), [[LRN-116]] release fix missing from develop, [[BDR-062]] density realign, [[EVAL-021]] the review, [[EVAL-022]] M5 pins trace. Noted un-back-merged release chores beyond A3: e65796f (SC1091 lint silence) — left for a future reconcile. diff --git a/.claude/memory/learnings.md b/.claude/memory/learnings.md index 33bcef5..a9838b0 100644 --- a/.claude/memory/learnings.md +++ b/.claude/memory/learnings.md @@ -128,6 +128,10 @@ rules: | LRN-110 | 2026-07-07 | job8: `21st_magic_component_builder` (magic MCP) opens unauth'd 127.0.0.1 callback server, CORS `*`, no token check, 10min window — any local POST lands verbatim in the tool result the model consumes = local prompt-injection channel | any MCP tool that opens a local callback/listener server to receive async results — check auth + origin scoping on the listener, not just the outbound call | | LRN-111 | 2026-07-07 | job8: empty permissions.allow for a risky MCP tool is a VALID posture (not a gap) when transcript census shows zero real invocations — pre-authorizing unused surface buys nothing, ask-gate costs nothing | deciding whether to allowlist any tool/command — check real usage before assuming "no entry = todo" | | LRN-112 | 2026-07-08 | job9: CC nested subagent dispatch SUPPORTED since v2.1.172 (cap 5 levels, `Agent` must be in subagent `tools:`) — "flattens to 1 level" is the pre-2.1.172 regime; live env v2.1.203. Contradicts the operating premise of the whole job1-9 series | a subagent-dispatches-subagent design is VERSION-CONTINGENT, not "broken" — check CC version before flagging; fix = raise floor or re-architect to bundle→L1 | +| LRN-113 | 2026-07-08 | partial-pattern-fix = recurring defect of the job1-9 series: fix the cited instance, leave the twins (trailer A1, YAML A4, attribution A5, hook A2). An adversarial review catches twins later; nothing catches them at commit time | any fix of a banned pattern: grep the ENTIRE surface + add a make-test guard (run-review-guards.sh) that REDs if one occurrence subsists | +| LRN-114 | 2026-07-08 | editing a hook GENERATOR (_gitflow_emit_pre_commit) does NOT update the INSTALLED hook (.githooks/pre-commit) — silent drift; T10 diffs the allow/block verdict not content, T16 emits fresh in a throwaway repo → job7 gitleaks backstop inert on the repo 8 days | after editing a template-generated artifact: reinstall (install-hook) + a gate that diffs installed==emit | +| LRN-115 | 2026-07-08 | analyzer Edit/Write grants (seo/geo/validator) are NOT dead: needed to write the REPORT (VALIDATE/SEO/GEO.md); the "never edit" rule targets CODE, instruction-level (same as the patron) — verified false-positive | do NOT re-flag as a tool-grant defect; a report-only agent keeps Write for its own report | +| LRN-116 | 2026-07-08 | memory backfill release→develop: a BLK marked "resolved" can have its RESOLUTION (code) missing from develop — BLK-016 resolved on release but rtk fix e58037c never back-merged → bug LIVE on develop | before backfilling a resolved blocker: verify the fix CODE is on the target branch, not just the registry entry | --- @@ -1149,3 +1153,30 @@ rules: - **context**: the whole job1-9 audit series ran on the premise *"Claude Code aplatit à 1 niveau → un design supposant 2 niveaux de sous-agents est cassé silencieusement."* job9 corrected it via `claude-code-guide` (official docs `code.claude.com/docs/en/agent-sdk/subagents.md`): a running subagent CAN spawn a further subagent IF `Agent` is in its `tools:` (omit it / add to `disallowedTools` to prevent nesting); hard cap **5 levels** ("a subagent 5 levels below main can't spawn further"); nesting **stabilized in v2.1.172** ("let subagents spawn their own subagents") — earlier versions did not support it at all. Live env confirmed **v2.1.203** (user). `claude --version` was unavailable in-sandbox so the report bracketed but could not pin it; the user pinned it. - **future application**: NEVER classify a subagent-dispatches-subagent design as "BROKEN" without checking the CC version. On ≥2.1.172 it works within the 5-level cap; on <2.1.172 it silently no-ops. The actionable finding is a VERSION-FLOOR ([[BDR-060]]) or a version-robust re-architecture (bundle→L1, [[BDR-061]]) — not "it's broken." When an agent must NOT nest, enforce it structurally: drop `Agent` from its `tools:` (done for seo/geo analyzers). Re-audit any prior job1-9 "nested = broken" finding through this lens. - **cousin**: [[BDR-060]] (version floor), [[BDR-061]] (path-b bundle pattern), [[LRN-057]] (subagent invocation idioms). + +## LRN-113 — Partial-pattern-fix is the job1-9 series' recurring defect: grep the whole surface + guard it +- **pattern**: fix one cited instance of a banned pattern, leave the twins. Review found 4: trailer stripped from commit-changer only (A1, twins in bugfixer/feater/hotfixer); YAML quoted elsewhere but seo/security-auditor left broken (A4); attribution scrubbed on 3 skills but geo-analyzer missed (A5); gitleaks added to the hook generator but the installed hook not regenerated (A2). +- **why it recurs**: the fixer greps for the reported line, fixes it, stops — never enumerates the pattern across the full surface. An adversarial review catches the twins later; nothing catches them at commit time. +- **fix**: every pattern-fix ends with (1) a whole-surface grep proving zero residue, (2) a deterministic make-test guard that REDs if any occurrence returns. Shipped `lib/tests/run-review-guards.sh` — G1 trailer, G2 false attribution, G3 strict-YAML, G4 reconcile hermeticity, G5 hook-drift; teeth-verified (planted violation REDs). This is the check that would have caught A1/A4/A5/A2 at make-test time instead of a review. +- **future application**: any "fix pattern X" task → grep agents/ lib/ hooks/ templates/ skills/, add/extend a review-guard with teeth. +- **cousin**: [[LRN-114]] (hook-drift class), [[LRN-047]] (silent degradation → measure/guard). + +## LRN-114 — Editing a hook generator does not touch the installed hook: reinstall + drift-guard +- **pattern**: job7 added the gitleaks scan to `_gitflow_emit_pre_commit` (the GENERATOR), but the installed `.githooks/pre-commit` is only (re)written by `gitflow init`/`install-hook`. job7 never re-installed → the repo's active hook stayed the pre-job7 version (620071b) for 8 days; `git commit` ran no secret scan while the team believed it did. +- **why undetected**: T10 (drift test) compares only the hook's allow/block VERDICT, not content; T16 emits a FRESH hook in a throwaway repo, validating the generator, never the installed file. Both green while the installed hook was stale. +- **fix**: after editing any template-generated artifact, regenerate the installed copy (`gitflow.sh install-hook`) AND add a content-drift gate — `run-review-guards.sh` G5 diffs installed `.githooks/pre-commit` against `emit-hook`. +- **future application**: any generator/template emitting an on-disk artifact needs an "installed == freshly-emitted" test, not just a behavioral one. +- **cousin**: [[LRN-113]] (partial-fix + guard), [[LRN-039]] (installers drift hand-curated config). + +## LRN-115 — Analyzer Edit/Write grants are not dead capability: they write the report (false-positive) +- **pattern**: a contract audit flagged seo/geo/validator-analyzer holding `Edit`/`Write` while instructed "do NOT apply any Edit/Write" as a defense-in-depth defect. Verified FALSE: those grants write the agent's own REPORT (`.claude/audits/VALIDATE.md`/`SEO.md`/`GEO.md`). The "never edit" rule targets CODE files (the fix-bundle is applied by the dispatcher) and is instruction-level — identical in the patron. Removing Write would break report generation. +- **why it matters**: don't "harden" a report-only agent by stripping Write — it needs it for its report. The code/report distinction is instruction-enforced, not tool-enforced, by design. +- **future application**: before flagging a tool-grant as dead, check whether the agent uses it for its own output artifact (report), not the forbidden target (code). +- **cousin**: [[BDR-061]] (analyzer bundle→L1 contract), [[LRN-113]]. + +## LRN-116 — A resolved blocker's FIX can be missing from develop even when the entry backfills cleanly +- **pattern**: backfilling release/1.0.0 memory into develop, BLK-016 (rtk PATH-dead) was marked "resolved" via fix e58037c. Checked before backfilling: e58037c (the `~/.cargo/bin`→`~/.local/bin` bridge in install-plugins.sh) was NOT on develop — develop still installed rtk to a cargo bin dir the tool shell can't see → rtk compression was LIVE-broken on develop (~460K tokens/30d). The registry entry looked safe to copy; the underlying fix wasn't there. +- **why it matters**: append-only registry backfill is "safe" only for the TEXT; a "resolved" status is a claim about CODE state that must be verified on the target branch, else you assert a resolution that isn't true. +- **fix**: ported e58037c to develop (13-line idempotent bridge), THEN backfilled BLK-016 resolved. General: before backmerging a resolved blocker, grep the target for the fix's code signature. +- **future application**: gitflow divergence review — enumerate release-only COMMITS that touch code, not just memory; a feature can be parallel-merged while its RC-branch fix is orphaned. +- **cousin**: [[LRN-036]] (PATH profile drift), [[LRN-047]] (silent degradation).