From 5f0d848ce42bf450cb8c19ba0e89b3c4e3cdbfef Mon Sep 17 00:00:00 2001 From: bchanot Date: Wed, 28 Oct 2026 18:40:31 +0100 Subject: [PATCH] =?UTF-8?q?chore(memory):=20BDR-129=20+=20LRN-236=20+=20LR?= =?UTF-8?q?N-237=20+=20EVAL-052=20=E2=80=94=20ship-feature=20memory-v2-wri?= =?UTF-8?q?te,=20TODO=20phase=204a=20checked=20+=20follow-ups?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/memory/decisions.md | 6 ++++++ .claude/memory/evals.md | 6 ++++++ .claude/memory/journal.md | 3 +++ .claude/memory/learnings.md | 8 ++++++++ .claude/tasks/TODO.md | 5 ++++- 5 files changed, 27 insertions(+), 1 deletion(-) diff --git a/.claude/memory/decisions.md b/.claude/memory/decisions.md index 8e2bf6c..b38e7c9 100644 --- a/.claude/memory/decisions.md +++ b/.claude/memory/decisions.md @@ -1420,3 +1420,9 @@ Branch feature/user-writing-web-rules, UNMERGED (human gate). - **Alternatives rejected**: block-list `paths:` parsing (spec shape inline; pre-flight `check` at S0/S1 moves the failure before any write); mod-based injection (BDR-125 says phase 0 no mod); dep-dir filter unconditional (vetoed `vendor/**`); `git add .claude/rules` dir (a pre-existing rule made visible by the negation would ride along); whole-file Write for the pointer (subtract-only). - **Residuals**: `###` granularity before faunosteo (12.6 KB path-bound content under always-on `## Project conventions`; `##`-only moves 1.5%), user decides (TODO 0b); `game` heading with `'` refused until renamed or extract by index; `extract -n` dry run deferred; T11b + AC2 hardcode 4 rules; `InstructionsLoaded` key on older CLIs unverified (check `claude --version` on work machine before pull); pre-commit `.claude/**` exemption lets a rules-only commit land on develop → phase 2. - **Refs**: spec `docs/superpowers/specs/2026-10-26-claude-md-split-design.md`, plan r3 `docs/superpowers/plans/2026-10-26-claude-md-split.md`, contract `.claude/tasks/contracts/2026-10-26-claude-md-split-0042.md`, commits abfa416..0b59687. [[BDR-125]], [[BDR-003]], [[BDR-065]], [[LRN-154]], [[LRN-232]], [[LRN-233]], [[EVAL-051]]. + +## BDR-129 — Memory registries v2: lib/memory.sh single writer, format: 2 without Index, shared parser, rc 6 chokepoint, oracle upgrade [accepted] (2026-10-28) +- **Decision**: `lib/memory.sh` = single writer + reader of `.claude/memory` registries (add/status/journal/index/check [--staged]/upgrade, `-C dir`). v2 = same file, `format: 2` frontmatter, no `## Index`. Status parser shared with `lib/reconcile.sh`: strict on v2 (heading `[tag]` or labelled status line, last wins, `memory_closed` = resolved|superseded|deprecated), legacy lexical reader kept on v1 (13 v1 repos: open sets identical). `memory-commit.sh` runs `check --staged`: staged v2 registry rewriting an entry line, new duplicate, bad heading, open fence, deletion or downgrade → unstaged, rc 6. Lock = bash noclobber file in git common dir, token unlock, serialized steal, verified backup restored on failure/signal. `upgrade`: Index-only dates/statuses → body lines, dialect headings → `## ID — title (date)`, one-way verdict repair (v1-closed→strict-open only), oracle else rc 2 file untouched; refuses duplicate IDs, non-table Index. +- **Why**: Index drifted (16 rows missing), status in 3 places, BLK-032..034 false-open (prose words), no single writer (ad hoc Edit/python on 357k files). Phase 4a of [[BDR-125]]. +- **Alternatives rejected**: `flock(1)` (absent on macOS, [[BDR-110]]); `mkdir` lock (uutils 0.10 not atomic, [[LRN-236]]); generated Index kept (drift by construction, [[LRN-055]]); upgrade via /prune-memory curation (no oracle); strict parser on v1 (regressed 4 of 13 repos: title-suffix, `date: · status:`, `**Statut** :`). +- **Refs**: contract `.claude/tasks/contracts/2026-10-27-memory-v2-write-2030.md`, plan + spec `docs/superpowers/{plans,specs}/2026-10-27-memory-v2-write*`, commits a423594..aab9e4d, upgrade 9698473. [[LRN-237]], [[EVAL-052]]. diff --git a/.claude/memory/evals.md b/.claude/memory/evals.md index a8ece0a..a376fa6 100644 --- a/.claude/memory/evals.md +++ b/.claude/memory/evals.md @@ -407,3 +407,9 @@ Dogfood: 3 blind lenses attacked the v1 plan for the plan-challenge feature itse - **Method**: 0d analyzer; brainstorm 4 user answers; plan r1 → r3 (correctness FATAL(5): SIGPIPE, rc 127, dry run on develop blocked, unchecked commit rc, 80 cols). SDD 12 tasks sonnet/medium, 12 task reviews clean (1 controller resolve avg, mostly attribution lines + oracle facts). RED (current skill stops at plugin-probe) → GREEN (S5 table, nothing written) → dry run on a branched scratch (2 rules in HEAD). Sonnet final review 3 Important → 89334f2. GATE 0 MET 14/20 runnable; verifier ECARTS(3) → ECARTS(1 = user MCP step); security PASS, 1 LOW. Opus/xhigh craft: 0 Critical, 2 Important, 8 Minor, ran the lib on the 6 real 0b targets (56 sections conserved) → 0b59687, re-review 0 open. `make test` all GREEN, suite PASS=93. - **Anomalies**: `extract` rc 0 on a failed rewrite passed 3 challengers + 12 task reviews + 1 final review: nobody tested an I/O failure (only logic refusals). `##`-only granularity misses faunosteo's payload (12.6 KB under `###`): found only because the reviewer ran the tool on the real targets. Headless proof 0 lines (model skipped Read). T10h-j named a guard the test could not observe. Brief tally arithmetic wrong (96 vs 93). Final review found 3 Important the 12 task reviews missed (dep-dir veto of `vendor/**`, misleading error, gitignore newline glue): per-task scope hides cross-task consequences. - **Action**: Review Focus lists a failure-path test per function that rewrites a file; a tool meant for other repos gets a reviewer run on their real inputs before merge; a load proof prompt names the tool call ([[LRN-232]]); test names state the observable. Links [[BDR-128]], [[LRN-233]]. + +## EVAL-052 — Memory v2: 12 review layers, each caught what green gates missed [keep] (2026-10-28) +- **Output**: phase 4a memory v2 (lib/memory.sh + 7 citer tasks + dialect migration), 2026-10-27/28. +- **Method**: 2 plan-challenge rounds (3 lenses each, reference lib run before the plan), 8 task reviews, 4 fix rounds on Task 8, verifier ×2, security gate, controller proof on 14 repo copies, opus xhigh final review + scoped re-review. +- **Anomalies**: each layer caught what earlier green layers missed. Challenge: two classifiers, BLK-019/020 false-open, uutils mkdir lock. Task reviews: G4 guard dodged by string split (rejected), `###` verdict flip ×4 shapes. Verifier: 3 EVAL Index actions lost. Controller proof: template path after `-C` (relative call only). Opus final, after all gates green: lock spin on unwritable `.git` (no rc), bare-prefix `status BLK` writing BLK-001, bullet-list Index facts dropped (trucks_at_work), 59+106 duplicate Date lines (`**Date** :` spacing). One implementer tried `--no-verify` in a temp repo (denied). +- **Action**: keep the full stack for libs that rewrite user data; add a real-data copy proof with the user's own invocation before the final review. [[BDR-129]], [[LRN-237]]. diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index ebd2e25..f134d5e 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -665,3 +665,6 @@ rules: - Phase 0b run (10 repos, faunosteo excluded: on the user's feature branch): 3 splits on chore/claude-md-split, merge on user go — zenquality 4 rules (Memory files 19.4k → 14.5k, project row 10k → 5k), game 2 rules (16k → 14.9k), angelscar356 1 rule (16.2k → 15.4k); 7 keep-everything (overview/stack/commands/structure/conventions only, S4 rule). `gitflow start` refreshed zenquality's own .githooks → S0 STOP on a dirty tree, committed as chore ed10340, re-dispatched. Three onboarder runs in parallel, one repo each, all clean first pass. - faunosteo split on the user's feature branch (user choice): 11 rules (10 ### under Project conventions + Docker), Memory files 31.9k → 27.2k, project row 22.3k → 17.7k, 7a75974. Phase 0b done: 4 repos split, 7 keep everything. - /feat extract-hardening (feature/extract-hardening 14a93e3 + docs 9160ce2): `_rules_out_path_ok` (absolute, `..`, exists or dangling link → rc 2, reason on stderr), mkdir + rule-write rc checked before the source rewrite, S0.3 prose in lib case order, 116 → 139 checks. Plan r1 `if ! { … } > "$out"` blind to a failed redirect (2 BLOCKER, bash 5.3.9 probe) → `|| { …; return 2; }`; r2 confirmation CONCERNS(1) (GNU mkdir stderr also says "cannot create"). Verifier CONFORME 8/8, security PASS. LRN-235. Three follow-ups in TODO (emit rc, rewrite keeps $tmp, lexical `..`). Merge on user go. + +## 2026-10-28 +- Phase 4a memory v2 shipped on feature/memory-v2-write: lib/memory.sh single writer, registries v2 (9698473), shared parser fixes BLK-032..034, upgrade migrates 12/14 repos on copies (game, trucks_at_work refused). 2 challenge rounds + 8 task reviews + 4 Task 8 fix rounds + verifier x2 + opus final (lock spin, bare-prefix status, bullet Index). BDR-129, LRN-236, LRN-237, EVAL-052. Merge on user go. diff --git a/.claude/memory/learnings.md b/.claude/memory/learnings.md index f609757..acc804e 100644 --- a/.claude/memory/learnings.md +++ b/.claude/memory/learnings.md @@ -1825,3 +1825,11 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s - **Date**: 2026-10-27 - **Context**: feat extract-hardening 2026-10-27. Plan r1 wrote `if ! { printf …; body; } > "$out"; then … return 2`. Two challengers + own probe on bash 5.3.9: redirect fails → bash returns 1 BEFORE applying `!`, `if` sees false, source rewritten, rc 0 = the bug unchanged. `! { echo a; } > bad; echo $?` → 1. shellcheck clean, floor guard blind. LRN-233 form works only because its redirects sit INSIDE the group. Same pass: `grep -c 'cannot create'` on captured stderr = 2 on GNU coreutils (mkdir prints `mkdir: cannot create directory …`), 1 on uutils (this host) → host-dependent test. mkdir rc check + write rc check give same rc and same fs state; only the stderr line tells them apart. - **Apply**: a redirected group is checked as `{ a && b; } > f || { cleanup; echo … >&2; return 2; }` (chain inside with `&&`, else the last command masks printf). Never `if ! { … } > f`. A guard whose only observable is a message gets a test on that message, `grep -c '^project-rules: …'` anchored on the lib's own prefix, never a bare word a system tool may also print. A fixture escape (`../x` from the fixture root, `ln -s /nonexistent`) runs from a subdir of the fixture and links inside it. Links [[LRN-233]], [[LRN-096]], [[BDR-128]]. + +## LRN-236 — uutils mkdir 0.10 is not atomic: lock with bash noclobber (O_EXCL), token unlock (2026-10-28) +- **Context**: memory v2 lock proof 2026-10-27. 5 concurrent `memory.sh add` lost 2-4 entries, silently. `/usr/bin/mkdir` = uutils coreutils 0.10.0: two concurrent `mkdir x` both rc 0 (`for i in $(seq 20); do (mkdir y && echo won) & done` → 2 wins). Lock-by-mkdir unsafe on this host. +- **Apply**: atomic create lock = bash noclobber: `(set -C; printf '%s\n' "$token" > "$lock") 2>/dev/null` (open O_EXCL by bash itself, no external tool, portable to macOS). Unlock only on token match; steal through a second noclobber guard. Race test with ≥5 writers is the flip test (red with mkdir). Never `rm -rf` a lock dir through a variable. [[BDR-129]], [[BDR-110]]. + +## LRN-237 — A migration oracle sharing the transform reader proves nothing: own range per reader, assert on the entry, prove with the user invocation (2026-10-28) +- **Context**: memory v2 `upgrade` verdict oracle, 2026-10-28. Oracle compared each blocker's open/closed verdict, legacy reader before vs strict reader after. Four review rounds each found a range leak the oracle shared: `###` entry invisible to v1 reader counted closed → `resolved` written on an open blocker; `## BLK-003` range swallowed a following `### BLK-001`; a `### ` heading quoted in a fence ended the range; status lines inside a fence read. Tests stayed green because they checked the reconcile enumerator, blind to `###`. Separately: template path derived after `-C` cd → upgrade rc 2 only when the lib is called by relative path; implementers always used absolute paths. +- **Apply**: an oracle sharing the transform's reader proves nothing: give each reader its own entry range (end at next heading of any level, fence-aware, skip fenced status). Assert on the entry itself, not via an enumerator. Prove a CLI on real data copies with the invocation users type (relative path + `-C`). Resolve a lib's own dir at source time (`cd -P` before any `cd`). [[BDR-129]], [[LRN-096]]. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index bbf810f..ce2c10b 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -28,7 +28,10 @@ Shared rules for every phase: - [ ] `extract` rewrite failure mid-`cat` (ENOSPC/EIO after the truncation of `$src`): keep `$tmp` and name it on stderr instead of `rm -f` (today "$src not rewritten" can be false, the two copies gone; onboarder S1 rollback covers the split path, not direct lib use). - [ ] shortcut `_rules_out_path_ok`: the `..` check is lexical, a symlinked parent component (`.claude` or `.claude/rules` pointing elsewhere) is not resolved; upgrade = `cd -P` on the parent dir, compare with `$PWD` (no portable `realpath -m`). -- [ ] **Phase 4a, memory v2 write side: shell, no mod** (shrunk 2026-10-25: the measured recall channels never read Meta; the measured status bug is a reconcile parser bug) +- [x] **Phase 4a, memory v2 write side: shell, no mod** (shrunk 2026-10-25: the measured recall channels never read Meta; the measured status bug is a reconcile parser bug) + - Done 2026-10-28 on feature/memory-v2-write (a423594..aab9e4d, registries v2 9698473): BDR-129, LRN-236, LRN-237, EVAL-052. memory suite 202/0, make test green. + - [ ] Phase 4a per-repo pass: `bash ~/.claude/lib/memory.sh -C upgrade` on a clean `.claude/memory`, one `chore/memory-v2` branch per repo, user approval each, commit via `/commit-change` when headings were rewritten (memory-commit rc 6 by design). Ready (proven on copies 2026-10-28): config, faunosteo, game-deploy ×2, integration_alphalink, robin_petier, zenquality, angelscar356, bchanot-cv, for_you, gitea-deploy, nuit-folle. Refused, user decision first: game (`## BDR-027/061 ADDENDUM` headings, duplicate BLK-008/014), trucks_at_work (bullet-list `## Index` holding BLK-001's closure). + - [ ] Phase 4a parked minors (ledger `.superpowers/sdd/2026-10-27-memory-v2-write/progress.md`): empty `memory.lock.steal` never stale (manual rm); `## Index` comment-edge mismatch; `upgrade` header doc lacks the non-table refusal; memory-commit rc 6 message misleading on a v1→v2 heading-rewrite upgrade; labelled "not resolved yet" reads resolved on v2; body `## ` line splits an entry (refuse at add); `-C` + relative `--body-file` (say absolute in § WRITE); tour/deploy/audit-delta § WRITE pointer; reconcile.sh:24-25 and :41-42 stale comments; `_mem_blob` → `git cat-file blob`; evals template "Later status" line; partial stage dropped on rc 6; 39 pre-existing >80-char lines in reconcile.sh / memory-commit.sh. - Facts 2026-10-24 (scratchpad memeval, 434 entries): registries ~1M chars, never in always-on context, read only by /feat /bugfix /ship-feature (analyze-before-plan); Index drift 6 rows; status in 3 places (title tag, body Status line, Index column) → reconcile false-open BLK-032..034; no single writer (Edit needs a Read of a 357k file, so real writes are ad hoc python/bash); 8 projects on this machine hold v1 registries. Verified 2026-10-25: `lib/reconcile.sh:60-67` drops the entry's OWN header while BLK-032..034 carry `[resolved]` there (the false-open); nothing machine-reads `## Index` (analyze-before-plan.md:42, reconcile.sh:9, test T1); 1 of 1,290 entries across the 8 projects above a 1% accent ratio. - `lib/memory.sh` = single writer, like gitflow.sh: `add --title --body-file [--status --supersedes]` allocates next ID from headings (never the Index) under `flock` (concurrent conversations, phase 5), appends the entry; `status ` appends a dated status line (append-only); `journal ""`; `index` prints the Index on demand from headings; `check [--staged]` = the validator (heading pattern, ID = max+1 and unique, no removed line from an existing entry outside a /prune-memory commit, new BLK quotes ≥1 error verbatim). The validator runs in `add` and `check` + make test, NOT in the generated pre-commit (undecidable there, and it exits 0 on working branches). No English check (doctrine stays). - Status: `reconcile_blk_current_status` reads the own-header `[tag]` first, then "last status-bearing line wins" (one parser, shared by memory.sh status and reconcile). Fixes BLK-032..034 false-open; regression test on those three.