diff --git a/.claude/memory/decisions.md b/.claude/memory/decisions.md index e3e23fc..eb1c0fa 100644 --- a/.claude/memory/decisions.md +++ b/.claude/memory/decisions.md @@ -1498,4 +1498,5 @@ Branch feature/user-writing-web-rules, UNMERGED (human gate). - **Alternatives rejected**: hide executor agents via `agent.offer` (breaks dispatch, proven); move writing-style/communication out (apply every turn, same cost); rewrite shown answer (`turn.complete` only appends below); Stop-hook enforcement (BDR-083). - **Refs**: probe `scratchpad/ctxprobe` (session-local, not committed), plugin-authoring types 2.1.306 (`prompt.context`, `prompt.compose`, `prompt.section`, `prompt.attachment`, `prompt.submit`, `tool.call`). Supersedes [[BDR-031]]; links [[BLK-009]], [[LRN-175]], [[LRN-054]], [[LRN-091]], [[BDR-115]]. - **Amendment (2026-10-24, user)**: phase 5 added: per-project virtual profiles, concurrent sessions (today `lib/profile.sh` global: one profile for all open sessions), auto-pick from fingerprint or pinned `/profile set`, per-skill advert of needed helpers; plugin project-scope `enabledPlugins` to verify. Advisor toast refused. Each phase deletes what it obsoletes (scripts, hooks, tests, doctrine). Full specs: TODO "Context mod queue". Probe bench kept at `.claude/tasks/plans/2026-10-24-context-mod-probe/`. +- **Amendment 2 (2026-10-24, user: full memory redesign allowed)**: phase 4 = memory v2 (write + capture + read + feedback), measured on 434 entries: capture is the weak link (5/7 real errors of 2026-10-23 had no entry), not search; haiku rerank of lexical top-30 = 14/22 vs BM25 9/22 on real FR prompts; error 4-gram match exact (429 → BLK-030/031). Storage: augment in place + Meta line + generated Index + single writer `lib/memory.sh` + pre-commit validator (Edit-level guard rejected: real writes go through Bash/python). One-file-per-entry rejected for now (44 citers + 8 project migrations for concurrency/history gains a validator+backfill mostly match). Registry guard moved from phase 2 to 4a; 4a needs no mod. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index bc22231..d8869e7 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -29,7 +29,7 @@ Shared rules for every phase: - Cleanup: `hooks/design-toolchain-reminder.sh` + its settings.json registration + `lib/tests/design-toolchain-reminder.test.sh` (the mod injects the section itself); the hook's "(already in context)" text; CLAUDE.global.md lines that only pointed at the moved text; BDR-031 citers. - [ ] **Phase 2, guards in the context mod (deny with reason, deterministic)** - - Registries: Edit/Write on `.claude/memory/{decisions,learnings,blockers,evals}.md` refused when the new heading is not `## - — `, the Index row is missing or mismatched, an existing entry line is removed or rewritten (append-only; status tag edits allowed by explicit rule), or the added text reads French (accent-ratio heuristic, threshold tested on the real registries: zero false positive required). + - Registry format: moved to phase 4a (measured 2026-10-24: registry writes go through Bash/python, never Edit, since Edit needs a Read of a 357k-char file; a `tool.call` guard on Edit/Write would miss them; a pre-commit validator sees every write path). - Merge gate: Bash running `gitflow.sh finish` refused unless the last user message (recorded at `prompt.submit` origin composer) carries an explicit signal (merge it / merge le / merger / fusionne / feature OK / ship it, FR+EN list, tested on this session's real phrasings) or an AskUserQuestion answer approving the merge in the same turn. - Commit message: `git commit -m/-F` over 50 words refused unless a project convention file exists (lib/commit-message.md lookup order). - Code limits on OUR code only (origin from phase 1): after Edit/Write, report lines > 80 cols and functions > 25 logic lines as result `context`, never deny (language support: sh, ts/js, py, rs first; others skipped). @@ -46,16 +46,19 @@ Shared rules for every phase: - Proof: canaries (hidden skill ABSENT, control PRESENT, typed hidden skill runs, trimmed agent dispatches), hint precision measured on a fixed prompt set (no hint on chat/bookkeeping), `/context` delta (~5-7.5k est.). - Cleanup: CLAUDE.global.md Skill routing section → 2 lines; LRN-043 routing-line rule superseded; doctor skill-listing budget check updated. -- [ ] **Phase 4, active memory** - - Session digest via `prompt.context` block (re-read on compaction/clear by the engine): open BLK (body status, last block wins, as lib/reconcile.sh reads it), actionable TODO items (Open § Actionable), last journal heading + line; only when `.claude/memory/` exists. - - Per-prompt retrieval: local score of registry headings (`## BDR|LRN|BLK|EVAL-` lines) against the typed prompt, top 3-5 `ID — title` lines as `prompt.submit` `context` above a threshold; no model call. Optional `memory_search` tool (`$.tool.register`, `isDeferred: false`) for skills (analyze-before-plan pass 1 calls it instead of grep). - - Facts 2026-10-24: registries never in always-on context (~1M chars: decisions 319k, learnings 357k, journal 168k, evals 85k, blockers 68k); Index tables drift (decisions 121/125 rows, evals 46/48); only /feat /bugfix /ship-feature read them (analyze-before-plan, at plan time, by heading); nothing fires on an action. "Session start: read .claude/memory/" is unrealistic as written. - - Action-triggered recall (user ask 2026-10-24: "pioché quand on fait l'action"): (a) error match: a tool result containing an error string quoted verbatim in a BLK/LRN body → inject that entry as result `context` (deterministic, entries already quote errors exact); (b) optional `- **Trigger**: ` line per BLK/LRN, checked at `tool.call` / `skill.prompt` → inject the entry; entries marked `critical` → deny the action once with the lesson as reason, then allow the retry. - - Generated index: rebuild each registry `## Index` from the `## ID —` headings (mod or script in make test), so it can never drift; readers keep reading headings. - - Selection quality: score = heading words + Trigger lines + quoted errors; precision measured on 10 real prompts and 10 real errors from journal history before shipping; threshold so nothing fires on chat/bookkeeping (LRN-091). - - Capitalization proposals on observed milestones only: a `feat`/`fix` commit or a `gitflow finish` seen in `tool.call` → one context note at the next prompt (never per turn, never at Stop). - - Proof: digest canary after start, after compaction, after /clear; retrieval precision on 10 real prompts from journal history; zero note on turns without milestone. - - Cleanup: CLAUDE.global.md "Session start" step 1 (read registries) → mod; "Proactive capitalization" paragraph KEPT (design choices, surprising discoveries, dead ends leave no commit: only the model sees them; the mod adds the commit/merge triggers, never replaces the judgment ones; writing always stays model draft + user approval, format checked by the phase 2 guard); analyze-before-plan pass 1 points at the tool; fix the reconcile engine gap seen 2026-10-23 (`reconcile_blk_open` ignores header `[resolved]`) since the digest reuses it. +- [ ] **Phase 4, memory v2: write, capture, read, feedback** (redesigned 2026-10-24 after measurement; 4a needs no mod and can run right after phase 0) + - Facts 2026-10-24 (scratchpad memeval, 434 entries): registries ~1M chars (decisions 319k, learnings 357k, journal 168k, evals 85k, blockers 68k), never in always-on context, read only by /feat /bugfix /ship-feature (analyze-before-plan); Index drift 6 rows (decisions 121/125, evals 46/48); status in 3 places (title tag, body Status line, Index column; LRN/EVAL none) → reconcile false-open BLK-032..034; no single writer (skill says Edit + manual Index row; Edit needs a Read of a 357k file, so real writes are ad hoc python/bash); 93-100% of entries carry a language-neutral artifact (path, command, quoted string); lesson field present in 98% LRN, 85% BLK, 98% EVAL (median 170-560 chars vs body 1.2-2k); 2 near-duplicate title pairs only; 325/434 entries have [[links]] (1080); growth 40-90 entries/month; 44 files cite registry paths; 8 projects on this machine hold v1 registries. "Session start: read .claude/memory/" is unrealistic; caveman premise "registries load every session" is false. + - Measured recall (5 real FR prompts, 22 relevant entries): BM25 titles 7/22, BM25 full text 9/22, haiku rerank of lexical top-30 14/22 (~8.5k chars), haiku over all titles 14/22 (~43k chars). Error 4-gram match on today's real errors: 429 credits → BLK-030 + BLK-031 exact, BLK-034 suite error exact, no false positive; 5 of 7 real errors of 2026-10-23 had NO entry (never captured): capture is the weak link, not search. + - Storage choice: augment in place (5 files, headings and paths unchanged) + per-entry Meta line + generated Index. Rejected for now: one file per entry with frontmatter (better concurrency and per-entry history, but 44 citers + 8 project migrations; a commit-time validator + backfill give the same machine fields). Revisit if same-repo concurrent writes conflict. + - **4a write side (shell, no mod).** `lib/memory.sh` = single writer and reader, like gitflow.sh: `add --title --apply --body-file [--status --scope --supersedes --critical]` allocates next ID from headings (never the Index), appends the entry ending with `- **Meta**: status=…; scope=project|global; triggers=path:…,cmd:…,skill:…,err:"…"; supersedes=…; critical=yes|no`, regenerates `## Index` from headings + Meta; `status ` appends a dated status line (append-only); `journal ""` appends under today's heading; `derive ` prints derived triggers (paths, commands, skill names, quoted errors normalized); `index`; `check [--staged]`. + - Validator: pre-commit (only when `.claude/memory/` is staged; diff-based on added/changed entries, so legacy entries in the 8 projects pass) + make test: heading pattern, ID = max+1 and unique, Meta on new entries, Index == generated, no removed line from an existing entry outside a /prune-memory curation commit, English (accent-ratio threshold proven zero false positive on all 8 projects' registries). New BLK must quote ≥1 error verbatim. + - Backfill via /prune-memory (deterministic script + user review): Meta for the 434 entries; status from the 3 legacy forms (last wins); triggers derived; Apply = LRN/BLK/EVAL lesson field, BDR = first sentence of **Decision**; scope proposed by the model, reviewed (global = applies in other projects too: Claude Code behaviour, permissions, mod API, CLI quirks). Coverage report is a criterion. + - Citers: capitalize, close, prune-memory, CAPITALIZE steps of feat/bugfix/hotfix/ship-feature/commit-change, lib/capitalize-commit.md, lib/analyze-before-plan.md, lib/reconcile.sh (Meta status first, legacy fallback: fixes BLK-032..034 false-open), templates/memory, onboard/init-project; doctrine-citers census. CLAUDE.global.md Memory registries: write only via lib/memory.sh; fix the false "load every session" premise (keep caveman: it keeps reads and injections small). + - **4b capture (mod).** Failure ledger: every tool result with isError, deny or non-zero exit carrying a distinctive error line → normalized + verbatim + command, kept per session in `$.store`, matched against the registry error index; unmatched = new failure modes. Milestones observed: feat/fix commit, `gitflow finish`, a suite red then green. `memory_pending` tool (`isDeferred: false`) returns unmatched failures + milestones verbatim; /capitalize, /close and every CAPITALIZE step call it first; one context note at the next prompt after a milestone, never per turn, never at Stop. Judgment triggers (design choice, surprising discovery, dead end) stay with the model: "Proactive capitalization" paragraph KEPT. Writing always = model draft + user approval + lib/memory.sh (validated). + - **4c read side (mod).** Tier 1 always-on: ≤10 user-marked `critical=yes` lessons, one line each (~1k chars) in `prompt.context`. Session digest in `prompt.context` (open BLK from Meta status, actionable TODO, last journal line; re-read by the engine on compaction and /clear). Intent recall at `prompt.submit` only for model-router classifier labels design/analysis or the first coding prompt after idle: BM25 top-30 over title + Apply + triggers + body → haiku keeps ≤3 (option: same call as the classifier, no added latency), bounded 3 s, fail open. Execution recall: Read path triggers (lands before the edit), Bash command triggers (after the result; `critical` → deny once with the Apply line, retry allowed), tool error 4-gram match. Injection = ID + title + Apply only; superseded entries never injected (successor shown); linked IDs as one-line see-also; EVAL excluded from auto recall; budget ~1.5k chars per turn; once per context (ledger reset on compact/clear). Config-repo entries with scope=global indexed in every project (repo resolved from the ~/.claude/lib link). `memory_search` tool for skills (analyze-before-plan pass 1 calls it). + - **4d feedback and promotion.** Log each injection (ID, trigger kind, turn) and whether the turn cited the ID or the retried action succeeded; /prune-memory reads it: never-recalled entries, noisy entries (injected often, never used) get triggers fixed; entries recalled ≥N times → proposed promotion to a mechanism (guard, test, skill step), BDR-100 "mechanisms, not lessons". + - Proof: validator zero false positive on the 8 projects; writer round-trip; backfill coverage; recall precision on ≥20 real prompts and ≥10 real errors (journal + transcripts), target ≥ the 14/22 baseline; digest canary after a compaction; haiku rerank latency inside the mod measured; `/context` delta; captured-failure demo: an error from session N recalled by match in session N+1. + - Cleanup: hand-maintained Index steps in skills, "add one row to the Index" instructions, ad hoc append snippets, the false doctrine premise, reconcile legacy-only status parsing once backfill lands. - [ ] **Phase 5, per-project profiles, native and concurrent (user ask 2026-10-24)** - Today `lib/profile.sh` is global (symlinks skills/ ↔ skills-disabled/, `claude plugin enable|disable` user scope): every open session shares one profile. Target: each session gets its own profile, so web in one project, full in another, backend in a third, at the same time.