From fb6cbf1e34848e7d518934e0bfde14a541c947a1 Mon Sep 17 00:00:00 2001 From: bchanot Date: Wed, 4 Nov 2026 19:15:34 +0100 Subject: [PATCH] docs(spec): phase 4c memory recall design, contract and spike --- .../2026-11-04-memory-recall-1859.md | 48 ++++++ .../2026-11-04-memory-recall-spike/REPORT.md | 27 +++ .../specs/2026-11-04-memory-recall-design.md | 158 ++++++++++++++++++ 3 files changed, 233 insertions(+) create mode 100644 .claude/tasks/contracts/2026-11-04-memory-recall-1859.md create mode 100644 .claude/tasks/plans/2026-11-04-memory-recall-spike/REPORT.md create mode 100644 docs/superpowers/specs/2026-11-04-memory-recall-design.md diff --git a/.claude/tasks/contracts/2026-11-04-memory-recall-1859.md b/.claude/tasks/contracts/2026-11-04-memory-recall-1859.md new file mode 100644 index 0000000..fb07ffa --- /dev/null +++ b/.claude/tasks/contracts/2026-11-04-memory-recall-1859.md @@ -0,0 +1,48 @@ +# CONTRACT — memory-recall (phase 4c) +- date: 2026-11-04 | flow: ship-feature | branch: feature/memory-recall (off develop 06d2576, created at STEP 4) +- status: active + +## REQUEST (verbatim — IMMUTABLE) +Phase 4c of the context mod queue (BDR-125), TODO.md "Phase 4b-4d, memory v2 capture, read, feedback" — the 4c read side: two channels only: intent recall at `prompt.submit` for classifier labels design/analysis or the first coding prompt after idle (BM25 top-30 over title + lesson + body → haiku keeps ≤3, bounded 3 s, fail open) and tool-error 4-gram match (PostToolUse; a classic hook is acceptable if the mod ledger is not needed). Session digest (open BLK via the status parser, actionable TODO, last journal line) in `hooks/session-start.sh` on `source: startup|compact|clear`; a SubagentStart twin if sub-agents need it. Injection = ID + title + lesson line; superseded never injected; budget ~1.5k chars per turn; once per context. Deferred: Read path triggers, Bash command triggers, "critical → deny once", Tier 1, cross-repo index, `memory_search` tool. Plus the 4b follow-ups queued for 4c: earlier-session milestones in `memory_pending`, common-phrase 4-gram collisions (every ENOENT reads known LRN-056), the lexical superseded reader `shortcut:`. Host: mods/memory (BDR-137). 4d stays deferred until an injection log exists. + +(user brief 2026-11-02, step 3: "Phases 4b à 4d"; launched after the 4b merge, user "merge it" 2026-11-04. Session constraints of that brief hold: default model opus; verifier, security-auditor and plan-challengers pinned on opus; never commit a `/model` or UI write of settings.json (LRN-253); plan text through Write/Edit (BLK-039); destructive steps on the live tree in the main loop after the user's go, `mv` over `rm` (BLK-040).) + +## CLARIFICATIONS +Pass A (gaps found by the STEP 0d read-before, put to the user at the STEP 1 brainstorm; answers land `[gated]`): +- The classifier label is private to model-router (`st.turnMain`, never stored or published) and the classifier never runs on prompts its keyword rules already map (`plan|design|architecture…` → plan, `why|explain|review|audit…` → reflect), nor on slash commands: a trigger "classifier labels design/analysis" read literally misses exactly the design/analysis prompts that say so. +- No uniform "lesson line" exists in the registries (BDR `**Decision**` median 643 chars, max 1989; LRN `Apply` / `Future application` / `Pattern`; BLK `Solution` / `Real cause`; EVAL `Action`): three BDR hits overflow the 1.5k budget unless the line is cut. +- Superseded readers diverge (memory.sh parser vs the 4b lexical reader: BDR-031 live for the lexical one, EVAL-041 falsely superseded for the shell one). +- Engine types 2.1.316 expose `$.state` and `$.process.run` (4b recorded "no `$.state`" from the 2.1.277 types); runtime availability unverified for both; mod `session.start` has no `source`, `classic.SessionStart` does. +- The 4-gram collision: `no such file or` / `such file or directory` belong to LRN-056 only, so every ENOENT reads known LRN-056 (document frequency does not fix it). +- Proof needs ≥ 20 real prompts and ≥ 10 real errors labelled for relevance; precision bar = the 14/22 haiku-rerank baseline (BDR-125 amendment 2, TODO.md:114). + +Brainstorm answers `[gated 2026-11-04]` (user: "continue" to recommendations 0-11, read as yes): +0. Spike first (done: `.claude/tasks/plans/2026-11-04-memory-recall-spike/REPORT.md`): `$.process.run` memory.sh index 128 ms; `classic.SessionStart` fires in a mod with source startup/resume/compact and its `additionalContext` reaches the model; `tool.call` error-answer `context` reaches the model; haiku rerank median ≈ 2 s, tail 5.7-7.3 s, ≈ 4k input tokens. +1. Intent-recall trigger without the router classifier: composer prompt, not mid-turn, not a slash command, ≥ 12 chars, AND (first prompt of a context [startup, /clear, compaction] OR first prompt after ≥ 30 min idle OR a design/analysis keyword match, the router's two default rule lists copied with their source commit). +2. Corpus = live BDR + LRN + BLK (EVAL excluded). Lesson line: BDR `Decision`; LRN `Apply` > `Future application` > `Pattern`; BLK `Solution` > `Real cause`; first sentence cut at 280 chars. Injection line = `ID, title (≤ 100), lesson`; ≤ 3 entries; hard cap 1500 chars. +3. Rerank: BM25 top-30 over title + lesson + body; haiku `claude-haiku-5-5`, effort low, 64 max tokens, gets the prompt + 30 candidates as ID + title + lesson, answers IDs validated against the candidates; own 3 s race (LRN-218); failure or timeout → nothing injected (no BM25-only fallback). +4. Superseded reader = lib/memory.sh (single reader, BDR-129) through `$.process.run(['bash', memory.sh, '-C', root, 'index'])` (spike: works, 128 ms); superseded and deprecated excluded; the 4b lexical `shortcut:` reader is replaced by it for the error index too (BDR-031 becomes superseded as memory.sh says). +5. Tool-error channel in the existing Bash hook (the ledger lives in the mod): a captured failure matching a live BLK/LRN entry attaches `memory: known , title, lesson` to the tool result context; an errno stop-phrase list (no such file or directory, permission denied, command not found, …) drops grams wholly inside a stop phrase at index and match time; main loop and sub-agents; noise measured with the 14-day replay before shipping. +6. Once per context: injected IDs kept in the closure per session and agent; reset at `session.end` and on `classic.SessionStart` compact/clear; the 1500-char per-turn budget covers both channels. +7. Digest in `hooks/session-start.sh`: reads the hook JSON on stdin without blocking under `, 'index'], {timeoutMs})` in a hook | works: exit 0, 508 lines (4 registries, `| ID | Date | Title | Status |`), 128 ms | +| `classic.SessionStart` in a mod | fires with `source` startup; `--continue "/compact"` fires `resume` then `compact`; keys: session_id, transcript_path, cwd, source, model, agent_id, agent_type, effort, … | +| `classic.SessionStart` returning `additionalContext` | reaches the model (it quoted `PROBE4C-SESSIONSTART-CONTEXT source=startup`) | +| `tool.call` Bash error answer `{...out, context: [...]}` | reaches the model (it quoted `PROBE4C-TOOL-CONTEXT` as tool-result context); the error answer has no `context` key before the hook adds one | +| `$.model.complete({model: 'claude-haiku-5-5', prompt, maxTokens: 64, effort})` | see below | + +haiku rerank latency (prompt = a design question + 30 candidates; same session, sequential): + +| variant | prompt chars | input tokens | ms (n=3, + 2 earlier full/low runs: 2494, 7332) | output tokens | +|---|---|---|---|---| +| effort low, ~350 chars per candidate | 10 865 | 3 965 | 1725, 2743, 1790 | 225-258 | +| no effort, same prompt | 10 865 | 3 965 | 2063, 5713, 1938 | 224-302 | +| effort low, ~150 chars per candidate | 4 865 | 2 045 | 2921, 2060, 2755 | 326-394 | + +- Median ≈ 2 s; tail 5.7-7.3 s (2 of 11 calls over 5 s, 3 of 11 over 2.7 s). A 3 s bound drops roughly one call in five on this sample. +- `maxTokens: 64` does not cap output tokens (220-400 reported): the model thinks; the request type has no thinking switch. A shorter candidate list does not cut latency. +- Answers were valid candidate IDs every time. +- Cost per recall ≈ 4k input + 0.3k output haiku tokens. + +Consequences for the design: decision 4 (superseded reader) = memory.sh through `$.process.run` (single reader, 128 ms, no port); the compact/clear ledger reset hooks `classic.SessionStart`; the 3 s bound is the user's number, its timeout share is measured by the proof harness on the 20 real prompts and put to the gate. diff --git a/docs/superpowers/specs/2026-11-04-memory-recall-design.md b/docs/superpowers/specs/2026-11-04-memory-recall-design.md new file mode 100644 index 0000000..c3b2725 --- /dev/null +++ b/docs/superpowers/specs/2026-11-04-memory-recall-design.md @@ -0,0 +1,158 @@ +# Phase 4c memory recall — design (2026-11-04) + +BDR-125 phase 4c, host `mods/memory` (BDR-137). Contract +`.claude/tasks/contracts/2026-11-04-memory-recall-1859.md` (brainstorm answers +0-11 `[gated 2026-11-04]`). Engine facts: +`.claude/tasks/plans/2026-11-04-memory-recall-spike/REPORT.md` (CLI 2.1.318). +Branch `feature/memory-recall`. + +## Goal +The registries hold the lessons; nothing brings them back when they matter. +4c brings back the right entries at three moments, without noise: when a +design or analysis prompt starts work, when a known error comes back, and at +the start of every context. Writing is untouched (4b capture, memory.sh). +Success: precision of the injected entries at or above the 14/22 baseline +(64 %), the error channel's noise measured before shipping, a digest seen +after a real compaction. + +## Unit 1 — shared reader (register.ts) +- Status: `lib/memory.sh -C index` through `$.process.run(['bash', + /.claude/lib/memory.sh, '-C', root, 'index'], {timeoutMs: 2000})`, + raced by an own 2.5 s timer; rows `| ID | Date | Title | Status |`; an ID is + closed for recall when its status word is `superseded` or `deprecated` + (memory.sh is the single reader, BDR-129; spike: 128 ms). The script path is + `$HOME/.claude/lib/memory.sh` (the installed lib, the same one the skills + call). A failed or timed-out read → the status map is unavailable for that + call: recall injects nothing, the error channel injects nothing, and + `memory_pending` answers as today with an `index: status reader unavailable` + line and every failure `NEW`. +- Entries: the registry files `decisions.md`, `learnings.md`, `blockers.md` + read with `$.fs.read` (stat first, 4 MiB cap per file, as 4b `loadIndex`); + the 4b `entriesOf` heading regex extends to `BDR|LRN|BLK`; each entry keeps + `{id, kind, title, lesson, body}`. +- Lesson line: BDR the `Decision` field; LRN `Apply`, else `Future + application`, else `Pattern`; BLK `Solution`, else `Real cause` (label match + case-insensitive, bold or plain, `- **Label**:` / `- Label:`); its first + sentence, cut at 280 chars with `…`; an entry without one has the empty + lesson (title only). +- The 4b lexical `isSuperseded` and its `shortcut:` comment are removed: the + error index (BLK + LRN spans, as 4b) takes its live set from the same status + map. + +## Unit 2 — intent recall (`prompt.submit`) +- Trigger: origin `composer`, `turnId` absent, not a slash command (text not + starting `/`), ≥ 12 code points, and one of: the first such prompt of the + context (set at `session.start` and on `classic.SessionStart` source + `clear` or `compact`, consumed by the first qualifying prompt); the first + after ≥ 30 min without a composer prompt in this session; a match of the + design or analysis keyword patterns, copied from model-router's default + prompt rules (`mods/model-router/hooks/register.ts`, commit 9bbd1d5): + `plan|planifie|planning|brainstorm|architecture|con[cç]ois|design` and + `pourquoi|why|explique|explain|analyse|analyze|comprendre|understand|review|audit`, + same word boundaries. +- Rank: BM25 (k1 1.2, b 0.75) over title + lesson + body of the live + BDR/LRN/BLK entries, tokens = lowercase letter/digit runs of ≥ 2 chars minus + a short FR+EN stop-word list; the query is the prompt's first 2000 chars, + redacted (4b `redact`). Entries already injected in this context are + removed before ranking. Top 30 with a score > 0. +- Rerank: `$.model.complete({model: 'claude-haiku-5-5', effort: 'low', + maxTokens: 64, timeoutMs: 3000, prompt})`, raced by an own 3 s timer + (LRN-218). The prompt: the redacted query, the candidates one per line as + `ID | title | lesson`, and the instruction to answer at most 3 IDs, comma + separated, or `none`. The answer is parsed for `(BDR|LRN|BLK)-\d+`, kept + only when the ID is a candidate, at most 3, order kept. +- Inject: one context block `memory recall:` then one line per entry + ` — <lesson>`; the block ≤ 1500 chars (an entry that + would overflow is dropped). No candidate, a `none`, a timeout, a refusal or + any error → nothing attached, the prompt passes unchanged. +- The 4b milestone note on the same prompt is attached first; recall follows. + +## Unit 3 — tool-error recall (Bash `tool.call`) +- After the 4b capture (same failure rule, same redacted line): the failure's + key is matched against the error index (live BLK/LRN, Unit 1); a match whose + ID was not injected in this context attaches `memory: known <ID> <title ≤ + 100> — <lesson>` to the tool answer's `context` (the answer keeps its shape, + LRN-205; spike: the model sees it). +- Common phrases: before gramming, both the entry spans and the failure key + replace each errno phrase by the placeholder `<errno>`; a gram made only of + placeholders is skipped (LRN-258, `<errno>` joins `#`, `<path>`, `<hex>`). + Phrases: no such file or directory, permission denied, command not found, + is a directory, not a directory, file exists, directory not empty, + operation not permitted, no space left on device, connection refused, + connection timed out, broken pipe, resource temporarily unavailable, + invalid argument, too many open files, read-only file system, device or + resource busy, input/output error. +- Main loop and sub-agents (the context key carries the agent id). + +## Unit 4 — ledger, budget, log +- Once per context: `st.seen: Map<contextKey, Set<ID>>`, key = session id + + (`agentId` or `main`); cleared at `session.end`, and the session's keys on + `classic.SessionStart` source `clear` or `compact`. +- Budget: the main loop has 1500 chars per turn, shared by both channels, + reset by each composer prompt; a sub-agent (error channel only) has 1500 + chars for its whole context. +- Log: each injection appends `{at, session, agent, channel: 'prompt' | + 'tool', ids}` to the project's store value (`log`, last 200 kept, 4b write + chain and size caps); never displayed; 4d reads it. + +## Unit 5 — session digest (`hooks/session-start.sh`) +- Reads the hook JSON from stdin only when stdin is not a terminal, with + `timeout 1`; `source` taken with jq when present, else empty. Prints the + digest for `startup`, `clear`, `compact`, and for an empty source (a manual + run); never for `resume`. +- Only in a project whose git toplevel (else cwd) has `.claude/memory/`: + `memory digest:` then up to three kinds of lines: + open BLK from `reconcile_blk_open` with status word `open` (`BLK-n title`, + ≤ 160 chars each, at most 3, then `+N more`), the upstream ones counted + (`+N upstream`); the first 3 unchecked `- [ ]` items under + `### Actionable` in `.claude/tasks/TODO.md` (≤ 160 chars each); the last + `- ` line of `.claude/memory/journal.md` (≤ 160 chars). Block ≤ 800 chars. + Any missing file or section → that line is omitted; nothing to show → no + block. Fail silent: an error in the digest never changes the banner or the + exit code. +- No SubagentStart twin. + +## Unit 6 — `memory_pending` +Milestones of the earlier stored sessions follow the current session's, +marked `(earlier session)`, same 3-session window; the answer order stays +header, `index:`, milestones, failures. + +## Unit 7 — off switch +`~/.claude/memory.json`: `"enabled": false` cuts everything (4b); +`"recall": false` cuts Units 2 and 3 injections only (capture, the note, the +tool and the log of past injections stay). Read like `enabled` (lazily per +session, a non-boolean logs one line and counts as absent). + +## Docs +README / USAGE (fr) memory sections gain recall, the error channel, the +digest, `"recall": false`, the residuals; CHANGELOG `[Unreleased]`; TODO (4c +ticked at merge, 4d stays deferred with the log now in place). + +## Proof +- Kit tests: reader (status map, lesson per kind, failure → fail open), + trigger (first prompt, idle, keyword, slash and mid-turn excluded), BM25 + order on fixtures, rerank parse (foreign IDs dropped, `none`, timeout, never + settles), budget and once-per-context (reset on compact/clear and + session.end), error channel (known match attaches, ENOENT does not, sub-agent + key), earlier milestones, `recall: false`, log cap; regex costs timed on + 64 KiB (LRN-259). +- Shell suite: the digest on fixture projects (sources, missing files, caps, + `</dev/null`), existing session-start callers unchanged. +- Measured before shipping (LRN-261, EVAL-056): a bun harness under + `.claude/tasks/plans/2026-11-04-memory-recall-spike/` imports the mod's own + functions (no port) and calls haiku through `claude -p --model haiku`: + (a) 20 real composer prompts from 14 days of this repo's transcripts that + meet the trigger: injected IDs per prompt, timeout share at 3 s; the + controller labels each injected entry relevant or not, the user checks the + labels; bar: relevant share ≥ 64 % (14/22). (b) the 14-day Bash failure + replay: injections per session and the noise share on the injected ones, + ENOENT lines included. Counts and shares only in the committed notes. +- Live: headless `-p` for the error channel (a failing command matching a + fixture BLK attaches the line); interactive by the user for recall (a design + prompt in a fresh session), and the digest after a real `/compact`. + +## Out of scope +Read path triggers, Bash command triggers, critical deny-once, Tier 1, +cross-repo index, `memory_search` tool, reading the router's classifier +label, a SubagentStart twin, injection citation tracking (4d), EVAL in the +recall corpus.