From 2a0d73da120fc62ecf7d69db6aa88ab79b732ea4 Mon Sep 17 00:00:00 2001 From: bchanot Date: Thu, 5 Nov 2026 17:51:19 +0100 Subject: [PATCH] docs: memory recall, known-error line and session digest --- .claude/tasks/TODO.md | 8 +- ARCHITECTURE.md | 4 +- CHANGELOG.md | 1 + README.md | 9 +- USAGE.md | 9 +- .../specs/2026-11-04-memory-recall-design.md | 97 +++++++++++-------- mods/memory/.claude-plugin/plugin.json | 2 +- mods/memory/hooks/register.ts | 2 + 8 files changed, 84 insertions(+), 48 deletions(-) diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 5a9e5d6..b99fe67 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -114,10 +114,12 @@ Shared rules for every phase: - Measured recall (5 real FR prompts, 22 relevant entries): BM25 full text 9/22, haiku rerank of lexical top-30 14/22 (~8.5k chars). Error 4-gram match on real errors: BLK-030/031/034 exact, no false positive; 5 of 7 real errors of 2026-10-23 had NO entry: capture is the weak link. - **4b capture (mod).** Failure ledger: tool results with isError, deny or non-zero exit carrying a distinctive error line → normalized + verbatim + command, REDACTED at capture (rtk-rewrite redaction + gitleaks patterns), bounded ring per session in `$.store` (4 MiB cap), matched against the registry error index; unmatched = new failure modes. Milestones: feat/fix commit, `gitflow finish`, a suite red then green. `memory_pending` tool returns unmatched failures + milestones; /capitalize, /close and every CAPITALIZE step call it first; one context note at the next prompt after a milestone, never per turn, never at Stop. "Proactive capitalization" paragraph KEPT. Writing always = model draft + user approval + lib/memory.sh. - Built 2026-11-04 on feature/memory-capture (f1d4f9d..HEAD): mod `mods/memory`, kit 100+ cases, consumers wired, noise measured (NOISE.md); merged 2026-11-04 into develop (dfadf8a): GATE 0 MET, verifier criteria 1-15 met (9 proven live), security PASS ×4, BDR-137 - - shortcut: `mods/memory/hooks/register.ts` `isSuperseded` is a lexical reader, not memory.sh's status parser; upgrade = share that parser when 4c needs it (final review M11) - - **4c read side (next).** Two channels only: intent recall at `prompt.submit` for classifier labels design/analysis or the first coding prompt after idle (BM25 top-30 over title + lesson + body → haiku keeps ≤3, bounded 3 s, fail open) and tool-error 4-gram match (PostToolUse; a classic hook is acceptable if the mod ledger is not needed). Session digest (open BLK via the status parser, actionable TODO, last journal line) in `hooks/session-start.sh` on `source: startup|compact|clear` (native matchers; the mod's `session.compact` only skips); a SubagentStart twin if sub-agents need it. Injection = ID + title + lesson line; superseded never injected; budget ~1.5k chars per turn; once per context. Deferred: Read path triggers, Bash command triggers, "critical → deny once", Tier 1, cross-repo scope=global index, `memory_search` tool. + - [x] shortcut (done in 4c, Task 1: the lexical reader is gone, statuses come from `memory.sh index`): `mods/memory/hooks/register.ts` `isSuperseded` is a lexical reader, not memory.sh's status parser; upgrade = share that parser when 4c needs it (final review M11) + - **4c read side (built 2026-11-05 on feature/memory-recall, merge pending).** Two channels only: intent recall at `prompt.submit` for classifier labels design/analysis or the first coding prompt after idle (BM25 top-30 over title + lesson + body → haiku keeps ≤3, bounded 3 s, fail open) and tool-error 4-gram match (PostToolUse; a classic hook is acceptable if the mod ledger is not needed). Session digest (open BLK via the status parser, actionable TODO, last journal line) in `hooks/session-start.sh` on `source: startup|compact|clear` (native matchers; the mod's `session.compact` only skips); a SubagentStart twin if sub-agents need it. Injection = ID + title + lesson line; superseded never injected; budget ~1.5k chars per turn; once per context. Deferred: Read path triggers, Bash command triggers, "critical → deny once", Tier 1, cross-repo scope=global index, `memory_search` tool. - 4c follow-up from the 4b live proof (2026-11-04): `memory_pending` lists milestones of the current session only, so a `/capitalize` run right after `/clear` misses the previous session's `fix:`/finish milestones (they stay in the store). Option: list earlier-session milestones marked `(earlier session)`. Doctrine (capitalize before /clear) covers the normal path. - - **4d feedback and promotion.** DEFERRED until a plain injection log (ID, trigger kind, turn, cited or not) shows misses or noise; then /prune-memory reads it and promotion to a mechanism follows BDR-100. + - **4d feedback and promotion.** The injection log now exists (4c, store key `l:`); 4d waits on its data. DEFERRED until a plain injection log (ID, trigger kind, turn, cited or not) shows misses or noise; then /prune-memory reads it and promotion to a mechanism follows BDR-100. + - [ ] shortcut: the injection log lives under `l:` store keys with no cross-root cap (each key holds 200 records and 64 KiB); upgrade = a global cap or sweep of stale roots when the store nears its 1 MiB cap (`mods/memory/hooks/register.ts`). + - [ ] residual: the pre-existing `memory.sh check` call in `hooks/session-start.sh` is not time-bounded (the digest's `memory.sh index blocker` call is); upgrade = wrap it in `timeout 2`. - Proof: recall precision on ≥20 real prompts and ≥10 real errors, target ≥ the 14/22 baseline; digest after a real compaction; redaction test on the ledger; `/context` delta; captured-failure demo: an error from session N recalled in session N+1. ### Actionable diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ed7b9d4..e06c19f 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -20,7 +20,7 @@ claude-config/ ├── update-all.sh # One-command update for all components ├── Makefile # Unified entry point: make install / doctor / update / test (make help) ├── plugins.lock.json # Version pinning for non-marketplace dependencies and vendored skills -├── hooks/ # Claude Code hooks: session start, statusline, RTK rewrite, ctx7 + design-toolchain reminders, attention notify, unpushed-work guard, manual-mode push guard, system-dir extraction guard, instruction-load log, code-limits line-width signal +├── hooks/ # Claude Code hooks: session start, statusline, RTK rewrite, ctx7 + design-toolchain reminders, attention notify, unpushed-work guard, manual-mode push guard, system-dir extraction guard, instruction-load log, memory digest at session start, code-limits line-width signal ├── githooks/ # Generated git hooks (pre-commit, post-commit, post-merge, reference-transaction), git's global core.hooksPath ├── .githooks/ # This repo's own copy of the same hooks ├── .claude/rules/ # This repo's project path rules (paths: frontmatter), loaded on Read of a matching file @@ -38,6 +38,6 @@ claude-config/ - `skills/` = entry points you invoke via `/skill-name` - `agents/` = execution units called by skills (never invoked directly by user) -- `mods/` = Claude Code mods (function-hooks plugins); each loads through the tracked symlink `skills/` as `@skills-dir`, live at the next session; `mods/model-router/routing.json` (tracked) holds the phase table (tier, effort and the `about` line the first-use dialog shows), every skill and agent row and the first-use decisions; the mod also redirects a main-loop code write made outside a skill run to a skill (README, "Write guard"); `mods/memory/` is the capture mod (Bash failures and milestones in the plugin store, read by `memory_pending`) +- `mods/` = Claude Code mods (function-hooks plugins); each loads through the tracked symlink `skills/` as `@skills-dir`, live at the next session; `mods/model-router/routing.json` (tracked) holds the phase table (tier, effort and the `about` line the first-use dialog shows), every skill and agent row and the first-use decisions; the mod also redirects a main-loop code write made outside a skill run to a skill (README, "Write guard"); `mods/memory/` is the capture mod (Bash failures and milestones in the plugin store, read by `memory_pending`) that also recalls registry entries and known errors - `templates/` = symlinked to `~/.claude/templates/` — copy into projects via `/onboard` or manually - **Graphify** builds a knowledge graph of any codebase (`/graphify query`), producing a navigable wiki in `graphify-out/wiki/`. This map helps Claude understand project structure, find relevant code faster, and reason across files. Essential for large-scope tasks (multi-file features, complex bugs, architectural changes). Small tasks should skip it and read files directly. Proposed only from 200 tracked code files: the session-start banner informs, the user decides; nothing builds a graph without that go. Where `graphify-out/graph.json` exists, the session-start hook prints the graphify usage rules. diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ea7201..5320979 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and this project - **Secure-by-default guardrails**: `rules/secure-by-default.md` (always on) makes six domains baseline on every project (the Supabase and cloud-function detail lives in the path rules `rules/supabase.md` and `rules/cloud-functions.md`), plus systematic API and access hygiene (auth on every route, CORS, secrets, passwords hashed with argon2id or bcrypt, never compared in clear, logged or hardcoded): hosting (rate limit, CDN and firewall, hard spend cap), Supabase (RLS on every table, owner-scoped policies, no service role key client-side), SMS and email (recorded consent, unsubscribe, STOP), cloud functions (no self-trigger, instance cap, budget stop), injection (bound values in raw escapes, `format()` for dynamic SQL, allowlisted filters, no shell into a DB tool), exposure (no data in the console, no production source map, no debug flag, no secret in hydrated state, data attributes, hidden fields or shipped comments). `/guardrails` audits them: a stdlib Python scanner (`skills/guardrails/scripts/scan.py`, FAIL / WARN / MANUAL, exit 1 on FAIL) then judgment, report in `.claude/audits/GUARDRAILS.md`. The security-auditor gate runs the scanner's `api` and `supabase` domains and blocks on FAIL. Items only a provider dashboard can set stay MANUAL, never claimed done. An optional read-only live probe (`skills/guardrails/scripts/probe.py`, `--live`) confirms a finding from outside on a target the user owns and names: anon-read, cross-user, no-auth route, CORS reflection and a capped rate-limit burst, verdicts OPEN / CLOSED / SKIPPED / ERROR. An `exposure` block in the plan adds three read-only probes, meant for a production build or a staging URL: a served or inline source map for the page's scripts, a stack trace in a 404 or a user-named error route, sensitive keys in a JSON response (identifier-shaped names printed, never values); a target the probe could not observe is SKIPPED or ERROR, never CLOSED, and every plan path must start with `/`. It sends GET, HEAD and OPTIONS only (one opt-in POST burst, capped at 25), requires `--authorized`, takes secrets from the environment never the command line, never follows a redirect to another host, and stops at a request budget. After the report, `/guardrails` proposes fixes (plan phase, so Fable when the router can switch), applies only the ones the user accepts through the `bugfixer` executor and the `lib/verify-secure-loop.md` gates, then re-scans: each accepted fix is a contract criterion whose oracle is the scanner (`--assert-absent RULE:path`, marker `GUARDRAILS-CLEAR`), and a full pass against the pre-fix baseline (`--baseline`, marker `GUARDRAILS-NO-NEW`) catches regressions; a live probe replays on the user's go; two rounds at most, residuals reported open; `--report-only` keeps the audit alone. Suites: `lib/tests/guardrails-scan.test.sh` (clean fixture plus one mutant per rule, check modes, `gates.sh` integration) and `lib/tests/guardrails-probe.test.sh` (probe logic against a localhost fixture server). - **/a11y skill**: accessibility leaves `/guardrails` and becomes its own skill. Same A11Y-* rules (img alt, field label, click without keyboard, focus outline removed, positive tabindex) on the shared scan engine (`skills/guardrails/scripts/scanner.py`, a `Catalogue` per CLI; `skills/a11y/scripts/scan.py`, markers `A11Y-CLEAR` / `A11Y-NO-NEW`), a Tab-only walk of the main flow on `--live` through `/browse` (marked not done otherwise), report `.claude/audits/A11Y.md`, fixes proposed with the alt and label wording as the user's choice, applied through the bugfix circuit (`bugfixer`, verify-secure gates, `bugfix/a11y-`), re-pass against a baseline. The keyboard, alt and label doctrine moves from `rules/secure-by-default.md` to the lazy `rules/a11y.md` (loads on UI files only; `rules/web-building.md` points at it). `/guardrails` scans `injection` (INJ-*) and `exposure` (EXP-*) instead, with per-language SQL patterns, shared `code_files` / `hits` / `dedupe_lines` / `client_reachable` helpers and a walker that lists files over 1 MB by path; `--domain a11y` is a usage error there. `/web-validate` keeps full WCAG conformance. Suite `lib/tests/a11y-scan.test.sh`; routing row `reflect`; a top-level skill in no pack, so on under every profile. - **memory mod**: `mods/memory/`, a Claude Code mod, captures Bash failures (`Exit code N` whose last line is error-shaped (a word like error, fatal, cannot, denied, a shellcheck code), redacted at capture (env secrets, vendor key prefixes, `--token` flags, API-key headers and cookies, JSON/YAML password/token/secret pairs, a 40-hex URL user; residual `user:p@ss@host` partial), sub-agent ones marked) and milestones (a `feat`/`fix` commit, a `gitflow.sh finish`, a `make test` red then green) in the plugin `$.store` (3 sessions per project, 50 failures each, 1 MiB). One context note at the first prompt after a milestone; the `mcp__memory__memory_pending` tool lists milestones and failures as `NEW` or `known ` against the BLK/LRN entries, and `/capitalize`, `/close` and the CAPITALIZE steps call it first. It never writes a registry. Off with `{"enabled": false}` in `~/.claude/memory.json`. Residuals: concurrent sessions last writer wins, non-Bash failures not captured, noise 83 % measured before the filter, cut by two thirds (`NOISE.md`). +- **memory recall**: the memory mod hands registry entries back. A composer prompt of 5 content words or more that opens a context, follows 30 minutes idle or holds a design or analysis word gets up to 3 live BDR, LRN or BLK entries (BM25 top 30, one haiku rerank bounded to 5 s), and a Bash failure matching a live BLK or LRN entry gets a `memory: known ` line; 1500 characters per turn, each entry once per context, sub-agents on the error channel only. `hooks/session-start.sh` prints a `memory digest:` (open BLK from `memory.sh index blocker`, 3 Actionable TODO items, last journal line, 800 characters, TODO dropped first) on startup, clear and compact. `"recall": false` in `~/.claude/memory.json` cuts the injections; each one is logged under the `l:` store key for phase 4d. Measured precision 54 to 61 % against the 64 % bar, so recall ships switchable. - **model-router mod**: `mods/model-router/`, a Claude Code mod (function-hooks plugin), routes every repo skill and agent from phase rows (`plan`, `reflect`, `orchestrate`, `escalate`, `judge`, `audit`, `implement`, `write`, `verify`, `explore`, `apply`, `mechanical`; built-ins Explore on sonnet/medium, Plan on opus/xhigh). A typed skill routes the main loop to its row, and a best-tier row holds across turns in a run slot until `/route clear`, `/route off`, a user `/model` or a typed skill on a non-best row. Agents get their row's model at spawn (within the tier, upward only) and its effort on every step; explicit Agent params win. Orchestrators declare their phases through the `mcp__model-router__route` tool. `ultrathink` in a prompt sets the turn's minimum effort and `/route effort=max` holds until `/route clear`; the built-in `/effort` is not a lever inside a run. `/route show` names the run slot (`main: run `), and a `null` row in the override drops a default row. The model gets a `route` tool and the user a `/route` command (`show|clear|off|on|reload|pending|ask on|off|forget ||model= effort=|switch on|off|verbose on|off`). Optional per-machine config `~/.claude/model-router.json`, where `"enabled": false` turns it off on that machine. The spinner suffix and the status line show the route in force. It loads in every session through the tracked symlink `skills/model-router` (`model-router@skills-dir`). `make doctor` gains a Mods section; suite `make test suite=lib/tests/mods.test.sh`. Known limits: a main-loop switch (a `/model` hand pick excepted) runs only while the cold read it costs (the context minus what the target model's prompt cache holds) fits `upgradeMaxTokens` (200000) or `downgradeMaxTokens` (120000), and a downgrade also needs `mainModelSwitch` (on by default, off on any non-boolean value); the hooks send full model ids, so the `models` table has to follow new versions. - **model-router first-use confirmation**: `mods/model-router/routing.json` (tracked) is the single source of the phase table and of every skill and agent row, and keeps the decisions: `confirmed`, `changed` (`from`/`to`), `projects` exceptions keyed by the origin remote reduced to `host/path` (credentials and local paths never stored), and `ask`. The first use of a rowed typed skill, a rowed agent spawn or a main-loop phase declared through the `route` tool opens a dialog with context: the skill's description (first sentence of its `SKILL.md` frontmatter) or the agent's, the phase with its `about` line (a new field on each of the 12 phases) and the model id and effort the next step really runs on. A row offers Later, Keep or Change; a main-loop phase Later or Keep. Change asks the model (fable, opus, sonnet, haiku with their tier), then the effort among those the phases of that model use (skipped when there is only one), then Everywhere or This project only; the pair maps to an existing phase (rows stay phase names, the same phase counts as Keep, a pair no phase offers is added by hand as a new phase). A skill-row change toasts the model and effort it now runs on, with the `/route switch on` hint when the main loop holds back a downgrade; a free-text answer counts as Later. One dialog at a time, never in headless (`-p`) runs or inside a sub-agent. Only a dialog answer, `/route ask on|off` or `/route forget` (which takes decisions back; a changed row returns to its shipped phase, the frontmatter floors stay yours to realign) writes the file (serialized, 64 KiB cap, never created when absent); each write asks you to commit it from the config repo. `/route pending` lists unconfirmed rows and phases. Layers: routing.json < `~/.claude/model-router.json` (its `ask` wins); a project's `.claude/model-router.json` is never read. Tests: `mods/model-router/hooks/register.test.ts`. - **model-router `/model` scope**: a `/model ` typed in the composer, or a pick in the `/model` picker, with a model the table knows asks where it applies. The picker's switch is held first (PreModelSwitch deny; the engine prints that a hook blocked it); a pick while another dialog is open is refused with `answer the open dialog first`. This project writes the full id as `model` into the project's `.claude/settings.local.json` (created if absent, refused through a link) and sets a sticky model route. This conversation sets the sticky route only. Everywhere runs the native `/model` (user `settings.json`). Both non-native answers drop the run slot and keep a sticky effort. With the picker's Default, an unknown name, a headless run or another origin, the native command runs with no question. A model chosen in that dialog always switches: it bypasses the cold-read caps and the `mainUpgrade` / `mainModelSwitch` flags, and while it is marked down or outside a known window it waits and the automatic chain decides. A native switch or `/clear` drops it; `/route model=` keeps the caps. Known limits: the `/config` Model row still writes user `settings.json`; a dialog never answered has no timeout. Tests: `mods/model-router/hooks/register.test.ts`. diff --git a/README.md b/README.md index be00e4d..53233b0 100644 --- a/README.md +++ b/README.md @@ -161,9 +161,14 @@ Limits: a main-loop switch re-reads cold the part of the context the target mode - Milestones: a successful `feat` or `fix` commit (plain `-m` or heredoc form), a `gitflow.sh finish` (not `finish chore`), a `make test ...` command red then green. Milestones are recorded on the main loop only: sub-agent commits and finishes are ignored. - Store: the plugin's `$.store`, keyed by project root: the 3 most recent sessions per project, 50 failures per session, 2 KiB per entry, 1 MiB for the whole store. It survives `/clear`. The store is the plugin's JSON file under the Claude configuration directory (observed here as `~/.claude/plugins/store/memory_skills-dir-.json`, an engine layout, not fixed by this repo); delete it to wipe the captures. - Note: one context line, `memory: