From 60f28a66a7de5474dc796a0736f91fb8f2980aa4 Mon Sep 17 00:00:00 2001 From: bchanot Date: Tue, 3 Nov 2026 11:03:54 +0100 Subject: [PATCH 01/43] chore(tasks): phase 3 closed with the user's /context measures; dev tool-search measure queued --- .claude/memory/journal.md | 3 +++ .claude/tasks/TODO.md | 4 +++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index de0a346..085d149 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -686,3 +686,6 @@ rules: - phase 5 profiles+packs shipped on feature/phase5-profiles-packs (13 tasks SDD, 3-lens challenge, gates MET/CONFORME/PASS after 3 passes): 9 packs, 9 profile fragments, profile.sh one-jq current --plain, cc [profile], census; BDR-134, LRN-248..253, BLK-039/040, EVAL-054; TODO: router ask-before-Fable queued - BDR-125 re-verification (gates x5, 31 SHAs, reconcile engine): 0 code defect, 5 done-but-open ticks → chore/todo-reconcile-bdr125 e58a617 (merge on signal); EVAL-055, LRN-255. /feat router-ask-tier (feature/router-ask-tier): plan r1 FATAL x2 → r3, feater 28 TIER tests, GATE 0 MET, verifier CONFORME 11/11, security PASS (1 LOW fixed); 9a24a62 + docs 789fd00; BDR-135, LRN-254. Merge on signal. Next: phase 2 guards via /ship-feature. - Phase 2 guards (feature/phase2-guards): bench 16 ask + 7 deny approved (quoted finish rule, gitflow_finish*, bash -s* not), rtk rc 3 passthrough, router write guard (skill latch), code-limits hook, settings via index, release finish in main loop. Plan r1 FATAL x2 → r3; SDD 11 tasks; final review fix wave; gates: verifier ECARTS → human rulings (16 replaces 11, criteria 1/5/7 amended, USAGE in scope), security PASS (MEDIUM rtk read -n .env queued). BDR-136, LRN-256/257, EVAL-056, BLK-018 resolved. Merge on user signal; /close click after merge. + +## 2026-11-03 +- Phase 3 closed, not built (user /context 2026-11-03: full 40k, dev 38.4k, web 40k; connector instr have a native lever, jit-subtractable doctrine ~1-2k). dev System tools 11.5k vs 7.6k full: tool-search threshold suspected, queued. Phase 2 merged 29624d7. Next: phase 4b capture on feature/memory-capture. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 3ba8ac4..7e22d19 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -104,10 +104,12 @@ Shared rules for every phase: - Proof: evasion matrix per rule (bench), the /close ritual finishes after one click, destructive commands never run live (plugin test with stubs only). - Cleanup: compress the gitflow section only for the forms now enforced. -- [ ] **Phase 3, context mod: CONDITIONAL** (user pick 2026-10-25). Measure the always-on residual after phases 0, 4a, 1, 5 and 2 (`/context` at repo root + the two biggest projects). Build only if a mod can carry conditionally a residual worth its cost (birth, off switch, canaries, doctor line); otherwise close the item with the measurement. +- [x] **Phase 3, context mod: CONDITIONAL — CLOSED 2026-11-03, not built** (measured by the user, repo root, Opus 5.5, after phases 0/4a/1/5/2: full 40k = memory files 8.2k, skills 16.5k, system tools 7.6k, MCP instr 2.4k, agents 2.2k, system 2.2k; dev 38.4k = memory 8.2k, skills 12.7k, system tools 11.5k, MCP instr 0.7k; web 40k. Connector instructions already have a native lever (profile `deniedMcpServers`: 2.4k → 0.7k on dev); the jit-subtractable doctrine left in memory files is ~1-2k (3-5%), not worth a mod birth, canaries, off switch and doctor line.) (user pick 2026-10-25). Measure the always-on residual after phases 0, 4a, 1, 5 and 2 (`/context` at repo root + the two biggest projects). Build only if a mod can carry conditionally a residual worth its cost (birth, off switch, canaries, doctor line); otherwise close the item with the measurement. - If built, scope = `prompt.compose` section deltas for doctrine a carrier already re-delivers (jit-marked, never deleted; mod off = full text; ledger in `$.state`, reset on compact/clear) + `mcp_instructions_delta` for connectors irrelevant in a coding repo. Shared infra (jit-marker subtraction, `$.state` ledger) lives here or in 4c, whichever ships first. - Dropped by the re-challenge: skill listing filter + routing.json `contexts` + census (phase 5 packs + the budget env already give 0 name-only); routing-hint classifier (no measured miss outside code intents the phase 2 guard catches; the "Skill routing" table lines move into our own skills' `description:`, ≤3 lines for vendored aliases stay in CLAUDE.global.md, done in phase 1 cleanup); agent listing rewrite (phase 1 diet); higgsfield deltas (phase 0/5); project fingerprint. `prompt.attachment` kinds (`skill_listing`, `agent_listing_delta`…) exist in the 2.1.306 types but not in the docs text: re-verify at build time. +- [ ] Measure: under profile `dev` System tools rise to 11.5k vs 7.6k on `full` (23 MCP tools vs 180): likely the deferred tool-search threshold no longer trips and tool definitions inline; dev saves 1.6k total instead of ~5k. Check the CLI threshold (ENABLE_TOOL_SEARCH / auto threshold) before tuning the dev profile. + - [ ] **Phase 4b-4d, memory v2 capture, read, feedback** (shrunk 2026-10-25 to the measured channels) - Measured recall (5 real FR prompts, 22 relevant entries): BM25 full text 9/22, haiku rerank of lexical top-30 14/22 (~8.5k chars). Error 4-gram match on real errors: BLK-030/031/034 exact, no false positive; 5 of 7 real errors of 2026-10-23 had NO entry: capture is the weak link. - **4b capture (mod).** Failure ledger: tool results with isError, deny or non-zero exit carrying a distinctive error line → normalized + verbatim + command, REDACTED at capture (rtk-rewrite redaction + gitleaks patterns), bounded ring per session in `$.store` (4 MiB cap), matched against the registry error index; unmatched = new failure modes. Milestones: feat/fix commit, `gitflow finish`, a suite red then green. `memory_pending` tool returns unmatched failures + milestones; /capitalize, /close and every CAPITALIZE step call it first; one context note at the next prompt after a milestone, never per turn, never at Stop. "Proactive capitalization" paragraph KEPT. Writing always = model draft + user approval + lib/memory.sh. From be4c4c49f2f8c4a20b822484c75ea1fdf6bc2c57 Mon Sep 17 00:00:00 2001 From: bchanot Date: Tue, 3 Nov 2026 11:18:12 +0100 Subject: [PATCH 02/43] chore(tasks): phase 4b contract seed and engine spike report --- .../2026-11-03-memory-capture-0100.md | 32 +++++++++ .../2026-11-03-memory-capture-spike/REPORT.md | 27 +++++++ .../probe-register.ts | 70 +++++++++++++++++++ 3 files changed, 129 insertions(+) create mode 100644 .claude/tasks/contracts/2026-11-03-memory-capture-0100.md create mode 100644 .claude/tasks/plans/2026-11-03-memory-capture-spike/REPORT.md create mode 100644 .claude/tasks/plans/2026-11-03-memory-capture-spike/probe-register.ts diff --git a/.claude/tasks/contracts/2026-11-03-memory-capture-0100.md b/.claude/tasks/contracts/2026-11-03-memory-capture-0100.md new file mode 100644 index 0000000..393e563 --- /dev/null +++ b/.claude/tasks/contracts/2026-11-03-memory-capture-0100.md @@ -0,0 +1,32 @@ +# CONTRACT — memory-capture (phase 4b) +- date: 2026-11-03 | flow: ship-feature | branch: feature/memory-capture (off develop 29624d7) +- status: active + +## REQUEST (verbatim — IMMUTABLE) +Phase 4b of the context mod queue (BDR-125), TODO.md "Phase 4b-4d, memory v2 capture, read, feedback" — the 4b capture part: failure ledger (tool results with isError, deny or non-zero exit carrying a distinctive error line → normalized + verbatim + command, redacted at capture, bounded ring per session in $.store, matched against the registry error index; unmatched = new failure modes), milestones (feat/fix commit, gitflow finish, a suite red then green), a `memory_pending` tool returning unmatched failures + milestones, called first by /capitalize, /close and every CAPITALIZE step; one context note at the next prompt after a milestone, never per turn, never at Stop; writing stays model draft + user approval + lib/memory.sh. Designed 2026-10-25 (measured: 5 of 7 real errors had no entry, error 4-gram match exact). 4c (read side) is the next ship-feature; 4d stays deferred. + +(user brief 2026-11-02, step 3: "Phases 4b à 4d". The TODO bullet "4b capture (mod)" at develop 29624d7 is the design seed.) + +Session constraints (same brief): default model opus; verifier, security-auditor and plan-challengers pinned on opus; never commit a `/model` or UI write of settings.json (LRN-253); plan text through Write/Edit (BLK-039); destructive steps on the live tree in the main loop after the user's go, `mv` over `rm` (BLK-040). + +## CLARIFICATIONS +Pass A (gaps found by the STEP 0d read-before, put to the user at the STEP 1 brainstorm; answers land `[gated]`): +- Engine facts that reshape the seed (types 2.1.277 on disk, CLI 2.1.317): a non-zero Bash exit is an `isError` result whose text starts `Exit code N` (no exit-code field; the typed record with stdout/stderr/gitOperation exists on success only); `$.store` is per plugin, machine-wide, cross-project, 4 MiB total (not per session); there is no `$.state`; a registered tool exists from the prompt after `session.start`; sub-agents with a `tools:` allowlist cannot call an MCP tool (commit-changer); gitleaks rules are compiled into the binary (no data to port as-is); `session.id` changes on /clear and resume. +- Spike 2026-11-03 (probe mod, headless, CLI 2.1.317; `.claude/tasks/plans/2026-11-03-memory-capture-spike/REPORT.md`): a non-zero exit, a deny rule and an unapproved ask all come back from `next(e)` as `isError: true` with the reason only in `text` (`Exit code N\n`, `Permission to use Bash with command … has been denied.`, `Claude requested permissions to use Bash, but you haven't granted it yet.`), never a `deny` field; `classic.PermissionDenied` never fired; `classic.PostToolUseFailure` fires on non-zero exits (`error` = text); success results carry `{stdout, stderr, interrupted}` and NO `gitOperation`, even on real commits (direct and in a script); stdout reaches the hook in clear (`TOKEN=abc123`), so redaction is needed. + +## ACCEPTANCE CRITERIA (seed, enriched at STEP 3 `[gated]`) +1. A mod captures failures (scope decided at the brainstorm) as entries {normalized line, verbatim excerpt, command, tool, time}, redacted BEFORE storage, in a bounded store layout decided at the brainstorm (entries per session, bytes per entry, eviction of stale sessions, total under the 4 MiB `$.store` cap). +2. Each captured failure is matched against an error index built from the project's `.claude/memory/` registries (rule decided at the brainstorm); matched ones carry the entry ID, unmatched ones are "new failure modes". +3. Milestones (definitions decided at the brainstorm) are recorded per session. +4. A `memory_pending` tool returns the unmatched failures and the milestones of the current session (shape decided at the brainstorm); it never writes a registry. +5. One context note at the first prompt after a milestone, at most once per milestone, never per turn, never at Stop. +6. /capitalize, /close and the CAPITALIZE steps of feat, bugfix, hotfix, ship-feature and commit-change call `memory_pending` first and feed its output to their candidate scan; the approval gate and `lib/memory.sh` writing are unchanged; every citer patched, doctrine-citers census green. +7. Redaction proven by a test whose oracle does not share the redactor (LRN-237): fixtures carrying secret-shaped strings (allowlisted for gitleaks) come out redacted. +8. The mod fails open on every hook, has the `~/.claude/.json` `"enabled": false` off switch, passes `claude plugin validate --strict` and `claude plugin test` (mods suite green), and `make test` is green. +9. Proof: a failure in session N is visible through `memory_pending` in the same session and survives a /clear as decided; the noise rate measured on real transcripts before shipping (LRN-047). + +## FILE SCOPE (seed; final at STEP 3) +- mods// (new mod: .claude-plugin/plugin.json, hooks/hooks.json, hooks/register.ts, hooks/register.test.ts), skills/ link — or mods/model-router if the brainstorm picks it as host +- skills/capitalize/SKILL.md, skills/close/SKILL.md, skills/{feat,bugfix,hotfix,ship-feature,commit-change}/SKILL.md, agents/commit-changer.md, lib/capitalize-commit.md +- CLAUDE.global.md ("Proactive capitalization", only if the brainstorm changes it), README.md, USAGE.md, CHANGELOG.md, ARCHITECTURE.md +- .claude/tasks/TODO.md, .claude/tasks/plans/ (spike notes), docs/superpowers/{specs,plans} (transient) diff --git a/.claude/tasks/plans/2026-11-03-memory-capture-spike/REPORT.md b/.claude/tasks/plans/2026-11-03-memory-capture-spike/REPORT.md new file mode 100644 index 0000000..d34c9db --- /dev/null +++ b/.claude/tasks/plans/2026-11-03-memory-capture-spike/REPORT.md @@ -0,0 +1,27 @@ +# probe4b report (CLI 2.1.317, claude -p, haiku, --setting-sources local) +Raw log: /home/bchanot/.claude/jobs/8573ce6f/tmp/probe4b/log.jsonl (per case: log.cN.jsonl; full streams: out.cN.jsonl) +Mod loaded: init event plugins list contains {"name":"probe4b","source":"probe4b@inline","version":"0.1.0"}. + +| case | isError | keys of out | deny | text | gitOperation | classic events | +|---|---|---|---|---|---|---| +| C1 false | true | ref,result,text,isError,isReadOnly | none (undefined) | "Exit code 1" | n/a (result is a string) | PostToolUseFailure (error "Exit code 1", is_interrupt false) | +| C2 echo SECRET (deny rule) | true | ref,result,text,isError | none | "Permission to use Bash with command echo SECRET has been denied." | n/a | none (no PermissionDenied) | +| C3 git merge -h (ask, auto) | true | ref,result,text,isError | none | "Claude requested permissions to use Bash, but you haven't granted it yet." | n/a | none | +| C4 ls /nonexistent-dir | true | ref,result,text,isError,isReadOnly | none | "Exit code 2\nls: cannot access '/nonexistent-dir': No such file or directory" | n/a | PostToolUseFailure (error = same text) | +| C5 echo>f && git add && git commit | true | ref,result,text,isError | none | "This Bash command contains multiple operations. The following parts require approval: echo hi, git add f.txt, git commit" | n/a (never ran) | none | +| C5b git commit --allow-empty -qm ... (extra, single command) | undefined (key absent) | ref,result,text | none | "(Bash completed with no output)" | absent (undefined) | PostToolUse | +| C6 bash commit.sh | undefined (absent) | ref,result,text | none | "(Bash completed with no output)" | absent | PostToolUse | +| C7 printf TOKEN=abc123 | undefined (absent) | ref,result,text,isReadOnly | none | "TOKEN=abc123" | absent | PostToolUse | + +Non-error result: typeof result = "object", with stdout, stderr, interrupted (false), returnCodeInterpretation absent, gitOperation absent. C7 stdout = "TOKEN=abc123" (trailing newline trimmed), stderr "". Error result: typeof result = "string" (same as text). +PostToolUse keys: session_id, transcript_path, cwd, scratchpad_dir, prompt_id, permission_mode, agent_id, agent_type, effort, hook_event_name, tool_name, tool_input, tool_response, tool_use_id, duration_ms. +PostToolUseFailure keys: same minus tool_response, plus error, is_interrupt. +Not logged: tool_response contents (only keys). + +Surprises +- The permission-denied results carry no `deny` field: they are isError results with the denial in text. PermissionDenied never fired (C2 rule deny, C3 ask). +- C3: init permissionMode was "auto", yet the ask rule yielded a "haven't granted" error, no classifier approval in -p. +- C5: Bash(*) did not cover the compound command (&&, redirect): split into parts needing approval, nothing ran. C5b added as a single-command substitute; commit succeeded, quiet output. +- gitOperation never present, even on real commits (C5b, C6). With -q the commit prints nothing; untested without -q. +- isError is absent (not false) on success; isReadOnly present only for some commands (false, ls, printf), absent for git commit / bash script. +- Hook parsing quirk: validator rejects on(name) with a variable event name and $ passed to non-top-level functions; needed literal event names and top-level function declarations. diff --git a/.claude/tasks/plans/2026-11-03-memory-capture-spike/probe-register.ts b/.claude/tasks/plans/2026-11-03-memory-capture-spike/probe-register.ts new file mode 100644 index 0000000..13ca6e3 --- /dev/null +++ b/.claude/tasks/plans/2026-11-03-memory-capture-spike/probe-register.ts @@ -0,0 +1,70 @@ +import type { Register } from 'claude-code' + +const LOG = '/home/bchanot/.claude/jobs/8573ce6f/tmp/probe4b/log.jsonl' + +function cut(v: any, n: number): any { + return v === undefined || v === null ? v : String(v).slice(0, n) +} + +async function append($: any, obj: any): Promise { + try { + const prev = (await $.fs.exists(LOG)) ? await $.fs.read(LOG) : '' + const text = typeof prev === 'string' ? prev : (prev?.text ?? '') + await $.fs.write(LOG, text + JSON.stringify(obj) + '\n') + } catch (err) { + // fail open + } +} + +async function classic($: any, name: string, e: any): Promise { + if (name === 'classic.PostToolUse' && e.tool_name !== 'Bash') return + await append($, { + ev: name, + keys: Object.keys(e), + tool_name: e.tool_name, + error: cut(e.error, 120), + reason: cut(e.reason, 120), + is_interrupt: e.is_interrupt, + }) +} + +async function toolRow($: any, e: any, out: any): Promise { + const r = out.result + const row: any = { + ev: 'tool.call', + command: cut(e.command, 80), + agentId: e.agentId !== undefined, + keys: Object.keys(out), + isError: out.isError, + deny: cut(out.deny, 120), + resultType: typeof r, + text: cut(out.text, 120), + } + if (r && typeof r === 'object' && !out.isError) { + row.stdout = cut(r.stdout, 80) + row.stderr = cut(r.stderr, 80) + row.interrupted = r.interrupted + row.rci = r.returnCodeInterpretation + row.gitOperation = r.gitOperation + } else if (r && typeof r === 'object') { + row.resultKeys = Object.keys(r) + } + await append($, row) +} + +export const register: Register = on => { + on('tool.call', { tool: 'Bash' }, async ($, e, next) => { + const out: any = await next(e) + await toolRow($, e, out).catch(() => {}) + return out + }).catch(($, e, next) => next(e)) + + on('classic.PostToolUseFailure', ($, e, next) => + classic($, 'classic.PostToolUseFailure', e).catch(() => {}).then(() => next(e))).catch(($, e, next) => next(e)) + + on('classic.PermissionDenied', ($, e, next) => + classic($, 'classic.PermissionDenied', e).catch(() => {}).then(() => next(e))).catch(($, e, next) => next(e)) + + on('classic.PostToolUse', ($, e, next) => + classic($, 'classic.PostToolUse', e).catch(() => {}).then(() => next(e))).catch(($, e, next) => next(e)) +} From fefcd9bfd6dddd169cdac50c93ee9041f6723a3e Mon Sep 17 00:00:00 2001 From: bchanot Date: Tue, 3 Nov 2026 14:55:13 +0100 Subject: [PATCH 03/43] docs(spec): phase 4b memory capture design --- .../specs/2026-11-03-memory-capture-design.md | 130 ++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 docs/superpowers/specs/2026-11-03-memory-capture-design.md diff --git a/docs/superpowers/specs/2026-11-03-memory-capture-design.md b/docs/superpowers/specs/2026-11-03-memory-capture-design.md new file mode 100644 index 0000000..9ccd97e --- /dev/null +++ b/docs/superpowers/specs/2026-11-03-memory-capture-design.md @@ -0,0 +1,130 @@ +# Phase 4b memory capture — design (2026-11-03) + +BDR-125 phase 4b. Contract `.claude/tasks/contracts/2026-11-03-memory-capture-0100.md` +(brainstorm answers 1-8 `[gated 2026-11-03]`). Engine facts: +`.claude/tasks/plans/2026-11-03-memory-capture-spike/REPORT.md`. Branch +`feature/memory-capture`. + +## Goal +Capture is the weak link of the memory loop (5 of 7 real errors of 2026-10-23 +had no registry entry). A mod records what failed during a session and the +moments worth capitalizing, and hands them to /capitalize and the CAPITALIZE +steps through one tool. Writing stays exactly as today: the model drafts, the +user approves, `lib/memory.sh` writes. The mod never writes a registry. + +## Unit 1 — mod `mods/memory/` +Layout like model-router (`.claude/rules/mods.md`): `.claude-plugin/plugin.json` +(name `memory`, version 0.1.0, description, author), `hooks/hooks.json` +(`{"modules": ["./register.ts"]}`), `hooks/register.ts`, +`hooks/register.test.ts`, tracked link `skills/memory -> ../mods/memory`. +Off switch: `~/.claude/memory.json` `{"enabled": false}` (read like +model-router's override; any read error → enabled). Fail open on every hook +(`.catch` → `next(e)`, one log line per hook per session). Closure state only; +persistence through `$.store`. + +## Unit 2 — failure capture +- Hook: `tool.call` on `Bash`, after `const out = await next(e)`; the answer + is returned unchanged. +- A failure: `out.isError === true` and `out.text` starts `Exit code ` + digits. + Excluded: texts starting `Permission to use`, containing `haven't granted it + yet`, starting `model-router:`, or containing ``. +- Distinctive line: the last non-empty line of `out.text` after the first line; + none → not captured (`false`, `grep` with no match). +- Entry: `{ line, excerpt, command, agent, at }` where `line` is the redacted + distinctive line (≤ 300 chars), `excerpt` the redacted last 1200 chars of the + text, `command` the redacted command (≤ 300 chars), `agent` true when + `e.agentId` is set, `at` the clock (ms). +- Normalized key: lowercase; numbers → `#`; absolute and relative paths + (`/x/y`, `./x`, `x/y.ext`) → ``; hex runs ≥ 7 → ``; whitespace + folded. The same key twice in a session is stored once (a `count` field + increments). +- Bounds: ≤ 50 entries per session (oldest dropped), entry ≤ 2 KiB after + truncation, the 3 most recent sessions per project kept, the whole store + value ≤ 1 MiB (oldest project sessions evicted first). Every regex over tool + text runs on input capped at 64 KiB (LRN-219). + +## Unit 3 — redaction (before anything is stored) +One function `redact(text)`, applied to line, excerpt and command: +- `NAME=value` where NAME matches `[A-Za-z_]*(TOKEN|API_KEY|SECRET|PASSWORD|PASSWD)[A-Za-z_]*` → `NAME=` (same rule as the rtk env-dump redaction). +- The 7 `/guardrails` SECRET_PATTERNS (skills/guardrails/scripts/checks/apis.py:10-19: Stripe, AWS access key, GitHub token, Slack token, SendGrid key, Supabase secret, private key block) → ``. +- `Bearer ` → `Bearer `; URLs `scheme://user:pass@host` → `scheme://@host`. +- 40-hex SHAs are not secrets and stay. +Test oracle: fixtures written as literal expected strings in the test, never +computed by the redactor (LRN-237); secret-shaped fixtures use the gitleaks +allowlisted shapes (`test-secret-…` and the fake AWS key named in +`.gitleaks.toml`), in files the allowlist covers. + +## Unit 4 — milestones +From successful Bash results (`out.isError` absent), on the main loop only: +- `commit`: the command's first simple command is `git commit` (optionally + `git -C commit`) and the text holds `-m "feat` / `-m 'feat` / `-m feat` + or `fix` (also `feat(` / `fix(` scopes). Other commits are not milestones. +- `finish`: the command runs `gitflow.sh finish` (bash or sh form) and not + `finish chore` (the /capitalize and /close auto-finish, BDR-068). +- `green`: a command containing `make test` that succeeded, when the same + command string (normalized whitespace) failed earlier in the session. +Stored per session with `{ kind, at, label }` (label: the commit subject's +first 80 chars, the branch named by finish if present, or the suite argument). +Note: at the first `prompt.submit` from the composer after a milestone, one +context line is attached: `memory: