forked from bchanot/claude
chore(memory): BDR-137 + LRN-258..261 + BLK-041 + EVAL-057 — ship-feature phase 4b memory capture
This commit is contained in:
1 parent
181bcfc7f2
commit
08e36dcc0b
5 files changed
+44
No files matched your search
@@ -1931,3 +1931,23 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
|
||||
- **Pattern**: headless permission bench (`claude -p --setting-sources local --settings <file>`): (1) `printf … | cd "$4" && claude -p` runs the cd in a pipe subshell and claude reads the caller's stdin (the cases file) → `cd` on its own line, every call `</dev/null`; (2) controls carry `allow Bash(*)` + `defaultMode: auto` (+ `skipAutoPermissionPrompt`), else default mode asks everything; (3) parse the stream-json `system/init` `permissionMode` per row, reject rows not `auto`; (4) verdict = DENIED ("has been denied") / ASKED ("requested permissions") / else RAN (a non-zero command exit is still RAN); (5) a rule is approved only when every row naming it hits its want and one row expects ASKED/DENIED; (6) append-only results, ids selectable, full run in background.
|
||||
- **Context**: phase 2 plan r1 bench broken (correctness BLOCKER), fixed r2/r3; run 2026-11-02, 41 cases + controls, all `auto`.
|
||||
- **Apply**: reuse `.claude/tasks/plans/2026-11-02-phase2-bench/run.sh` + `bench.sh` for any future rule (docker mount spellings, `rtk read * .env*` twins).
|
||||
|
||||
## LRN-258 — All-placeholder n-grams in a normalized error index match everything: skip grams with no content word (2026-11-04)
|
||||
- **Pattern**: normalization turns values into placeholders (`#`, `<path>`, `<hex>`); a word 4-gram made only of placeholders (`# # # #`: timestamps, IP:port, version quads) indexes a registry ID and every such failure line reads `known`. Measured on this repo: `# # # #` → LRN-074, so `connect ECONNREFUSED 127.0.0.1:5432` and `Error at 2026-11-04 12:00:00.123` read known.
|
||||
- **Context**: 4b final review I1 (`mods/memory/hooks/register.ts` `grams`), fixed c81a7a2 at index and match time, test IDX9. Open sibling: common-phrase grams (`no such file or directory` → LRN-056) for 4c.
|
||||
- **Apply**: any n-gram index over normalized text drops grams with no content word; test with an IP span and a timestamp line against the real registries, not fixtures only.
|
||||
|
||||
## LRN-259 — Regex over free text: measure on 64 KiB repeats, lookbehind the start token, never \s* under /m (2026-11-04)
|
||||
- **Pattern**: three quadratic or costly regex shapes landed in one branch, each caught by a reviewer who RAN it: STATUS `^\s*-\s*` under /m (3.1 s on 80k newlines: `\s*` crosses newlines → `[ \t]*`); JWT `eyJ…\.eyJ…` rescanning at every `eyJ` (1.58 s on 64 KiB → prefix `(?<![A-Za-z0-9_-])`, 2 ms); URL scheme `[a-z][a-z0-9+.-]{0,30}:\/\/` backtracking at every letter (7 ms, linear but 8× the rest → `(?<![a-z0-9+.-])`, 1 ms). A fix round introduced two of them.
|
||||
- **Context**: 4b Task 4 review, final re-review, security re-scan; LRN-219 sets the 64 KiB cap, this is the method: `'x'.repeat(65536)` per suspicious rule in a bun one-liner, 64 KiB → 256 KiB must grow ×4.
|
||||
- **Apply**: a TIME1-style test per regex family; a reviewer measures regex cost by running, never by reading; a lookbehind on the start token is the cheap cure for "every position is a candidate start".
|
||||
|
||||
## LRN-260 — Headless mod proof: claude -p haiku with allowedTools naming the mcp tool and ToolSearch, four scripted steps (2026-11-04)
|
||||
- **Pattern**: headless proof of a mod with a tool: scratch repo `mktemp -d` on a feature branch, `GITFLOW_NO_PUSH=1` exported in the creating command, `claude -p "<4 steps>" --model haiku --output-format stream-json --verbose --allowedTools 'Bash(ls *)' 'Bash(git commit *)' 'mcp__memory__memory_pending' 'ToolSearch'`; prompt "exactly these four steps, one tool call each, no retry, reply with the tool result verbatim" → haiku complied first try, rc 0, $0.011; `stream-json` holds the `tool_use` records to assert the call sequence.
|
||||
- **Context**: 4b Task 9 (`$CLAUDE_JOB_DIR/tmp/proof9/run.sh`); the note half cannot be proven in `-p` (prompt origin `sdk`, no composer) → interactive check by the user; the run leaves a `p:<tmp>` store key, harmless.
|
||||
- **Apply**: split any mod proof into a headless capture half and an interactive UI half; name the mcp tool and `ToolSearch` in `--allowedTools`, else the tool is "not granted".
|
||||
|
||||
## LRN-261 — Noise measured before shipping a capture rule: python port, 14-day replay, 30 labelled samples, alternatives on the whole corpus (2026-11-04)
|
||||
- **Pattern**: before shipping a rule that fires on tool output, measure its noise: sonnet ports the rule to Python (`noise.py`, stdlib, read-only, `--seed`), replays 14 days of `~/.claude/projects/<repo>/*.jsonl` + `*/subagents/*.jsonl`, prints 30 redacted samples; the controller labels useful/noise. Result 4b: 83 % noise, concentrated in sub-agents (23/25) vs main loop (2/5); candidate tightenings (`probe`, `errorish`) measured on the WHOLE corpus (81 → 60 / 27) before the gate; the user picked one, amending a gated answer.
|
||||
- **Context**: 4b Task 7, LRN-047 lineage; NOISE.md holds counts and shares only (contract check forbids `excerpt:`/`command:` lines); samples stay in the job tmp.
|
||||
- **Apply**: a capture or injection rule ships with its measured noise share and the numbers of the alternatives; a port claims "1:1" only while the TS rule is frozen (ours drifted twice).
|
||||
Reference in new issue
Block a user