forked from bchanot/claude
chore(memory): BLK-043 fork bomb resolved in 49aa2ff, BLK-044 open, contract and plans
This commit is contained in:
1 parent
fb2a4ff5ba
commit
fd646a611a
6 files changed
+321
-1
No files matched your search
@@ -0,0 +1,138 @@
|
||||
# Phase 4d memory feedback and promotion: plan (2026-11-05)
|
||||
|
||||
BDR-125 phase 4d, host `mods/memory` (BDR-137). Input: the 4c injection log,
|
||||
store key `l:<root>` (`LogRecord` in `mods/memory/hooks/register.ts`).
|
||||
Status: draft r1, not challenged, not gated. Starts after the 4c merge.
|
||||
|
||||
## What the log holds (read 2026-11-05, 18:30)
|
||||
|
||||
- Real use: 1 record. This repo, trigger `fresh`, outcome `injected`, IDs
|
||||
BDR-125, BDR-137, LRN-261: the prompt that asked for this plan.
|
||||
- Plus 1 headless proof record (tool channel, fixture BLK-001, temp repo).
|
||||
- Expected rate in this repo (RECALL.md, 14 days, after the content floor):
|
||||
about 15 triggered asks per 14 days, 3 tool-channel injections.
|
||||
|
||||
No decision can come from this. The TODO gate ("deferred until the log shows
|
||||
misses or noise") is not met, and the log alone can never meet it, for the
|
||||
three reasons below.
|
||||
|
||||
## Gaps between the log and the 4d questions
|
||||
|
||||
1. Usefulness is not recorded. The 4d design (queue challenge 2026-10-25)
|
||||
asked for "cited or not"; a record holds `at, session, agent, channel,
|
||||
ids, trigger, outcome`. Relevance needs the prompt and the reply, and
|
||||
both live in the transcript, not in the store.
|
||||
2. Misses leave no trace. The log lists what came, never what should have
|
||||
come. Example: this plan's prompt did not bring BDR-100, the promotion
|
||||
rule the TODO 4d line cites, and nothing records that.
|
||||
3. Transcripts live 7 days (`cleanupPeriodDays: 7`, job7 step D, secret
|
||||
hygiene). A single log-to-transcript join at the gate, about 3 weeks
|
||||
out, would find most turns gone. Older transcripts survive today; the
|
||||
setting does not promise it.
|
||||
|
||||
Minor: a prompt-channel `none` mixes "BM25 found no candidate" and "haiku
|
||||
declined" (both paths in `recallFor`); rerank latency is not logged, only
|
||||
the `timeout` outcome.
|
||||
|
||||
## 4d.1 Two log fields (mod, /feat after the 4c merge)
|
||||
|
||||
- Prompt records gain `cands` (BM25 candidates after the seen filter, 0-30)
|
||||
and `ms` (rerank wall time, absent when no rerank ran). Both optional in
|
||||
`isLogRecord`: records written before stay valid.
|
||||
- Size: about 40 bytes more per record, 200 records stay under 40 KiB
|
||||
(cap 64 KiB).
|
||||
- Nothing else enters the store: no prompt text, no reply text (LRN-124),
|
||||
no runtime cited flag (it would need a Stop hook reading the transcript
|
||||
every turn, for a signal the offline join gets at no runtime cost).
|
||||
- Proof: kit tests for both fields, an old-shape record still loads, the
|
||||
`none` split visible on a fixture with zero candidates.
|
||||
|
||||
## 4d.2 Weekly reader (harness, no runtime change)
|
||||
|
||||
- `feedback.ts` (bun) beside this plan. It reuses `recall-bench.ts`'s
|
||||
transcript loader (`findFiles`, `records`, `composerText`, exported, not
|
||||
copied) and the mod's own functions (no port, LRN-261).
|
||||
- Reads every `l:` key of the store file, all roots. The file path is
|
||||
engine-internal (`~/.claude/plugins/store/memory_skills-dir-*.json`):
|
||||
a renamed file fails loudly, acceptable for a measurement tool.
|
||||
- Joins each record to its transcript by `session` and `at`:
|
||||
- prompt channel: the composer prompt it answered and the assistant turn
|
||||
that followed;
|
||||
- tool channel: the failing command and the next 3 Bash results of the
|
||||
same session (retry succeeded or not).
|
||||
- Per injected entry: `cited` (the ID appears in the turn's assistant
|
||||
text, a lower bound) and a relevance label (controller labels, the user
|
||||
spot-checks: the 4c method).
|
||||
- Misses: on up to 5 triggered asks per run, the controller searches the
|
||||
registries for relevant entries the recall did not bring, and names the
|
||||
cause (no lexical overlap, BM25 rank over 30, haiku declined). The
|
||||
FR-prompt vs EN-entry gap is the expected first cause.
|
||||
- Persists `labels.tsv` (at, root name, channel, trigger, outcome, cands,
|
||||
ID, cited, label): IDs and enums only, committed. Labelled records are
|
||||
skipped on the next run; prompt and reply text stay in the scratchpad.
|
||||
- Writes `FEEDBACK.md`: counts and shares only.
|
||||
- Cadence: once a week, inside the 7-day window, about 10 minutes of label
|
||||
checks for the user.
|
||||
|
||||
## 4d.3 Gate and decisions fixed before the data
|
||||
|
||||
Gate: at least 20 labelled injected prompt asks and 10 tool injections (the
|
||||
BDR-125 proof line), or 5 weeks after the 4c merge, whichever comes first.
|
||||
At the date cap with a smaller n: report "insufficient sample" with the n,
|
||||
extend once or close.
|
||||
|
||||
| Signal | Threshold | Action |
|
||||
|---|---|---|
|
||||
| prompt precision (relevant / injected entries) | at least 64 % | keep |
|
||||
| same, below 64 % | worst trigger kind below 50 % | tighten or drop that trigger (idle first: the 4c noise sat there) |
|
||||
| precision of the 3rd pick | below 40 % | cap at 2 entries |
|
||||
| timeout + paused share (asks with `cands` > 0) | above 25 % | revisit `RERANK_MS` or skip the rerank on short lists |
|
||||
| `ms` p90 | below 2500 | lower `RERANK_MS`: every triggered prompt waits that long |
|
||||
| tool precision | below 50 % | require 2 matched grams, or exclude the entry |
|
||||
| misses in the sample | 1 ask in 3 or more | diagnose by cause; no fix built ahead |
|
||||
| entry injected 3 times or more, 0 relevant | any | noisy: /prune-memory rewrites it |
|
||||
| entry relevant 3 times or more, in 2 sessions or more | any | promotion candidate (BDR-100) |
|
||||
|
||||
Each action is its own /feat or /hotfix with the measurement attached. 4d
|
||||
builds no tuning before the gate.
|
||||
|
||||
## 4d.4 Promotion and /prune-memory (conditional)
|
||||
|
||||
Built only if the gate shows at least one noisy entry or one promotion
|
||||
candidate; otherwise closed with the measurement, like phase 3.
|
||||
|
||||
- Promotion stays a proposal: the reader lists each candidate with its
|
||||
counts, the user picks, the mechanism (guard, test, skill step) becomes
|
||||
its own TODO run, and the entry links the mechanism's commit. Never
|
||||
automatic.
|
||||
- /prune-memory STEP 1 gains "D. Recall feedback": reads `labels.tsv`
|
||||
through one lib script that holds the thresholds (doctrine "After code
|
||||
changes" step 4); noisy entries join section C, promotion candidates go
|
||||
to STEP 2 as TODO lines.
|
||||
- Open at that point: where other repos keep their labels (the mod logs
|
||||
every root, the reader lives in this repo).
|
||||
|
||||
## Open choices (user)
|
||||
|
||||
1. Cadence vs retention: weekly reader runs (recommended); or raise
|
||||
`cleanupPeriodDays` for the window (reverses job7 step D, transcripts
|
||||
carry secrets); or one run at the gate on a single week.
|
||||
2. 4d.1 fields `cands` and `ms`: add them (recommended) or measure with the
|
||||
current record.
|
||||
3. Gate numbers and the 5-week cap.
|
||||
4. Promotion threshold: relevant 3 times in 2 sessions.
|
||||
|
||||
## Prerequisite: 4c merge
|
||||
|
||||
- Recall half of the interactive proof seen in this session: a fresh design
|
||||
prompt brought 3 entries, all relevant to the ask by the controller's
|
||||
reading (labeller and beneficiary are the same model: unchecked).
|
||||
- Still owed: the digest after a real `/compact`, the 4c gates, the user's
|
||||
"merge it".
|
||||
|
||||
## Out of scope
|
||||
|
||||
Runtime cited flag, prompt text in the store, the 4c deferrals (Read and
|
||||
Bash command triggers, critical deny-once, Tier 1, cross-repo index,
|
||||
`memory_search`), the `l:<root>` cross-root shortcut and the unbounded
|
||||
`memory.sh check` residual (own TODO lines).
|
||||
@@ -0,0 +1,110 @@
|
||||
# Bugfix plan: detect_plan fork bomb (2026-11-06)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
BUG: running `lib/tests/session-digest.test.sh` (uncommitted SD13 cases)
|
||||
filled 14 GiB of RAM in 45 s; 2026-11-05 23:49 → OOM → reboot 23:55.
|
||||
Kernel OOM dump: 126 `claude`, 396 `bash`, 124 `sh`, all
|
||||
`oom_score_adj=-1000`.
|
||||
|
||||
ROOT CAUSE: `lib/detect-plugins.sh:85` `detect_plan` falls back to
|
||||
`claude config get planType` when `$HOME/.claude.json` lacks the plan
|
||||
markers or is absent. claude 2.1.318 has no `config` subcommand
|
||||
(`claude config --help` prints the main usage), so the words are read
|
||||
as a prompt and a full session starts. Its SessionStart hook
|
||||
(`hooks/session-start.sh:158` → `detect_plan`) runs with the same HOME,
|
||||
finds the same missing markers, starts another `claude`: unbounded
|
||||
recursion. SD13's `mkhome` builds a HOME that links `~/.claude/*` but
|
||||
not `~/.claude.json`, so the fallback fires.
|
||||
|
||||
EVIDENCE: journal boot -1 (earlyoom + kernel OOM table), transcript
|
||||
90b2c305 (`make test suite=lib/tests/session-digest.test.sh` at
|
||||
23:49:07, result at 23:51:28), `claude config --help` output.
|
||||
|
||||
BLAST RADIUS: production too. Real sessions survive only because
|
||||
`~/.claude.json` holds `tengu_cobalt_compass` + `tengu_harbor` (internal
|
||||
feature flags). If upstream renames them, every session start recurses.
|
||||
`detect_plan` is called only from `hooks/session-start.sh:158`. No other
|
||||
hook or lib calls the `claude` CLI (`lib/cc.sh` is the user launcher).
|
||||
The hook header already promises "Filesystem only, except one git fetch".
|
||||
|
||||
RELATED MEMORY: no BLK/BDR/LRN bears (grep detect_plan, fork, recurs,
|
||||
oom, planType).
|
||||
|
||||
## Fix plan
|
||||
|
||||
1. `lib/detect-plugins.sh` `detect_plan`: delete the `claude config get
|
||||
planType` fallback and its `case`; markers absent → `echo "pro"`
|
||||
(the existing default). Comment states why the CLI is never called
|
||||
(a hook-launched `claude` re-enters this hook).
|
||||
2. `hooks/session-start.sh`: recursion guard at the top, before stdin
|
||||
is read: `[ -n "${CLAUDE_SS_RUNNING:-}" ] && exit 0; export
|
||||
CLAUDE_SS_RUNNING=1`. The variable lives only in the hook's process
|
||||
tree: a sibling session never sees it; any `claude` a future change
|
||||
spawns from the hook gets a silent no-op hook instead of a cascade.
|
||||
3. `lib/tests/session-digest.test.sh` (keeps the uncommitted SD13/SD14
|
||||
cases intact):
|
||||
- suite-wide stub: `$WORK/stub/claude` writes one line to
|
||||
`$WORK/claude-calls` and exits 0; `PATH="$WORK/stub:$PATH"` exported
|
||||
before any case. A regression then records a call instead of
|
||||
forking a real session. SD13h's `PATH="$bin"` already holds no
|
||||
claude.
|
||||
- SD15a: `HOME` without `.claude.json` (mkhome as is), hook runs →
|
||||
`claude-calls` absent/empty.
|
||||
- SD15b: `CLAUDE_SS_RUNNING=1` set → hook prints nothing, rc 0.
|
||||
- SD15c: `detect_plan` sourced alone with an empty HOME prints `pro`.
|
||||
Replaces the `ulimit -u` idea: the per-user process cap counts every
|
||||
process of the user (hundreds of live ones), so a value is either
|
||||
too tight (spurious red) or too loose (no protection); the stub is
|
||||
deterministic.
|
||||
|
||||
ORDER: fix 1 lands before the suite runs again (DO NOT run the suite
|
||||
on the current tree).
|
||||
|
||||
RISK: low. Plan detection for users without the markers now always
|
||||
reports Pro (banner budget label only). Guard: a user who runs the hook
|
||||
by hand from inside a hook-spawned shell sees nothing (no such use).
|
||||
|
||||
Out of scope: origin of `oom_score_adj=-1000` on claude processes
|
||||
(recorded as an open BLK), plan detection accuracy.
|
||||
|
||||
## Revision r2 (after 3-lens challenge, 2026-11-06)
|
||||
|
||||
Challenge: simplicity SOLID (4 MINOR), correctness CONCERNS (1 MAJOR),
|
||||
robustness CONCERNS (3 MAJOR). Named changes:
|
||||
|
||||
- R1 (correctness 1, robustness 3: landing). The fallback is on main and
|
||||
develop. The fix splits in two:
|
||||
- Part A, on its own branch off main or develop (user choice), in a
|
||||
separate `git worktree` (so `~/.claude/hooks` and `~/.claude/lib`,
|
||||
symlinked to the main working tree, never flip to another branch and
|
||||
the uncommitted 4c work stays untouched): fix 1, fix 2, and a
|
||||
standalone regression suite `lib/tests/detect-plan.test.sh` that
|
||||
needs nothing from feature/memory-recall.
|
||||
- Part B, on feature/memory-recall after Part A merges: suite-wide
|
||||
stub in session-digest.test.sh + final empty-log assertion.
|
||||
- R2 (robustness 1+2, correctness 2+4: proof). The stub is written with
|
||||
the log path expanded at creation, `chmod +x`, and a positive control
|
||||
first calls `claude` through PATH and asserts 1 logged line (stub
|
||||
reached). The regression case calls `detect_plan` directly (no hook,
|
||||
no recursion path) with an empty HOME and asserts output `pro` AND
|
||||
0 new log lines. Red proof: that case runs once against the pre-fix
|
||||
lib (safe: stub catches the single call) and must fail, then pass
|
||||
after fix 1. Copied into the contract as [gated]: never run the hook,
|
||||
`make test` or the session-digest suite before fix 1 is on disk.
|
||||
- R3 (correctness 3, robustness 5: guard). Guard kept as
|
||||
defense-in-depth (the -1000 adj made the runaway unkillable), renamed
|
||||
`SESSION_START_RUNNING` (simplicity 4, repo convention). The hook
|
||||
comment says it has no live caller and that it relies on claude
|
||||
passing env to its hooks. Test: hook run with the variable set prints
|
||||
nothing, rc 0; and a PATH stub `git` writes `$SESSION_START_RUNNING`
|
||||
seen by the hook's child, asserting `1` (proves the export).
|
||||
- R4 (simplicity 2, correctness 5: dead code). detect_plan docstring
|
||||
"max or pro"; the `free)` arm at session-start.sh:162 dropped.
|
||||
- R5 (simplicity 3): SD15a/SD15c dropped from Part B; one final
|
||||
cumulative empty-log check covers every hook run of the suite.
|
||||
- R6 (robustness 6: containment). The first run of each suite after the
|
||||
fix goes through `systemd-run --user --scope -p TasksMax=256` (cgroup
|
||||
pids cap counting only that scope).
|
||||
- Deferred to TODO (robustness 4, MINOR): stub in memory, graphify-gate
|
||||
and push-guard suites, which run the hook with the real HOME.
|
||||
Reference in new issue
Block a user