chore(memory): H4b journal, safety LOWs closed

This commit is contained in:
bchanot committed 2026-11-13 18:27:28 +01:00
1 parent 189c1f916b
commit f0908e74ea
5 files changed
+126 -1

No files matched your search

+1
View File
@@ -745,3 +745,4 @@ rules:
- correction: the CLI 2.1.323 validator blocker above is BLK-049, not BLK-046.
- /bugfix mods-strict-types (BLK-049): 17 TS sites narrowed for CLI 2.1.323 validate --strict, kits 288/640 unchanged, make test all green again. An analyzer corrected my premise (the 15 '.catch' lines are info, the TS errors fail strict). Other session's Fable 5.2 rename found uncommitted in this checkout: split out, ships as its own branch.
- feature/fable-5-2 (user ask mid-run): fable alias → claude-fable-5-2 (ca1dc9f), rename done by another session in this checkout, split from BLK-049, id probed live, kit 640/0. Live CLI warns: deny Bash(git push * :**) matches as * wildcard (pre-existing, H item).
- /bugfix safety-lows (block H4b): code-limits private state dir (mkdir -m 700 no -p + one find test), session-start runner timeout→gtimeout→perl alarm, pre-commit fails closed on git diff and filter errors (generator lib/gitflow.sh + both copies), memory.sh rewrap, claude-stub env log path (1f107c3). 4 rounds: generator was the hook source (decision a), GATE 2 MEDIUM stat/lstat race closed, lint regressions fixed. Residual: perl alarm kills the exec'd process only (TODO).
+2 -1
View File
@@ -42,7 +42,8 @@ Shared rules for every phase:
- [x] BLK-043 Part B: suite-wide `claude` PATH stub + final empty-log check in `lib/tests/session-digest.test.sh` (SD0/SD15, done 2026-11-06 via /hotfix; security PASS, MEDIUM folded into the shortcut below).
- [x] shortcut (BLK-043 gate 2 LOWs): `lib/tests/detect-plan.test.sh` unchecked `mktemp -d` (empty WORK → real `claude probe`), `bash -c "source '$LIB'"` interpolation, no `GIT_CEILING_DIRECTORIES` on hook runs; and the memory, graphify-gate, push-guard suites run the hook with no `claude` stub. Upgrade = `|| exit 1` + `command -v claude` assert, positional args, ceiling dir, shared stub helper. Same gap in `lib/tests/session-digest.test.sh` SD0 (gate MEDIUM 2026-11-06): `claude probe` runs before any check that the stub is first on PATH (stub write failure or noexec TMPDIR → real CLI), a red SD0 does not stop the suite; LOW: log path interpolated into the generated stub unquoted. Done 2026-11-06 dc5f127 + 7ca6b18 (shared `lib/tests/claude-stub.sh`, 6 suites, BLK-045).
- [ ] BLK-044: trace origin of `oom_score_adj=-1000` on claude processes; set `OOMScoreAdjust` on the `cc` scope so a runaway claude dies first.
- [ ] safety-residuals gate LOWs (2026-11-06, bugfix/safety-residuals): `rtk read -n server.key` / `rtk read id_rsa` at repo root pass (`.key`/`.pem`/`id_*` rtk denies all need `*/`; pre-existing, bench before adding); session-start `memory.sh check` and digest unbounded when `timeout` is absent (stock macOS); pre-commit allowlist fails open if `git diff --cached` errors (pre-existing, Gitea backstop); `lib/memory.sh` `_mem_path_safe` line > 80 chars; `claude-stub.sh` interpolates the mktemp log path into the generated stub (test-only).
- [x] safety-residuals + phase 2 remaining LOWs done 2026-11-13 (block H4b, bugfix/safety-lows 1f107c3); `rtk read -n server.key`/`id_rsa` root forms done in H4a (a43ba1b).
- [ ] session-start perl fallback (no timeout/gtimeout, stock macOS): `alarm` kills only the exec'd bash; a hung child of memory.sh keeps the pipe open. Upgrade: fork + setpgrp + kill the group (GATE 2 LOW 2026-11-13).
- Proof (closed 2026-11-12, block B): precision = EVAL-058 (prompt gate; the ≥10-error half folded into the tool gate, BDR-140); redaction, `/context` delta and the known-error recall proven live (journal 2026-11-12); digest after a real compaction = pending this run's first auto-compact.
- Gate 1 result (2026-11-12, EVAL-058): prompt precision 57 % (n=75), under the 64 % target; fresh 78 %, idle 44 %, keyword 55 %; tool gate 0 records.
- [x] 4d gate 1: idle tightened to ≥ 8 content words (BDR-146, 7e87d77)
@@ -0,0 +1,34 @@
# CONTRACT — feedback-followups (phase 4d follow-ups, block H5)
- date: 2026-11-13 | flow: bugfix | branch: bugfix/feedback-followups (off develop)
- status: active
## REQUEST (verbatim — IMMUTABLE)
metricsOf: split rows by the 2026-11-12 fix date and retire the 3rd-pick signal (pre-fix rows keep re-emitting the idle and cap actions; BDR-146 residual). 4d follow-ups (final review + task reviews 2026-11-08, non-blocking): transcript cache holds every parsed file per run (~1 GB at steady state: process per (root, session, agent) and drop); image prompts (array content) become `lost` (counted); no boundary tests for the threshold actions (3rd pick, late, tool, miss, noisy/promotion, idle tie-break), eviction note, torn count, toolItem; `replyAfter` hides `seen.add` in a filter; no `tsc` type check on lib/memory-feedback; kit test pinning the exact field set of a prompt log record (IR11 is toMatchObject since 4d.1, so a future text field would pass); labels.ts minors (fromLine checks column count only, readState accepts non-object JSON, .bak keeps one state, misleading error on an entry label with a cause).
## CLARIFICATIONS
- [gated 2026-11-13, challenge r1-r3; plan .claude/tasks/plans/2026-11-13-feedback-followups-1800.md § r2 + r3 FINAL] fix instant = merge d3d5e6c 2026-11-12T19:53:59Z; split scope and floor per r3; image prompts DROPPED (0 instances, measured); `tsc` not done (no compiler on the machine, a new dependency) and `l:<root>` shortcut kept (49.7 KB of 4 MiB measured): both closed in the TODO with those reasons; `.bak` one state accepted.
## ACCEPTANCE CRITERIA
1. thirdPick retired, fix instant constant present.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'thirdPick' lib/memory-feedback/metrics.ts lib/memory-feedback/thresholds.ts | grep -c ':0$')$(grep -c "GATE1_FIX_AT = Date.parse('2026-11-12T19:53:59Z')" lib/memory-feedback/thresholds.ts)
EXPECT: 21
EVIDENCE: PENDING
2. memory-feedback suite green.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/memory-feedback.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: PENDING
3. New tests exist (post-fix floor, A,B,A torn read-once, every label kind, two IR11 key-set pins).
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'GATE1_FIX_AT' lib/memory-feedback/metrics.test.ts | grep -c '^[1-9]')$(grep -c 'Skipped 1 torn' lib/memory-feedback/reader.test.ts | grep -c '^[1-9]')$(grep -c 'Object.keys' mods/memory/hooks/register.test.ts | grep -c '^[2-9]')
EXPECT: 111
EVIDENCE: PENDING
4. Memory kit green, strict validation green.
CHECK: cd /home/bchanot/Documents/claude && claude plugin validate --strict mods/memory >/dev/null 2>&1 && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/mods.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: PENDING
5. The live labels.tsv still reads (worksheet on a COPY of ~/.claude/memory-feedback under mktemp, never the real dir).
(manual: executor runs the reader on a copy and quotes rc 0 + the head lines)
6. Peak RSS of the reader before/after on a copy of the real population (manual, quoted).
7. make test green (manual, bun on PATH); functions ≤ 25 logic lines / 5 params / 5 locals; lines ≤ 80.
## FILE SCOPE
lib/memory-feedback/{metrics,thresholds,transcripts,reader,labels}.ts and their .test.ts, mods/memory/hooks/register.test.ts. CHANGELOG/README → doc sync; TODO lines → STEP 8.
@@ -0,0 +1,61 @@
# CONTRACT — safety-lows (phase 2 + safety-residuals LOWs, block H4b)
- date: 2026-11-13 | flow: bugfix | branch: bugfix/safety-lows (off develop)
- status: active
## REQUEST (verbatim — IMMUTABLE)
Phase 2 security gate follow-ups: LOW code-limits state dir (`mkdir -m 700`, owner check). safety-residuals gate LOWs (2026-11-06): session-start `memory.sh check` and digest unbounded when `timeout` is absent (stock macOS); pre-commit allowlist fails open if `git diff --cached` errors (pre-existing, Gitea backstop); `lib/memory.sh` `_mem_path_safe` line > 80 chars; `claude-stub.sh` interpolates the mktemp log path into the generated stub (test-only).
## CLARIFICATIONS
- [gated 2026-11-13, internal] (1) hooks/code-limits.sh:91: `mkdir -m 700 -p "$dir"`, then require `[ -O "$dir" ] && [ ! -L "$dir" ]` else return 0 (skip the signal, never fail the tool). (2) session-start: ONE bounded runner chosen once near the top: `timeout 2`, else `gtimeout 2`, else `perl -e 'alarm shift; exec @ARGV' 2` (stock macOS has perl), else none; the three sites (:252, :311, :414) use it. (3) githooks/pre-commit:41: capture `staged=$(git diff --cached --name-only --no-renames) || { echo "gitflow pre-commit: cannot list staged paths: BLOCKED" >&2; exit 1; }` then filter. (4) memory.sh `_mem_path_safe` rewrapped ≤ 80 cols, same logic. (5) lib/tests/claude-stub.sh: the generated stub reads the log path from an env var set at install (`CLAUDE_STUB_LOG`) instead of a printf-interpolated literal; `claude_stub_install` exports it.
- [gated 2026-11-13, NEED-DECISION scope → (a), user brief: take the root-cause answer] githooks/pre-commit is generated by lib/gitflow.sh `_gitflow_emit_hook` (~:492): the fail-closed change goes into the generator, and the tracked .githooks/pre-commit copy is regenerated to match (gitflow-test T19d, run-review-guards G5). FILE SCOPE += lib/gitflow.sh (emit block only), .githooks/pre-commit.
- [gated 2026-11-13, round 3 executor deviation accepted] `[ -n "$staged" ] || exit 0` before the filter keeps an empty staged list allowed (grep -v on one empty line returns 0; gitflow-test T21g).
- [gated 2026-11-13, round 3: GATE 1 gaps + GATE 2 MEDIUM/LOWs] (1) state dir without `-p` and without any shellcheck directive: `mkdir -m 700 "$dir" 2>/dev/null` (success = atomic proof), else accept an existing dir only through ONE lstat-based test `[ -n "$(find "$dir" -maxdepth 0 -type d -user "$(id -u)" -perm 700 2>/dev/null)" ]` (find does not follow a symlink operand without -H/-L), else return 0; the parent TMPDIR is created by the session, never by this hook. (2) perl runner `perl -e 'alarm shift; exec { $ARGV[0] } @ARGV' 2` (never a shell). (3) pre-commit filter stage also fails closed: `blocked=$(printf '%s\n' "$staged" | grep -vE '…')`, grep rc 0 = blocked paths, rc 1 = none, rc ≥ 2 = BLOCKED with a message; generator, emitted hook and .githooks copy stay identical. (4) shellcheck clean on every changed .sh, pre-existing findings in touched files included (session-start.sh:186 SC2001, session-digest.test.sh:325 SC2016), without new directives: claude-stub and the precommit shim write their generated script through a quoted heredoc reading an env var (no interpolated path, no SC2016); code-limits.test uses `stat`/`find` instead of `ls` (SC2012) with a portable form (the portability census forbids `stat -c`: use `find -perm`).
## ACCEPTANCE CRITERIA
1. code-limits state dir 700 + owner/symlink check.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'mkdir -m 700 "\$dir"' hooks/code-limits.sh)$(grep -c -- '-type d -user "\$(id -u)" -perm 700' hooks/code-limits.sh)
EXPECT: 11
EVIDENCE: MET exit=0 marker-found :: 11
2. One bounded runner in session-start, with a perl fallback.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c "alarm shift" hooks/session-start.sh)$(grep -c 'command -v timeout' hooks/session-start.sh)
EXPECT: 11
EVIDENCE: MET exit=0 marker-found :: 11
3. pre-commit fails closed on a git diff error.
CHECK: cd /home/bchanot/Documents/claude && grep -c 'cannot list staged paths' githooks/pre-commit
EXPECT: 1
EVIDENCE: MET exit=0 marker-found :: 1
4. memory.sh has no line over 80 cols.
CHECK: cd /home/bchanot/Documents/claude && awk 'length > 80' lib/memory.sh | wc -l
EXPECT: 0
EVIDENCE: MET exit=0 marker-found :: 0
5. claude-stub no longer interpolates the log path.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'CLAUDE_STUB_LOG' lib/tests/claude-stub.sh | grep -c '^[2-9]')$(grep -c "printf '#!/bin/sh.*%s" lib/tests/claude-stub.sh)
EXPECT: 10
EVIDENCE: MET exit=0 marker-found :: 10
6. Regression tests: code-limits (dir mode 700, a pre-planted symlink dir → no signal, no write through it), session-digest (runner falls back to perl when timeout and gtimeout are absent from PATH), a pre-commit case where `git diff --cached` fails → rc 1 + BLOCKED (find the suite that tests githooks/pre-commit; add one if none, under lib/tests/).
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/code-limits.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: MET exit=0 marker-found :: all suites green
7. session-digest suite green.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/session-digest.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: MET exit=0 marker-found :: all suites green
9. Generator, emitted hook and tracked copy agree.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'cannot list staged paths' lib/gitflow.sh githooks/pre-commit .githooks/pre-commit | grep -c ':1$')
EXPECT: 3
EVIDENCE: MET exit=0 marker-found :: 3
10. gitflow suite green.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/gitflow-test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: MET exit=0 marker-found :: all suites green
11. No shellcheck directive added; shellcheck clean on every changed .sh.
CHECK: cd /home/bchanot/Documents/claude && echo $(git diff HEAD | grep -c '^+.*shellcheck disable')$(shellcheck -x hooks/code-limits.sh hooks/session-start.sh lib/gitflow.sh lib/memory.sh lib/tests/claude-stub.sh lib/tests/code-limits.test.sh lib/tests/session-digest.test.sh lib/tests/precommit-failclosed.test.sh >/dev/null 2>&1 && echo OK)
EXPECT: 0OK
EVIDENCE: MET exit=0 marker-found :: 0OK
8. make test green (manual, bun on PATH).
(manual: run 2026-11-13 on round 3, `make test` → all suites green; round 4 is lint-only, its 4 suites green, shellcheck githooks/pre-commit clean)
## FILE SCOPE
hooks/code-limits.sh, hooks/session-start.sh, githooks/pre-commit, lib/memory.sh (:436-439), lib/tests/claude-stub.sh, lib/tests/code-limits.test.sh, lib/tests/session-digest.test.sh, + the pre-commit suite (existing or new lib/tests/*.test.sh).
@@ -0,0 +1,28 @@
# PLAN — feedback-followups (phase 4d follow-ups, block H5) r1
Scope: lib/memory-feedback/{metrics,thresholds,transcripts,reader,labels}.ts + their .test.ts, mods/memory/hooks/register.test.ts (one pin).
- (a) metricsOf date split + retire the 3rd pick (BDR-146 residual). thresholds.ts: drop `T.thirdPick`; add `export const GATE1_FIX_AT = Date.parse('2026-11-12T00:00:00Z')` with a comment (idle ≥ 8 words + cap 2 shipped 7e87d77). metrics.ts: `metricsOf` keeps rows with `at >= GATE1_FIX_AT` for every precision and action; the report prints one line `rows before 2026-11-12 excluded: N` (N may be 0). Remove the `thirdPick` field, `n.third`, the `3rd pick` action (:165-166) and the report row (:212). Every citer of `thirdPick` / `T.thirdPick` / "3rd pick" in skills/memory-feedback/SKILL.md and docs patched in the same commit (grep).
- (b) reader.ts transcript memory: `loaderFor` caches every parsed transcript for the whole run. Process logs grouped by (root, session, agent): sort the joined work by that key, keep only the current group's parsed transcript, drop it when the key changes. Output order of the worksheet unchanged (re-sort by the original index before writing).
- (c) transcripts.ts image prompts: a user prompt whose `message.content` is an array (text + image blocks) is joined from its `type: "text"` blocks' `.text` (image blocks skipped, `[image]` placeholder appended once when any image block exists); today such prompts end `lost`. :76 `a.content.map(String)` (which turns objects into `[object Object]`) uses the same text-block extraction.
- (d) transcripts.ts:127-137 `replyAfter`: the `seen.add` inside `filter` becomes an explicit loop (same output).
- (e) labels.ts minors: `fromLine` also requires an ID `^(BDR|LRN|BLK|EVAL)-[0-9]+$` and a label in {relevant, irrelevant} (error names the line); `readState` rejects non-object JSON (array, null, scalar) with the existing error path; the error for an entry label carrying a cause names the expected shape. `.bak` keeps one state: accepted (stated in a comment, no change).
- (f) tests: metrics.test.ts boundary cases for every remaining threshold action (lateShare, toolKeep, missShare, noisyHits, promoteHits+promoteSessions, idle tie-break of the worst trigger): exactly at the threshold and one step past; the fix-date filter (a row at GATE1_FIX_AT - 1 excluded, at GATE1_FIX_AT kept); reader.test.ts eviction note, torn count, toolItem shapes; transcripts.test.ts array-content prompt joins (text only, text+image, image only); labels.test.ts the three minors; mods/memory register.test.ts: the prompt log record's exact key set (`Object.keys(rec).sort()` toEqual the documented list) next to IR11.
- Not done, justified (TODO removal): `tsc` on lib/memory-feedback (no TypeScript compiler on this machine, no node_modules: adding one = a new dependency; the mods are type-checked by `claude plugin validate --strict` since BLK-049); `l:<root>` shortcut (measured 2026-11-13: 6 `l:` + 7 `p:` keys = 49.7 KB of the 4 MiB cap, 1.2 %; `sweepLogs` tested; no redesign without a measured need).
- Disposition: honors BDR-140 (one thresholds file, spec §4), BDR-146 (gate 1 fixes; pre-fix rows must stop re-emitting the idle and cap actions), BDR-100.
## r2 (correctness FATAL(4), robustness FATAL(5), simplicity FATAL(3); all folded) — r2 WINS over r1
- (a) `GATE1_FIX_AT = Date.parse('2026-11-12T19:53:59Z')` (merge d3d5e6c of 7e87d77; the last pre-fix labelled row is 15:34Z, the 8 records after the merge hold no 3-id injection and no idle trigger). SCOPE of the split (BDR-146 residual = idle + cap): only the prompt precision, `byTrigger` and their actions (promptKeep, triggerFloor) read rows `at >= GATE1_FIX_AT`; they are emitted only when those post-fix prompt entries reach `PROMPT_GATE_N` (gate-2 sample floor), else the report says `post-fix prompt precision: k/20 entries, too few to act`. Gate counts, gate clock, tool, late, ms, miss, noisy and promotion stay on ALL rows (BDR-140 noisy/promotion unchanged; /prune-memory step D keeps its items). Report line placed after the gate lines: `precision rows since ${day(GATE1_FIX_AT)}: N (pre-fix excluded: M)`. state.json untouched. `thirdPick` retired (field, `n.third`, action, row, `T.thirdPick`); no live citer (grep 2026-11-13: only CHANGELOG [3.0.0] and append-only registries); `pick` column KEPT for the 12-column format; CHANGELOG [Unreleased] line via doc sync.
- (b) one-slot transcript cache keyed by the resolved file path (what `read` keys on); work sorted by that path, item `n` = original index + 1 assigned BEFORE sorting, output re-sorted by `n`; `torn` summed at each load (not over the live cache). Proof: peak RSS (`/usr/bin/time -v`) of `worksheet` on a copy of the real population, before and after, quoted in the report.
- (c) DROPPED. Measured: 115/115 hook attachments are string arrays (`.map(String)` exact), 0 human prompts with array content in the 636 MB corpus, 0 `lost` rows of 112; a lost record is already visible ("Lost records: N"). :76 unchanged. TODO line closed with that measurement.
- (d) `replyAfter`: `[...new Map(…flatMap(textsOf)).values()].join('\n')` (same output, no side effect).
- (e) `fromLine`: label ∈ the full `Label` set (one const: DRAFT_LABELS + ask + lost …); id = '' for ask/lost/test/checked, `ID_RE` (labels.ts:14, reused) for relevant/irrelevant/missed; channel ∈ {prompt, tool}; `at` finite. Returning `undefined` keeps the existing line-numbered error; its text stops advising a .bak restore as the fix and says what the .bak holds (the state before the LAST append). `readState` rejects non-object JSON through the existing error path.
- (f) tests: metrics.test clock starts at GATE1_FIX_AT (+ one pre-cutoff 3-id row at 2026-11-12T15:00Z excluded from precision, kept in noisy/promote); one table-driven at-threshold test per comparator not yet tested at its boundary (lateShare 1/4, missShare 1/3, toolKeep 1/2, msP90 2500, promptKeep 16/25, triggerFloor, promote 3 hits/2 sessions, idle tie-break), noisyHits already covered (M3); the gate-2 floor (19 vs 20 post-fix entries); reader.test T0 moved after GATE1_FIX_AT, an interleaved A,B,A session test (order, `n`, header numbers), torn across 2 files, R8 "each file read once" kept; labels.test readRows on a fixture with every label kind (round-trip of `rowsOf`), the three checks; register.test.ts key-set pin of the prompt log record next to IR11.
## r3 (confirmation CONCERNS(3), folded; FINAL, no further challenge)
- Post-fix split: prompt precision, `byTrigger`, `n.prompt` AND the sampled-miss share (`missShare`, `sampled`) read entries/asks `at >= GATE1_FIX_AT` (the 5 pre-fix sampled misses were diagnosed and accepted by BDR-146). Gate counts, clock, tool, late, ms, noisy, promotion stay on all rows (step D dedupes noisy/promotion).
- The floor gates the ACTIONS only (promptKeep, triggerFloor, missShare): it is `itemsOf(postFix, 'prompt') >= PROMPT_GATE_N` in ASKS; the report line reads `post-fix prompt asks: k/20 (since <ISO of GATE1_FIX_AT>), pre-fix entries excluded: M` placed after the gate lines (ISO timestamp, never the bare day). Precision and byTrigger are always computed (M2/M10 keep their semantics on rebased clocks).
- state.json gains one OPTIONAL key `postFixAt` (absent in today's file = not reached; old files stay readable): `nextState` sets it once when the floor is first crossed and prints `NEW: post-fix prompt sample reached (k/20 asks)` so /memory-feedback STEP 6 proposes the gate-2 EVAL.
- Read-once proof: interleaved A,B,A test with ONE torn line in A only → `Skipped 1 torn transcript lines` (2 = re-read); the "R8 kept" claim is withdrawn (no such test existed).
- labels.ts: `at` must match `/^\d+$/`; the line error keeps the literal `labels.tsv.bak` (L7) and says the .bak holds the state before the last append.
- thirdPick citer: CHANGELOG.md:13 is under [Unreleased] and stays true (historical measure); the doc sync adds the retirement note next to it.
- IR11 pin: two key sets pinned, an injected record (with cands + ms) and a `none` outcome record.