chore(memory): index fail-closed journal

This commit is contained in:
bchanot committed 2026-11-15 20:40:53 +01:00
1 parent 038670e4e8
commit 1a4759d752
4 files changed
+179 -3

No files matched your search

+1
View File
@@ -761,3 +761,4 @@ rules:
## 2026-11-15
- Docker host-root mount deny shipped 0f73d7e: 28 rows (7 shapes x 4 prefixes), bench-proven echo analogue, verifier CONFORME, security PASS. Legacy rows dead, kept. Wrapper/podman residual added TODO:90. Docs committed.
- A2 line width: 4 lines >80 cols re-wrapped (register.ts 591/683/3765, transcripts.ts 133; audit line numbers had drifted), no behavior change, make test green, security PASS.
- A3 index fail-closed: memory_index rc 2 on non-regular/unreadable registry with kind, lax skip+stderr without kind (register.ts recall keeps rc 0); digest unknown on unparsable BLK row; SD9b/c re-pinned, SD9f/g/h + T20* added (87aecc1). Residuals BUGS-FOUND 07f9a7f. Next A4.
@@ -0,0 +1,27 @@
# CONTRACT — index-fail-closed (BDR-125 gap audit A3, BUGS-FOUND:87)
- date: 2026-11-15 | flow: bugfix | branch: bugfix/index-fail-closed (off develop)
- status: active
## REQUEST (verbatim — IMMUTABLE)
A3 (BDR-125 gap audit): memory_index on an unreadable or non-regular blockers.md (permission denied, a directory, a FIFO) must return non-zero so the session digest prints `- open: unknown (memory.sh index failed)` instead of `- open: none`. Today an unreadable blockers.md can yield `- open: none`, a false all-clear. Re-pin session-digest tests SD9b/SD9c to the new behavior.
## CLARIFICATIONS
- [internal 2026-11-15, challenge r1-r3] plan .claude/tasks/plans/2026-11-15-index-fail-closed-1200.md: r2 + r3 win over the original FIX PLAN. Strict (kind named) → rc 2; lax (no kind, used live by mods/memory register.ts) → skip bad registry with stderr line, rc 0. Existing unknown line reused verbatim. Empty/unknown status words stay not-open (v1 rows in game/config).
## ACCEPTANCE CRITERIA
1. session-digest suite green (SD9b/SD9c re-pinned to unknown, SD9f/g/h added).
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/session-digest.test.sh </dev/null 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: MET exit=0 marker-found :: all suites green
2. memory suite green (strict rc 2 on dir/FIFO, lax rc 0 with stderr, healthy rc 0 and byte-identical output).
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/memory.test.sh </dev/null 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: MET exit=0 marker-found :: all suites green
3. SD9b and SD9c assert the unknown line, not none.
CHECK: cd /home/bchanot/Documents/claude && grep -cE 'check SD9[bc]-unknown' lib/tests/session-digest.test.sh
EXPECT: 2
EVIDENCE: MET exit=0 marker-found :: 2
4. Full make test green; shellcheck clean on touched shell files; plan r3 items all implemented; regression tests fail on the pre-fix lib (manual).
## FILE SCOPE
lib/memory.sh (memory_index, usage line :12), hooks/session-start.sh (_digest_blockers), lib/tests/session-digest.test.sh, lib/tests/memory.test.sh, agents/handover-doc-writer.md (:120 one sentence).
@@ -1,9 +1,9 @@
# BDR-125 gap audit 2026-11-14 (3 analyzers, all phases vs original asks) — consolidated
## Batch A — code (one skill each)
- A1 /bugfix docker host-root mount deny (phase 2, TODO:89): plan .claude/tasks/plans/2026-11-14-docker-mount-deny-1400.md r2 (confirmation pending).
- A2 /hotfix line width regressions: mods/memory/hooks/register.ts:976 (86 col, breaks 4b c10 / 4c c9 oracles), lib/memory-feedback/transcripts.ts:133 (follow-ups c7), register.test.ts:71 optional.
- A3 /bugfix memory_index rc 0 on unreadable/non-regular blockers.md → digest `- open: none` (4c digest defect, BUGS-FOUND:87): non-zero rc → `- open: unknown`; SD9b/SD9c re-pinned.
- [x] A1 DONE merged b03aced (0f73d7e). /bugfix docker host-root mount deny (phase 2, TODO:89): plan .claude/tasks/plans/2026-11-14-docker-mount-deny-1400.md r2.
- [x] A2 DONE merged 2315a0b (21ef47f). /hotfix line width regressions: mods/memory/hooks/register.ts:976 (86 col, breaks 4b c10 / 4c c9 oracles), lib/memory-feedback/transcripts.ts:133 (follow-ups c7), register.test.ts:71 optional.
- [x] A3 DONE 87aecc1 (docs 038670e, residuals BUGS-FOUND 07f9a7f). /bugfix memory_index rc 0 on unreadable/non-regular blockers.md → digest `- open: none` (4c digest defect, BUGS-FOUND:87): non-zero rc → `- open: unknown`; SD9b/SD9c re-pinned.
- A4 memory.sh `check --staged`: new entry ID = max+1 (4a ask TODO:170; today comment "gap harmless" :162).
- A5 memory.test.sh T18 hard-fails without bun on PATH → SKIP line like other bun-gated tests.
- A6 (user choice) routing guard fail mode (phase 2 shared rule "guards fail closed"; today fail-open).
@@ -0,0 +1,148 @@
# PLAN — index-fail-closed (BDR-125 gap audit A3, BUGS-FOUND:87)
## DIAGNOSIS
BUG: session digest prints `- open: none` when blockers.md is a directory,
a FIFO, unreadable, or when the index emits unparsable `| BLK-` rows: a
false all-clear.
ROOT CAUSE:
1. lib/memory.sh:594-603 `memory_index`: `[ -f "$file" ] || continue`
treats a present non-regular registry like an absent one; the awk rc of
`_mem_index_rows` is dropped (the function's rc is the trailing `echo`).
So `memory.sh index blocker` exits 0 with no rows.
2. hooks/session-start.sh:331-351 `_digest_blockers`: a `| BLK-` row whose
fields do not parse is skipped (`continue`), so malformed output with
rc 0 also ends as `- open: none`.
EVIDENCE: SD9b (stub prints malformed rows, rc 0) and SD9c (blockers.md is a
directory) both pin `- open: none` today (lib/tests/session-digest.test.sh:
206-213). BUGS-FOUND.md:87 states the same.
RELATED MEMORY: BDR-147/BDR-144 (digest is the start-of-session source:
a false none hides blockers, so fail closed honors them); CLAUDE.global.md
"`- open: unknown` → read blockers.md yourself" (the existing line is the
fail-closed output, reused verbatim, no new public string); LRN-274/LRN-233
(pipefail callers, capture output). No BLK names this cause.
## FIX PLAN
1. lib/memory.sh `memory_index`:
- `[ -e "$file" ] || continue` (absent stays silent, as today; a
dangling symlink counts as absent: `-e` false).
- present but not (`-f` and `-r`) → `echo "memory: $file: not a readable
regular file" >&2; return 1` (never opens it: a FIFO cannot block).
- `_mem_index_rows "$file" "${spec#* }" || return 1`.
Applies to every kind (`index` with no kind too): a bad registry fails
the whole call, the callers (digest, prune-memory, memory-feedback) then
say unknown instead of reading a partial table.
2. hooks/session-start.sh `_digest_blockers`: buffer the `- open: BLK…`
lines in a local variable instead of echoing inside the loop; a `| BLK-`
row that does not split into id | date | title | status (the existing
`[[ "$rest" == *" | "* ]] || continue` branch) → print the existing
`- open: unknown (memory.sh index failed)` line and `return 0`. Only on
a clean pass print the buffered lines, then the none/more lines as today.
Banner and exit code unchanged (rc 0).
3. Tests:
- lib/tests/session-digest.test.sh SD9b-none → SD9b-unknown (exact
unknown line, count 1, and `- open: none` count 0); SD9c-none →
SD9c-unknown likewise. New SD9f: blockers.md is a FIFO (mkfifo in the
test temp project) → unknown line, run returns (bounded by the
harness's existing timeout), rc 0. New SD9g: blockers.md mode 000 →
unknown, skipped with a SKIP line when `id -u` is 0.
- lib/tests/memory.test.sh: new T-index-dir: `index blocker` with
blockers.md a directory → rc 1 and stderr names the file; T-index-ok
unchanged rows still rc 0 (existing T1f covers it).
- Fix the stale comment at session-digest.test.sh:245 only if it is the
SD9 block (it is not: out of scope, C5 handles it).
RISK: low. A caller that relied on rc 0 with a broken registry now gets
rc 1; grep shows the only runtime caller is the digest (already handles
non-zero). Skill callers run it interactively and print the error.
## FILES
lib/memory.sh, hooks/session-start.sh, lib/tests/session-digest.test.sh,
lib/tests/memory.test.sh.
## r2 (challenge r1: correctness CONCERNS(1), robustness CONCERNS(2), simplicity SOLID) — SUPERSEDES FIX PLAN 1-3 and RISK above
BLOCKERs landed: (a) `index` with no kind is a live runtime caller
(mods/memory/hooks/register.ts:671, stdout dropped on rc != 0 at :657 →
recall blackout); (b) `-e` false ≠ absent (dangling symlink, unsearchable
.claude/memory). Minors folded: rc 2 per lib convention (memory.sh:528,598),
`-f` guard only and awk rc propagation tested by mode 000, no buffer in the
digest, SD9f replaced by a timed memory.test.sh FIFO case, explicit rc 0
asserts on healthy trees.
1. lib/memory.sh `memory_index` (extract a per-registry helper if the
function passes 25 logic lines):
- strict = a kind was named (`index blocker`); lax = no kind.
- `.claude/memory` absent → every registry absent (silent, rc 0, as today).
- `.claude/memory` present but not searchable (`! -x`) → BAD.
- file absent = `! -e` AND `! -L` → silent skip (as today).
- present but `! -f` (dir, FIFO, dangling symlink, socket) → BAD, never
opened.
- rows captured first: `rows=$(_mem_index_rows …)`; awk rc != 0 (mode 000:
gawk exits 2) → BAD. Header and rows printed only after a clean read,
so no partial table reaches stdout.
- BAD → stderr `memory: <file>: not a readable regular file`; strict →
`return 2`; lax → skip this registry, continue, final rc 0
(register.ts keeps the healthy registries; it already reports bad files
itself via tryRead).
2. hooks/session-start.sh `_digest_blockers`: the `[[ "$rest" == *" | "* ]]
|| continue` branch → `{ echo "- open: unknown (memory.sh index
failed)"; return 0; }`. No buffering (real memory.sh never emits such a
row; this guards version skew/stubs; an earlier printed BLK line plus
unknown still fails closed). Empty or unknown status words stay "not
open" (game has 14, config 3 empty-status rows from v1: treating them as
unknown would flag every session) → residual in BUGS-FOUND.
3. Tests:
session-digest.test.sh: SD9b → exact unknown line count 1 and
`- open: none` count 0; SD9c (dir) → same; new SD9f dangling symlink
blockers.md → unknown; SD9g blockers.md mode 000 → unknown; SD9h
.claude/memory mode 600 (not searchable) → unknown, restore 700 after
the run; SD9g/SD9h print SKIP when `id -u` is 0. chmod only on literal
paths inside the test's own temp project, never -R.
memory.test.sh: healthy `index blocker` rc 0 and healthy `index` rc 0
(explicit); blockers.md a directory → `index blocker` rc 2 + stderr names
blockers.md; same tree `index` (no kind) → rc 0, stderr warning, a
learnings row still printed; blockers.md a FIFO (mkfifo in the test's
mktemp dir) → `timeout 5 bash "$MEM" index blocker` rc 2 (not 124).
4. Out of the executor's scope, handled by the run's closing steps: README.md
:167 / USAGE.md:220 wording (doc sync), BUGS-FOUND:87 marked FIXED + a new
residual line (unknown/empty status words counted not-open; an unclosed
``` fence hides later entries with no banner warning).
RISK (corrected): runtime callers are the digest (`index blocker`, strict)
and register.ts (`index`, lax: unchanged rc 0, bad registry now skipped with
a stderr line instead of silently). Skill/agent callers (prune-memory with
a kind, memory-feedback without) get rc 2 / a stderr line on a broken
registry, which they print.
## r3 (re-challenge: correctness SOLID, robustness SOLID; minors folded) — amends r2
- Dir rule: `.claude/memory` present (`-e` or `-L`) but `! -d` or `! -x` →
BAD with its own stderr line `memory: .claude/memory: not a searchable
directory`; absent only when neither `-e` nor `-L` holds AND `.claude`
(if present) is searchable, else BAD.
- Healthy output byte-identical: print a registry's table only after the
read; rows printed only when non-empty (`[ -n "$rows" ] && printf
'%s\n' "$rows"`), then the trailing blank line as today. Test: full
`index` output on the shared healthy $FIX equals a capture taken with the
pre-fix logic (store the expected text literally in the test).
- Usage line lib/memory.sh:12: name the contract `index [<kind>]` → with a
kind rc 2 on an unreadable registry; without, bad registries skipped
with a stderr line, rc 0.
- agents/handover-doc-writer.md:120: one sentence: `index decision` rc != 0
→ write `MEMORY: decisions reader unavailable` instead of an empty §6.1.
(5th file in scope.)
- Test placement: SD9f/g/h each on its own `mkproj`, run with `run "$p" -`
(real lib, as SD9c), never `"$S9"` (still the malformed stub). The
memory.test.sh dir/FIFO/no-kind cases use their own `-C` subtree (pattern
v1h, memory.test.sh:320) seeded with one BLK and one LRN entry; the shared
$FIX blockers.md is never touched.
- Skip idiom: after chmod, SKIP when the read still works (`[ -r
blockers.md ]` for SD9g, `[ -x .claude/memory ]` for SD9h), as
push-guard.test.sh:220-222. SD9h restores 700 right after the run
(`chmod 700 … || true`) AND in the EXIT trap before `rm -rf`.
- Residuals (BUGS-FOUND, closing step): empty/unknown status words counted
not-open; unclosed ``` fence hides later entries without a banner line;
`index` reads without the lock while `status`/`_mem_restore` rewrite in
place (truncate then write, memory.sh:262,319): a racing digest can read
an empty file.
FILES (final): lib/memory.sh, hooks/session-start.sh,
lib/tests/session-digest.test.sh, lib/tests/memory.test.sh,
agents/handover-doc-writer.md.