chore(memory): staged next-ID journal

This commit is contained in:
bchanot committed 2026-11-15 22:15:58 +01:00
1 parent eb801b7620
commit 9a8d4ae0ef
4 files changed
+238 -1

No files matched your search

+1
View File
@@ -762,3 +762,4 @@ rules:
- Docker host-root mount deny shipped 0f73d7e: 28 rows (7 shapes x 4 prefixes), bench-proven echo analogue, verifier CONFORME, security PASS. Legacy rows dead, kept. Wrapper/podman residual added TODO:90. Docs committed.
- A2 line width: 4 lines >80 cols re-wrapped (register.ts 591/683/3765, transcripts.ts 133; audit line numbers had drifted), no behavior change, make test green, security PASS.
- A3 index fail-closed: memory_index rc 2 on non-regular/unreadable registry with kind, lax skip+stderr without kind (register.ts recall keeps rc 0); digest unknown on unparsable BLK row; SD9b/c re-pinned, SD9f/g/h + T20* added (87aecc1). Residuals BUGS-FOUND 07f9a7f. Next A4.
- A4 staged next-ID: check --staged refuses new ## entry not = max+1 (line-order replay of _mem_next_id, body IDs feed max only); runs on v2 HEAD with entries + no open fence, so upgrade/first commit/template import pass (e0311f4). Challenge r2 caught upgrade false refusal (trucks_at_work 17536ce replay). Residuals BUGS-FOUND 074cdde. Next A5.
@@ -0,0 +1,25 @@
# CONTRACT — staged-next-id (BDR-125 gap audit A4)
- date: 2026-11-15 | flow: feat | branch: feature/staged-next-id (off develop)
- status: active
## REQUEST (verbatim — IMMUTABLE)
A4 (BDR-125 gap audit, phase 4a ask): `memory.sh check --staged` must reject a staged registry whose new entry ID is not max+1 of the IDs already committed (today a comment in lib/memory.sh says the gap is harmless and only duplicates are refused). A new entry with a skipped ID (e.g. BLK-052 when max is BLK-050) or a reused one fails the check with a named error; entries written by `memory.sh add` (next-free ID) still pass. Regression tests in lib/tests/memory.test.sh.
## CLARIFICATIONS
- [internal 2026-11-15, challenge r1] plan .claude/tasks/plans/2026-11-15-staged-next-id-2030.md section r2 wins over the original APPROACH/EDGE CASES/TESTS. Rule replays `_mem_next_id` in line order: a new entry = unfenced `## P-n` heading absent from HEAD's `## ` set, must equal 1 + the max ID seen at any level (fenced or not) in HEAD and above it in the staged file; body lines feed the max, never count as entries, so whatever `add` writes passes. A registry absent from HEAD skips the sequence rule (first commit of a populated registry with holes passes; renumbering is forbidden). A reused ID present in HEAD's `## ` set stays the duplicate rule's job. Only v2 staged files are checked (existing rule). lib/capitalize-commit.md rc 6 cause list gains the new cause. The other 4a validator item (new BLK quotes an error verbatim) is out of scope.
- [internal 2026-11-15, re-challenge r2] plan section r3 amends r2: the sequence rule runs only on a v2 HEAD with no open fence and at least one shape-valid `## ` entry (an `upgrade` commit from v1, a first commit, or an import onto a committed empty template skips it); known entries match by exact ID token; the rc 6 remedy says a committed ID is never renamed. Tests T8m (upgrade passes) and T8n (template import passes) added.
- [internal 2026-11-15, gate 0] criterion 2 CHECK wrapped in `|| true`: `grep -c` exits 1 on zero matches while printing the expected 0; the criterion itself is unchanged.
## ACCEPTANCE CRITERIA
1. memory suite green, with the new skipped-ID, lower-ID and multi-add cases.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/memory.test.sh </dev/null 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: MET exit=0 marker-found :: all suites green
2. The "gap is harmless" comment is gone from lib/memory.sh.
CHECK: cd /home/bchanot/Documents/claude && { grep -c 'gap is harmless' lib/memory.sh || true; }
EXPECT: 0
EVIDENCE: MET exit=0 marker-found :: 0
3. Full make test green; shellcheck clean on lib/memory.sh; the refusal tests (skipped ID, gap fill, body-only reuse) fail on the pre-change lib while the pass tests pass on both; reused-ID test asserts the duplicate refusal; existing T7k (merge duplicate, staged append) and T8 (single add) still pass (manual).
## FILE SCOPE
lib/memory.sh (_mem_next_id comment :163-165, _mem_check_staged :536-556, a new helper, usage lines :15-17), lib/tests/memory.test.sh (new T8f.. cases after T8e), lib/capitalize-commit.md (rc 6 cause list :79-91).
@@ -4,7 +4,7 @@
- [x] A1 DONE merged b03aced (0f73d7e). /bugfix docker host-root mount deny (phase 2, TODO:89): plan .claude/tasks/plans/2026-11-14-docker-mount-deny-1400.md r2.
- [x] A2 DONE merged 2315a0b (21ef47f). /hotfix line width regressions: mods/memory/hooks/register.ts:976 (86 col, breaks 4b c10 / 4c c9 oracles), lib/memory-feedback/transcripts.ts:133 (follow-ups c7), register.test.ts:71 optional.
- [x] A3 DONE 87aecc1 (docs 038670e, residuals BUGS-FOUND 07f9a7f). /bugfix memory_index rc 0 on unreadable/non-regular blockers.md → digest `- open: none` (4c digest defect, BUGS-FOUND:87): non-zero rc → `- open: unknown`; SD9b/SD9c re-pinned.
- A4 memory.sh `check --staged`: new entry ID = max+1 (4a ask TODO:170; today comment "gap harmless" :162).
- [x] A4 DONE e0311f4 (tests strengthened in same commit, CHANGELOG eb801b7, residuals BUGS-FOUND 074cdde). memory.sh `check --staged`: new entry ID = max+1 (4a ask TODO:170; today comment "gap harmless" :162).
- A5 memory.test.sh T18 hard-fails without bun on PATH → SKIP line like other bun-gated tests.
- A6 (user choice) routing guard fail mode (phase 2 shared rule "guards fail closed"; today fail-open).
@@ -0,0 +1,211 @@
# PLAN — staged-next-id (BDR-125 gap audit A4)
## CONTEXT
- lib/memory.sh:163 `_mem_next_id <file> <PREFIX>`: max over every heading
level (`^#+ *PREFIX-[0-9]+`), fenced or not, + 1. Comment :164 says "a gap
is harmless, a duplicate is not".
- lib/memory.sh:536 `_mem_check_staged <file>`: copies HEAD blob ($h) and
staged blob ($s); on a v2 staged file runs `memory_check_file "$s" PREFIX
"$h" "$1"` (new culprits: shape, duplicate, Index, open fence) and
`_mem_removed_lines`. Caller: lib/memory-commit.sh:99 (rc 6 chokepoint,
BDR-129). No ID-sequence rule today.
- TODO.md:171 (phase 4a ask): `check [--staged]` = validator "ID = max+1
and unique".
## APPROACH
1. New helper in lib/memory.sh, next to `_mem_check_staged`:
`_mem_check_next_ids <name> <PREFIX> <head copy> <staged copy>` → rc 2
(stderr) on a bad new ID, rc 0 otherwise.
- One awk over both files. ID extraction = the `_mem_next_id` regex
(`match($0, "^#+ *" p "-[0-9]+")`, every level, fenced or not), numeric
value of the digits.
- HEAD: set `old[n]`, `max` = largest n (0 when the HEAD blob is empty or
absent).
- Staged: new = n not in `old`, deduplicated (a duplicate heading is
already the culprit rule's job).
- Sort new numerically (awk has no sort in POSIX: collect into an array
indexed by n and walk n from 1 to the largest new n; IDs are small).
Hmm: walk only from min(new) to max(new) — bounded by the staged file's
largest ID.
- expected = max + 1; for each new n in ascending order: n != expected →
print `memory: <name>: new entry <PREFIX>-<n padded %03d> is not the
next free ID <PREFIX>-<expected %03d>` and set bad; then expected =
n + 1 when n == expected, else keep expected (report every bad one,
the next good one would also mismatch: simpler = report the first
mismatch only, then stop). Decision: report the first mismatch only,
one line, rc 2.
- The printed ID uses the heading's own digits (keep zero padding as
written) for the bad one, `%03d` for the expected one (what `add`
writes).
2. `_mem_check_staged`: in the v2 branch, after `memory_check_file`, add
`_mem_check_next_ids "$1" "$(_mem_prefix "$1")" "$h" "$s" || rc=2`.
HEAD absent → $h empty → max 0, first new entry must be 001.
3. Comment :164 → "every heading level, fenced or not: check --staged
reads IDs the same way, so a new ID must be this max + 1; a duplicate
is refused too" (keep the trucks_at_work / stray fence reason).
4. Usage lines :15-17: `--staged` adds "new IDs = HEAD max + 1, in order".
## EDGE CASES
- Several `add` calls staged together: new = max+1..max+k, consecutive →
pass.
- Merge leftover duplicate (T7k): the duplicate ID is in HEAD → not new;
the appended entry is max+1 → pass.
- A new ID already present in HEAD (reuse): not "new" by set membership;
the duplicate culprit rule already refuses it when both are `## ` level.
A new ID below max and absent from HEAD (fills an old gap) → refused.
- v1 staged file → skipped as today (branch only runs for v2).
- HEAD v1, staged v2 (`upgrade`): headings keep their IDs → no new IDs.
- Fenced heading-like line in a new entry body (`## BLK-999` in a code
block): counted, like `_mem_next_id` counts it, so refused unless it is
the next ID. Accepted: such a line would also move `add`'s next ID.
- IDs with 4+ digits: numeric compare.
## TESTS (lib/tests/memory.test.sh, after T8e, same fixture $B, committed
base restored after each case with `git reset -q && cat b.keep > "$B"`)
- T8f-two-adds-pass: two `memory.sh add blocker`, stage, `check --staged`
rc 0.
- T8g-skip-refused: append by printf a v2 heading with ID max+2, stage →
rc 2, stderr names `is not the next free ID` once.
- T8h-gap-fill-refused: build a HEAD with a gap (commit a file whose IDs
are 001 and 003 via printf), stage a new `## BLK-002 — …` → rc 2, named.
- T8i-new-file-first-id: a registry absent from HEAD, staged v2 with only
BLK-002 → rc 2; with BLK-001 → rc 0. Use its own `-C` subtree or a
separate repo dir under $FIX so $B stays untouched.
- Existing T7k and T8 must stay green.
## DISPOSITION
Honors BDR-129 (memory.sh single writer, `check --staged` = rc 6
chokepoint in memory-commit.sh): the rule lives in the same validator, no
new caller. LRN/BLK: none bears on ID allocation.
## FILES
lib/memory.sh, lib/tests/memory.test.sh.
## r2 (challenge r1: correctness CONCERNS(1 MAJOR), robustness SOLID (5 minor), simplicity CONCERNS(1 MAJOR)) — SUPERSEDES APPROACH, EDGE CASES and TESTS above
Landed: (a) body lines (`### BLK-n`, fenced `## BLK-n`) were read as new
entries → a valid `add` refused (correctness 1, robustness 2); (b) the
ascending walk was bounded by a file-controlled number (corr 3, rob 1);
(c) a registry absent from HEAD with ID holes would be refused and
renumbering is forbidden (rob 3); (d) no test for the reused-ID case the
request names (simp 1, corr 4); (e) T8h moved the shared fixture's HEAD
(corr 5, rob 5, simp 4); (f) rc 6 guide lacks the new cause (rob 4).
1. Rule = replay of `_mem_next_id` as it ran when `add` wrote each entry.
New helper `_mem_check_next_ids <name> <PREFIX> <head copy> <staged
copy>` → rc 2 + one stderr line on the first bad entry, rc 0 otherwise.
One awk over the two files, `FILENAME == ARGV[1]` for the HEAD pass
(memory_check_file idiom; an empty HEAD file yields no lines):
- HEAD pass: `top` = max ID over every `^#+ *P-[0-9]+` line, every
level, fenced or not (the `_mem_next_id` reading); `entry[n]` set for
each unfenced `^## P-[0-9]+` heading (the `_mem_culprits` reading:
toggle `fence` on `^```` lines).
- Staged pass, in line order, `top` carried over from HEAD:
a line matching `^## P-[0-9]+` outside a fence whose n is not in
`entry` is a NEW entry: n != top + 1 → print `memory: <name>:<line>:
new entry <P>-<n as written> is not the next free ID <P>-<top+1 as
%03d>` and exit 2 (first bad entry only). Then, for every line
matching `^#+ *P-[0-9]+` (fenced or not, any level, new entry heading
included), top = max(top, n).
- Consequence: body lines feed `top` but are never entries, so whatever
`add` writes passes by construction; a reused ID absent from HEAD's
`## ` set (body-only in HEAD, or a gap fill) is n <= top → refused; a
reused ID present in HEAD's `## ` set is left to the existing
duplicate culprit. No sort, no walk.
2. `_mem_check_staged`: in the v2 branch, after `memory_check_file`, call
it only when the HEAD blob exists (`hrc` = 0): `[ "$hrc" = 1 ] ||
_mem_check_next_ids "$1" "$(_mem_prefix "$1")" "$h" "$s" || rc=2`.
A registry absent from HEAD (first commit of a populated or imported
registry, holes allowed) skips the sequence rule; shape and duplicate
rules still apply.
3. Comment lib/memory.sh:164 → say that check --staged replays this same
reading (new `## ` entry = this max + 1), keep the trucks_at_work /
stray fence reason; drop "a gap is harmless".
4. Usage lines lib/memory.sh:15-17: `--staged` also refuses a new entry
whose ID is not the next free ID (registry already in HEAD).
5. lib/capitalize-commit.md rc 6 cause list (:79-91): add "a new entry
whose ID is not the next free ID (stderr names the expected one: rename
that uncommitted heading, or remove it and re-add with memory.sh add)".
Keep the paragraph's wrapping style.
TESTS (lib/tests/memory.test.sh, after T8e):
- In the shared fixture ($B, HEAD = BLK-001..003, restore with the
existing `git reset -q && cat b.keep > "$B"`, never commit):
- T8f-two-adds-pass: two `memory.sh add blocker`, stage, rc 0.
- T8g-skip-refused: printf-append `## BLK-005 — skip (2026-10-27)` +
a body line, stage → rc 2, stderr has `not the next free ID BLK-004`
once.
- T8h-reuse-dup: append `## BLK-002 — again (2026-10-27)`, stage → rc
2, `duplicate BLK-002` once, `not the next free ID` 0 times.
- T8i-body-ids-pass: `memory.sh add blocker` with a body holding a line
`### BLK-099 note` and a fenced block with `## BLK-001 — quoted`;
then a second `add`; stage → rc 0 (second entry is BLK-100).
- In a separate repo under $FIX (git init like t17, `reg` helper or
printf with v2 frontmatter; run `(cd <dir> && bash "$MEM" check
--staged)`):
- T8j-first-commit-holes-pass: blockers.md absent from HEAD, staged with
BLK-001 and BLK-003 → rc 0.
- commit it; T8k-gap-fill-refused: append `## BLK-002 — fill`, stage →
rc 2, `not the next free ID BLK-004`.
- T8l-body-only-reuse-refused: HEAD entry body holds `### BLK-010
note` (commit), stage a new `## BLK-010 — x` → rc 2 (expected
BLK-011).
Refusal cases (T8g, T8k, T8l) must fail on the pre-change lib; pass cases
pass on both.
FILES (final): lib/memory.sh, lib/tests/memory.test.sh,
lib/capitalize-commit.md.
## r3 (re-challenge: correctness CONCERNS(1 MAJOR), robustness CONCERNS(1 MAJOR), same finding) — amends r2
Landed: r2 refused a real `memory.sh upgrade` commit (v1 HEAD with
`### BDR-n` headings rewritten to `## BDR-n`; replayed on trucks_at_work
17536ce: passes today, r2 → rc 2) and its remedy steered toward
renumbering. Minors folded below.
1. Gate (in `_mem_check_staged`, v2 branch): run `_mem_check_next_ids`
only when the HEAD blob exists (hrc 0) AND `memory_is_v2 "$h"` AND
`! _mem_fence_open "$h"`. A v1 HEAD (upgrade commit), an absent HEAD
(first commit) or a HEAD with an open fence (the open-fence culprit
already speaks) skips the sequence rule; shape and duplicate rules still
apply. Inside the awk: no shape-valid `## ` entry in HEAD (empty or
template-only registry) → exit 0 (import onto a committed template with
ID holes passes).
2. Known entry = exact ID token. HEAD pass: `entry[id]` for each unfenced
heading matching the shape `^## P-[0-9][0-9][0-9]+ — .` (the
`_mem_culprits` shape), id = the `P-digits` token. Staged pass: an
unfenced line matching `^## P-[0-9]+` whose `P-digits` token is not in
`entry` is NEW (catches re-padded `BLK-0002` and a reuse of a HEAD
heading that had a bad shape: n <= top → refused).
3. `FNR == 1 { fence = 0 }` in the awk so the staged pass never inherits
the HEAD pass's fence state.
4. Comment replacing lib/memory.sh:164 states the reading and its two
assumptions in a few lines: check --staged replays this max in line
order on a v2 HEAD, so a new `## ` entry must be this max + 1; holds
when entries are appended by memory.sh (a hand-inserted ID line above
a new entry, or an ID line removed above the first entry, can refuse a
valid add).
5. lib/capitalize-commit.md rc 6 cause text: "a new entry whose ID is not
the next free ID (stderr names the expected one). A committed ID is
never renamed: fix only the uncommitted new heading and every
uncommitted mention of its ID (the `superseded by` line `add
--supersedes` wrote, the journal line), or drop the new entry and that
line and re-add it with memory.sh add."
6. Tests, amended:
- Separate repos under $FIX: create files with printf or `(cd <dir> &&
reg …)` inside the subshell only, never `reg` from $FIX (it would
overwrite the shared fixture). After T8k: `git reset -q && git
checkout -- .` in that repo before T8l's commit.
- New T8m-upgrade-pass: own repo, HEAD v1 blockers.md with `### BLK-001
— a` and `### BLK-002 — b` (plus whatever v1 shape `memory.sh
upgrade` accepts; read `_mem_upgrade_headings` / existing T9 `up/`
fixture at memory.test.sh:203 for a valid v1 input), run `memory.sh
upgrade` there, stage, `check --staged` rc 0.
- New T8n-template-import-pass: own repo, HEAD = v2 blockers.md with
frontmatter and no entry, staged adds `## BLK-001 — a (2026-10-27)`
and `## BLK-005 — b (2026-10-27)` → rc 0.
- T8j (absent from HEAD, holes) stays a pass.
Refusal cases that must fail on the pre-change lib: T8g, T8k, T8l.
7. Out of scope, logged to BUGS-FOUND at close: memory-commit.sh checks
the index (:99) but `git commit -- <paths>` (:118) re-reads the
working tree, unlocked: a write between the two is committed
unchecked (pre-existing).
FILES unchanged: lib/memory.sh, lib/tests/memory.test.sh,
lib/capitalize-commit.md.