Merge bugfix/feedback-followups into develop

This commit is contained in:
bchanot committed 2026-11-13 19:51:17 +01:00
commit a967fd93dc
22 files changed
+584 -78

No files matched your search

+1
View File
@@ -1562,6 +1562,7 @@ Supersedes [[BDR-030]].
- **Alternatives rejected**: drop idle (user: tighten); index body (already done); add `promoted` to `_MEM_WORDS` (classifier change for annotation only).
- **Residual**: metricsOf has no fix-date split, pre-fix rows keep 3rd-pick + idle actions in later FEEDBACK.md (TODO); tool-channel noisy entries not fixed by title rewrite (body spans).
- **Reference**: feature/recall-gate1-fixes 7e87d77 + 39f9790; contract .claude/tasks/contracts/2026-11-12-recall-gate1-fixes-1830.md; plan r3; gates: 3 lenses (BLOCKER x3) + confirmation, GATE 0 MET, verifier ECARTS(2)→CONFORME x2, security PASS x2. See [[BDR-140]], [[EVAL-058]], [[BDR-100]].
- **Status (2026-11-13)**: accepted, residual closed 9f9412a (block H5): precision/triggers/miss share post-fix only (cutoff = merge d3d5e6c 2026-11-12T19:53:59Z), actions wait for 20 post-fix asks, 3rd-pick signal retired, reminder waits for the post-fix sample
## BDR-147 — D1b: the session digest is the TODO source at start; Actionable section else first 3 items, overflow marker [accepted] (2026-11-12)
- **Decision**: D1b: session digest = only TODO source at session start. `_digest_todo` reads `## Actionable` or `### Actionable` (`/^###? Actionable([ \t\r]|$)/`), section empty/absent → file's first 3 top-level `- [ ] ` items; headings + items inside `<!-- -->` ignored; `- todo: +N not shown, read .claude/tasks/TODO.md` marker when 800 cap drops todo lines (room reserved only on overflow). Seeds (init-project, capitalize:83 guarded, onboard STEP 9 fresh format) emit `## Actionable` + comment. CLAUDE.global.md Session start step 1 no longer orders a TODO.md Read (digest carries it; no digest → read it). Local repos migrated insert-only: empty `## Actionable` + comment under H1, `chore/todo-actionable` per repo.
+1
View File
@@ -746,3 +746,4 @@ rules:
- /bugfix mods-strict-types (BLK-049): 17 TS sites narrowed for CLI 2.1.323 validate --strict, kits 288/640 unchanged, make test all green again. An analyzer corrected my premise (the 15 '.catch' lines are info, the TS errors fail strict). Other session's Fable 5.2 rename found uncommitted in this checkout: split out, ships as its own branch.
- feature/fable-5-2 (user ask mid-run): fable alias → claude-fable-5-2 (ca1dc9f), rename done by another session in this checkout, split from BLK-049, id probed live, kit 640/0. Live CLI warns: deny Bash(git push * :**) matches as * wildcard (pre-existing, H item).
- /bugfix safety-lows (block H4b): code-limits private state dir (mkdir -m 700 no -p + one find test), session-start runner timeout→gtimeout→perl alarm, pre-commit fails closed on git diff and filter errors (generator lib/gitflow.sh + both copies), memory.sh rewrap, claude-stub env log path (1f107c3). 4 rounds: generator was the hook source (decision a), GATE 2 MEDIUM stat/lstat race closed, lint regressions fixed. Residual: perl alarm kills the exec'd process only (TODO).
- /bugfix feedback-followups (block H5, 4d follow-ups): post-fix split + 3rd pick retired + one-slot transcript cache + strict labels/state + key-set pins (9f9412a, docs). 3 challenge lenses FATAL on r1 (midnight cutoff, validator rejecting own rows, text-block extraction losing every join); 3 executor rounds (locals, at bound, legacy closed flag). RSS 279.8 → 267.7 MB on 16 records. tsc not done (no compiler, new dep), l:<root> kept (49.7 KB of 4 MiB), image prompts dropped (0 instances). LRN-288.
+5
View File
@@ -2087,3 +2087,8 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
- **Context**: lib/memory.sh upgrade (block H2 2026-11-13); BUGS-FOUND "stale Index cell over a `**Statut** :` line" UNVERIFIED → real; 3 lenses + confirmation found the r1 fix would have reopened resolved blockers.
- **Pattern**: a predicate like "does the Index win" lives at several sites (flush, Index-stated skip list, oracle); a fix at one site forks the predicate: the others keep the old answer and the oracle, built on the same predicate, goes blind. "Keep the body line" is wrong when the strict reader cannot read that line: the entry ends with NO status (= open). The fix = compute ONE effective status (strict word, else the v1 body word) once, feed every consumer, and WRITE the v1 word as the dated status so the strict reader can see it; keep `mem_want` first so an Index `superseded`/`deprecated` still wins; a mixed/unknown label = "" everywhere except excluded from the skip list so the existing refusal fires.
- **Apply**: before changing a status rule, grep every consumer of the same predicate and a fixture per consumer (Index open + body resolved, Index resolved + body resolved, Index superseded + body accepted, mixed). Test oracles that count substrings shape the executor's code (a rename + `${!re}` indirection appeared to satisfy a count): count behaviours, not names.
## LRN-288 — Post-fix cutoff = the merge instant; derived gate flags recomputed on read; reader and writer validators round-trip (2026-11-13)
- **Context**: block H5 2026-11-13: the BDR-146 residual fix (post-fix precision split) went through 3 lenses + confirmation + 3 executor rounds; every round found a defect only live data showed.
- **Pattern**: a time cutoff for "after the fix" = the MERGE instant of the fix (git committer time of the merge), not the fix date at midnight (36 pre-fix rows sat between midnight and the merge). A gate derived from several marks must be recomputed on read, never trusted from a stored flag an older version wrote (a legacy `"closed":true` silenced the reminder for the new gate). Sample floors are counted in the unit the gate already uses (asks, not entries). A validator added on read must accept every row kind the writer emits (round-trip test over `rowsOf`), and the writer must not accept what the reader refuses (`at` 1e21).
- **Apply**: measure on the live data before planning a metrics change (labels.tsv, store, transcripts read-only); pin the reader/writer round trip; derive closed/reached flags from their inputs.
+3 -3
View File
@@ -34,9 +34,9 @@ Shared rules for every phase:
- **4c read side (built 2026-11-05 on feature/memory-recall, merged 2026-11-06 ddb04f4).** Two channels only: intent recall at `prompt.submit` for classifier labels design/analysis or the first coding prompt after idle (BM25 top-30 over title + lesson + body → haiku keeps ≤3, bounded 3 s, fail open) and tool-error 4-gram match (PostToolUse; a classic hook is acceptable if the mod ledger is not needed). Session digest (open BLK via the status parser, actionable TODO, last journal line) in `hooks/session-start.sh` on `source: startup|compact|clear` (native matchers; the mod's `session.compact` only skips); a SubagentStart twin if sub-agents need it. Injection = ID + title + lesson line; superseded never injected; budget ~1.5k chars per turn; once per context. Deferred: Read path triggers, Bash command triggers, "critical → deny once", Tier 1, cross-repo scope=global index, `memory_search` tool.
- [x] 4c follow-up from the 4b live proof: `memory_pending` lists earlier-session milestones marked `(earlier session)` (done 78eff1c 2026-11-05, mods/memory/hooks/register.ts milestoneLines).
- **4d feedback (4d.1-4d.3 built 2026-11-08 on feature/memory-feedback, BDR-140, merged 2026-11-08 dcb593a).** Mod logs `cands`+`ms`; bun reader `lib/memory-feedback/`; `/memory-feedback` skill; banner reminder; two gates (prompt 20 or 35 d, tool 10 or 35 d +1 extension). Plan r3 `docs/superpowers/plans/2026-11-06-memory-feedback.md` (purged at finish; git keeps it), contract `.claude/tasks/contracts/2026-11-06-memory-feedback-4d-1825.md`. 4d.4 (/prune-memory step D, BDR-100 promotion) stays conditional: built only when a gate reports a noisy entry or a promotion candidate.
- [ ] metricsOf: split rows by the 2026-11-12 fix date and retire the 3rd-pick signal (pre-fix rows keep re-emitting the idle and cap actions; BDR-146 residual)
- [ ] 4d follow-ups (final review + task reviews 2026-11-08, non-blocking): transcript cache holds every parsed file per run (~1 GB at steady state: process per (root, session, agent) and drop); image prompts (array content) become `lost` (counted); no boundary tests for the threshold actions (3rd pick, late, tool, miss, noisy/promotion, idle tie-break), eviction note, torn count, toolItem; `replyAfter` hides `seen.add` in a filter; no `tsc` type check on lib/memory-feedback; kit test pinning the exact field set of a prompt log record (IR11 is toMatchObject since 4d.1, so a future text field would pass); labels.ts minors (fromLine checks column count only, readState accepts non-object JSON, .bak keeps one state, misleading error on an entry label with a cause).
- [ ] shortcut: the injection log lives under `l:<root>` store keys (about 28 KiB each, 200 records and 64 KiB at most); `sweepLogs` counts them with the `p:` keys and evicts the oldest log key (by its newest record, never the current root's) once the total nears the engine's 4 MiB store cap; upgrade = one global log key or a smaller per-root cap (`mods/memory/hooks/register.ts`).
- [x] metricsOf split + 3rd pick retired: done 2026-11-13 (block H5, 9f9412a; cutoff = merge instant d3d5e6c)
- [x] 4d follow-ups done 2026-11-13 (block H5, 9f9412a): one-slot transcript cache (RSS 279.8 → 267.7 MB on 16 records), boundary tests, eviction/torn/toolItem tests, replyAfter Map, key-set pins, labels.ts checks. NOT done, justified: `tsc` (no TypeScript compiler on the machine, adding one = a new dependency; the mods are type-checked by `claude plugin validate --strict` since BLK-049); image prompts (0 human array prompts in the 636 MB corpus, 115/115 hook attachments are string arrays, lost records stay visible)
- [x] `l:<root>` shortcut kept, justified 2026-11-13: 6 `l:` + 7 `p:` keys = 49.7 KB of the 4 MiB cap (1.2 %), `sweepLogs` tested; no redesign without a measured need
- [x] residual: the pre-existing `memory.sh check` call in `hooks/session-start.sh` is not time-bounded (the digest's `memory.sh index blocker` call is); upgrade = wrap it in `timeout 2`. Done 2026-11-06 d93f06c (SD16).
- [x] BLK-043 hotfix merged 2026-11-06 (`bd5e661` main, `7e47bea` develop, back-merged into feature/memory-recall `3564e3e`, worktree removed).
- [x] BLK-043 Part B: suite-wide `claude` PATH stub + final empty-log check in `lib/tests/session-digest.test.sh` (SD0/SD15, done 2026-11-06 via /hotfix; security PASS, MEDIUM folded into the shortcut below).
@@ -0,0 +1,41 @@
# CONTRACT — bugs-found-prose (BUGS-FOUND agent/skill/doc items, block H6a)
- date: 2026-11-13 | flow: bugfix | branch: bugfix/bugs-found-prose (off develop)
- status: active
## REQUEST (verbatim — IMMUTABLE)
Fix the BUGS-FOUND entries unfixed since 2026-11-09 that live in agent, skill, lib-prose and doc files: handover-doc-writer §6.1 status filter; handover STEP 10 paths not rooted on the PACKAGE root + space-split PROJECT line; handover/client-handover ROOT double-quoted literal + unquoted `-C <root>` remedy; plugin-advisor `embedded` from REQUEST keywords alone; plugin-gate accepts pack `extras`; onboard/plugin-advisor "pinned ctx7" wording; `make plugin` cwd-dependent remedy + undefined "A) fix plugins"; plugin-gate templates double quotes, linked-worktree apply-check path, PROJECT_ROOT = $HOME; "gstack ON/OFF" vocabulary in onboard/tour + unwrapped lines; stale graphify claims (.gitignore, plugins.lock.json, update-all.sh, install-plugins.sh labels/comment, LRN-154).
## CLARIFICATIONS
- [gated 2026-11-13, orchestrator per user brief: the fix that answers each finding, no new design]
1. handover-doc-writer §6.1: only BDRs whose current word (from `memory.sh -C "$ROOT" index decision`, the reader STEP 9 already uses) is accepted/implemented/proposed enter the vulgarized list; superseded/deprecated are left out (one sentence says so in the deliverable's method note).
2. handover STEP 10 + every other git/file read in that agent: `git -C "$ROOT" …` / `"$ROOT/…"`; the PACKAGE carries the root on its own line `PROJECT_ROOT: <path>` read whole-line (spaces kept); client-handover-writer emits that line.
3. ROOT assignment: the agents take the root from that whole line into a variable via a read (no pasted double-quoted literal), and every printed remedy quotes it: `memory.sh -C "<root>" status …`.
4. plugin-advisor: `embedded` → `↻ SET : minimal` only from project-level signals (files, manifests), never REQUEST keywords alone; REQUEST keywords may only add a NOTE.
5. plugin-gate.md: `extras` removed from the accepted advisor tokens (same reason as higgsfield: 21st-ai metered); the user can still `profile.sh enable extras` by hand.
6. ctx7 wording: "installs ctx7 (latest, plugins.lock.json pins `latest`)" in onboard SKILL.md and plugin-advisor (no pin added).
7. `make plugin` remedies: `make -C ~/Documents/claude plugin` is not portable: write "run `make plugin` in your claude-config checkout" (print-only); plugin-gate "A) fix plugins" defined as exactly that print.
8. plugin-gate templates: tokens and `<PROJECT_ROOT>` in single quotes; the apply check compares the parent of `git rev-parse --git-common-dir` (what profile.sh writes) and refuses PROJECT_ROOT equal to `$HOME`.
9. onboard SKILL.md:387-762 and tour SKILL.md:155 "gstack ON/OFF" → the packs vocabulary used since D3 (`pack gstack-dev on/off`, `profile.sh current`); rewrap plugin-check SKILL.md:17, plugin-gate.md:45, USAGE.md:182; USAGE.md:1022 transcript gains the ↻ glyph.
10. graphify: .gitignore:54-56, plugins.lock.json:37, update-all.sh:310-312 (point at `make plugin`, no manual `graphify claude install`), install-plugins.sh:688/690/1269 labels and :694-695 comment say what the steps do now (the trap reverts the CLAUDE.md block and hook). LRN-154 gets a `memory.sh status` correction line in STEP 8 (registry append-only), not an edit.
## ACCEPTANCE CRITERIA
1. Each of the 10 items changed in its file (read check by the verifier, item by item).
CHECK: cd /home/bchanot/Documents/claude && git diff --stat develop -- agents/handover-doc-writer.md agents/client-handover-writer.md agents/plugin-advisor.md lib/plugin-gate.md skills/onboard/SKILL.md skills/tour/SKILL.md skills/plugin-check/SKILL.md USAGE.md .gitignore plugins.lock.json update-all.sh install-plugins.sh | tail -1 | grep -c '12 files changed'
EXPECT: 1
EVIDENCE: PENDING
2. No "gstack ON/OFF" left in onboard/tour; no `extras` among plugin-gate's accepted tokens.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'gstack ON/OFF\|gstack ON\b' skills/onboard/SKILL.md skills/tour/SKILL.md | grep -vc ':0$')
EXPECT: 0
EVIDENCE: PENDING
3. plugins.lock.json valid JSON; shellcheck clean on update-all.sh and install-plugins.sh changes (no new finding vs develop).
CHECK: cd /home/bchanot/Documents/claude && jq . plugins.lock.json >/dev/null && echo JSON-OK
EXPECT: JSON-OK
EVIDENCE: PENDING
4. Suites that read these files stay green: handover-memory-contract, doctrine-citers, skill-routing-census, curated-config-guard, portability-census.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/handover-memory-contract.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: PENDING
5. make test green (manual, bun on PATH).
## FILE SCOPE
agents/handover-doc-writer.md, agents/client-handover-writer.md, agents/plugin-advisor.md, lib/plugin-gate.md, skills/onboard/SKILL.md, skills/tour/SKILL.md, skills/plugin-check/SKILL.md, USAGE.md, .gitignore, plugins.lock.json, update-all.sh, install-plugins.sh. LRN-154 + BUGS-FOUND → STEP 8.
@@ -0,0 +1,37 @@
# CONTRACT — bugs-found-scripts (BUGS-FOUND script/test items, block H6b)
- date: 2026-11-13 | flow: bugfix | branch: bugfix/bugs-found-scripts (off develop)
- status: active
## REQUEST (verbatim — IMMUTABLE)
Fix the BUGS-FOUND entries unfixed since 2026-11-09 that live in scripts and tests: lib/profile-state.jq newline in a settings value can forge a `plugin:<id>=on` line; skills/prune-memory/tests/run-deterministic.sh:13 unchecked mktemp; session-start digest blocker line fails open (print `- open: none` after a successful read); curated-config-guard does not lock the Step 7 `(cd "$REPO" && graphify claude install)` subshell; push-guard T42c passes if the probe jq write fails; deny_lost has no pipefail; T62F mktemp outside $WORK without trap. Plus the GATE 2 LOW of 2026-11-13: push-guard T63 presence check reads ask+deny, so a deny row demoted to ask still passes.
## CLARIFICATIONS
- [gated 2026-11-13, orchestrator per user brief]
1. profile-state.jq: every value printed into the plain text goes through one sanitizer (`gsub("[\n\r]"; " ")` or `@json` on free-text values) so no value can start a new line; plugin ids are printed only when they match the id shape `^[A-Za-z0-9._-]+@[A-Za-z0-9._-]+$`. Test: a settings fixture whose value holds `\nplugin:evil@x=on` shows no `evil` line (in the existing profile or session-banner suite).
2. run-deterministic.sh: `SANDBOX="$(mktemp -d …)" || exit 1` and an empty-value guard; SKILL.md:34 rewrap ≤ 80 (cosmetic part of the same finding).
3. session-start `_digest_blockers`: when the `index blocker` read succeeds with zero open rows, print `- open: none`; a failed or timed-out read prints `- open: unknown (memory.sh index failed)`. Test in session-digest.test.sh (both cases).
4. curated-config-guard.test.sh: one lock asserting install-plugins.sh Step 7 still runs `graphify claude install` inside `(cd "$REPO" && …)` (fixed-string grep, flip-tested).
5. push-guard T42c: assert the probe file exists and parses before using it (`jq -e . "$WORK/s-probe.json"` → check); deny_lost: `set -o pipefail` local to the function (or explicit rc checks) so a missing list or jq failure fails the check, flip-tested; T62F: settings copy under `$WORK` (cleaned by the suite trap).
6. T63: split the presence list into a deny list checked against `.permissions.deny` only and an ask list checked against `.permissions.ask` only (rows assigned by where they sit in settings.json today).
## ACCEPTANCE CRITERIA
1. Suites green: profile (or session-banner), session-digest, curated-config-guard, push-guard, prune-memory deterministic.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/push-guard.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: PENDING
2. session-digest green with the two new cases.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'open: none' hooks/session-start.sh lib/tests/session-digest.test.sh | grep -c ':[1-9]')$(PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/session-digest.test.sh 2>&1 | tail -1)
EXPECT: 2all suites green
EVIDENCE: PENDING
3. curated-config-guard and prune-memory suites green.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite="lib/tests/curated-config-guard.test.sh skills/prune-memory/tests/run-deterministic.sh" 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: PENDING
4. Forged-line test present and green.
CHECK: cd /home/bchanot/Documents/claude && grep -rlc 'evil@' lib/tests/*.sh | wc -l
EXPECT: 1
EVIDENCE: PENDING
5. shellcheck -x clean on every changed .sh (no new directive); make test green (manual, bun on PATH).
## FILE SCOPE
lib/profile-state.jq, lib/detect-plugins.sh (only if the sanitizer belongs there), hooks/session-start.sh (`_digest_blockers`), skills/prune-memory/tests/run-deterministic.sh, skills/prune-memory/SKILL.md (:34 rewrap), lib/tests/{session-digest,curated-config-guard,push-guard,profile or session-banner}.test.sh. BUGS-FOUND/TODO → STEP 8.
@@ -8,27 +8,34 @@ metricsOf: split rows by the 2026-11-12 fix date and retire the 3rd-pick signal
## CLARIFICATIONS
- [gated 2026-11-13, challenge r1-r3; plan .claude/tasks/plans/2026-11-13-feedback-followups-1800.md § r2 + r3 FINAL] fix instant = merge d3d5e6c 2026-11-12T19:53:59Z; split scope and floor per r3; image prompts DROPPED (0 instances, measured); `tsc` not done (no compiler on the machine, a new dependency) and `l:<root>` shortcut kept (49.7 KB of 4 MiB measured): both closed in the TODO with those reasons; `.bak` one state accepted.
- [gated 2026-11-13, round 2 after GATE 1/2] (r2-1) `closed` = `promptAt && toolAt && postFixAt` so lib/memory-feedback-remind.sh keeps prompting until the post-fix sample is measured (BDR-146 gate 2), old state files without postFixAt read as not closed; (r2-2) `itemError` bounds `at` with `Number.isSafeInteger` so every appended row survives `fromLine(toLine(row))`; (r2-3) R11 and R14 split to ≤ 5 locals; (r2-4) tests: the floor withholds the missShare action below 20 post-fix asks while late/msP90 still fire; an ask with 2 entries counts once toward the floor; a pre-fix row still counts toward promote; the no-candidate `none` record key set pinned (no `ms`).
- [gated 2026-11-13, round 3] `readState` derives `closed` from promptAt && toolAt && postFixAt instead of trusting the stored flag (an old `"closed":true` without postFixAt reads open); the next apply rewrites the flag, so lib/memory-feedback-remind.sh resumes. FILE SCOPE += comments only in lib/memory-feedback-remind.sh:4-5 and lib/tests/memory-feedback-remind.test.sh:4 ("silent once all three gate marks are recorded"); transcripts.ts:133 comment → "first position kept; the value is derived from the key".
## ACCEPTANCE CRITERIA
1. thirdPick retired, fix instant constant present.
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'thirdPick' lib/memory-feedback/metrics.ts lib/memory-feedback/thresholds.ts | grep -c ':0$')$(grep -c "GATE1_FIX_AT = Date.parse('2026-11-12T19:53:59Z')" lib/memory-feedback/thresholds.ts)
EXPECT: 21
EVIDENCE: PENDING
EVIDENCE: MET exit=0 marker-found :: 21
2. memory-feedback suite green.
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/memory-feedback.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: PENDING
EVIDENCE: MET exit=0 marker-found :: all suites green
3. New tests exist (post-fix floor, A,B,A torn read-once, every label kind, two IR11 key-set pins).
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'GATE1_FIX_AT' lib/memory-feedback/metrics.test.ts | grep -c '^[1-9]')$(grep -c 'Skipped 1 torn' lib/memory-feedback/reader.test.ts | grep -c '^[1-9]')$(grep -c 'Object.keys' mods/memory/hooks/register.test.ts | grep -c '^[2-9]')
EXPECT: 111
EVIDENCE: PENDING
EVIDENCE: MET exit=0 marker-found :: 111
4. Memory kit green, strict validation green.
CHECK: cd /home/bchanot/Documents/claude && claude plugin validate --strict mods/memory >/dev/null 2>&1 && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/mods.test.sh 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: PENDING
EVIDENCE: MET exit=0 marker-found :: all suites green
5. The live labels.tsv still reads (worksheet on a COPY of ~/.claude/memory-feedback under mktemp, never the real dir).
(manual 2026-11-13: reader on a HOME copy under mktemp, `bun reader.ts worksheet` rc 0, "16 records to label", live labels.tsv reads under the new checks; real dir untouched)
(manual: executor runs the reader on a copy and quotes rc 0 + the head lines)
6. Peak RSS of the reader before/after on a copy of the real population (manual, quoted).
(manual 2026-11-13: HEAD 279812 kB / 0.35 s, tree 267724 kB / 0.22 s on 16 records; worksheet and items byte-identical)
7. make test green (manual, bun on PATH); functions ≤ 25 logic lines / 5 params / 5 locals; lines ≤ 80.
(manual: full `make test` run 2026-11-13 on round 2 → all suites green)
## FILE SCOPE
lib/memory-feedback/{metrics,thresholds,transcripts,reader,labels}.ts and their .test.ts, mods/memory/hooks/register.test.ts. CHANGELOG/README → doc sync; TODO lines → STEP 8.
+1
View File
@@ -14,6 +14,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and this project
- **memory digest TODO lines**: with no `## Actionable` (or `### Actionable`) section, the digest now shows the TODO file's first 3 unchecked top-level items instead of none, and items inside HTML comments are ignored. When the 800-character cap drops TODO lines, a `- todo: +N not shown, read .claude/tasks/TODO.md` line counts them. `/init-project`, `/onboard` and `/capitalize` seed new TODO files with a `## Actionable` section. The session-start doctrine (`CLAUDE.global.md`) now trusts the digest and reads `.claude/tasks/TODO.md` only when no digest prints or the task needs it.
- **Skill routing (`CLAUDE.global.md`)**: routing lines a skill's own description already covers move there (bugfix, feat/hotfix split, ship-feature, tour, reconcile, capitalize, close); the section keeps the gstack cases (investigate on explicit ask, gstack ship retired per BDR-105), the pack-off SEO/GEO/W3C line (`profile.sh enable seo|web-validate`), `cso` and media. `/bugfix` triggers gain "error", "500", "stack trace + fix"; `/ship-feature` gains "ship this feature".
- **model-router**: the `fable` alias, head of the best tier, resolves to `claude-fable-5-2` (was `claude-fable-5-1`); every best-tier phase, row and skill pin naming `fable` follows. Kit tests updated.
- **memory feedback**: precision, trigger precision and the miss share count only rows after the gate-1 fix (merge 2026-11-12T19:53:59Z); their actions wait for 20 post-fix asks (`Post-fix prompt asks: k/20` in FEEDBACK.md, one `NEW: post-fix prompt sample reached` line) and the session reminder keeps prompting until then. The third-pick signal is retired (recall keeps 2 picks). The reader holds one parsed transcript at a time; `labels.tsv` rows and `state.json` are validated on read.
### Removed
- `hooks/design-toolchain-reminder.sh`, its `UserPromptSubmit` entry in `settings.json` (`ctx7-reminder` stays the only one) and its suite `lib/tests/design-toolchain-reminder.test.sh`. Measured 2026-10-31: 59 fires, 0 Reads of `rules/web-building.md`. Design routing stays with the `rules/web-building.md` path rule and the `CLAUDE.global.md` pointer. The fire log `~/.claude/logs/design-toolchain-fires.log` is left in place.
+1 -1
View File
@@ -167,7 +167,7 @@ Limits: a main-loop switch re-reads cold the part of the context the target mode
- Digest: `hooks/session-start.sh` prints `memory digest:` on `startup`, `clear` and `compact` (never `resume`) in a project that has `.claude/memory/`: up to 3 open BLK from `memory.sh index blocker` (the rest counted, upstream ones counted), up to 3 unchecked TODO items (from a `## Actionable` or `### Actionable` section, else the file's first 3 unchecked top-level items; items inside HTML comments ignored), the last journal line, 800 characters at most. When the cap bites, TODO lines drop first and a `- todo: +N not shown, read .claude/tasks/TODO.md` line counts them; nothing else drops. Without `timeout` and `jq` it prints only from a terminal.
- Recall switch: `{"enabled": true, "recall": false}` in `~/.claude/memory.json` cuts the injections (recall and known-error line) and keeps capture. Every ask is logged under the `l:<root>` store key (last 200 records, 64 KiB), never shown: when, session, agent, channel, the IDs injected, the `trigger` (`fresh`, `idle`, `keyword`, `tool`) and the `outcome` (`injected`, `none`, `timeout`, `refused`, `paused`; an ask that injects nothing is logged with no IDs); a prompt ask also logs `cands` (BM25 candidates after the seen filter) and `ms` (rerank wall time, when a rerank ran). `/memory-feedback` reads it.
- Recall residuals: measured on 14 days of this repo's transcripts, precision is 54 to 61 % against a 64 % bar, so recall ships on and switchable, and the log decides. The router's classifier label is not used. EVAL entries are not recalled. Each failing command that matches an entry spawns one `memory.sh index`. The haiku bound drops the recalls that answer slower than 5 seconds.
- Feedback: `/memory-feedback` (phase 4d) labels each injected entry from its transcript, with a 5-label spot check by you, and reports precision, misses, noisy entries and promotion candidates against `lib/memory-feedback/thresholds.ts`. Two gates: prompt at 20 labelled asks or 35 days, tool at 10 injections (one 35-day extension, then closed). Data stays local in `~/.claude/memory-feedback/` (IDs, enums and repo tags only, never in git). The session banner prints `memory feedback: … → /memory-feedback` when a run is due, and goes silent once both gates are recorded or with `"recall": false`.
- Feedback: `/memory-feedback` (phase 4d) labels each injected entry from its transcript, with a 5-label spot check by you, and reports precision, misses, noisy entries and promotion candidates against `lib/memory-feedback/thresholds.ts`. Two gates: prompt at 20 labelled asks or 35 days, tool at 10 injections (one 35-day extension, then closed). Precision, trigger precision and the miss share count only rows after the gate-1 fix; their actions wait for 20 post-fix asks (`Post-fix prompt asks: k/20` in FEEDBACK.md, pre-fix entries excluded). Data stays local in `~/.claude/memory-feedback/` (IDs, enums and repo tags only, never in git). The session banner prints `memory feedback: … → /memory-feedback` when a run is due, and goes silent once the prompt gate, the tool gate and the post-fix sample are recorded, or with `"recall": false`.
- Off switch: `{"enabled": false}` in `~/.claude/memory.json`, read at the first event of a session, so it applies at the next session or after `/reload-plugins`. Off, the tool answers `memory capture is off (~/.claude/memory.json)`. A switch file that cannot be read, or whose `enabled` is not a boolean, keeps capture on and logs one line.
- Residuals: cross-process writes are unverified (two concurrent sessions: last writer wins). Non-Bash failures are not captured. `-b` counts as curl's cookie flag only when `curl` opens its command segment, after optional env assignments, `sudo`, `rtk` or `$(`: `time curl -b …` keeps its cookies. 40-hex Gitea tokens are kept by design: they look like commit SHAs. A session root that moves with `/cd` or a worktree move hides earlier captures of the same repo from /capitalize. A bare `gitflow.sh finish` on a chore branch counts as a milestone. Measured at 83 % noise before the error-shaped filter over 14 days of this repo's transcripts; the filter keeps every useful sample and cuts captures by two thirds (`.claude/tasks/plans/2026-11-03-memory-capture-spike/NOISE.md`).
+1 -1
View File
@@ -219,7 +219,7 @@ Le mod `mods/memory/` note les échecs Bash et les jalons de la session pour que
- Digest : `hooks/session-start.sh` affiche `memory digest:` sur `startup`, `clear` et `compact` (jamais `resume`) dans un projet qui a `.claude/memory/` : jusqu'à 3 BLK ouverts issus de `memory.sh index blocker` (le reste compté, les « upstream » comptés), jusqu'à 3 items non cochés du TODO (section `## Actionable` ou `### Actionable`, sinon les 3 premiers items de premier niveau du fichier ; items en commentaire HTML ignorés), la dernière ligne du journal, 800 caractères au plus. Quand le plafond mord, les lignes TODO tombent d'abord et une ligne `- todo: +N not shown, read .claude/tasks/TODO.md` les compte ; rien d'autre ne tombe. Sans `timeout` ni `jq`, il ne s'affiche que depuis un terminal.
- Interrupteur du rappel : `{"enabled": true, "recall": false}` dans `~/.claude/memory.json` coupe les injections (rappel et ligne d'erreur connue) et garde la capture. Chaque demande est journalisée sous la clé de store `l:<racine>` (200 enregistrements, 64 Kio), jamais affichée : date, session, agent, canal, IDs injectés, `trigger` (`fresh`, `idle`, `keyword`, `tool`) et `outcome` (`injected`, `none`, `timeout`, `refused`, `paused` ; une demande qui n'injecte rien est journalisée sans ID) ; une demande sur prompt note aussi `cands` (candidats BM25 après le filtre des entrées déjà vues) et `ms` (durée du rerank, quand il a tourné). `/memory-feedback` la lit.
- Limites du rappel : mesurée sur 14 jours de transcripts de ce dépôt, la précision est de 54 à 61 % pour une barre à 64 %, donc le rappel est livré actif et coupable, et le journal tranchera. L'étiquette du classifieur du routeur n'est pas utilisée. Les entrées EVAL ne sont pas rappelées. Chaque commande en échec qui correspond à une entrée lance un `memory.sh index`. La borne haiku écarte les rappels qui répondent après 5 secondes.
- Retour d'usage : `/memory-feedback` (phase 4d) étiquette chaque entrée injectée depuis son transcript, avec un contrôle de 5 étiquettes par toi, et donne la précision, les manqués, les entrées bruyantes et les candidates à promotion face aux seuils de `lib/memory-feedback/thresholds.ts`. Deux gates : prompt à 20 demandes étiquetées ou 35 jours, outil à 10 injections (une prolongation de 35 jours, puis fermé). Les données restent locales dans `~/.claude/memory-feedback/` (IDs, enums et étiquettes de dépôt seulement, jamais dans git). La bannière de session affiche `memory feedback: … → /memory-feedback` quand un passage est dû, et se tait une fois les deux gates enregistrés ou avec `"recall": false`.
- Retour d'usage : `/memory-feedback` (phase 4d) étiquette chaque entrée injectée depuis son transcript, avec un contrôle de 5 étiquettes par toi, et donne la précision, les manqués, les entrées bruyantes et les candidates à promotion face aux seuils de `lib/memory-feedback/thresholds.ts`. Deux gates : prompt à 20 demandes étiquetées ou 35 jours, outil à 10 injections (une prolongation de 35 jours, puis fermé). La précision, la précision par trigger et la part de manqués ne comptent que les lignes postérieures au correctif du gate 1 ; leurs actions attendent 20 demandes post-correctif (`Post-fix prompt asks: k/20` dans FEEDBACK.md, entrées antérieures exclues). Les données restent locales dans `~/.claude/memory-feedback/` (IDs, enums et étiquettes de dépôt seulement, jamais dans git). La bannière de session affiche `memory feedback: … → /memory-feedback` quand un passage est dû, et se tait une fois le gate prompt, le gate outil et l'échantillon post-correctif enregistrés, ou avec `"recall": false`.
- Interrupteur : `{"enabled": false}` dans `~/.claude/memory.json`, lu au premier événement d'une session : effet à la session suivante ou après `/reload-plugins`. Coupé, l'outil répond `memory capture is off (~/.claude/memory.json)`. Un fichier illisible, ou dont `enabled` n'est pas un booléen, laisse la capture active et journalise une ligne.
- Limites connues : les écritures entre processus ne sont pas vérifiées (deux sessions simultanées : la dernière écrit). Les échecs hors Bash ne sont pas captés. `-b` ne vaut drapeau de cookies de curl que si `curl` ouvre son segment de commande, après d'éventuelles affectations de variables, `sudo`, `rtk` ou `$(` : `time curl -b …` garde ses cookies. Un jeton de 40 hexadécimaux nu est gardé à dessein : il ressemble à un SHA de commit (expurgé seulement comme utilisateur d'URL). Une racine de session qui bouge avec `/cd` ou un changement de worktree cache à /capitalize les captures précédentes du même dépôt. Un `gitflow.sh finish` nu sur une branche chore compte comme un jalon. Bruit mesuré à 83 % avant le filtre de forme d'erreur sur 14 jours de transcripts de ce dépôt ; le filtre garde tous les échantillons utiles et réduit les captures des deux tiers (`.claude/tasks/plans/2026-11-03-memory-capture-spike/NOISE.md`).
+4 -3
View File
@@ -2,9 +2,10 @@
# lib/memory-feedback-remind.sh — the phase 4d reminder line of the
# session-start banner (BDR-057: transcripts live 7 days). One line when
# /memory-feedback never ran, or ran REMIND_DAYS+ days ago and the memory
# store changed since. Silent once both gates are recorded ("closed":true),
# with "recall": false, or on any error; always rc 0. bash 3.2 safe,
# portable (find -mtime, no stat -c), no JSON parse.
# store changed since. Silent once all three gate marks (prompt, tool,
# post-fix sample) are recorded ("closed":true), with "recall": false, or
# on any error; always rc 0. bash 3.2 safe, portable (find -mtime, no
# stat -c), no JSON parse.
REMIND_DAYS=5
state="$HOME/.claude/memory-feedback/state.json"
+60
View File
@@ -105,6 +105,15 @@ test('L8: state defaults, round trip, a corrupt file throws', () => {
expect(() => readState(f)).toThrow('state.json')
})
test('L8b: closed is derived from the three marks, not the stored flag', () => {
const f = path.join(tmp(), 'state.json')
fs.writeFileSync(f, '{"lastRun":1,"promptAt":1,"toolAt":1,"closed":true}')
expect(readState(f).closed).toBe(false)
fs.writeFileSync(f,
'{"lastRun":1,"promptAt":1,"toolAt":1,"postFixAt":1,"closed":false}')
expect(readState(f).closed).toBe(true)
})
test('L9: repo tags keep same-named repos apart', () => {
expect(tagOf('/a/x')).toMatch(/^x#[0-9a-f]{6}$/)
expect(tagOf('/a/x')).toBe(tagOf('/a/x'))
@@ -116,3 +125,54 @@ test('L10: the skeleton draft lists each injected id once', () => {
item(3, { lost: true })]).split('\n').filter(l => !l.startsWith('#'))
expect(lines.filter(l => l !== '')).toEqual(['1\tBDR-001\t', '1\tLRN-002\t'])
})
test('L11: readRows round trips every label kind', () => {
const file = path.join(tmp(), 'mf', 'labels.tsv')
const full = item(1)
const drafts = parseDraft([OK, '2\tBDR-009\tmissed\tno-overlap',
'1\t-\tchecked', '2\t-\ttest'].join('\n'), [full,
item(2, { ids: [] })])
if (typeof drafts === 'string') throw new Error(drafts)
const rows = [...rowsOf(full, drafts), ...rowsOf(item(2, { ids: [] }),
drafts), ...rowsOf(item(3, { lost: true }), [])]
appendRows(file, rows)
expect(new Set(rows.map(r => r.label))).toEqual(new Set(['ask', 'lost',
'relevant', 'irrelevant', 'missed', 'checked', 'test']))
expect(readRows(file)).toEqual(rows)
})
test('L12: a line with a wrong id, label, channel or at is refused', () => {
const good: Row = { at: 5, root: 'r#000000', session: 's',
channel: 'prompt', trigger: 'fresh', outcome: 'injected', cands: 1,
ms: 2, pick: 1, id: 'BDR-001', label: 'relevant', cause: '' }
expect(fromLine(toLine(good))).toEqual(good)
const bad: Partial<Row>[] = [{ label: 'maybe' as never },
{ id: 'BDR-1' }, { id: '' }, { label: 'ask' }, // ask carries no id
{ channel: 'x' as never }, { at: Number.NaN }, { at: -5 },
{ at: 1.5 }]
for (const b of bad) expect(fromLine(toLine({ ...good, ...b })))
.toBeUndefined()
expect(fromLine(toLine({ ...good, id: '', label: 'ask' }))).toBeDefined()
})
test('L13: a damaged line names itself and what the .bak holds', () => {
const file = path.join(tmp(), 'labels.tsv')
appendRows(file, rowsOf(item(1), []))
fs.appendFileSync(file, '1\tr\ts\tprompt\t\t\t\t\t\tNOPE\trelevant\t\n')
expect(() => readRows(file)).toThrow(/line 3 unreadable.*labels\.tsv\.bak/)
})
test('L14: state.json that is not an object is refused', () => {
const f = path.join(tmp(), 'state.json')
for (const raw of ['[1]', 'null', '7', '"x"']) {
fs.writeFileSync(f, raw)
expect(() => readState(f)).toThrow('state.json')
}
writeState(f, { lastRun: 3, postFixAt: 3, closed: false })
expect(readState(f).postFixAt).toBe(3)
})
test('L15: an entry label with a cause says it takes none', () => {
const d = parseDraft(`${OK}\n1\tBDR-001\trelevant\toverlap`, ITEMS)
expect(d).toContain('takes no cause')
})
+32 -8
View File
@@ -12,6 +12,8 @@ const DRAFT_LABELS = ['relevant', 'irrelevant', 'test', 'missed', 'checked']
const ENTRY_LABELS = ['relevant', 'irrelevant']
const CAUSES = ['no-overlap', 'overlap']
const ID_RE = /^(BDR|LRN|BLK)-\d{3,}$/
const ROW_LABELS = [...DRAFT_LABELS, 'ask', 'lost']
const BARE_LABELS = ['ask', 'lost', 'test', 'checked'] // carry no entry id
/** One worksheet item: a log record and what the join found. */
export type Item = {
@@ -25,9 +27,11 @@ export type Row = {
pick?: number; id: string; label: Label; cause: string
}
export type Draft = { item: number; id: string; label: Label; cause: string }
/** promptAt/toolAt: when each gate verdict was first reached. */
/** promptAt/toolAt/postFixAt: when each gate verdict or floor was first
* reached. */
export type State = {
lastRun: number; promptAt?: number; toolAt?: number; closed: boolean
postFixAt?: number // the post-fix prompt floor, first crossed (optional)
}
const COLS = ['at', 'root', 'session', 'channel', 'trigger', 'outcome',
@@ -52,10 +56,20 @@ export function toLine(r: Row): string {
.join('\t')
}
/** A labels.tsv line as a row; the header or a short line: undefined. */
/** Whether the fields hold a known label, its id shape, a channel and
* an integer `at`. */
function sound(f: string[]): boolean {
const label = f[10] ?? ''
const id = f[9] ?? ''
const idOk = BARE_LABELS.includes(label) ? id === '' : ID_RE.test(id)
return ROW_LABELS.includes(label) && idOk &&
(f[3] === 'prompt' || f[3] === 'tool') && /^\d+$/.test(f[0] ?? '')
}
/** A labels.tsv line as a row; the header or a damaged line: undefined. */
export function fromLine(line: string): Row | undefined {
const f = line.split('\t')
if (f.length !== COLS.length || f[0] === 'at') return undefined
if (f.length !== COLS.length || !sound(f)) return undefined
return {
at: Number(f[0]), root: f[1] ?? '', session: f[2] ?? '',
channel: f[3] as Channel, trigger: f[4] ?? '', outcome: f[5] ?? '',
@@ -87,11 +101,12 @@ export function readRows(file: string): Row[] {
const r = fromLine(line)
if (r !== undefined) return [r]
throw new Error(`labels.tsv line ${i + 1} unreadable: fix it by hand ` +
'or restore labels.tsv.bak')
'(labels.tsv.bak holds the state before the last append)')
})
}
/** Appends rows; the previous file is copied to labels.tsv.bak first. */
/** Appends rows; the previous file is copied to labels.tsv.bak first (one
* state: the one before the last append). */
export function appendRows(file: string, rows: Row[]): void {
mkdir(file)
if (fs.existsSync(file)) fs.copyFileSync(file, `${file}.bak`)
@@ -104,9 +119,17 @@ export function readState(file: string): State {
if (!fs.existsSync(file)) return { lastRun: 0, closed: false }
try {
const s = JSON.parse(fs.readFileSync(file, 'utf8'))
if (typeof s !== 'object' || s === null || Array.isArray(s)) {
throw new Error('state.json is not an object')
}
const at = (v: unknown) => typeof v === 'number' ? v : undefined
return { lastRun: Number(s.lastRun) || 0, promptAt: at(s.promptAt),
toolAt: at(s.toolAt), closed: s.closed === true }
const [promptAt, toolAt, postFixAt] =
[at(s.promptAt), at(s.toolAt), at(s.postFixAt)]
// closed is derived: a stored flag without all three marks reads open
const closed = promptAt !== undefined && toolAt !== undefined &&
postFixAt !== undefined
return { lastRun: Number(s.lastRun) || 0, promptAt, toolAt, postFixAt,
closed }
} catch {
throw new Error(`state.json unreadable (${file}): fix or delete it`)
}
@@ -135,7 +158,8 @@ function fitError(d: Draft, it: Item | undefined): string {
? '' : `item ${d.item}: only checked or test take id -`
}
if (ENTRY_LABELS.includes(d.label)) {
return plain && it.ids.includes(d.id)
if (!plain) return `item ${d.item}: ${d.label} takes no cause`
return it.ids.includes(d.id)
? '' : `item ${d.item}: ${d.id} was not injected there`
}
const ok = d.label === 'missed' && it.channel === 'prompt' &&
+126 -2
View File
@@ -1,12 +1,12 @@
import { expect, test } from 'bun:test'
import { DAY_MS, GATE_DAYS } from './thresholds.ts'
import { DAY_MS, GATE1_FIX_AT, GATE_DAYS } from './thresholds.ts'
import {
actionsOf, clockStart, gateVerdict, metricsOf, p90, precision,
renderFeedback, type Metrics,
} from './metrics.ts'
import type { Label, Row } from './labels.ts'
let clock = 1000
let clock = GATE1_FIX_AT + 1000 // every default row is post-fix
/** A labels.tsv row; each call gets its own `at` unless given. */
function row(over: Partial<Row> & { label: Label }): Row {
clock += 1
@@ -129,3 +129,127 @@ test('M10: FEEDBACK.md shows n per channel and trigger', () => {
expect(text).toContain('| prompt precision | 33 % (n=3) |')
expect(text).toContain('| tool precision | n/a (n=0) |')
})
const shift = (rs: Row[], at: number) => rs.map(r => ({ ...r, at }))
const PRE = Date.parse('2026-11-12T15:00:00Z')
test('M11: the fix instant splits prompt precision, not noisy', () => {
const old = shift(ask(['irrelevant', 'irrelevant', 'irrelevant']), PRE)
const mm = metricsOf([...old, ...ask(['relevant'])])
expect(mm.precision).toBe(1)
expect(mm.n.prompt).toBe(1)
expect(mm.preFix).toBe(3)
expect(mm.promptN).toBe(2) // the gate count reads every row
const thrice = [1, 2, 3].flatMap(() => shift(ask(['irrelevant']), PRE))
expect(metricsOf(thrice).noisy).toEqual(['r#000000 LRN-001'])
const edge = [...shift(ask(['irrelevant']), GATE1_FIX_AT - 1),
...shift(ask(['relevant']), GATE1_FIX_AT)]
expect(metricsOf(edge).precision).toBe(1)
expect(metricsOf(edge).n.prompt).toBe(1)
})
test('M12: the miss share reads post-fix asks only', () => {
const miss = (at: number) => [row({ at, id: '', label: 'ask' }),
row({ at, id: 'BDR-009', label: 'missed', cause: 'overlap' })]
const mm = metricsOf([...miss(GATE1_FIX_AT - 1), ...miss(GATE1_FIX_AT)])
expect(mm.sampled).toBe(1)
expect(mm.missShare).toBe(1)
})
/** Metrics whose every gate is reached, with `over` on top. */
const reached = (over: Partial<Metrics>): Metrics => ({
...metricsOf([]), start: 0, promptN: 20, toolN: 10, postAsks: 20, ...over,
})
const acts = (over: Partial<Metrics>) => {
const mm = reached(over)
return actionsOf(mm, gateVerdict(mm, 1))
}
const sized = (i: number, k: number, f: number) => ({
idle: { rate: i, n: 5 }, keyword: { rate: k, n: 5 },
fresh: { rate: f, n: 5 },
})
test('M13: each comparator at its threshold and one step past', () => {
const has = (over: Partial<Metrics>, part: string) =>
acts(over).some(a => a.includes(part))
expect(has({ late: 0.25 }, 'timeout + paused')).toBe(false)
expect(has({ late: 0.26 }, 'timeout + paused')).toBe(true)
expect(has({ msP90: 2500 }, 'rerank p90')).toBe(false)
expect(has({ msP90: 2499 }, 'rerank p90')).toBe(true)
expect(has({ tool: 0.5 }, 'tool precision')).toBe(false)
expect(has({ tool: 0.49 }, 'tool precision')).toBe(true)
expect(has({ missShare: 1 / 3 }, 'misses in 1 ask')).toBe(true)
expect(has({ missShare: 0.33 }, 'misses in 1 ask')).toBe(false)
const by = sized(0.4, 0.9, 0.9)
expect(has({ precision: 0.64, byTrigger: by }, 'prompt precision'))
.toBe(false)
expect(has({ precision: 0.63, byTrigger: by }, 'prompt precision'))
.toBe(true)
const low = (r: number) => sized(r, 0.9, 0.9)
expect(has({ precision: 0.5, byTrigger: low(0.5) }, 'prompt precision'))
.toBe(false)
expect(has({ precision: 0.5, byTrigger: low(0.49) }, 'prompt precision'))
.toBe(true)
})
test('M14: the worst trigger ties go idle, keyword, fresh', () => {
const pick = (by: ReturnType<typeof sized>) => acts({ precision: 0.3,
byTrigger: by }).find(a => a.startsWith('prompt precision'))
expect(pick(sized(0.25, 0.25, 0.25))).toContain('the idle trigger')
expect(pick(sized(0.9, 0.25, 0.25))).toContain('the keyword trigger')
expect(pick(sized(0.25, 0.2, 0.9))).toContain('the keyword trigger')
expect(pick(sized(0.9, 0.9, 0.1))).toContain('the fresh trigger')
})
test('M15: promotion needs 3 hits in 2 sessions, noisy 3 never hit', () => {
const rel = (s: string) => ask(['relevant'], { session: s })
expect(metricsOf([...rel('a'), ...rel('a'), ...rel('b')]).promote)
.toEqual(['r#000000 LRN-001'])
expect(metricsOf([...rel('a'), ...rel('b')]).promote).toEqual([])
const irr = () => ask(['irrelevant'])
expect(metricsOf([...irr(), ...irr(), ...irr()]).noisy).toHaveLength(1)
expect(metricsOf([...irr(), ...irr()]).noisy).toEqual([])
})
test('M16: the post-fix floor gates the prompt actions, 19 vs 20', () => {
const idle = (n: number) => Array.from({ length: n }, () =>
ask(['irrelevant'], { trigger: 'idle' })).flat()
const run = (n: number) => {
const mm = { ...metricsOf(idle(n)), start: 0, promptN: 20 }
return actionsOf(mm, gateVerdict(mm, 1))
}
expect(run(19).some(a => a.startsWith('prompt precision'))).toBe(false)
expect(run(20).some(a => a.startsWith('prompt precision'))).toBe(true)
const text = renderFeedback(metricsOf(idle(19)), gateVerdict(
metricsOf(idle(19)), 1), 0)
expect(text).toContain('Post-fix prompt asks: 19/20 (since ' +
'2026-11-12T19:53:59.000Z), pre-fix entries excluded: 0')
})
test('M17: below the floor the miss action waits, late and p90 fire', () => {
const asks = Array.from({ length: 19 }, () =>
ask(['irrelevant'], { trigger: 'idle', ms: 2000 })).flat()
const miss = row({ id: 'BDR-009', label: 'missed', cause: 'overlap',
at: asks[0]!.at })
const late = Array.from({ length: 8 }, () =>
row({ id: '', label: 'ask', outcome: 'timeout' }))
const mm = { ...metricsOf([...asks, miss, ...late]), start: 0, promptN: 20 }
expect(mm.missShare).toBeGreaterThanOrEqual(1 / 3)
const out = actionsOf(mm, gateVerdict(mm, 1))
expect(out.some(a => a.startsWith('misses in 1 ask'))).toBe(false)
expect(out.some(a => a.startsWith('timeout + paused'))).toBe(true)
expect(out.some(a => a.startsWith('rerank p90'))).toBe(true)
})
test('M18: an ask with 2 entries counts once toward the floor', () => {
const two = Array.from({ length: 19 }, () =>
ask(['relevant', 'relevant'])).flat()
expect(metricsOf(two).postAsks).toBe(19)
expect(metricsOf([...two, ...ask(['relevant'])]).postAsks).toBe(20)
})
test('M19: a pre-fix row still counts toward promote', () => {
const old = (s: string) => shift(ask(['relevant'], { session: s }), PRE)
expect(metricsOf([...old('a'), ...old('a'), ...old('b')]).promote)
.toEqual(['r#000000 LRN-001'])
})
+24 -17
View File
@@ -2,7 +2,7 @@
// FEEDBACK.md text, computed from labels.tsv rows only (transcripts expire
// at 7 days, the rows stay). Spec §4.
import {
DAY_MS, GATE_DAYS, PROMPT_GATE_N, T, TOOL_GATE_N,
DAY_MS, GATE1_FIX_AT, GATE_DAYS, PROMPT_GATE_N, T, TOOL_GATE_N,
} from './thresholds.ts'
import { keyOf, type Row } from './labels.ts'
@@ -13,10 +13,12 @@ type Rate = number | undefined
type Sized = { rate: Rate; n: number }
export type Metrics = {
start?: number; promptN: number; toolN: number; lost: number
precision: Rate; byTrigger: Record<string, Sized>; thirdPick: Rate
precision: Rate; byTrigger: Record<string, Sized>
late: Rate; msP90: Rate; tool: Rate; missShare: Rate; sampled: number
noisy: string[]; promote: string[]
n: { prompt: number; third: number; tool: number } // entries
n: { prompt: number; tool: number } // entries
postAsks: number // post-fix prompt asks labelled: the floor of the actions
preFix: number // pre-fix prompt entries left out of the prompt signals
}
const share = (k: number, n: number): Rate => n === 0 ? undefined : k / n
@@ -66,7 +68,6 @@ function byTrigger(entries: Row[]): Record<string, Sized> {
/** Entry counts behind the three channel precisions. */
const countsOf = (entries: Row[]) => ({
prompt: entries.filter(r => r.channel === 'prompt').length,
third: entries.filter(r => r.channel === 'prompt' && r.pick === 3).length,
tool: entries.filter(r => r.channel === 'tool').length,
})
@@ -109,25 +110,29 @@ function promote(entries: Row[]): string[] {
const itemsOf = (entries: Row[], ch: string) =>
new Set(entries.filter(r => r.channel === ch).map(keyOf)).size
/** Every 4d signal, from labels.tsv rows. */
const postFix = (r: Row) => r.at >= GATE1_FIX_AT
/** Every 4d signal, from labels.tsv rows. Prompt precision, triggers and
* the miss share read post-fix rows (BDR-146); the rest read all rows. */
export function metricsOf(all: Row[]): Metrics {
const rows = liveRows(all)
const entries = rows.filter(isEntry)
const prompt = entries.filter(r => r.channel === 'prompt')
const recent = entries.filter(r => r.channel === 'prompt' && postFix(r))
const asks = rows.filter(r => r.channel === 'prompt' &&
(r.label === 'ask' || r.label === 'lost'))
const miss = misses(rows)
const miss = misses(rows.filter(postFix))
return {
start: clockStart(all, []),
promptN: itemsOf(entries, 'prompt'), toolN: itemsOf(entries, 'tool'),
lost: rows.filter(r => r.label === 'lost').length,
precision: precision(prompt), byTrigger: byTrigger(prompt),
thirdPick: precision(prompt.filter(r => r.pick === 3)),
precision: precision(recent), byTrigger: byTrigger(recent),
late: lateShare(asks), msP90: p90(asks.flatMap(r => r.ms ?? [])),
tool: precision(entries.filter(r => r.channel === 'tool')),
missShare: miss.rate, sampled: miss.sampled,
noisy: noisy(entries), promote: promote(entries),
n: countsOf(entries),
n: { ...countsOf(entries), prompt: recent.length },
postAsks: itemsOf(recent, 'prompt'),
preFix: countsOf(entries).prompt - recent.length,
}
}
@@ -158,13 +163,11 @@ function worstTrigger(by: Record<string, Sized>): string | undefined {
function promptActions(m: Metrics): string[] {
const out: string[] = []
const worst = worstTrigger(m.byTrigger)
if (under(m.precision, T.promptKeep) && worst !== undefined) {
const floor = m.postAsks >= PROMPT_GATE_N
if (floor && under(m.precision, T.promptKeep) && worst !== undefined) {
out.push(`prompt precision under ${pct(T.promptKeep)}: ` +
`tighten or drop the ${worst} trigger`)
}
if (under(m.thirdPick, T.thirdPick)) {
out.push(`3rd pick under ${pct(T.thirdPick)}: cap recall at 2 entries`)
}
if ((m.late ?? 0) > T.lateShare) {
out.push(`timeout + paused over ${pct(T.lateShare)}: ` +
'revisit RERANK_MS or skip the rerank on short lists')
@@ -172,7 +175,7 @@ function promptActions(m: Metrics): string[] {
if (under(m.msP90, T.msP90)) {
out.push(`rerank p90 under ${T.msP90} ms: lower RERANK_MS`)
}
if ((m.missShare ?? 0) >= T.missShare) {
if (floor && (m.missShare ?? 0) >= T.missShare) {
out.push('misses in 1 ask in 3 or more: diagnose by cause')
}
return out
@@ -197,6 +200,11 @@ function toolLine(m: Metrics, v: Verdict): string {
: `Tool gate: ${v.tool} (${m.toolN}/${TOOL_GATE_N})`
}
const postFixLine = (m: Metrics) =>
`Post-fix prompt asks: ${m.postAsks}/${PROMPT_GATE_N} (since ` +
`${new Date(GATE1_FIX_AT).toISOString()}), pre-fix entries excluded: ` +
`${m.preFix}`
/** FEEDBACK.md: counts and shares; repo tags only in the last lists. */
export function renderFeedback(m: Metrics, v: Verdict, now: number) {
const acts = actionsOf(m, v)
@@ -204,12 +212,11 @@ export function renderFeedback(m: Metrics, v: Verdict, now: number) {
`# Memory feedback (${day(now)})`, '',
`Gate clock start: ${m.start === undefined ? 'not started' : day(m.start)}`,
`Prompt gate: ${v.prompt} (${m.promptN}/${PROMPT_GATE_N})`,
toolLine(m, v), `Lost records: ${m.lost}`, '',
toolLine(m, v), `Lost records: ${m.lost}`, postFixLine(m), '',
'| Signal | Value |', '|---|---|',
`| prompt precision | ${pct(m.precision)} (n=${m.n.prompt}) |`,
...Object.entries(m.byTrigger).map(([k, x]) =>
`| precision, ${k} | ${pct(x.rate)} (n=${x.n}) |`),
`| 3rd pick precision | ${pct(m.thirdPick)} (n=${m.n.third}) |`,
`| timeout + paused | ${pct(m.late)} |`,
`| rerank ms p90 | ${m.msP90 ?? 'n/a'} |`,
`| tool precision | ${pct(m.tool)} (n=${m.n.tool}) |`,
+136 -2
View File
@@ -3,6 +3,8 @@ import { spawnSync } from 'node:child_process'
import * as fs from 'node:fs'
import * as os from 'node:os'
import * as path from 'node:path'
import { GATE1_FIX_AT } from './thresholds.ts'
import { tagOf } from './labels.ts'
import { slugOf } from './transcripts.ts'
import {
apply, checkDir, population, readLogs, safePath, sample, storeFile,
@@ -17,7 +19,7 @@ const mk = (pfx: string) => {
dirs.push(d)
return d
}
const T0 = Date.parse('2026-11-10T10:00:00.000Z')
const T0 = GATE1_FIX_AT + 20 * 3600000 // post-fix: the floor counts it
const NOW = T0 + 2 * 3600000 // every fixture record is 2 h old
const iso = (ms: number) => new Date(T0 + ms).toISOString()
// built at run time: no secret-shaped literal in the repo (BLK-041)
@@ -156,7 +158,7 @@ test('R8: the CLI refuses a dir inside a git work tree', () => {
test('R10: a tampered items.json is refused, nothing written', () => {
for (const bad of [{ root: '/repo/x' }, { root: 'a\tb#abcdef' },
{ session: '../s' }, { channel: 'x' }, { ids: ['../x'] },
{ cited: ['BDR-999'] }, { at: -1 }, { lost: 'no' }]) {
{ cited: ['BDR-999'] }, { at: -1 }, { at: 1e21 }, { lost: 'no' }]) {
const { p, work } = fixture(STORE)
worksheet(work, p, NOW)
const f = path.join(work, 'items.json')
@@ -181,3 +183,135 @@ test('safePath keeps plain paths and withholds shell-active ones', () => {
for (const bad of ['/tmp/$(id)', '/tmp/`id`', "/tmp/a'b", '/a/../b'])
expect(safePath(bad)).toBe(w)
})
/** The 3 transcript records of the prompt asked at `T0 + ms`. */
const turn = (ms: number, tag: string) => [
{ type: 'user', timestamp: iso(ms - 50), origin: { kind: 'human' },
message: { content: `prompt-${tag}` } },
{ type: 'attachment', timestamp: iso(ms + 400), attachment: {
type: 'hook_additional_context', hookName: 'prompt.submit',
content: ['memory recall:\nBDR-001 x'] } },
{ type: 'assistant', timestamp: iso(ms + 900),
message: { id: `m${ms}`, content: [{ type: 'text', text: 'ok' }] } },
]
const jsonl = (recs: object[], torn = 0) => recs.map(r => JSON.stringify(r))
.concat(Array(torn).fill('{"torn')).join('\n') + '\n'
const put = (p: Paths, session: string, text: string) => fs.writeFileSync(
path.join(p.projects, slugOf('/repo/x'), `${session}.jsonl`), text)
const sheetOf = (work: string) =>
fs.readFileSync(path.join(work, 'worksheet.md'), 'utf8')
const promptAt = (sheet: string, t: string) =>
sheet.indexOf(`Prompt: prompt-${t}`)
const itemsOf = (work: string): { n: number }[] => JSON.parse(
fs.readFileSync(path.join(work, 'items.json'), 'utf8'))
test('R11: A,B,A sessions: each file read once, order kept', () => {
const { p, work } = fixture({ 'l:/repo/x': [
REC(T0, { session: 'sA' }), REC(T0 + 60000, { session: 'sB' }),
REC(T0 + 120000, { session: 'sA' })] })
put(p, 'sA', jsonl([...turn(0, 'A1'), ...turn(120000, 'A2')], 1))
put(p, 'sB', jsonl(turn(60000, 'B1')))
expect(worksheet(work, p, NOW)).toBe(3)
const sheet = sheetOf(work)
expect(sheet).toContain('Skipped 1 torn transcript lines.')
const heads = sheet.split('\n').filter(l => l.startsWith('## Item'))
expect(heads.map(h => h.slice(0, 9))).toEqual(['## Item 1',
'## Item 2', '## Item 3'])
expect(heads.map(h => /s:(s.)/.exec(h)?.[1])).toEqual(['sA', 'sB', 'sA'])
expect(promptAt(sheet, 'A1')).toBeLessThan(promptAt(sheet, 'B1'))
expect(promptAt(sheet, 'B1')).toBeLessThan(promptAt(sheet, 'A2'))
expect(itemsOf(work).map(i => i.n)).toEqual([1, 2, 3])
})
test('R12: torn lines are summed across files', () => {
const { p, work } = fixture({ 'l:/repo/x': [
REC(T0, { session: 'sA' }), REC(T0 + 60000, { session: 'sB' })] })
put(p, 'sA', jsonl(turn(0, 'A'), 1))
put(p, 'sB', jsonl(turn(60000, 'B'), 2))
worksheet(work, p, NOW)
expect(sheetOf(work)).toContain('Skipped 3 torn transcript lines.')
})
test('R13: a root at the log cap, newer than the last run, is noted', () => {
const full = Array.from({ length: 200 }, (_, i) =>
REC(T0 + i * 1000, { ids: [], outcome: 'none' }))
const { p, work } = fixture({ 'l:/repo/x': full })
fs.mkdirSync(p.out)
fs.writeFileSync(path.join(p.out, 'state.json'),
JSON.stringify({ lastRun: T0 - 1, closed: false }))
worksheet(work, p, NOW)
expect(sheetOf(work)).toContain(
`Possible eviction before labelling: ${tagOf('/repo/x')}`)
})
const use = (ms: number, id: string, command: string) => ({
type: 'assistant', timestamp: iso(ms), message: { id: `a${id}`,
content: [{ type: 'tool_use', id, name: 'Bash',
input: { command } }] } })
const result = (ms: number, id: string, err: boolean) => ({
type: 'user', timestamp: iso(ms), message: { content: [
{ type: 'tool_result', tool_use_id: id, is_error: err }] } })
const toolRec = (at: number, session: string) => REC(at, { session,
channel: 'tool', ids: ['BLK-003'], trigger: undefined,
outcome: undefined, cands: 1, ms: undefined })
test('R14: a tool item shows command, known line, next results', () => {
const { p, work } = fixture({ 'l:/repo/x': [
toolRec(T0, 'sA'), toolRec(T0 + 60000, 'sB')] })
put(p, 'sA', jsonl([use(-100, 'u1', 'make test'), result(-50, 'u1', true),
{ type: 'attachment', timestamp: iso(400), attachment: {
type: 'hook_additional_context', hookName: 'tool.call',
toolUseID: 'u1-context', content: ['memory: known BLK-003 x'] } },
use(500, 'u2', 'ls'), result(600, 'u2', false)]))
expect(worksheet(work, p, NOW)).toBe(2)
expect(sheetOf(work)).toContain('Command: make test')
expect(sheetOf(work)).toContain('Known: memory: known BLK-003 x')
expect(sheetOf(work)).toContain('Next Bash results: ok')
expect(sheetOf(work)).toContain('LOST') // sB has no transcript
})
test('R15: the post-fix floor is NEW once, state keeps postFixAt', () => {
const recs = Array.from({ length: 20 }, (_, i) =>
REC(T0 + i * 1000, { session: `s${i}`, ids: ['BDR-001'] }))
const { p, work } = fixture({ 'l:/repo/x': recs })
recs.forEach((r, i) => put(p, r.session, jsonl(turn(i * 1000, 'f'))))
worksheet(work, p, NOW)
draft(work, Array.from({ length: 20 }, (_, i) =>
`${i + 1}\tBDR-001\trelevant`).join('\n'))
const first = apply(work, p, NOW)
expect(first).toContain('NEW: post-fix prompt sample reached (20/20 asks)')
expect(first).toContain('Post-fix prompt asks: 20/20 (since ' +
'2026-11-12T19:53:59.000Z)')
const st = JSON.parse(fs.readFileSync(path.join(p.out, 'state.json'),
'utf8'))
expect(st.postFixAt).toBe(NOW)
worksheet(work, p, NOW + 1)
draft(work, '')
expect(apply(work, p, NOW + 1)).not.toContain('post-fix prompt sample')
})
/** Applies `n` relevant asks over an old state file: both gate-1 gates
* recorded, no postFixAt. Returns the state written. */
function closedAfter(n: number, closed = false): { closed: boolean } {
const recs = Array.from({ length: n }, (_, i) =>
REC(T0 + i * 1000, { session: `s${i}`, ids: ['BDR-001'] }))
const { p, work } = fixture({ 'l:/repo/x': recs })
fs.mkdirSync(p.out)
fs.writeFileSync(path.join(p.out, 'state.json'), JSON.stringify(
{ lastRun: T0 - 1, promptAt: T0 - 2, toolAt: T0 - 2, closed }))
recs.forEach((r, i) => put(p, r.session, jsonl(turn(i * 1000, 'f'))))
worksheet(work, p, NOW)
draft(work, recs.map((_, i) => `${i + 1}\tBDR-001\trelevant`).join('\n'))
apply(work, p, NOW)
return JSON.parse(fs.readFileSync(path.join(p.out, 'state.json'), 'utf8'))
}
test('R16: closed waits for the post-fix floor, old state included', () => {
expect(closedAfter(19).closed).toBe(false)
expect(closedAfter(20).closed).toBe(true)
})
test('R17: a legacy closed:true without postFixAt is rewritten open', () => {
expect(closedAfter(1, true).closed).toBe(false)
})
+53 -27
View File
@@ -16,8 +16,10 @@ import {
tagOf, writeSafe, writeState, type Draft, type Item, type Row,
type State,
} from './labels.ts'
import { PROMPT_GATE_N } from './thresholds.ts'
import {
clockStart, gateVerdict, metricsOf, renderFeedback, type Verdict,
clockStart, gateVerdict, metricsOf, renderFeedback, type Metrics,
type Verdict,
} from './metrics.ts'
import {
joinPrompt, joinTool, records, sessionFiles, type Loaded,
@@ -158,23 +160,39 @@ function toolItem(
return { item: { ...b, cited: [], lost: false }, text }
}
/** A per-run transcript loader: each file read once (R8); a record
* reads its main thread or its own sub-agent file only; torn lines
* summed once per file. */
const fileOf = (projects: string, l: Logged): string =>
sessionFiles(projects, l.root, l.rec.session, l.rec.agent)[0] ?? ''
/** A per-run transcript loader with one slot: the work is sorted by file,
* so each file is parsed once and dropped when the next one loads (memory
* stays at one transcript); a record reads its main thread or its own
* sub-agent file only; torn lines are summed at each load. */
function loaderFor(projects: string): Loader {
const files = new Map<string, Loaded>()
const read = (f: string) => {
const got = files.get(f) ?? records(f)
files.set(f, got)
return got
}
let slot: { file: string; got: Loaded } | undefined
let torn = 0
const load = (l: Logged): Loaded => {
const r = l.rec
const fs_ = sessionFiles(projects, l.root, r.session, r.agent)
return { torn: 0, recs: fs_.flatMap(f => read(f).recs) }
const file = fileOf(projects, l)
if (file === '') return { torn: 0, recs: [] }
if (slot?.file !== file) {
slot = { file, got: records(file) }
torn += slot.got.torn
}
return slot.got
}
const torn = () => [...files.values()].reduce((t, x) => t + x.torn, 0)
return { load, torn }
return { load, torn: () => torn }
}
const byFile = (a: { file: string; n: number }, b: typeof a) =>
a.file === b.file ? a.n - b.n : a.file < b.file ? -1 : 1
/** Every record joined, file by file; the items come back in `n` order. */
function joinAll(pop: Logged[], projects: string) {
const loader = loaderFor(projects)
const work = pop.map((l, i) =>
({ l, n: i + 1, file: fileOf(projects, l) })).sort(byFile)
const joined = work.map(w => joinOne(w.l, w.n, loader.load))
.sort((a, b) => a.item.n - b.item.n)
return { joined, torn: loader.torn() }
}
/** One worksheet item and its redacted text. */
@@ -212,9 +230,8 @@ function collect(p: Paths, now: number) {
const logs = readLogs(storeFile(p.store))
const pop = population(logs, new Set(rows.map(keyOf)),
clockStart(rows, logAtsOf(logs)), now)
const loader = loaderFor(p.projects)
const joined = pop.map((l, i) => joinOne(l, i + 1, loader.load))
return { joined, head: sheetHead(pop.length, loader.torn(),
const { joined, torn } = joinAll(pop, p.projects)
return { joined, head: sheetHead(pop.length, torn,
evictions(logs, readState(stateOf(p)).lastRun)) }
}
@@ -238,21 +255,29 @@ const draftOf = (dir: string) => {
return fs.existsSync(f) ? fs.readFileSync(f, 'utf8') : ''
}
/** The state after this run, and a NEW line per gate first reached. */
function nextState(st: State, v: Verdict, now: number) {
/** The state after this run, and a NEW line per gate or floor first
* reached. */
function nextState(st: State, m: Metrics, v: Verdict, now: number) {
const news: string[] = []
let { promptAt, toolAt } = st
let { promptAt, toolAt, postFixAt } = st
if (promptAt === undefined && v.prompt === 'reached') {
promptAt = now
news.push('NEW: prompt gate reached')
}
const toolDone = v.tool === 'reached' || v.tool === 'insufficient'
if (toolAt === undefined && toolDone) {
if (toolAt === undefined &&
(v.tool === 'reached' || v.tool === 'insufficient')) {
toolAt = now
news.push(`NEW: tool gate ${v.tool}`)
}
const closed = promptAt !== undefined && toolAt !== undefined
return { state: { lastRun: now, promptAt, toolAt, closed }, news }
if (postFixAt === undefined && m.postAsks >= PROMPT_GATE_N) {
postFixAt = now
news.push('NEW: post-fix prompt sample reached ' +
`(${m.postAsks}/${PROMPT_GATE_N} asks)`)
}
const closed = promptAt !== undefined && toolAt !== undefined &&
postFixAt !== undefined
return { state: { lastRun: now, promptAt, toolAt, postFixAt, closed },
news }
}
const assertFresh = (items: Item[], file: string) => {
@@ -275,7 +300,7 @@ function commit(
): string {
assertFresh(items, labelsOf(p))
const { fresh, m, v } = projected(items, drafts, p, now)
const next = nextState(readState(stateOf(p)), v, now)
const next = nextState(readState(stateOf(p)), m, v, now)
const feedback = renderFeedback(m, v, now)
if (fresh.length > 0) appendRows(labelsOf(p), fresh)
writeState(stateOf(p), next.state)
@@ -285,6 +310,7 @@ function commit(
const isCount = (v: unknown) =>
typeof v === 'number' && Number.isInteger(v) && v >= 0
const isSafeCount = (v: unknown) => isCount(v) && Number.isSafeInteger(v)
const optCount = (v: unknown) => v === undefined || isCount(v)
const idsOk = (v: unknown): v is string[] => Array.isArray(v) &&
v.every(x => typeof x === 'string' && ENTRY_ID.test(x))
@@ -305,7 +331,7 @@ function itemError(raw: unknown): string {
['ids', idsOk(i.ids)],
['cited', idsOk(i.cited) && i.cited.every(c =>
(i.ids as string[]).includes(c))],
['numbers', isCount(i.at) && isCount(i.n) && optCount(i.cands) &&
['numbers', isSafeCount(i.at) && isCount(i.n) && optCount(i.cands) &&
optCount(i.ms)],
['lost', typeof i.lost === 'boolean'],
]
+3 -1
View File
@@ -5,11 +5,13 @@ export const DAY_MS = 86400000
export const GATE_DAYS = 35 // the 5-week cap from the gate clock start
export const PROMPT_GATE_N = 20 // labelled injected prompt asks
export const TOOL_GATE_N = 10 // labelled tool injections
// Merge d3d5e6c of 7e87d77 (idle trigger >= 8 words, cap 2 entries): prompt
// rows before it still reflect the old triggers (BDR-146 residual).
export const GATE1_FIX_AT = Date.parse('2026-11-12T19:53:59Z')
export const T = {
promptKeep: 0.64, // relevant / labelled entries (4c baseline 14/22)
triggerFloor: 0.5, // worst trigger kind under this gets tightened
thirdPick: 0.4, // 3rd pick precision under this caps picks at 2
lateShare: 0.25, // timeout + paused among asks with candidates
msP90: 2500, // rerank p90 under this lowers RERANK_MS
toolKeep: 0.5, // tool precision under this narrows the match
+2 -5
View File
@@ -126,14 +126,11 @@ const textsOf = (r: Rec): [string, string][] => blocksOf(r)
/** Main-thread assistant text after `ts` up to the next prompt. */
function replyAfter(recs: Rec[], ts: number): string {
const end = nextPromptAt(recs, ts)
const seen = new Set<string>()
return recs
const texts = recs
.filter(r => r.type === 'assistant' && r.isSidechain !== true)
.filter(r => tsOf(r) > ts && tsOf(r) < end)
.flatMap(textsOf)
.filter(([key]) => !seen.has(key) && !!seen.add(key))
.map(([, text]) => text)
.join('\n')
return [...new Map(texts).values()].join('\n') // first position kept; the value is derived from the key
}
export type PromptJoin = { prompt: string; injected: string; reply: string }
+2 -2
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env bash
# lib/tests/memory-feedback-remind.test.sh — the phase 4d banner line:
# shown when /memory-feedback never ran, or ran 5+ days ago and the store
# changed since; silent once both gates are recorded, with recall off, or
# on any error.
# changed since; silent once all three gate marks are recorded, with recall
# off, or on any error.
set -u
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
LIB="$ROOT/lib/memory-feedback-remind.sh"
+36
View File
@@ -2658,6 +2658,42 @@ test('IR11: an injection leaves one log record; 205 keep 200', async (
expect(await loadLog(storeApi(w), ROOT)).toHaveLength(200)
})
/** The key set of a log record is pinned: a text field must fail. */
const KEYS = ['agent', 'at', 'cands', 'channel', 'ids', 'ms', 'outcome',
'session', 'trigger']
test('IR11b: an injected record holds exactly these keys', async (
$, on,
) => {
const { w } = await bootRecall($, on)
await start($)
await say($, GO)
const log = await loadLog(storeApi(w), ROOT)
expect(log.map(r => Object.keys(r).sort())).toEqual([KEYS])
})
test('IR11c: a none record holds exactly these keys', async ($, on) => {
const { w } = await bootRecall($, on)
w.model = { text: 'none' }
await start($)
await say($, GO)
const log = await loadLog(storeApi(w), ROOT)
expect(log.map(r => Object.keys(r).sort())).toEqual([KEYS])
})
test('IR11d: a no-candidate record holds these keys, no ms', async (
$, on,
) => {
const { w } = await bootRecall($, on)
w.proc = { stdout: rowsOut([['BLK-001', 'x'], ['LRN-002', 'y']])
.replace(/open/g, 'superseded') }
await start($)
await say($, GO)
const log = await loadLog(storeApi(w), ROOT)
expect(log.map(r => Object.keys(r).sort()))
.toEqual([KEYS.filter(k => k !== 'ms')])
})
test('IR12: a pending milestone and a trigger give [note, block]', async (
$, on,
) => {
+4 -2
View File
@@ -72,11 +72,13 @@ MFD="$(cd -P "$HOME/.claude/lib" && pwd)/memory-feedback"
```
rc 1 `draft refused: <reason>` → fix the draft, rerun. It prints
FEEDBACK.md: show the gate lines and the Actions list to the user.
FEEDBACK.md: show the gate lines (`Post-fix prompt asks: k/20` included:
actions wait for 20 post-fix asks) and the Actions list to the user.
## STEP 6 — A GATE REACHED FOR THE FIRST TIME
Only when the output ends with a `NEW: …` line:
Only when the output ends with a `NEW: …` line (a prompt gate, a tool gate
or `NEW: post-fix prompt sample reached`):
1. Propose the verdict as an EVAL entry (counts and shares from
FEEDBACK.md, no prompt text, no other repo's name) and each Action as a
TODO line of this config repo; the user approves each. Actions naming