From c43b18cdcd7e0f4ee936dd70db513e36d08ce965 Mon Sep 17 00:00:00 2001 From: bchanot Date: Thu, 5 Nov 2026 17:46:15 +0100 Subject: [PATCH] chore(bench): phase 4c re-measure after the content floor, 5 s rerank bound gated --- .claude/tasks/contracts/2026-11-04-memory-recall-1859.md | 2 ++ .claude/tasks/plans/2026-11-04-memory-recall-spike/RECALL.md | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/.claude/tasks/contracts/2026-11-04-memory-recall-1859.md b/.claude/tasks/contracts/2026-11-04-memory-recall-1859.md index 26e85f1..1f48be8 100644 --- a/.claude/tasks/contracts/2026-11-04-memory-recall-1859.md +++ b/.claude/tasks/contracts/2026-11-04-memory-recall-1859.md @@ -48,6 +48,8 @@ Digest cap order `[gated 2026-11-05]` (user: "Retirer d'abord les TODO"): over 8 Measurement gate `[gated 2026-11-05]` (RECALL.md: prompt channel 19/38 = 50 % relevant < 64 % bar, noise on content-free idle prompts; timeout share 4/20 = 20 %; user: "Plancher de contenu"): no intent recall on a prompt with fewer than 5 content words (`tokensOf` minus stop words); recall ships on and switchable; the 3 s bound stays; 4d reads the injection log. +Rerank bound gate `[gated 2026-11-05]` (three harness runs: 16/55 haiku calls over 3 s = 29 % > the 25 % bar; 1/55 over 4.5 s and over 5 s; user: "5 s"): `RERANK_MS` = 5000 (own race and `timeoutMs`). After the content floor: 25/41 = 61 % relevant counting entries written after the prompt, 19/35 = 54 % excluding them (RECALL.md). + 9. `[gated 2026-11-05]` Mod tests: `mods/memory/hooks/register.test.ts` holds the plan's named tests and the mods suite is green; no line of register.ts over 80 columns. CHECK: cd /home/bchanot/Documents/claude && T=mods/memory/hooks/register.test.ts && for n in SR1 SR3 SR5 LS1 LS4 ER1 ER3 RB1 RB3 RB5 IR1 IR5 IR6 IR7 IR8 IR9 IR11 TE1 TE2 TE4 PM1 PM2; do /usr/bin/grep -q "$n" "$T" || exit 1; done && [ "$(awk 'length($0) > 80' mods/memory/hooks/register.ts | wc -l)" = 0 ] && a=$(make test suite=lib/tests/mods.test.sh 2>&1) && printf '%s\n' "$a" | /usr/bin/grep -q 'all suites green' && echo MOD4C-OK EXPECT: MOD4C-OK diff --git a/.claude/tasks/plans/2026-11-04-memory-recall-spike/RECALL.md b/.claude/tasks/plans/2026-11-04-memory-recall-spike/RECALL.md index fa8e6dc..325f675 100644 --- a/.claude/tasks/plans/2026-11-04-memory-recall-spike/RECALL.md +++ b/.claude/tasks/plans/2026-11-04-memory-recall-spike/RECALL.md @@ -25,3 +25,7 @@ The noise sits on content-free prompts triggered by idle time (short approvals): ## Decision (2026-11-05) User: "Plancher de contenu" — no intent recall on a prompt with fewer than 5 content words (stop words excluded); recall ships on and switchable (`"recall": false`); the injection log (4c) feeds the 4d measurement on real use. The 3 s rerank bound stays (timeout share 20 % < 25 %). + +## After the content floor (debdead) and the bound decision + +Same harness, same seed: 15 prompts meet the trigger (was 24), all sampled; 14 injections; n = 41 entries; relevant 25 / 41 = 61 % counting entries written after the prompt (temporal leakage, 6 entries), 19 / 35 = 54 % excluding them. Still under the 64 % bar; the misses are vague follow-ups, the hits are topical prompts. haiku `duration_api_ms` > 3000 over the three runs: 16 / 55 = 29 % (> 25 % bar); > 4500: 1 / 55; > 5000: 1 / 55. User decision: `RERANK_MS` = 5000.