chore(bench): phase 4c re-measure after the content floor, 5 s rerank bound gated
This commit is contained in:
1 parent
debdead5e0
commit
c43b18cdcd
2 files changed
+6
No files matched your search
@@ -48,6 +48,8 @@ Digest cap order `[gated 2026-11-05]` (user: "Retirer d'abord les TODO"): over 8
|
||||
|
||||
Measurement gate `[gated 2026-11-05]` (RECALL.md: prompt channel 19/38 = 50 % relevant < 64 % bar, noise on content-free idle prompts; timeout share 4/20 = 20 %; user: "Plancher de contenu"): no intent recall on a prompt with fewer than 5 content words (`tokensOf` minus stop words); recall ships on and switchable; the 3 s bound stays; 4d reads the injection log.
|
||||
|
||||
Rerank bound gate `[gated 2026-11-05]` (three harness runs: 16/55 haiku calls over 3 s = 29 % > the 25 % bar; 1/55 over 4.5 s and over 5 s; user: "5 s"): `RERANK_MS` = 5000 (own race and `timeoutMs`). After the content floor: 25/41 = 61 % relevant counting entries written after the prompt, 19/35 = 54 % excluding them (RECALL.md).
|
||||
|
||||
9. `[gated 2026-11-05]` Mod tests: `mods/memory/hooks/register.test.ts` holds the plan's named tests and the mods suite is green; no line of register.ts over 80 columns.
|
||||
CHECK: cd /home/bchanot/Documents/claude && T=mods/memory/hooks/register.test.ts && for n in SR1 SR3 SR5 LS1 LS4 ER1 ER3 RB1 RB3 RB5 IR1 IR5 IR6 IR7 IR8 IR9 IR11 TE1 TE2 TE4 PM1 PM2; do /usr/bin/grep -q "$n" "$T" || exit 1; done && [ "$(awk 'length($0) > 80' mods/memory/hooks/register.ts | wc -l)" = 0 ] && a=$(make test suite=lib/tests/mods.test.sh 2>&1) && printf '%s\n' "$a" | /usr/bin/grep -q 'all suites green' && echo MOD4C-OK
|
||||
EXPECT: MOD4C-OK
|
||||
|
||||
@@ -25,3 +25,7 @@ The noise sits on content-free prompts triggered by idle time (short approvals):
|
||||
## Decision (2026-11-05)
|
||||
|
||||
User: "Plancher de contenu" — no intent recall on a prompt with fewer than 5 content words (stop words excluded); recall ships on and switchable (`"recall": false`); the injection log (4c) feeds the 4d measurement on real use. The 3 s rerank bound stays (timeout share 20 % < 25 %).
|
||||
|
||||
## After the content floor (debdead) and the bound decision
|
||||
|
||||
Same harness, same seed: 15 prompts meet the trigger (was 24), all sampled; 14 injections; n = 41 entries; relevant 25 / 41 = 61 % counting entries written after the prompt (temporal leakage, 6 entries), 19 / 35 = 54 % excluding them. Still under the 64 % bar; the misses are vague follow-ups, the hits are topical prompts. haiku `duration_api_ms` > 3000 over the three runs: 16 / 55 = 29 % (> 25 % bar); > 4500: 1 / 55; > 5000: 1 / 55. User decision: `RERANK_MS` = 5000.
|
||||
Reference in new issue
Block a user