Merge chore/memory-feedback-gate1 into develop

This commit is contained in:
bchanot committed 2026-11-12 18:07:33 +01:00
commit 1b82a25f2b
4 files changed
+29 -1

No files matched your search

+7
View File
@@ -447,3 +447,10 @@ Dogfood: 3 blind lenses attacked the v1 plan for the plan-challenge feature itse
- **Result**: plan r1 FATAL(9) (note unprovable in `-p`) before code. Task reviews caught 5 Important (quadratic STATUS regex, heredoc commit form never matched, unguarded `fs.exists`, whole-command milestone regexes, fs.exists missing/unreadable). Final opus review still found 2 Important + 13 minors (all-placeholder grams on real registries, redaction gaps, `> \d+` letterless, success path reading the store per call). Security gate PASS ×3 but each coverage round found new shapes (cookie tail from the fix itself, `-b`, JSON affixes). Fix rounds introduced 2 regressions (quadratic JWT, cookie 400-cap), both caught by scoped re-reviews.
- **Anomalies**: controller ruling "letterless rule subsumed by ERRORISH" was wrong (`> \d+` alternative) and cost one revert; "1:1 port" claim in noise.py drifted as the TS rule grew; the verifier's second pass refused `gates.sh run` (it rewrites the contract) and re-ran the CHECKs read-only instead.
- **Action**: a ruling that removes a check lists the counterexample search over the alternatives first; reviewers measure regex cost by running; keep a scoped re-review after every fix round, a round of its own can regress; a port of a live rule carries the commit it was ported from.
## EVAL-058 — Memory recall gate 1: 57 % precision (n=75), idle 44 %, 3rd pick 35 %, tool channel empty (2026-11-12)
- **Checked**: phase 4c recall quality at first `/memory-feedback` prompt gate (BDR-140), window 2026-11-08 → 2026-11-12.
- **Method**: 28 records, 2 repos (config + 1 project), 75 injected entries labelled relevant/irrelevant against the ask; 5 labels spot-checked by user (0 flip); misses searched on 5 asks picked where a miss was suspected (biased, overstates the miss rate); gate output `FEEDBACK.md`.
- **Result**: prompt precision 57 % (n=75), under the 64 % (14/22) 4b-4d proof target. By trigger: fresh 78 % (n=23), idle 44 % (n=32), keyword 55 % (n=20). 3rd pick 35 % (n=23). Timeout + paused 0 %. Rerank p90 3355 ms. Tool gate 0/10 records. Misses: 7 on 5 asks, 4 no-overlap (BM25 cannot reach, title+lesson share no word with the prompt), 3 overlap (rerank dropped them). 1 noisy entry + 1 promotion candidate in the project repo, to run there via /prune-memory.
- **Anomalies**: idle trigger fires on short follow-up prompts ("ok on y va") where all 3 entries miss; tool channel logged 0 records in 4 days, unproven live; rerank p90 above the 3 s bound.
- **Action**: 4 TODO lines under Phase 4b-4d (idle trigger, cap at 2, tool channel check, no-overlap misses). Decisions deferred to the user.
+1
View File
@@ -727,3 +727,4 @@ rules:
- 2026-11-12: /feat dead-deny lock on feature/dead-deny-lock: 79f1a3f push-guard T41 drops dead Write lines, T42 justified list until 3.0.0 (T42c/d/e). 2 challenge rounds (r2 FATAL: base-keyed expiry no green path -> version expiry re-gated). Gates MET/CONFORME/PASS, MEDIUM fail-open fixed. User hand edit of settings.json pending (TODO:92).
- 2026-11-12: /doc 3.0.0 on chore/docs-3.0.0: ed67c52 MIGRATION 'Upgrading to 3.0.0' (7 breaking items), CHANGELOG gitflow deny 14 entries, Fixed bullet dead 2.0.0 deny forms, [2.0.0] Linux residual fixed in place, '(98 checks)' dropped. Next: /release-candidate.
- 2026-11-12: /reconcile before 3.0.0: 75 SHAs + 31 merges verified, nothing blocks the cut; TODO restructured (9 stale boxes fixed, shipped BDR-125 phase blocks to History, residuals grouped, post-3.0.0 items: empty push-guard removed-ok at next bump, D3 MEDIUM /bugfix). CHANGELOG internal headings -> [internal X.Y.Z].
- 2026-11-12: /memory-feedback gate 1 (prompt 26/20): 28 records, 75 labels, 5 spot-checked (0 flip), 7 misses on 5 asks. Precision 57 % under the 64 % target, idle 44 %, tool channel empty. EVAL-058 + 4 TODO lines on chore/memory-feedback-gate1.
+19
View File
@@ -3,6 +3,20 @@
## Open (reconciled 2026-11-12 before 3.0.0, /reconcile on the BDR-125 phases; full pass 2026-10-23)
Done history before 2026-10-08 removed 2026-10-23 (git keeps it: `git log -p -- .claude/tasks/TODO.md`); decisions live in `.claude/memory/`.
### BDR-125 close-out run (user 2026-11-12): no phase 0/0b/4a/1/5/2/4b/4c/4d line left in TODO or BUGS-FOUND
Decisions at start (asked once): faunosteo memory v2 on `chore/memory-v2` off develop in a fresh worktree (split already on develop 7a75974, 11 rules); settings.json `npm -g` deny lines removed on the block E branch, classifier test = `npm install -g --dry-run cowsay`; 3.0.0 cut right after block A (/release-candidate), later blocks land under the next Unreleased; LRN-270 bisect = user-run interactive sessions, one `--settings` file per pack. Merge prompts (`gitflow finish` ask) = the human signal per block once gates are green.
- [ ] A merge chore/memory-feedback-gate1 (b7505d1 + routing `confirmed` row) into develop
- [ ] A' /release-candidate 3.0.0 (scope as ticked 2026-11-11)
- [ ] B chore/phase4-proofs: known-error replay (`memory: known BLK-xxx` + tool record), live redaction (fake secret → ledger REDACTED), digest after a real /compact, `/context` full+dev delta vs the 2026-11-09 base; each = command + output + journal line; a failure → /bugfix
- [ ] C /feat gate 1: idle trigger tightened (≥ 8 words AND a real idle gap), recall cap 2, body indexed for BM25 with p90 before/after (> 3 s → back to title+lesson, "accepted"), 4d.4 built (/prune-memory step D: noisy rewrite + BDR-100 promotion)
- [ ] D /feat D1b: digest falls back on the first unchecked items when `### Actionable` is absent; init-project + onboard templates emit the heading; local repos' TODO migrated on `chore/*` (one per repo, merge on ask); Session-start step 1 "Read TODO.md" dropped from the doctrine, citers census
- [ ] E phase 1 + 2 decisions: design hook DELETED (web-building.md loads by path); "Skill routing" lines → skills' `description:`; `npm -g` deny lines out + dry-run test; rtk `cat|head|tail *.env*` forms benched, every form kept without proof of obsolescence, T42 list updated; BLK-022 buried (phase 2 covers it): guard-bash.test.sh SKIP spec deleted, BLK-022 → superseded, TODO "Blocked" block removed
- [ ] F faunosteo `chore/memory-v2`: `memory.sh upgrade`, `/context` before/after, /prune-memory there for BDR-080 (noisy) + LRN-080 (promotion); never merged by me
- [ ] G BLK-044: trace `oom_score_adj` origin (unit, cc.sh), `OOMScoreAdjust` on the `cc` scope, proof via `/proc/<pid>/oom_score_adj`
- [ ] H residuals in order: phase 0 (project-rules.sh emit/extract/`..`), 4a (BLK-038, MIGRATION.md:209), 5 (fragment key allowlist), 2 LOWs (awk, `mkdir -m 700`, `cat x.key`, doctor.sh:255), 4d follow-ups (transcripts per (root, session, agent), image prompts, threshold boundary tests, `tsc`, `l:<root>` one global key), BUGS-FOUND unfixed since 11-09, LRN-270 bisect (user-run), doctor skillOverrides residue; a residual NOT done = justified in the TODO line removal
- [ ] I model-router live checks: W4-B AC9 + parentModel done from transcripts/logs; Fable-out-of-credits + StopFailure closed "not reproducible, reopen on incident"
- [ ] J /reconcile, CHANGELOG (gate 1 at 57 % delivered + each block), 0 BDR-125 line left, EVAL of the run, /close
### Context mod queue (BDR-125): one /ship-feature per phase, order (user 2026-10-25): 0 → 4a → 1 → 5 → 2 → 3 (conditional) → 4b-4d
Re-challenged 2026-10-25 on Fable (4 lenses + native-features guide, synthesis in `.claude/tasks/plans/2026-10-25-context-mod-queue-challenge.md` § RE-CHALLENGE; user picks: native ask merge gate, phase 3 conditional, rest in block). The plan halves: native carriers and existing files first, a mod only where a measured gain has no native lever.
Shared rules for every phase:
@@ -29,6 +43,11 @@ Shared rules for every phase:
- [ ] BLK-044: trace origin of `oom_score_adj=-1000` on claude processes; set `OOMScoreAdjust` on the `cc` scope so a runaway claude dies first.
- [ ] safety-residuals gate LOWs (2026-11-06, bugfix/safety-residuals): `rtk read -n server.key` / `rtk read id_rsa` at repo root pass (`.key`/`.pem`/`id_*` rtk denies all need `*/`; pre-existing, bench before adding); session-start `memory.sh check` and digest unbounded when `timeout` is absent (stock macOS); pre-commit allowlist fails open if `git diff --cached` errors (pre-existing, Gitea backstop); `lib/memory.sh` `_mem_path_safe` line > 80 chars; `claude-stub.sh` interpolates the mktemp log path into the generated stub (test-only).
- Proof: recall precision on ≥20 real prompts and ≥10 real errors, target ≥ the 14/22 baseline; digest after a real compaction; redaction test on the ledger; `/context` delta; captured-failure demo: an error from session N recalled in session N+1.
- Gate 1 result (2026-11-12, EVAL-058): prompt precision 57 % (n=75), under the 64 % target; fresh 78 %, idle 44 %, keyword 55 %; tool gate 0 records.
- [ ] 4d gate 1 (2026-11-12, EVAL-058): idle trigger at 44 % precision: tighten (skip prompts under N words, or only after a real idle gap) or drop; decide before the next gate
- [ ] 4d gate 1: cap recall at 2 entries (3rd pick 35 %)
- [ ] 4d gate 1: tool channel logged 0 records in 4 days: check the PostToolUse error match actually fires (live error in a session, then memory-feedback)
- [ ] 4d gate 1: misses 4/7 no-overlap: BM25 over title+lesson cannot reach them; option = index body too, or accept
### BDR-125 residuals (after 3.0.0)
Open children of the shipped phases (History 2026-11-12), text verbatim.
+2 -1
View File
@@ -203,7 +203,8 @@
"deploy": "verify",
"feat": "reflect",
"bugfix": "reflect",
"capitalize": "apply"
"capitalize": "apply",
"memory-feedback": "reflect"
}
},
"changed": {},