chore(memory): BDR-116 amendment + EVAL-044 + journal/TODO/contract w3b — feat model-router wave 3-B

This commit is contained in:
bchanot
2026-10-11 14:25:53 +02:00
parent 70416222b1
commit 563a154446
7 changed files with 231 additions and 1 deletions
+4 -1
View File
@@ -26,7 +26,10 @@ migration of shifters/pins/model-gate in wave 2 after proof; names model-router
## 2026-10-10 — model-router wave 3-A: first-use route confirmation + decision memory (feature/model-router-confirm)
Plan `.claude/tasks/plans/2026-10-10-model-router-w3a-confirm-1201.md` r4, contract `2026-10-10-model-router-w3a-confirm-1201`. User decisions 2026-10-10: tracked routing.json reached through the plugin dir (no new link); blocking `$.ui.ask` dialog at first use; per skill row, agent row, main phase.
- [x] W3-A landed (22455c0, 2026-10-11): routing.json = phases + rows + decisions; T1/T2/T3 dialogs; project exceptions `projects[<normalized remote>]` in the tracked file (project-tree layer dropped: security); writers = dialog + `/route ask`; `/route pending`; census from the file with the `changed` WARN exemption; kit 86 → 190. Gates: 3 lenses + 1 confirmation (BLOCKER census), feater + 4 rounds, verifier CONFORME + re-verify, security BLOCK(1) credential leak fixed → PASS, full make test green (design-tool-gate env red). Live: T2 dialogs answered by the user (verifier Keep, feater Keep, security-auditor → implement then reset to verify on user go).
- [ ] W3-A live checks still open: T1 (typed `/feat` etc.) and T3 (first `route(phase=…)` call) dialogs; a parallel same-agent dispatch answered after > 10 s; what an unanswered dialog resolves to on the terminal (must be Later or nothing written). Record in the contract as `[gated]` when seen.
- [x] W3-B dialog with context (2026-10-11, 604a6c4, contract `2026-10-11-model-router-w3b-dialog-1240`, plan r3): descriptions (skill frontmatter / agent.offer / phase `about`), real id + effort in the question, Later / Keep / Change, model → effort (derived from the phases of that model) → scope, rows stay phases, T3 Later/Keep. 3 lenses (2 BLOCKERs on my r1 avoided) + 1 confirmation, feater + 2 rounds, verifier CONFORME (3rd), security PASS (4 LOW: symlinked SKILL.md read, TOCTOU, raw key in a log, any plugin hooking AskUserQuestion could answer). Live decisions: security-auditor → judge (kept, floor aligned `model: opus`), plan-challenger → verify (reset on user go), orchestrate Keep (T3 live check done).
- [ ] W3-B residuals: a Change answered Everywhere can store a machine-only phase name (defined in `~/.claude/model-router.json`) into the tracked file (security note, correctness); SKILL.md symlink follow (LOW); `descs` cap and "(custom)" label untested; the security-auditor's `changed.from = verify` entry stays in routing.json although the floor now equals the row (harmless; prune at release).
- [ ] trace the "auto mode: use Bash/sed instead of Edit/Write" instruction block that sub-agents see attributed to the model-router MCP instructions (security-auditor 2026-10-11): it is NOT in mods/model-router source; likely the harness's own auto-mode text rendered next to the mod's MCP block. Confirm the origin.
- [ ] W3-A live checks still open: T1 (typed `/feat` etc.) dialog (T3 seen live 2026-10-11: orchestrate Keep); a parallel same-agent dispatch answered after > 10 s; what an unanswered dialog resolves to on the terminal (must be Later or nothing written). Record in the contract as `[gated]` when seen.
- [ ] W3-A residuals (security LOW, accepted): non-atomic cross-process write (two sessions, crash mid-write → file reads as failed, previous config kept); `host/path` of a private remote in the tracked file; a persistent write failure re-asks every use (`asked.delete` in the catch); `out.length` vs byte size at the cap; `changePatch` scope 'project' with a vanished key writes Everywhere (cwd change mid-dialog). Deferred: `/route set`, `/route confirm`, `(unconfirmed)` in show, dialog edits of phases, frontmatter auto-alignment, session-start warning when routing.json is dirty, per-machine `projects`.
- [ ] routing.json ships `confirmed` for verifier/feater (user Keeps) and phases verify/implement: every clone inherits them (by design: decisions travel). Revisit at release if unwanted.
@@ -0,0 +1,38 @@
# CONTRACT — model-router-w3b-dialog
- date: 2026-10-11 | flow: feat | branch: feature/model-router-confirm (continues W3-A before its merge)
- status: active
## REQUEST (verbatim — IMMUTABLE)
j'aimerais qu'on vois pour que quand je le prompt de demande de confirmation ou changement pour un model et un effort, il faudrait qu'il soit un peu plus complet. par exemple expliquer brievement ce que fait ce pour quoi on demqnde de choisir plutot que juste le nom. car juste verifier en vrai ca peut etre pleins de chose, ou dire feature : auditor. car c'est pas tres clair on sait pas pour quoi on prend la decision. Surtout que lam auditor security je croism ca proposait sonnet alors que nonm un audit securite ca devrait etre le plus performant du model. bref metre un peu de contextm dire oui ca me vam ou changer et si on choisis de changerm proposer quel model puis quel effort, puis pour userscope et projet ca c'est bien, et l'enregistrer.
## CLARIFICATIONS
(pass A: none — outcome, scope and flow given in the request; W3-A contract and plan r4 give the base)
Q: public shape (orchestrator default after three lenses, user may veto): ASK1 options Later / Keep / Change; ASK2 model = the four aliases labelled with their tier role (`fable (best)`, `opus (big)`, `sonnet (work)`, `haiku (cheap)`; Other = Later); ASK3 effort = the efforts offered by the phases headed by that model (fable: medium high xhigh max; opus: xhigh, skipped; sonnet: low medium high xhigh; haiku: low, skipped); the pair maps to that phase (rows stay phase names; a pair no phase offers is added by hand as a new phase in routing.json); ASK4 scope as W3-A; T3 (main-loop phase) = Later / Keep with context, no edit (BDR-116) [stated 2026-10-11]
Q: live decisions during this run (plan-challenger judge → verify Everywhere; orchestrate phase Keep = the T3 live check) / A: user "Revenir à judge" → row reset, its confirmed/changed entries removed; orchestrate Keep kept [gated 2026-10-11]
Q: live W3-B dialog answered during the security scan: security-auditor → Change → opus → xhigh → Everywhere = row `judge` (the request's own example; the first T2 answer through the new dialog) / A: kept; my reset of `changed` had wiped its from/to entry (census red at 604a6c4) → restored in the follow-up data commit [gated 2026-10-11]
Q: GATE 1 ran 3 verifiers (ECARTS(2) coverage → ECARTS(1) one non-discriminating test → CONFORME), security PASS (4 LOW) [recorded 2026-10-11]
## ACCEPTANCE CRITERIA
1. ASK1 text carries context: for a skill row "`/<name>` — <description ≤ 140 chars from the SKILL.md frontmatter, five YAML forms, first sentence>. Routed to <phase> (<about>): <model id> at <effort>. OK?"; for an agent row "`<name>` — <description from agent.offer ≤ 140 chars>. Routed to …"; for a phase "Phase `<name>` — <about>; used by <n> rows [+ its non-row consumers]: <model id> at <effort>. OK?"; descriptions and `about` pass `clean()` (Cc/Cf stripped, newlines folded, code-point truncation with "…"); a missing or unreadable description, an unset HOME or a name outside `^[A-Za-z0-9][A-Za-z0-9._:-]*$` degrades to the name alone (never blocks, never reads outside `${HOME}/.claude/skills/<name>/SKILL.md`, stat kind `file`, size-capped). Options: Later / Keep / Change (T3: Later / Keep); Other at ASK1 = Later + toast. One kit test per clause.
2. Change flow (rows only): ASK2 model = four fixed labels (Other or unknown = Later + toast); ASK3 effort = the efforts of the phases headed by the chosen alias, sorted ascending by level (≤ 4 labels, one → skipped, zero → Later + toast; Other = Later); target phase = the row's current phase when it matches, else the first matching phase in `cfg.phases` order; same phase as now = Keep (no `changed` entry); ASK4 as W3-A ([Everywhere, This project only], or [Everywhere, Later] with no repo key); any non-matching answer = Later, nothing written; storage as W3-A (`changed` from/to strings, `confirmed` = to); the new route applies to the current decision at once; after a main-row change the toast reports the real decision and the `/route switch on` hint when the pick is a downgrade. T3 never changes anything. One kit test per clause.
3. Rows stay phase names in every layer; `ALTS`, the alt options and the free-text phase branch of the W3-A dialog are removed (no dead code); the W3-A dialog tests are migrated to the new answers and every existing test stays green.
CHECK: ! grep -qE 'ALTS\b' mods/model-router/hooks/register.ts && echo W3B-NO-ALTS
EXPECT: W3B-NO-ALTS
EVIDENCE: MET exit=0 marker-found :: W3B-NO-ALTS
4. `routing.json` phases gain an `about` string (≤ 120 chars) for the 11 phases, loaded into a separate map (never on `Route`, never in `DEFAULT_CONFIG`), a non-string or overlong value dropped + logged once; descriptions never read from the project tree.
CHECK: bash .claude/tasks/contracts/w3b-about.sh
EXPECT: W3B-ABOUT
EVIDENCE: MET exit=0 marker-found :: W3B-ABOUT
5. Census unchanged and green (rows are phases; the DEFAULT == file phase lock keeps matching with `about` present); `lib/effort-shift.md` dialog lines updated (≤ 4 lines changed, Later/Keep/Change and the model-then-effort flow named).
CHECK: bash lib/tests/effort-routing.test.sh >/dev/null 2>&1 && grep -q 'Change' lib/effort-shift.md && [ "$(wc -l < lib/effort-shift.md)" -le 66 ] && echo W3B-CENSUS
EXPECT: W3B-CENSUS
EVIDENCE: MET exit=0 marker-found :: W3B-CENSUS
6. Kit suite, mods suite and validate green.
CHECK: cd mods/model-router && out="$(claude plugin test . 2>&1)" && printf '%s\n' "$out" | grep -qE '[0-9]+ pass' && ! printf '%s\n' "$out" | grep -qE '[1-9][0-9]* fail' && claude plugin validate . 2>&1 | grep -q 'passed' && cd ../.. && make test suite=lib/tests/mods.test.sh 2>&1 | grep -q 'all suites green' && echo W3B-MOD-GREEN
EXPECT: W3B-MOD-GREEN
EVIDENCE: MET exit=0 marker-found :: W3B-MOD-GREEN
7. Security posture unchanged: writers, paths, caps, no model-originated write; descriptions truncated and control characters stripped before they enter a question. Judged by reading + one kit test (a description with control characters and 500 chars → ≤ 140 clean chars).
## FILE SCOPE
mods/model-router/routing.json, mods/model-router/hooks/register.ts, mods/model-router/hooks/register.test.ts, lib/tests/effort-routing.test.sh, lib/effort-shift.md; .claude/tasks/contracts/w3b-about.sh (oracle, orchestrator)
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# GATE 0 oracle, contract w3b: every routing.json phase carries a 1-120 char `about`.
set -u
python3 -I - <<'PY'
import json,sys
r=json.load(open('mods/model-router/routing.json'))
ok=all(isinstance(v.get('about'),str) and 0<len(v['about'])<=120 for v in r['phases'].values())
print('W3B-ABOUT' if ok else 'W3B-ABOUT-MISSING'); sys.exit(0 if ok else 1)
PY
@@ -0,0 +1,170 @@
# PLAN r3 — model-router wave 3-B: a dialog with context, model then effort on change (2026-10-11)
r1 → r2 after simplicity CONCERNS(3), robustness FATAL(5), correctness
FATAL(8): all three rejected inline-route rows and dialog edits of phases.
r2 → r3 after the confirmation (correctness CONCERNS(2)). Precedence r3 > r2 > r1. Contract `.claude/tasks/contracts/2026-10-11-model-router-w3b-dialog-1240.md`.
Base = W3-A (22455c0, plan r4, BDR-116). Facts: `agent.offer` payload carries
`description` and `source`; `$.ui.ask` takes 2-4 labels + Other; `cfg.models`
maps the four aliases to ids; each default alias heads one tier (best fable,
big opus, work sonnet, cheap haiku); `readCapped` exists (size cap, stat).
## Decisions
- D1 rows stay PHASE NAMES everywhere (routing.json, projects, override).
No inline route rows. The dialog's model+effort choice maps to a phase.
- D2 the dialog never edits a phase: T3 offers Later / Keep only, with
context. Phases change by hand in routing.json + DEFAULT_CONFIG (census
lock kept). BDR-116 unchanged.
- D3 one description rule for every kind, hardened read, no cache.
## Texts (ASK1)
- `clean(s, n)`: FIRST fold every `\s` and `\p{Z}` (incl. U+2028/2029) to
a space, THEN strip `\p{Cc}` and `\p{Cf}`, collapse spaces, trim; when
longer than n code points keep n-1 and append "…" (total ≤ n). Applied to
every description and `about` (no answer echo anywhere).
- Leads are KEPT as the first words (existing assertions stay valid):
"First route for /feat — <desc>. Routed to reflect (<about>): claude-fable-5-1
at high. OK?", "First dispatch of security-auditor — <desc>. Routed to
verify (<about>): claude-sonnet-5-5 at xhigh. OK?", "First use of
orchestrate on the main loop — <about>; used by …: … OK?". Without a
description the " — <desc>" part is omitted; without `about` the
parenthesis is omitted.
- Skill row desc = `skillDesc($, st, name)`: name must match
`^[A-Za-z0-9][A-Za-z0-9._:-]*$` (else name only); `readCapped` of
`${HOME}/.claude/skills/<name>/SKILL.md` (stat kind must be `file`, size
cap as the override; HOME from `$.env.get`, unset → name only); parse the
frontmatter `description:` in its five forms (plain, `|`, `>`/`>-`,
single-quoted with `''`, double-quoted): unquote, join the block lines
with spaces, cut at the first sentence end (`. ` / `.` at end), then
clean(…, 140); any failure → name only. Read at ask time, once (the key
is asked at most once per session). The kind check lives in a
skill-specific wrapper around `readCapped` (config reads untouched).
- Agent row: the offer description goes through the SAME rule (sentence
cut + clean 140) WHEN STORED at `agent.offer` (bounded cache
`Map<type, {source, description}>`); absent → name only.
- Phase (T3): "First use of orchestrate on the main loop — <about>; used by
<n> rows[, the derived route of dispatches][, <k> prompt rule(s)]:
claude-fable-5-1 at medium. OK?"; consumers derived: `orchestrate` →
"the derived route of dispatches" (hardcoded, pushOrchestrate), prompt
rules counted from `cfg.prompt.filter(r => r.phase === name)` (floor
rules named "floor rule"). Options ["Later", "Keep"].
- Row options: ["Later", "Keep", "Change"]. At EVERY step the `askOr` LATER
(dismissed, headless) is checked first and is silent; any other
non-label answer = Later + toast "answer not recognised, default kept"
(no echo). `ALTS`, the alt slice in `optionsFor` and the free-text phase
branch of `decide` are removed.
- Main-row texts keep the real-decision rule (decideFor/mainEffort); spawn
texts keep spawnTarget + explicit effort.
## Change flow (rows only)
- ASK2 "Which model for `<name>`?": four FIXED labels from the default
aliases, each with its tier role ("fable (best)", "opus (big)", "sonnet
(work)", "haiku (cheap)"; role = the tier headed by the alias in
`cfg.tiers`, "(custom)" when none); Other or an unknown label = Later +
toast.
- ASK3 "Which effort on <alias>?": the efforts of the phases (in
`cfg.phases`) whose tier head is the chosen alias, deduplicated and
sorted ASCENDING by LEVELS (fable: medium high xhigh max; opus: xhigh;
sonnet: low medium high xhigh; haiku: low) → at most 4 labels (the
first 4); exactly one → ASK3 skipped; zero → Later + toast "no phase
uses <alias>"; Other = Later.
- Target phase = the row's CURRENT phase when it matches (alias, effort),
else the first matching phase in `cfg.phases` order. Same phase as now →
treated as Keep (no `changed` entry).
- ASK4 scope as W3-A: ["Everywhere", "This project only"], or ["Everywhere",
"Later"] when there is no repo key.
- After a CHANGE on a MAIN row (T1, never on Keep), computed in
`confirmSkill` from `skillRow` + `decideFor`: toast "`/<name>` now runs
<id> at <effort>" plus ", main moves up only: `/route switch on` allows
a downgrade" when the chosen alias ranks below the session model and the
switch is off; the new route is re-picked into the current decision
(T1/T2 as W3-A).
- Storage unchanged from W3-A: `changed.<kind>.<name> = {from (first), to}`,
`confirmed.<kind>.<name> = to` (strings), `projects[key]` rows.
## `about`
- routing.json: `phases.<name>.about` (≤ 120 chars, clean) on the 11
phases; loaded into `mem.about: Record<phase, string>` (part of the
memory rebuilt with the config, so it survives /clear with cfg; never on
`Route`, never in DEFAULT_CONFIG; the census `code_phase` regex
untouched); a non-string or overlong `about` is dropped + logged once;
a phase without `about` → no parenthesis. acceptPhase ignores the key.
## Steps
- [ ] S1 routing.json `about` ×11 (one line each from the W2 phase table);
loader + map + validation.
- [ ] S2 descriptions: offer cache `{source, description}`; `skillDesc`
(allowlist, readCapped, kind check, YAML forms, sentence cut); `clean`.
- [ ] S3 ASK1 texts + options Later/Keep/Change; Other → Later + toast;
dead code removed (`ALTS`, alt slice, free-text branch).
- [ ] S4 change flow ASK2 → ASK3 (derived efforts) → ASK4; target-phase
rule; same-phase = Keep; main toast with the real decision.
- [ ] S5 `lib/effort-shift.md` dialog lines (≤ 4 changed).
- [ ] S6a migrate the W3-A dialog tests: every `asks([...])` answering a
phase name at ASK1 becomes Keep or a Change chain (ASK1 → ASK2 →
[ASK3] → ASK4; the `asked[n]` indexes shift accordingly); option-list
assertions become Later/Keep/Change (T3 Later/Keep); the lead-text
assertions (:2266 "First dispatch of feater", :2395 "First use of
orchestrate on the main loop") stay valid by construction; :1914
expects the new toast "answer not recognised, default kept"; the two
free-text tests (:1871 "a free-text phase counts as an alt", :1878
"equal to the current phase is a Keep") are DELETED (authorized:
they test the removed branch); "a phase name answer is a Later" kept.
World/helper extensions: `offerOf` takes a description; the World
gains a per-path stat `kinds` override and a HOME override (unset).
- [ ] S6b new tests, one per clause: skill text with each YAML form (plain,
`|`, `>-`, 'sq' with `''`, "dq"), missing SKILL.md → name only, a bad
name (`../x`) → name only, stat kind ≠ file → name only, a SKILL.md
over the cap → name only, HOME unset → name only, a 500-char
description with control, bidi and U+2028 chars → ≤ 140 clean code
points ending in "…"; a Keep on a main row → no "now runs" toast;
a Change → the toast with the switch hint; zero-effort alias → Later
+ toast; dismissed ASK2 → silent Later; agent text from the offer
description; T3 text with `about` and consumers; ASK2 labels; ASK3
derived efforts per alias (fable 4, opus → skipped, sonnet 4, haiku →
skipped); security-auditor Change → opus → (skipped) → Everywhere →
row `judge`, spawn on claude-opus-5-5 at xhigh now; feater Change →
sonnet → high → `write`; Explore Change → sonnet → medium → stays
`explore` (current phase wins) → treated as Keep, no `changed`;
`/feat` Change → sonnet → medium → `implement`, toast names "moves up
only" with the switch off; Other at ASK1/ASK2/ASK3 → Later + toast,
nothing written; project scope with the derived phase; `about`
overlong → dropped + logged.
- Disposition: honors BDR-116 rules (1), (3)-(6) unchanged (rows = phases,
dialog never edits phases, writers/paths/caps as is); AMENDS its clause
(2): options become Later/Keep/Change with a model-then-effort chain,
Other = Later (successor amendment written at CAPITALIZE, append-only); BDR-115 (full ids from `cfg.models`),
LRN-210 (one clause per test), LRN-212 (live mod: announce dialogs).
- Docs (README/USAGE/CHANGELOG lines on the dialog) drift after this wave:
STEP 6 doc-sync, out of this contract's scope.
## Challenge ledger (r1 → r2)
- simp 1/2, rob 2/3/4/7/11/12, corr 1/2/3/9/10/12 (inline rows) → D1:
rows stay phases; efforts derived from the alias's phases; target-phase
rule with tie-break; same-phase = Keep.
- simp 3, rob 1 BLOCKER/5/6/13, corr 4/5 (T3 phase edits) → D2: T3
Later/Keep with context; BDR-116 unchanged.
- simp 6, rob 8/9, corr 8 (descriptions) → D3 single rule, five YAML
forms, hardened read, allowlist, clean() with Cc/Cf and code points.
- simp 7, corr 14 (`about` on Route) → separate map, validated at load.
- simp 8, corr 13 (ASK2 Other, aliases) → fixed four labels, Other = Later.
- simp 9 (dead code) → S3 removal.
- rob 10, corr 6 (main model pick is a placebo) → toast with the real
decision and the switch hint.
- corr 7 (W3-A tests) → S6a migration step.
- corr 11 (no-key scope) → W3-A behaviour kept ([Everywhere, Later]).
- corr 15 (docs, AC5 oracle) → doc-sync noted; contract AC5 reworded.
## Confirmation ledger (r2 → r3)
- conf 1 (S6a under-listed) → leads kept by construction; toast text,
deleted free-text tests, `asked[n]` shifts listed.
- conf 2 (read-hardening clauses untested) → S6b tests + World extensions.
- conf 3 (ASK3 order) → ascending by LEVELS; tie-break in `cfg.phases` order.
- conf 4 (clean order, length) → fold first, strip, n-1 + "…".
- conf 5 (dismissed ASK2/3 toasted) → LATER checked first, silent.
- conf 6 (zero efforts, texts, echo) → Later + toast; ASK2/ASK3 texts; no echo.
- conf 7 (toast on Keep) → Change only, computed in confirmSkill.
- conf 8 (BDR-116 clause 2) → amendment at CAPITALIZE.
- conf 9 (`about` across /clear) → in `mem`, rebuilt with cfg.
- conf 10 (agents vs skills rule) → same rule at offer time, bounded.
- conf 11 (hardcoded consumers) → derived from cfg.prompt.
- conf 12 (kind check placement) → skill wrapper around readCapped.