From 5389ee53e7555ca173fbf277de465b220733727d Mon Sep 17 00:00:00 2001 From: bchanot Date: Sun, 11 Oct 2026 21:07:04 +0200 Subject: [PATCH] =?UTF-8?q?chore(memory):=20BDR-116=20amendment=20(classif?= =?UTF-8?q?ier)=20+=20EVAL-045=20+=20journal/TODO/contract=20w4b=20?= =?UTF-8?q?=E2=80=94=20feat=20model-router=20wave=204-B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/memory/decisions.md | 6 +- .claude/memory/evals.md | 7 + .claude/memory/journal.md | 1 + .claude/tasks/TODO.md | 5 +- ...-10-11-model-router-w4b-classifier-1858.md | 32 ++++ ...-10-11-model-router-w4b-classifier-1858.md | 172 ++++++++++++++++++ 6 files changed, 220 insertions(+), 3 deletions(-) create mode 100644 .claude/tasks/contracts/2026-10-11-model-router-w4b-classifier-1858.md create mode 100644 .claude/tasks/plans/2026-10-11-model-router-w4b-classifier-1858.md diff --git a/.claude/memory/decisions.md b/.claude/memory/decisions.md index 8d2da24..b62e75c 100644 --- a/.claude/memory/decisions.md +++ b/.claude/memory/decisions.md @@ -1414,4 +1414,8 @@ Branch feature/user-writing-web-rules, UNMERGED (human gate). - **Amendment (2026-10-11, wave 3-B, commit 604a6c4)**: clause (2) superseded: the dialog carries CONTEXT (skill description = first sentence of its SKILL.md frontmatter, five YAML forms, hardened read: name allowlist, stat kind file, size cap, `clean()` Cc/Cf + code points; agent description from `agent.offer`, bounded cache; phase `about` line, new 120-char field on the 11 routing.json phases, kept in memory not on Route) and the REAL next model id + effort. Options Later / Keep / Change (T3 Later / Keep). Change = model (4 fixed alias labels with tier role) → effort among those the phases headed by that alias offer (ascending, ≤ 4, one → skipped, zero → Later) → scope; the pair maps to an existing PHASE (rows stay phase names; row's current phase wins a tie; same phase = Keep); a pair no phase offers = a new phase by hand. Other anywhere = Later + toast (no echo). A main-row change toasts the real decision + `/route switch on` hint on a held downgrade. Three lenses rejected inline-route rows and dialog phase edits (2 BLOCKERs). First real use: user moved security-auditor to `judge` (opus/xhigh) through it; floor aligned (`model: opus`, 2026-10-11). Links [[EVAL-044]]. - **Amendment (2026-10-11, wave 3-C, commit cd8d72f)**: clause (6) extended: `/route forget ` (composer only) clears decisions in the tracked file through the same writer: a name in every table (`confirmed`, `changed` → row restored to its recorded `from` when it is a string, the row still equals `to` and `from` is a known phase; `projects[*]` pruned), asked again at once; `all`/`projects` confirm through the engine dialog (Cancel first, anything else = cancelled) and hold the single-dialog slot; nothing written when nothing to forget; the answer names a restored row's file + values to realign (a floor aligned to the decision makes the census FAIL after the restore). No per-user decisions file (user: "seulement le forget"): another user inherits the shared memory and may forget. Residuals (security LOW/MEDIUM): raw error text in the answer, file fragments unsanitized in the answer, `floorFile` name not allowlisted, `__proto__` plan/apply mismatch, `changed.phases` restorable. Links [[EVAL-044]]. - **Amendment (2026-10-11, wave 4-A, commit 68d7ea6)**: logging rule: `verbose` (boolean) logs TRANSITIONS only, built from the decision itself inside `safely` before `next`: main "fable/high → fable/xhigh (: [, ])" once per turn per transition (causes: skill /, route tool, prompt rule, prompt floor, dispatch, sticky /route, session; model: upgrade, switch, fallback: down); one hand-off line per agent dispatch deduped per turn per (type, target) "main … → agent / (row |explicit|fallback|frontmatter, row|explicit|engine)"; later agent effort moves; an answered-by anomaly (agents only, canonical ids, once per agent+model). The host prefixes the plugin name: new lines carry none. The four stream lines are gone. Residuals W3-A/B/C closed: failure texts to the log, file fragments cleaned at render, floorFile allowlisted, `__proto__` refused, changed scan = rows, dialog Change offers shipped phases only (`mem.filePhases` = validated seed layer), post-answer write failure keeps the key asked. Rejected: a `trace` level (unrequested surface); detecting agent moves at turn.step (decided at spawn). Links [[EVAL-044]]. - +- **Amendment (2026-10-11, wave 4-B, commit e54a2b5)**: prompt classifier closes the main-loop gap (plain prompt, no skill, no agent, no route in force). Composer `prompt.submit`, trimmed text ≥ 12 chars not starting with `/`, no prompt rule matched, no sticky/turn/run/floor route → one `$.model.classify(sample, [design, analysis, coding, bookkeeping, chat])`, sample = whole text ≤ 2000 code points else head 500 + tail 1500, no `model` option (engine small model). Label → phase (design→plan, analysis→reflect, coding→implement, bookkeeping→apply, chat/unknown → nothing) written as the TURN route, EFFORT ONLY (no tier: main model never moves on a guess), may lower, replaced by any later route tool call or typed skill, ignored at a step whose engine effort ≠ session baseline (`engineBase`: a user `/effort` beats the guess). Bounded: race with `$.clock.sleep(3000)`; timeout/rejection count, user abort neither counts nor writes, every guard re-checked after the await (a route landing meanwhile wins); 3 failures in a row → pause `cooldownMinutes` (×60000, kept across `/clear`, cleared by `/route reload`, one toast + status marker ` · classifier: paused`); skipped while the haiku id is marked down or main runs on haiku. No dialog, no routing.json write on a classified route. Config key `classify` (default true, `pickSwitch`), `/route classify on|off` session toggle surviving a rebuild, `/route show` `classify:` field, status `route: classifier `, verbose line `(: classifier)`. +- **Why**: user 2026-10-11: routing must also cover the main conversation when no skill or agent is invoked. Effort-only because a model flip on a 3-s guess would fight the breaker and the user's `/model`; turn-scoped because a guess must never outlive its prompt. +- **Alternatives rejected**: first-use dialog on classified phases (the TODO line said so; a dialog per plain prompt is noise and the phase is already confirmed through T3); writing the guess into routing.json; classifying mid-turn or sub-agent prompts; unbounded await (robustness BLOCKER r1: the classify call sat on the critical path with no clock race); `classify` as a tracked-file writer (session toggle + config key instead). +- **Gates**: plan r1 → r3 (simplicity CONCERNS(4), robustness FATAL(6) BLOCKER unbounded await, correctness CONCERNS(4); confirmation robustness CONCERNS(4)); feater DONE (423) + 1 round (trim, argumentHint, repeated undefined: 428, each mutation-proven); GATE 0 MET; verifier ECARTS(2) coverage → CONFORME on #2; security PASS (0 findings; LOW accepted: pasted text can steer one turn's effort up, README says so). Abort paths judged by reading (`[gated]` clarification). Residual: `smallModelBusy` checks `cfg.models.haiku` while classify uses the engine's default small model (mismatch only when the two differ). Live AC9 open. +- **Refs**: contract `.claude/tasks/contracts/2026-10-11-model-router-w4b-classifier-1858.md`, plan r3 same slug. Links [[EVAL-045]], [[LRN-210]]. diff --git a/.claude/memory/evals.md b/.claude/memory/evals.md index 1d0222e..18f7b5e 100644 --- a/.claude/memory/evals.md +++ b/.claude/memory/evals.md @@ -65,6 +65,7 @@ rules: | EVAL-042 | 2026-10-10 | model-router W2 plan r1 → r4: 3 lenses (1 BLOCKER), 2 confirmations (1 BLOCKER then 0); W2-A verifier 3× ECARTS on coverage clauses only, W2-B ECARTS(7) → CONFORME; gate A→B read from engine records | second confirmation paid again (BLOCKER on my own r2 slot); compound coverage criterion = endless ECARTS; engine jsonl replaces the live log | | EVAL-043 | 2026-10-11 | model-router W3-A: 3 lenses + 1 confirmation (census BLOCKER, project-tree layer dropped), feater DONE + 4 rounds, verifier 3× then re-verify after a REAL security BLOCK (credential fragment in the tracked key) | challenge + security gates both earned their cost; coverage-shaped criteria still cost 3 verifier rounds; live mod side effects misread as a test leak | | EVAL-044 | 2026-10-11 | model-router W3-B (dialog with context): 3 lenses converged on 2 BLOCKERs of my r1 (inline rows, phase edits) + 1 confirmation; feater DONE + 2 coverage rounds; verifier 3× to CONFORME; security PASS; user's first real decision through the new dialog | the lenses pay most when the plan adds a data shape; coverage criteria still cost rounds; live dialogs during the gates must be announced and their data reconciled before commit | +| EVAL-045 | 2026-10-11 | model-router W4-B (prompt classifier): 3 lenses (BLOCKER: unbounded classify await) + 1 confirmation; feater DONE + 1 round; verifier ECARTS(2) → CONFORME; security PASS 0 findings; LRN-210 applied (one coverage clause per criterion) | first wave since W2 with no verifier cap: per-clause criteria + abort paths declared unverifiable up front cut the rounds to one | --- @@ -411,3 +412,9 @@ Dogfood: 3 blind lenses attacked the v1 plan for the plan-challenge feature itse - **Anomaly**: my r1 proposed a second row shape to satisfy "model then effort" literally; the derived-effort list satisfies it with zero new data shapes. During the gates the user answered the NEW dialog live: security-auditor → opus/xhigh (the request's example), plan-challenger → verify (reset on user go); my reset of `changed` wiped a legitimate entry and left the committed file census-red for one commit. - **Action**: when a request says "choose X then Y", look for the existing shape that already encodes (X, Y) before adding one; reconcile routing.json decisions one by one (never `changed = {}`), and run the census before every data commit. Links [[EVAL-043]], [[BDR-116]], [[LRN-212]]. +## EVAL-045 — model-router W4-B: per-clause criteria and a declared-unverifiable path ended the verifier loop in one round +- **Date**: 2026-10-11 +- **Output checked**: plan `.claude/tasks/plans/2026-10-11-model-router-w4b-classifier-1858.md` r1 → r3; diff e54a2b5 (register.ts +211, 428 kit tests, 4 docs). +- **Method**: simplicity CONCERNS(4), robustness FATAL(6) (BLOCKER: the classify await sat unbounded on the prompt's critical path → `$.clock.sleep` race, abort not counted, post-await re-check), correctness CONCERNS(4) → r2; confirmation robustness CONCERNS(4) → r3. Feater DONE (423, 5 mutation proofs); GATE 0 MET; verifier #1 ECARTS(2) (no test on `trim`, none on the argumentHint) → feater (+5 tests, each failing under its mutation; first trim tests passed without the trim because the length guard caught them, lengthened) → verifier #2 CONFORME 8/8, criterion 9 gated. Security PASS, 0 findings, 1 LOW accepted. +- **Anomaly**: none on the loop. Verifier observations worth keeping: the sub-agent test proves only spawn+step never calls classify (`prompt.submit` has no agent form in the API types); the 3-s tests pin the upper bound only; the post-await re-check is tested through the route-landing case alone. Security note: `smallModelBusy` guards `cfg.models.haiku`, classify uses the engine small model. +- **Action**: keep the W4-B contract shape as the template (one clause per criterion, kit-untestable paths listed in CLARIFICATIONS before the first verifier); the live AC9 check is the real oracle for label quality (drop `bookkeeping` if it misfires). Links [[EVAL-044]], [[BDR-116]], [[LRN-210]], [[LRN-212]]. diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index b75eed8..96c57aa 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -600,3 +600,4 @@ rules: - model-router W3-B (dialog with context) landed 604a6c4 + data 32b71d2 + floor chore. User asked after W3-A: explain what is decided, OK/change/later, model then effort then scope. Plan r1→r3: all three lenses killed my inline-row format and the T3 phase edit (2 BLOCKERs avoided); efforts derived from the phases of the chosen model. Feater DONE first pass (229 tests), verifier ECARTS(2)→(1)→CONFORME, security PASS. GATE 0 heredoc CHECK not runnable again → script. Live during the gates: user answered the NEW dialog for security-auditor → opus/xhigh (the request's example; kept, floor aligned to opus, model-routing lock updated), plan-challenger → verify (reset on user go), orchestrate Keep (T3 check done). My `changed = {}` reset wiped a legit entry → census red for one commit → restored. Docs P1-P6 approved (patch in flight), BDR-116 amendment + EVAL-044 written. Next: doc commit, memory commit, user merge of feature/model-router-confirm (W3-A + W3-B) + publish by hand; T1 live check still open. - model-router W3-C `/route forget` landed cd8d72f (+ data commit: escalate Keep). User asked whether decisions can be cleared and what another user can do; chose the command only, no per-user file. Plan r1→r3: three lenses CONCERNS (false `st.mem` premise, kind order, docs scope, census red after restoring an aligned floor, no confirmation on `all`), confirmation CONCERNS(3). Feater DONE; verifier ECARTS(2) with a REAL typo (`ForgetPlan`), then coverage only ×2 → cap → user accepted; security PASS (1 MEDIUM + 4 LOW parked). My own `route(escalate)` call opened the T3 dialog the user Kept. Branch feature/model-router-confirm now carries W3-A/B/C (14 commits), unmerged, unpushed; merge on signal; full make test running. - model-router W4-A landed 68d7ea6 on feature/model-router-w4 (+ data commit). User: fewer logs (one line per real change with a cause), residuals, then the classifier. Plan r1→r3: three lenses (BLOCKER: an agent's model move is decided at spawn; `trace` level rejected as unrequested surface; reuse lastPlan/Loop; cause from the decision), confirmation CONCERNS(5) (hand-off line per dispatch deduped per type+target). Feater DONE (341 tests) + 4 rounds; verifier ECARTS(3) real (dedupe on text, double host prefix) then coverage ×2 → cap → user accepted; security PASS + 2 MEDIUM (raw skill names) fixed. Live: the terminal logs changed as soon as the feater saved; user Kept plan-challenger (judge) and the judge phase. Next: W4-B classifier; push-guard hotfix queued. +- model-router W4-B prompt classifier landed e54a2b5 on feature/model-router-w4. Plain composer prompt with no route in force → `$.model.classify` (5 labels, head 500 + tail 1500) → turn effort only, 3-s clock race, 3 failures pause, `/route classify on|off`. Plan r1→r3 (robustness BLOCKER: unbounded await), feater DONE + 1 round (trim, argumentHint), verifier ECARTS(2) → CONFORME, security PASS 0 findings. No live routing.json change this wave. Branch carries W4-A + W4-B (5 commits), unmerged, unpushed: merge on signal, user publishes. Live AC9 (plain prompts labelled) open. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index cbd2c8a..e371d3f 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -34,13 +34,14 @@ Plan `.claude/tasks/plans/2026-10-10-model-router-w3a-confirm-1201.md` r4, contr - [x] W4-A transition logs + residuals (2026-10-11, 68d7ea6, contract `2026-10-11-model-router-w4a-logs-residuals-1637`, plan r3): verbose = transitions only (main line, hand-off per dispatch, agent moves, answered-by anomaly), stream lines removed, host prefix respected; residuals of W3-A/B/C closed (a)-(g). 3 lenses (BLOCKER: agent moves decided at spawn) + 1 confirmation, feater + 4 rounds, verifier at the cap on coverage (user accepted), security PASS + 2 MEDIUM fixed. Kit 284 → 358. - [ ] W4-A residuals (LOW, accepted): `readLayer` logs `String(err)` of a JSON parse error (a file fragment can appear in the local log); `logCall`/`noteMain` ui calls on the step path sit outside `safely` (fail-open on routing, no security impact). - [ ] push-guard hotfix: the PreToolUse hook denies a read-only command whose TEXT contains the push verb next to `git` (grep pattern, heredoc); match the command's own argv only. -- [ ] W4-B classifier (next): haiku `$.model.classify` on typed prompts → turn default route (plan/reflect/implement/apply/chat), first-use confirmation per phase, `/route classify on|off`, transition cause `classifier`. +- [x] W4-B classifier (2026-10-11, e54a2b5, contract `2026-10-11-model-router-w4b-classifier-1858`, plan r3): `$.model.classify` on plain composer prompts with no route in force → turn effort only (design→plan, analysis→reflect, coding→implement, bookkeeping→apply, chat→nothing), 3-s race, abort not counted, 3 failures pause `cooldownMinutes`, no dialog, no file write, `classify` config key, `/route classify on|off`, status + verbose line. 3 lenses (BLOCKER: unbounded await) + 1 confirmation, feater + 1 round, verifier CONFORME (2nd), security PASS. Kit 358 → 428. Deviation from the TODO line: no first-use confirmation on classified phases (noise per prompt; T3 already confirms the phase). +- [ ] W4-B residuals: `smallModelBusy` checks `cfg.models.haiku` but classify uses the engine's default small model (guard misses when they differ); abort paths judged by reading only; the 3-s tests pin the upper bound only. Live AC9 open: type a few plain prompts, read `route: classifier ` (verbose line with `/route verbose on`), drop `bookkeeping` if it misfires. - [ ] W3-A live checks still open: T1 (typed `/feat` etc.) dialog (T3 seen live 2026-10-11: orchestrate Keep); a parallel same-agent dispatch answered after > 10 s; what an unanswered dialog resolves to on the terminal (must be Later or nothing written). Record in the contract as `[gated]` when seen. - [ ] W3-A residuals (security LOW, accepted): non-atomic cross-process write (two sessions, crash mid-write → file reads as failed, previous config kept); `host/path` of a private remote in the tracked file; a persistent write failure re-asks every use (`asked.delete` in the catch); `out.length` vs byte size at the cap; `changePatch` scope 'project' with a vanished key writes Everywhere (cwd change mid-dialog). Deferred: `/route set`, `/route confirm`, `(unconfirmed)` in show, dialog edits of phases, frontmatter auto-alignment, session-start warning when routing.json is dirty, per-machine `projects`. - [ ] routing.json ships `confirmed` for verifier/feater (user Keeps) and phases verify/implement: every clone inherits them (by design: decisions travel). Revisit at release if unwanted. - [ ] W2 residuals: probe 4 unobserved (above); the push-guard hook denies a read-only grep (or a heredoc) whose TEXT contains the push verb next to `git` (security-auditor + orchestrator 2026-10-10, false positives, reworded); ~27 loose "sonnet pin"/"opus pin" shorthand sites kept (true by census); README config key list omits `tiers`/`fallback`/`cooldownMinutes`/`mainUpgrade`/`upgradeMaxTokens` (doc audit item 6, pre-existing); MIGRATION.md "Upgrading to 3.0.0" at release time; `skillCalls`/`spawning` counters global; offers keyed by name. -- [ ] W3 optional: step heuristics, haiku classifier, quota-aware downgrade, A/B +- [ ] W3 optional: step heuristics, quota-aware downgrade, A/B (haiku classifier: done W4-B) ## 2026-09-30 — Higgsfield pack: CLI + skills in the install process, off by default (feature/higgsfield-pack) Contract `.claude/tasks/contracts/2026-09-30-higgsfield-pack-1412.md`, spec + plan under diff --git a/.claude/tasks/contracts/2026-10-11-model-router-w4b-classifier-1858.md b/.claude/tasks/contracts/2026-10-11-model-router-w4b-classifier-1858.md new file mode 100644 index 0000000..d415cb6 --- /dev/null +++ b/.claude/tasks/contracts/2026-10-11-model-router-w4b-classifier-1858.md @@ -0,0 +1,32 @@ +# CONTRACT — model-router-w4b-classifier +- date: 2026-10-11 | flow: feat | branch: feature/model-router-w4 +- status: active + +## REQUEST (verbatim — IMMUTABLE) +okm j'aimerais que tu corrige donc les residus techniques, et aussi que tu fasse cette wave optionnelle du classifieur. [earlier, same day: "est-ce quil y a une comande pour clear…"; "tu me dit le routing de tout skills et agents, mais c'est valable aussi pour la conversatio principale ? Si tu invoaue ni skill ni agebt il faut que ca route aussi on est d'accord ?" → orchestrator: the plain conversation routes only through prompt keywords, ultrathink, derived orchestrate and the route tool; the missing piece is the planned haiku classifier on each typed prompt → user chose it] + +## CLARIFICATIONS +(pass A: none — the W1 plan § W3.2 and the 2026-10-11 exchange define outcome and scope) +Q: public shape (after three lenses, orchestrator default, user may veto): labels `design | analysis | coding | bookkeeping | chat` mapped to plan / reflect / implement / apply / none; the classified phase becomes the TURN DEFAULT route carrying its EFFORT ONLY (never a model move), and only when NO route is in force (no typed skill, no run slot, no route tool, no sticky `/route`, no floor, no keyword rule, not mid-turn, composer only); no first-use dialog on it (visibility: status line `route: classifier ` and the transition line with `/route verbose on`); `classify` is a machine config key (default `true`, `"classify": false` in `~/.claude/model-router.json` turns it off) with a session toggle `/route classify on|off`; the call is bounded to 3 s and pauses for `cooldownMinutes` after 3 failures; one small-model call per typed prompt of ≥ 12 characters [stated 2026-10-11] + +Q: the abort paths (`next.signal.aborted` first in the catch and after the await; classify's own `aborted` cause not counted) cannot be driven by the kit (no abortable dispatch, a throwing bottom hook becomes "no implementation") / A: judged by reading; the live check covers the rest [gated 2026-10-11] + +## ACCEPTANCE CRITERIA +1. On a composer `prompt.submit` with `classify` on, not mid-turn, a trimmed text ≥ 12 chars not starting with `/`, no prompt rule matched and NO route in force on main (no sticky, turn, run or floor), the mod calls `$.model.classify(, [design, analysis, coding, bookkeeping, chat])` once, with no `model` option, and writes the turn route `{ effort }` of the mapped phase with source `classifier` (`chat` or an unknown label → nothing); it never runs for a sub-agent prompt, a non-composer origin (so never under `-p`), mid-turn, with the mod off, with `classify` off, when a rule matched, or when a route is in force. One kit test per clause (a bottom `model.classify` mock). +2. The classified route is turn-scoped (gone at `turn.complete`), carries the effort only (the main model never moves because of it: no tier), may lower the effort (coding → medium, bookkeeping → low), is replaced by any later route tool call or typed skill in the turn, and is ignored at a step whose engine effort differs from the session's baseline (a user `/effort` beats it). One kit test per clause. +3. Bounded and fail-safe: the call races a 3 s `$.clock.sleep` (timeout → no route, counts as a failure); a user interrupt (`next.signal.aborted`) neither counts nor writes; every guard is re-checked after the await (a route landing meanwhile wins); a rejection or timeout counts, three in a row pause the classifier for `cooldownMinutes` (×60000, kept across `/clear`, cleared by `/route reload`) with one toast and a status marker, a settled answer resets the counter; an unknown label neither warns nor counts; the classifier is skipped while the haiku id is marked down or the main loop runs on haiku. The prompt always proceeds. One kit test per clause. +4. No dialog and no file write: a classified route opens no first-use dialog and never writes routing.json. One kit test (no `ui.ask`, no `fs.write` on a classified prompt). +5. Config and command: `classify` is a Config key (default `true`) read from the machine override through `pickSwitch` (invalid → default + one log); `/route classify on|off` is a session toggle (composer only) that survives a rebuild like `switch`/`verbose`; `/route show` prints `classify: on|off` on the `router:` line; the argumentHint lists it. One kit test per clause. +6. Visibility: the status line reads `route: classifier ` and the main transition line "(: classifier)" (with verbose on). One kit test per clause. +7. Kit suite, mods suite, validate and census green. + CHECK: cd mods/model-router && out="$(claude plugin test . 2>&1)" && printf '%s\n' "$out" | grep -qE '[0-9]+ pass' && ! printf '%s\n' "$out" | grep -qE '[1-9][0-9]* fail' && claude plugin validate . 2>&1 | grep -q 'passed' && cd ../.. && make test suite=lib/tests/mods.test.sh 2>&1 | grep -q 'all suites green' && bash lib/tests/effort-routing.test.sh >/dev/null 2>&1 && echo W4B-GREEN + EXPECT: W4B-GREEN + EVIDENCE: MET exit=0 marker-found :: W4B-GREEN +8. Docs: README (`classifier` paragraph in the model-router section: what, cost, `/route classify`), USAGE (`/route` row + one sentence in "Niveau d'effort"), CHANGELOG Unreleased Added, `lib/effort-shift.md` (≤ 2 lines). + CHECK: grep -qi 'classif' README.md && grep -qi 'classif' USAGE.md && grep -qi 'classif' CHANGELOG.md && grep -qi 'classif' lib/effort-shift.md && echo W4B-DOCS + EXPECT: W4B-DOCS + EVIDENCE: MET exit=0 marker-found :: W4B-DOCS +9. Live check (orchestrator + user, after the gates, `[gated]`): a few plain prompts typed in the terminal, each with the label it got (status line `route: classifier `, transition line with verbose on); drop `bookkeeping` if it misfires. + +## FILE SCOPE +mods/model-router/hooks/register.ts, mods/model-router/hooks/register.test.ts, README.md, USAGE.md, CHANGELOG.md, lib/effort-shift.md, lib/tests/effort-routing.test.sh (only if a lock needs it) diff --git a/.claude/tasks/plans/2026-10-11-model-router-w4b-classifier-1858.md b/.claude/tasks/plans/2026-10-11-model-router-w4b-classifier-1858.md new file mode 100644 index 0000000..8b23072 --- /dev/null +++ b/.claude/tasks/plans/2026-10-11-model-router-w4b-classifier-1858.md @@ -0,0 +1,172 @@ +# PLAN r3 — model-router wave 4-B: prompt classifier (2026-10-11) + +r1 → r2 after simplicity CONCERNS(4), robustness FATAL(6) (BLOCKER: the +classify await is unbounded on the critical path), correctness CONCERNS(4). +r2 → r3 after the confirmation (robustness CONCERNS(4)). Precedence +r3 > r2 > r1. Contract `.claude/tasks/contracts/2026-10-11-model-router- +w4b-classifier-1858.md`. Base: feature/model-router-w4 at 86845fc. +Corrected facts: default keyword rules never run mid-turn and only write +plan/reflect (they never lower); `mainEffort` ranks sticky > turn > run > +floor > engine and `mainModel` ranks turn above floor; `runMain` holds a +typed best-tier skill's route across the run's turns; `$` calls do not +count against the hook budget (an unbounded `classify` await freezes the +prompt), while `$.clock.sleep` does; `ClassifyOptions` has `model` only; +`claude -p` prompts arrive with origin `sdk`; the breaker is fed by +`classic.StopFailure` only; `warnOnce` fires once per session; config +switches (`enabled`, `verbose`, `mainModelSwitch`) are Config keys read +through `pickSwitch` with session toggles; `confirmPhase` (T3) is +confirm-only, so a dialog on a classified phase would decide nothing. + +## Decisions +- D1 no first-use dialog on classified phases (Later and Keep would apply + the same route). Visibility = the status line (`route: classifier + `) and the W4-A transition line (needs `/route verbose on`). +- D2 `classify` is a Config key (default `true`), read from the machine + override through `pickSwitch` (invalid → default + one log), with a + session toggle `/route classify on|off` next to `switch`/`verbose`, + printed on the `router:` line of `/route show`. Persistent off = + `"classify": false` in `~/.claude/model-router.json`. routing.json is + NOT touched; no new writer. +- D3 the classifier fills only an EMPTY route: it runs only when no route + is in force on main (`mainRoute(st) === null && st.turnFloor === null`: + no sticky `/route`, no turn route from a rule/skill/tool, no run slot + from a typed skill, no floor), evaluated after `routeFromPrompt`. Real + coverage, stated in the docs: a typed best-tier skill (feat, bugfix, + ship-feature…) holds its run slot across turns, so nothing is + classified until `/route clear`, `/route off`, a user `/model` or a + typed non-best skill; a non-best skill's row is turn-scoped, so its + follow-up prompts ARE classified (accepted: those rows are low/medium + and a complex follow-up may rightly rise). Never mid-turn (`e.turnId !== + undefined` → skip, as the rules). Never for a non-composer origin (so + never under `-p`). Never for a sub-agent prompt (prompt.submit is main + only; a test proves an agent's prompt is not classified). +- D4 a classified route carries the EFFORT ONLY: `turnMain = { phase, + route: { effort: cfg.phases[phase].effort }, source: 'classifier' }`. + No tier, so the main model never moves because of a guess (no cold + context re-reads on opus/sonnet sessions, no cost surprise). It may + lower the effort (coding = medium, bookkeeping = low); turn-scoped. + An explicit user `/effort` beats it: `st.engineBase` = the engine + effort of the first main step of the session (kept across /clear); + at `turn.step`, when `e.effort !== st.engineBase` (the user moved the + builtin effort) a `classifier`-sourced turn route is ignored for that + step (the engine effort applies) — "un choix explicite bat la phase + déduite" (BDR-115 amendment b). +- D5 bounded call: `label = await Promise.race([classify, $.clock.sleep( + CLASSIFY_MS = 3000, { signal: next.signal }).then(() => TIMEOUT)])` + inside try/catch: in the catch, `next.signal.aborted` is tested FIRST + (a user interrupt: no count, no write, return); a TIMEOUT counts as a + failure (D7) and gives no route; after the await every guard of D8 is + RE-CHECKED (`st.off`, the switch, no route in force, not paused) before + the write, and `next.signal.aborted` skips it (a late result never + lands in a later turn). The losing classify promise is left to settle + (the kit offers no cancel); stated. +- D6 labels are descriptive and mapped: `design` → plan, `analysis` → + reflect, `coding` → implement, `bookkeeping` → apply, `chat` → none + (lookup with `hasKey`; a mapped phase without an effort → no route). + The call passes no `model` option (the engine's small fast model; no + alias from a hook, LRN-203). The sample = the whole trimmed text when + ≤ `CLASSIFY_MAX = 2000` code points, else the first `CLASSIFY_HEAD = + 500` + last `CLASSIFY_TAIL = 1500` code points (cut by code points, + never a lone surrogate). A pasted tail can steer the label for one + turn (effort only, no file write): accepted and documented. +- D7 failures: a rejected call (classify's own `aborted` cause excepted) + or a TIMEOUT increments `st.classifyFails`; a settled answer resets it; + at `CLASSIFY_FAILS = 3` the classifier pauses until `clock.now() + + cooldownMinutes * 60000` (`st.classifyUntil`) with one toast + "classifier paused min (3 failures)" and the status marker + `classifier: paused`; both fields are in `resetSession`'s kept set and + cleared by `/route reload` (like the breaker); the expiry check runs + BEFORE the route-in-force guard (the marker never stays stale); an + unknown label neither warns nor counts; the classifier is skipped while + `cfg.models.haiku` (a stated proxy for the engine's small model) is in + `st.down`, and when the main loop itself runs on haiku (the effort is + stripped there anyway). +- D8 guards summary (in order): `st.off` → `cfg.classify` (+ toggle) → + composer → `e.turnId === undefined` → not a slash → trimmed length ≥ 12 + → no rule matched this prompt → no route in force → not paused, model + not down → classify. + +## Steps +- [ ] S1 config: `classify` key (DEFAULT true, pickSwitch), toggle, + `/route classify on|off`, show `router:` line, argumentHint. +- [ ] S2 `classifyPrompt($, st, e, next)` in the prompt.submit hook after + `routeFromPrompt` and the typed marker; `LABELS`/`LABEL_PHASE` + tables; the sample helper; the race; the failure counter and pause; + the effort-only route write with source `classifier`; `Source` + gains `'classifier'`; `slotWord` maps it to `classifier`; the + status line already prints `route: `. +- [ ] S3 docs: README (classifier paragraph: what, one small-model call + per typed prompt when no route is in force, effort only, `/route + classify off` or `"classify": false` per machine, where to see it: + status line, `/route verbose on`); the `/route` subcommand lists + (README ×2, USAGE) gain `classify on|off`; the machine keys lists + (README, USAGE) gain `classify`; USAGE "Niveau d'effort" one + sentence; CHANGELOG Unreleased Added; `lib/effort-shift.md` ≤ 2 + lines (the plain conversation routes by content now). +- [ ] S4 kit tests (a bottom `model.classify` mock; `$.clock` mock for + the race), one per clause: called once with the head+tail sample + and the five labels, no model option; each label → its phase effort + only (no tier on the route) and `chat`/`undefined` → none; never + resolving → no route after 3 s, prompt proceeds; rejection ×3 → + pause + toast + status marker, then no call during the cooldown, + a success resets the counter; `aborted` → no count, no warn; small + model down → not called; not called: short text, slash, sdk origin, + mid-turn (`turnId`), rule matched (keyword and ultrathink), sticky + `/route`, run slot held, floor carried, `classify` off (config, + toggle), mod off, an agent's prompt; turn-scoped (gone at + turn.complete); effort lowered to low for bookkeeping; the main + model unchanged on a sonnet session for `design`; `/route show` + prints `classify: on`; status line `route: classifier reflect`; + W4-A line "(reflect: classifier)"; invalid config value; 3 timeouts + → pause; a dispatch abort mid-race → no count, no write; a `/route + ` landing during the await → the classifier result dropped; + pause expiry at cooldown-1 ms (still paused) and +1 ms (resumes); + `/clear` keeps the pause, `/route reload` clears it; a user + `/effort` change (engine effort ≠ baseline) → the classified effort + ignored at the step; a 2000-code-point text sent whole, a longer + one head+tail with no lone surrogate; `constructor` label → none; + a phase without effort → none; main on haiku → not called; a + non-best skill's follow-up prompt classified; a plain prompt after + a finished best-tier run not classified until `/route clear`. +- Edge cases: a prompt that is only a URL or code (the classifier + decides; no special-casing); every classified prompt waits one small- + model round trip (typically under a second, 3 s at most: stated in the + README and measured in the AC9 live check); a `[1m]` session (no + effect: effort only). +- Disposition: honors BDR-115 (no alias from a hook, no model write, no + file write), BDR-116 (writers unchanged; no new dialog), LRN-203, + LRN-210, LRN-212 (live mod: the first plain prompt after the feater + saves is classified). +- Docs note for the user: a misclassification lowers one turn's effort + only; `/route ` or `ultrathink` overrides it at once. + +## Challenge ledger (r1 → r2) +- simp 1, corr 3/7, rob 6 (dialog decides nothing / misstates) → D1 dropped. +- simp 2, corr 11, rob 11 (flag plumbing, new writer) → D2 Config key. +- simp 3/4, corr 1/2/12, rob 3/4 (mid-turn, run slot, floor) → D3. +- rob 5 (model flips per turn) → D4 effort only. +- rob 1 BLOCKER, corr 9 (unbounded await, late write) → D5. +- simp 5, corr 4 (alias from a hook) → D6 no model option. +- corr 8, simp 8 (label words) → D6 descriptive labels mapped. +- rob 7 (head-only sample) → D6 head + tail. +- rob 2, corr 5 (failure handling, warn spent) → D7. +- corr 6, rob 9 (`-p`) → D3 never. +- corr 10 (sub-agent clause) → S4 test. +- simp 6, rob 10 (visibility) → status line + verbose note. +- simp 7 (doc lists) → S3 lists named. + +## Confirmation ledger (r2 → r3) +- conf 1 (timeouts uncounted) → timeouts count toward the pause. +- conf 2 (abort lands in the counted branch) → `next.signal.aborted` + tested first in the catch. +- conf 3 (classifier overrides a user `/effort`) → `engineBase` guard at + the step: an explicit builtin effort beats a classified route. +- conf 4 (run coverage) → D3 states the real coverage; docs name it. +- conf 5 (guards not re-checked after the await) → re-checked. +- conf 6 (pause bookkeeping) → ×60000, kept across /clear, cleared on + reload, expiry checked first, boundary tests. +- conf 7 (sample, labels) → whole text ≤ 2000 cp, code-point cuts, + hasKey, no-effort phase skipped. +- conf 8 (down check) → haiku id as a stated proxy; skip on a haiku main. +- conf 9 (pasted tail) → accepted, documented. +- conf 10 (latency) → stated in the README and measured live.