fix(model-router): close the accepted residuals of waves 1-4
Kill switch fails closed: a present-but-unreadable override at first load or a non-boolean enabled turns the router off (logged); /route reload says when the previous config was kept. Prompt rules with nested quantifiers are dropped; a config path that is not a regular file reads as failed; caught errors log kind + code only (errWord); rejected keys are masked in logs (keyWord), alias keys match KEY_NAME, prototype names are never keys; offers capped at 256; a models alias without a fallback list is ranked at the end of the default chain; canonical matches a table id at a '-' boundary only; logCall/noteMain cannot unroute a step; route-tool failure text no longer claims nothing was routed. Repo side: mods suite probes the 'claude plugin test' usage line, doctor bounds plugin list to 60 s, update-all skips @skills-dir mods. Kit 431/431, full make test green.
This commit is contained in:
@@ -7,14 +7,14 @@ migration of shifters/pins/model-gate in wave 2 after proof; names model-router
|
||||
- [x] W0 spike in dev-mods (hot reload): facts a-d established 2026-10-08 (plan file § Spike facts); e moved to W1.10
|
||||
- [x] W1-A the mod in `mods/model-router/` (b721c94, contract `2026-10-08-model-router-w1a-1533`, plan r3): challenge 3 lenses + 1 confirmation (2 BLOCKER + 10 MAJOR closed by named changes), feater DONE first pass, GATE 0 MET 5/5, verifier CONFORME 6/6, security PASS (4 MEDIUM + 5 LOW reported, below)
|
||||
- [x] W1-A hardening (user go "oui durcis", contract criteria 7-11): `/route` composer-only; no agent model axis (effort only after spawn); pattern ≤ 200 / scan ≤ 4096 / phase keys `^[a-z][a-z0-9_-]{0,31}$` / config ≤ 64 KB / `additionalProperties: false` / `typeof e.skill`; `warnOnce` in all 14 catches + config-drop logs; `safely` around post-`next` bookkeeping. feater DONE, gap round (dead `Loop.frozen`, spawn returns `started` verbatim, tool description), GATE 0 MET 9/9, verifier CONFORME 11/11, security PASS.
|
||||
- [ ] W1-A residuals (security, accepted, none exploitable from outside the user's own files): ReDoS is size-bounded only (`(a+)+$` in `~/.claude/model-router.json` + a 4 KB paste hangs the hook; fix = nested-quantifier rejection or a far lower scan cap); `stat().size` trusted (FIFO/device path in ~/.claude); unrestricted `models`/`agents`/`skills` KEYS echoed raw in logs (log flood); `agentId` read from the flat tool event (engine strip unverified); 5 closures without a kit test (no fs in the kit, LRN-206); "nothing routed" catch text after a state write.
|
||||
- [x] W1-A residuals CLOSED 2026-10-11 (bugfix/model-router-residuals): nested-quantifier patterns rejected (`NESTED_QUANT`); `stat.kind !== 'file'` → layer failed; rejected keys masked in logs (`keyWord`), alias keys `KEY_NAME`; route-tool catch text says the route may be partly applied. Left (accepted): `agentId` read from the flat tool event; closures without a kit test.
|
||||
- [x] W1-B1 floor precedence (contract `2026-10-08-model-router-floor-1835`, 2026-10-09): `ultrathink` + typed `/effort-<l>` = the main turn's default AND minimum (r2 after 3 lenses: a pure floor made /effort-low a no-op); one helper `mainEffort`; per-axis precedence; mid-turn prompt floors now + next turn; `"enabled": false` per machine, kept across /clear and failed reload; typed slash attested. 30 tests; verifier CONFORME 6/6 (after 1 gap round); security PASS ×2
|
||||
- [ ] W1-B1 residuals (security 2026-10-09, accepted): first-load failure of the override falls to defaults (`enabled: true`); non-boolean `enabled` drops to the default on reload; `skill.prompt` preload guard is a heuristic (`loops.size > 0`; a preload during spawn or a non-composer `/effort-*` while idle still writes the floor); marker not bound to a valid level; `/route reload` answers "config reloaded" even when the previous config was kept; transient missing override lifts a config-set off; `String(err)` of a JSON parse in the local log. Display: `/route show` folds the floor into the effort while off; model-axis text with a model-only sticky and the switch on.
|
||||
- [x] W1-B1 residuals CLOSED 2026-10-11: broken override at first load → router off (fail closed, logged); non-boolean `enabled` → off; `/route reload` answers `config unreadable; previous config kept`; `errWord` (name + code only) replaces every `String(err)` log. Superseded by W2-B: `skill.prompt` preload guard, `/effort-*` marker (bridge removed). Left (accepted): transient missing override lifts a config-set off (ENOENT must mean defaults on most machines); `/route show` display items (no defect named).
|
||||
- [x] W1-B2 wiring (contract `2026-10-08-model-router-wiring-1835`, 2026-10-09): tracked symlink `skills/model-router` → `../mods/model-router` loads as `model-router@skills-dir` (fresh-process `claude plugin list --json` proves it), `.gitignore` `mods/*/tsconfig.json`, `lib/tests/mods.test.sh` (4 checks, capability probe, bounded, SKIP), doctor `── Mods ──` fail-soft, CLAUDE.md `## mods/`. Plan r2 from 3 lenses (5 MAJOR), feater DONE, gap round (4b label + unbounded probe), verifier CONFORME 8/8, security PASS
|
||||
- [ ] W1-B2 residuals (accepted): `claude plugin <unknown> --help` returns 0 → the suite's capability probe can pass on a CLI without `plugin test` and then FAIL instead of SKIP (fail-closed; fix = grep the probe output for the test usage line); doctor `claude plugin list --json` unbounded; doctor `echo -e` helpers interpolate `$_mod` (tracked folder names only); fallback timeout guard orphans grandchildren (only without coreutils timeout); `update-all.sh` runs `claude plugin update` over `@skills-dir` → one recurring warn (needs an update-all edit)
|
||||
- [x] W1-B2 residuals CLOSED 2026-10-11: mods suite probe greps the `claude plugin test` usage line; doctor `plugin list --json` under `timeout 60`; update-all skips `@skills-dir` mods. Left (accepted): doctor `echo -e` helpers (tracked names only); fallback timeout guard orphans grandchildren without coreutils timeout.
|
||||
- [x] W1 close-out (2026-10-09): doc-sync opus audit SIGNIFICANT → user go all 10 → patched (README effort routing + Explore row + /route, USAGE, ARCHITECTURE mods/, CHANGELOG Unreleased) b22f894; hot-reload link removed from `~/.claude/dev-mods/<session>/`; BDR-115 amended (a6e2003). Pending user: `/reload-plugins` in this session (new sessions load the skills-dir copy by themselves); merge decision on feature/model-router-mod (`gitflow finish`, human signal)
|
||||
- [x] W1-C adaptive tiers (user 2026-10-09, contract `2026-10-09-model-router-tiers-1237`, plan r4 after 3 lenses + 2 confirmations: 6 BLOCKER + 29 MAJOR closed by named changes): phases name absolute tiers (best fable>opus>sonnet · big opus>fable>sonnet · work sonnet>opus · cheap haiku>sonnet); breaker fed by `classic.StopFailure` kinds rate_limit|overloaded|billing_error|model_not_found + `PostModelSwitch` auto (episode backoff 15→300 min, `/model` clears, `/clear` keeps, `/route reload` clears); fallback chain fable→opus→sonnet→haiku; main UPGRADE by default under `upgradeMaxTokens` 200k (fails closed on unknown usage), DOWNGRADE gated by `mainModelSwitch`; sticky `turnModel` per turn; derived `orchestrate` on background dispatches; prompt default rules (plan/reflect FR+EN, Unicode guards, skipped on `/…`, floor match, mid-turn). 58 tests; verifier CONFORME then 3 gap/hardening rounds; security PASS
|
||||
- [ ] W1-C accepted-by-design (security 2026-10-09, MEDIUM, not coded around): (1) the model itself can raise the main loop to fable for the rest of a turn through the `route` tool (plan/reflect/escalate/judge) or a background dispatch (derived orchestrate = best tier) — bounded by the turn and `upgradeMaxTokens`, no sticky route is model-callable; (2) `ultrathink` and the default keyword rules now mean "best tier" (fable) at the phase's effort, so an incidental keyword in a pasted composer prompt costs a fable turn (origin composer only). Residuals: `PostModelSwitch` `auto` covers "other programmatic change" (a healthy model could be marked 15 min); strikes never decay inside a session; a `[1m]` variant's `model_not_found` marks the base model until reload; a `models` alias added by the override without a `fallback` key stays unranked (`withEveryAlias` not applied to the default chain) so `leaveDown` skips the upgrade gates for it; `canonical` prefix match has no segment boundary (`claude-sonnet-5-50` would map to sonnet); classifier deferred to W2
|
||||
- [ ] W1-C accepted-by-design (security 2026-10-09, MEDIUM, not coded around; 2026-10-11 closed: unranked `models` alias → `withEveryAlias` on the default chain too, `canonical` segment boundary `known + '-'`): (1) the model itself can raise the main loop to fable for the rest of a turn through the `route` tool (plan/reflect/escalate/judge) or a background dispatch (derived orchestrate = best tier) — bounded by the turn and `upgradeMaxTokens`, no sticky route is model-callable; (2) `ultrathink` and the default keyword rules now mean "best tier" (fable) at the phase's effort, so an incidental keyword in a pasted composer prompt costs a fable turn (origin composer only). Residuals: `PostModelSwitch` `auto` covers "other programmatic change" (a healthy model could be marked 15 min); strikes never decay inside a session; a `[1m]` variant's `model_not_found` marks the base model until reload; a `models` alias added by the override without a `fallback` key stays unranked (`withEveryAlias` not applied to the default chain) so `leaveDown` skips the upgrade gates for it; `canonical` prefix match has no segment boundary (`claude-sonnet-5-50` would map to sonnet); classifier deferred to W2
|
||||
- [x] W1-C live verification part 1 (2026-10-09 after `/reload-plugins`, skills-dir copy loaded, 12 hooks): derived orchestrate on a real background Explore (main high → medium while it ran → high after its end); Explore on sonnet/medium; `route plan` → next request xhigh (engine record); `Skill(effort-low)` bridged → next request low (engine record); `/route show` resolves all 10 phases to full ids, `down: none`; steps arrive as bare `claude-fable-5-1` (no `[1m]`) so the suffix carry never fires on this session
|
||||
- [ ] W1-C live verification part 2 (needs a real incident): StopFailure vs turn.complete order; `PostModelSwitch` `auto` semantics and its `from_model` after a router upgrade; whether a router rewrite raises `auto`; `[1m]` carry validity on opus (only on a session whose steps carry it)
|
||||
- [x] W2-A mod (2026-10-09, bb56f3e on feature/model-router-w2, contract `2026-10-09-model-router-w2a-1546`, plan r4 `2026-10-09-model-router-w2-1546`): user decisions — effort-* skills deleted (W2-B), pins REWORKED not deleted (rows = phases by role; frontmatter `model:`/`effort:` kept as census-locked off-state floor after a robustness BLOCKER), orchestrators declare phases, slim model gate. Phases `write` work/high + `apply` work/low; 56 skill rows, 21 agent rows; agents' model at spawn within tier, upward only, project-defined agents skipped (agent.offer); typed slash → name-bound marker (composer|sdk|bridge) + pending slot + idle fallback; best-tier rows in a `runMain` slot surviving turn end; unrowed skill leaves the route; route answer always names the id; `null` override rows. 3 lenses + 2 confirmations (1 BLOCKER each round closed), feater + 4 gap rounds, GATE 0 MET, verifier 3× ECARTS on coverage clauses only → user accepted at the cap, security PASS (2 MEDIUM pre-existing: first-load kill switch fails open, ReDoS size-bounded). Kit 58 → 88 tests. Doc-sync skipped for A (mod-only, docs at B).
|
||||
@@ -22,25 +22,25 @@ migration of shifters/pins/model-gate in wave 2 after proof; names model-router
|
||||
- [x] W2-B repo migration (2026-10-10, 1f2d33b, contract `2026-10-10-model-router-w2b-1045`): 15 citers → `mcp__model-router__route` per phase, `effort=` on built-in judgment dispatches + SDD sonnet implementers (`effort="medium"`), `lib/effort-shift.md` + `lib/model-gate.md` rewritten (witness = route answer, `/route on`), deletions (effort-*, effort-pins.*, model-check.sh, tests, installer blocks), bridge removed from the mod, census rewritten (140 checks, drift lock rows ↔ frontmatter), analyzer xhigh, CLAUDE.global.md Design-work line. Verifier ECARTS(7) → CONFORME 10/10, security PASS, full make test green (design-tool-gate env red). Docs P1-P8 user-approved; SemVer: breaking → next release 3.0.0 (MIGRATION section at release).
|
||||
- [ ] (was) W2 gate A→B (user): `/reload-plugins`, then the live probe of plan § Gate (4 points: typed `/status` marker vs fallback in the verbose log; a real rowed spawn line + `step 0 agent` effort = spawn/first-step ordering; sonnet session typed `/feat` self-check + route answer; probe 1 again with a background agent alive). `typed-marker` never seen → W2-B blocked, A4 re-planned.
|
||||
- [x] (was) W2-B repo migration (plan § W2-B B0-B7 + STEP 6/7): bridge removal, `lib/effort-shift.md` rewrite, 15 citers → `route`, `effort=` on opus general-purpose dispatches, slim `lib/model-gate.md`, delete `model-check.sh` + `effort-pins.*` + their tests + install/update blocks, delete `skills/effort-*`, analyzer `effort: xhigh`, census rewrite (drift lock rows ↔ frontmatter), docs + registries (BDR-115 amendment, LRN typed-slash/run slot, EVAL)
|
||||
- [ ] W2-A residuals (security, accepted): first-load failure of the override activates the router despite `enabled:false` (fix = treat a failed first load as off); ReDoS on a self-authored prompt pattern (size-bounded); error text in the local log; model alias keys unvalidated (PHASE_KEY would do); `offers` map uncapped; `__proto__`/`constructor` override keys untested. Known limits: `skillCalls`/`spawning` counters are global; offers keyed by name only; builtin `/effort` is not a lever inside a run.
|
||||
- [x] W2-A residuals CLOSED 2026-10-11 (first load fail closed, nested quantifiers, `errWord`, alias keys, `offers` capped at 256, `__proto__`/`constructor` keys tested). Known limits (unchanged): `skillCalls`/`spawning` counters are global; offers keyed by name only; builtin `/effort` is not a lever inside a run.
|
||||
## 2026-10-10 — model-router wave 3-A: first-use route confirmation + decision memory (feature/model-router-confirm)
|
||||
Plan `.claude/tasks/plans/2026-10-10-model-router-w3a-confirm-1201.md` r4, contract `2026-10-10-model-router-w3a-confirm-1201`. User decisions 2026-10-10: tracked routing.json reached through the plugin dir (no new link); blocking `$.ui.ask` dialog at first use; per skill row, agent row, main phase.
|
||||
- [x] W3-A landed (22455c0, 2026-10-11): routing.json = phases + rows + decisions; T1/T2/T3 dialogs; project exceptions `projects[<normalized remote>]` in the tracked file (project-tree layer dropped: security); writers = dialog + `/route ask`; `/route pending`; census from the file with the `changed` WARN exemption; kit 86 → 190. Gates: 3 lenses + 1 confirmation (BLOCKER census), feater + 4 rounds, verifier CONFORME + re-verify, security BLOCK(1) credential leak fixed → PASS, full make test green (design-tool-gate env red). Live: T2 dialogs answered by the user (verifier Keep, feater Keep, security-auditor → implement then reset to verify on user go).
|
||||
- [x] W3-B dialog with context (2026-10-11, 604a6c4, contract `2026-10-11-model-router-w3b-dialog-1240`, plan r3): descriptions (skill frontmatter / agent.offer / phase `about`), real id + effort in the question, Later / Keep / Change, model → effort (derived from the phases of that model) → scope, rows stay phases, T3 Later/Keep. 3 lenses (2 BLOCKERs on my r1 avoided) + 1 confirmation, feater + 2 rounds, verifier CONFORME (3rd), security PASS (4 LOW: symlinked SKILL.md read, TOCTOU, raw key in a log, any plugin hooking AskUserQuestion could answer). Live decisions: security-auditor → judge (kept, floor aligned `model: opus`), plan-challenger → verify (reset on user go), orchestrate Keep (T3 live check done).
|
||||
- [ ] W3-B residuals: (machine-only phase into the tracked file: CLOSED in W4-A); SKILL.md symlink follow (LOW); `descs` cap and "(custom)" label untested; the security-auditor's `changed.from = verify` entry stays in routing.json although the floor now equals the row (harmless; prune at release).
|
||||
- [ ] W3-B residuals: SKILL.md symlink follow (LOW, kept: `skills/` itself is a symlink); `descs` cap and "(custom)" label untested. (security-auditor `changed` entry purged 2026-10-11.)
|
||||
- [x] "auto mode: use Bash/sed" block attributed to the mod: traced 2026-10-11 by reading, it is the harness's own auto-mode instruction rendered after the mod's MCP block; not in the mod source. Closed.
|
||||
- [x] W3-C `/route forget` (2026-10-11, cd8d72f, contract `2026-10-11-model-router-w3c-forget-1457`, plan r3): name/all/projects, guarded restore, confirmation dialog for all/projects, no-write paths, docs. 3 lenses + 1 confirmation, feater + 3 rounds (1 real defect: `ForgetPlan` typo), verifier at the cap on coverage (user accepted), security PASS. Kit 232 → 284. Live: orchestrate + escalate T3 Keeps seen.
|
||||
- [x] W3-C residuals closed in W4-A (68d7ea6). Live smoke of `/route forget` from the terminal still unseen.
|
||||
- [x] W4-A transition logs + residuals (2026-10-11, 68d7ea6, contract `2026-10-11-model-router-w4a-logs-residuals-1637`, plan r3): verbose = transitions only (main line, hand-off per dispatch, agent moves, answered-by anomaly), stream lines removed, host prefix respected; residuals of W3-A/B/C closed (a)-(g). 3 lenses (BLOCKER: agent moves decided at spawn) + 1 confirmation, feater + 4 rounds, verifier at the cap on coverage (user accepted), security PASS + 2 MEDIUM fixed. Kit 284 → 358.
|
||||
- [ ] W4-A residuals (LOW, accepted): `readLayer` logs `String(err)` of a JSON parse error (a file fragment can appear in the local log); `logCall`/`noteMain` ui calls on the step path sit outside `safely` (fail-open on routing, no security impact).
|
||||
- [x] W4-A residuals CLOSED 2026-10-11: `errWord`; `logCall` → `tryLog`, `noteMain` refresh under `safely`.
|
||||
- [ ] push-guard hotfix: the PreToolUse hook denies a read-only command whose TEXT contains the push verb next to `git` (grep pattern, heredoc); match the command's own argv only.
|
||||
- [x] W4-B classifier (2026-10-11, e54a2b5, contract `2026-10-11-model-router-w4b-classifier-1858`, plan r3): `$.model.classify` on plain composer prompts with no route in force → turn effort only (design→plan, analysis→reflect, coding→implement, bookkeeping→apply, chat→nothing), 3-s race, abort not counted, 3 failures pause `cooldownMinutes`, no dialog, no file write, `classify` config key, `/route classify on|off`, status + verbose line. 3 lenses (BLOCKER: unbounded await) + 1 confirmation, feater + 1 round, verifier CONFORME (2nd), security PASS. Kit 358 → 428. Deviation from the TODO line: no first-use confirmation on classified phases (noise per prompt; T3 already confirms the phase).
|
||||
- [ ] W4-B residuals: `smallModelBusy` checks `cfg.models.haiku` but classify uses the engine's default small model (guard misses when they differ); abort paths judged by reading only; the 3-s tests pin the upper bound only. Live AC9 open: type a few plain prompts, read `route: classifier <phase>` (verbose line with `/route verbose on`), drop `bookkeeping` if it misfires.
|
||||
- [ ] W4-B residuals: `smallModelBusy` assumes the engine's small model is the `haiku` alias (`$.model.classify` takes no model: documented in code 2026-10-11, no fix possible); abort paths judged by reading only; the 3-s tests pin the upper bound only. Live AC9 open: type a few plain prompts, read `route: classifier <phase>` (verbose line with `/route verbose on`), drop `bookkeeping` if it misfires.
|
||||
- [ ] W3-A live checks still open: T1 (typed `/feat` etc.) dialog (T3 seen live 2026-10-11: orchestrate Keep); a parallel same-agent dispatch answered after > 10 s; what an unanswered dialog resolves to on the terminal (must be Later or nothing written). Record in the contract as `[gated]` when seen.
|
||||
- [ ] W3-A residuals (security LOW, accepted): non-atomic cross-process write (two sessions, crash mid-write → file reads as failed, previous config kept); `host/path` of a private remote in the tracked file; a persistent write failure re-asks every use (`asked.delete` in the catch); `out.length` vs byte size at the cap; `changePatch` scope 'project' with a vanished key writes Everywhere (cwd change mid-dialog). Deferred: `/route set`, `/route confirm`, `(unconfirmed)` in show, dialog edits of phases, frontmatter auto-alignment, session-start warning when routing.json is dirty, per-machine `projects`.
|
||||
- [ ] routing.json ships `confirmed` for verifier/feater (user Keeps) and phases verify/implement: every clone inherits them (by design: decisions travel). Revisit at release if unwanted.
|
||||
|
||||
- [ ] W2 residuals: probe 4 unobserved (above); the push-guard hook denies a read-only grep (or a heredoc) whose TEXT contains the push verb next to `git` (security-auditor + orchestrator 2026-10-10, false positives, reworded); ~27 loose "sonnet pin"/"opus pin" shorthand sites kept (true by census); README config key list omits `tiers`/`fallback`/`cooldownMinutes`/`mainUpgrade`/`upgradeMaxTokens` (doc audit item 6, pre-existing); MIGRATION.md "Upgrading to 3.0.0" at release time; `skillCalls`/`spawning` counters global; offers keyed by name.
|
||||
- [ ] W2 residuals: probe 4 unobserved (above); the push-guard hook denies a read-only grep (or a heredoc) whose TEXT contains the push verb next to `git` (security-auditor + orchestrator 2026-10-10, false positives, reworded); ~27 loose "sonnet pin"/"opus pin" shorthand sites kept (true by census); (README key list: already complete since W2-B docs, verified 2026-10-11); MIGRATION.md "Upgrading to 3.0.0" at release time; `skillCalls`/`spawning` counters global; offers keyed by name.
|
||||
- [ ] W3 optional: step heuristics, quota-aware downgrade, A/B (haiku classifier: done W4-B)
|
||||
|
||||
## 2026-09-30 — Higgsfield pack: CLI + skills in the install process, off by default (feature/higgsfield-pack)
|
||||
|
||||
Reference in New Issue
Block a user