42 lines
12 KiB
Markdown
42 lines
12 KiB
Markdown
# CONTRACT — model-router-tiers (wave 1-C: absolute tiers, availability fallback, derived phases)
|
|
- date: 2026-10-09 | flow: feat | branch: feature/model-router-mod
|
|
- status: active
|
|
|
|
## REQUEST (verbatim — IMMUTABLE)
|
|
User (fr): "j'aimerais ne pas avoir a reflechir a tout ca, donc ne pas lancer les /route moi meme par exemple. On vois le reflect, orchestratm escalate et plan session (fable) par du preincipe que la sessino est sur fable de base ? Si on est sur haiku j'aimerias que ca fonctionne aussi avec les models adapte. et d'ailleurs que se passe til quand on a plus de credit fable, cela passe sur opus xhigh ? il faut se fallback. ET surtout oui avec un systeme adaptatif et pour la reflexion pour trouver une solution ou des idees, il faut le meilleur, pour en faire le plan en se basant sur cette reflexion. Bref un systeme logique et optimise. /ultrathink . ensuite je reload puginm tu test, on commit puis on passe a la vague 2"
|
|
|
|
## CLARIFICATIONS
|
|
Q: "session" phases / A: no phase keeps "the session model" any more: every phase names a TIER (`best`, `big`, `work`, `cheap`), an ordered list of aliases; the first AVAILABLE alias wins. plan/reflect/orchestrate/escalate → `best` (fable, then opus, then sonnet). A haiku session asked to plan runs the plan on fable. [user: "si on est sur haiku j'aimerais que ca fonctionne aussi avec les models adaptés"]
|
|
Q: what is "available" / A: the mod cannot read per-model quota (rateLimits are account windows: five_hour, seven_day). Availability = a circuit breaker: a model is DOWN for `cooldownMinutes` (default 15) after a main or agent turn ends with `reason: 'error'` or `'refusal'` on it, or when the engine itself switched away from it (`classic.PostModelSwitch`, `source: 'auto'`). A down model is skipped in every tier and in the `fallback` chain; `/route show` lists down models with their reset time; `/route reload` clears the breaker. [orchestrator — derived from the engine's declarations]
|
|
Q: no credit left on fable / A: main loop on a down model → next available alias of the `fallback` chain (`fable, opus, sonnet, haiku`), effort unchanged (plan stays xhigh → "opus xhigh"), always allowed (a down model yields nothing), logged and shown. [user: "il faut se fallback"]
|
|
Q: main-loop model moves / A: UPGRADE (phase tier ranks above the current model) allowed by default (`mainUpgrade: true`); DOWNGRADE (cheaper model) still gated by `mainModelSwitch` (default false: one cold-cache step per switch, and haiku's window may not fit); same rank → no switch. [orchestrator — cost/quality trade-off, stated to the user]
|
|
Q: no `/route` typed by the user / A: phases come from (1) prompt rules at turn start (keyword → phase as the turn's DEFAULT route, source 'prompt', overridable; `ultrathink` stays a FLOOR), (2) the model's own `route` calls and the skills table, (3) derived: an Agent dispatch from main pushes `orchestrate` and the previous route is restored when the last live agent ends, (4) optional classifier (`classifier: false` by default) that asks the engine's small model for a phase label when no rule matched. [user: "ne pas lancer les /route moi-même"]
|
|
Q: default prompt rules / A: floor: `\bultrathink\b` → escalate. Defaults: `\b(plan|planifie|planning|brainstorm|architecture|con[cç]ois|design)\b` → plan; `\b(pourquoi|why|explique|explain|analyse|analyze|comprendre|understand|review|audit)\b` → reflect. No default rule lowers a turn (no `mechanical` rule): lowering is explicit (route tool, skills). [orchestrator — conservative defaults, user-editable in the override]
|
|
Q: typed `/effort-<l>` / A: unchanged (floor + default of the turn, B1).
|
|
Q (r2): breaker signal / A: `classic.StopFailure` error kinds `rate_limit | overloaded | billing_error | model_not_found` (+ the engine's own fallback detected at the step), NOT `turn.complete` reasons (a context-limit or network error is not unavailability); backoff 15 → 300 min; `/clear` keeps the marks (account-wide), `/route reload` and a user `/model` clear them. [orchestrator — challenge r1]
|
|
Q (r2): classifier / A: deferred to wave 2 (dead code by default, no test can reach it, no timeout on the call). [orchestrator — challenge r1]
|
|
Q (r2): upgrade cost / A: an upgrade is skipped above `upgradeMaxTokens` (default 200000 context tokens): one cold read of the whole context per switch into a model. [orchestrator — LRN-204]
|
|
Q (r2): superseded clauses / A: floor AC4 (`turnMain` sources gain 'derived' and 'prompt'), W1-A AC6 (the switch gates downgrades only), BDR-115 (6) (window guard on every switch). [orchestrator]
|
|
|
|
## ACCEPTANCE CRITERIA
|
|
1. Suite green: `claude plugin test` passes with at least 43 `test(` calls; `claude plugin validate` passes with no warning; no line over 80 chars; no `any` type.
|
|
CHECK: cd mods/model-router && out=$(claude plugin test . 2>&1); rc=$?; echo "$out" | tail -n 3; [ $rc -eq 0 ] && [ "$(grep -cE '^\s*test\(' hooks/register.test.ts)" -ge 43 ] && v=$(claude plugin validate . 2>&1) && echo "$v" | grep -q 'Validation passed' && ! echo "$v" | grep -qi 'warning' && ! grep -nE '.{81,}' hooks/register.ts hooks/register.test.ts && ! grep -nE ':\s*any\b|<any>|as any\b' hooks/register.ts && echo TIERS-SUITE-OK
|
|
EXPECT: TIERS-SUITE-OK
|
|
EVIDENCE: pending
|
|
2. Type-check clean against this build's declarations.
|
|
CHECK: T=/Users/b.chanot/Documents/claude/mods/model-router/.claude-plugin/types; W=$(mktemp -d) && printf '{"compilerOptions":{"target":"es2023","lib":["es2023"],"types":[],"module":"esnext","moduleResolution":"bundler","strict":true,"noUncheckedIndexedAccess":true,"noEmit":true,"skipLibCheck":true,"jsx":"react","jsxFactory":"h","jsxFragmentFactory":"Fragment"},"include":["%s/claude-code/index.d.ts","%s/claude-code-tools/index.d.ts","%s/hooks"]}' "$T" "$T" "$PWD/mods/model-router" > "$W/tsconfig.json" && (cd "$W" && npx --yes -p typescript@5 tsc -p tsconfig.json) && echo TSC-OK
|
|
EXPECT: TSC-OK
|
|
EVIDENCE: pending
|
|
3. Tiers in the config: `tiers` (best/big/work/cheap), `fallback`, `cooldownMinutes`, `mainUpgrade`, `upgradeMaxTokens` exist in DEFAULT_CONFIG; every default phase names a tier, none a bare model; `/route show` prints the resolved model of each phase and a `down:` line; no `classifier` (deferred to wave 2); no `Loop.model` / `spawnModel` / `loop.model` identifier (W1-A AC8).
|
|
CHECK: cd mods/model-router/hooks && D=$(awk '/^const DEFAULT_CONFIG/,/^}/' register.ts) && echo "$D" | grep -q "tiers:" && echo "$D" | grep -q "fallback:" && grep -q "cooldownMinutes" register.ts && grep -q "mainUpgrade" register.ts && grep -q "upgradeMaxTokens" register.ts && ! grep -q "classifier" register.ts && [ "$(awk '/^const DEFAULT_CONFIG/,/^}/' register.ts | grep -cE "tier: '(best|big|work|cheap)'")" -ge 10 ] && ! awk '/^const DEFAULT_CONFIG/,/^}/' register.ts | grep -qE "model: '(haiku|sonnet|opus|fable)'" && ! grep -qE "loop\.model|explicitModel|spawnModel" register.ts && grep -q "down:" register.ts && echo TIERS-CONFIG-OK
|
|
EXPECT: TIERS-CONFIG-OK
|
|
EVIDENCE: pending
|
|
4. Tests prove (names contain the quoted word; plan r2 R14 lists them): `tier` — a `plan` route on a session model `claude-haiku-4-5-20251001` makes the main step run on `claude-fable-5-1` at xhigh (upgrade, default on); `downgrade` — a `mechanical` route on a fable session leaves the model unchanged while `mainModelSwitch` is off; `fallback` — with a `plan` route, a `classic.StopFailure` `rate_limit` on main after a fable step makes the next main step run on `claude-opus-5-5` at xhigh, and after `/route reload` fable is used again; `breaker` — an `invalid_request` failure never marks a model down, backoff expiry restores it, a `/model` command (`PostModelSwitch` source `command`) clears it; `engine fallback` — a `PostModelSwitch` with source `auto` marks the model the engine left (one strike, idempotent within the hold) and a plan route does not go back to it; `unknown` — a session model absent from the table is never switched; `spawn` — `Explore` spawns on `claude-opus-5-5` while sonnet is down (agent StopFailure with `agent_id`); `derived` — a main Agent call sets `orchestrate` and the previous `plan` route is back when the spawned agent ends; a route declared after the dispatch is not overwritten by the pop; `default rule` — "planifie la migration" sets `plan` as the turn default and a later route call overrides it; a typed `/analyze …` and a `/effort-low pourquoi …` prompt get no default rule; `per axis` — a model-less sticky never hides a turn route's tier; `floor` tests from B1 still pass.
|
|
CHECK: cd mods/model-router/hooks && for w in tier downgrade fallback breaker "engine fallback" unknown spawn derived "default rule" "per axis"; do grep -qE "test\('[^']*$w" register.test.ts || { echo "missing test: $w"; exit 1; }; done && [ "$(grep -cE "test\('[^']*(breaker|derived|default rule)" register.test.ts)" -ge 7 ] && echo TIERS-TESTS-OK
|
|
EXPECT: TIERS-TESTS-OK
|
|
EVIDENCE: pending
|
|
5. Judged by reading (plan r2 R1-R15, r3 S1-S11 and r4 T1-T8 are binding; r4 wins over r3, r3 over r2 where they conflict: two fields `turnModel` (sticky, per turn) and `lastPlan` (breaker target, kept), unrouted steps pass `e.model` verbatim, auto marks target the model actually sent and skip when the engine landed where the router was, `sessionModel` preserved across /clear and never prefix-matched when empty, tokens `number | undefined` with windowOk failing closed, episode strikes with `model_not_found` lengthening a hold; no per-step engine-fallback detection, `PostModelSwitch` auto DOES mark one idempotent strike, the main model is sticky within a turn, `sessionModel` cached at start and on PostModelSwitch, D1 for background dispatches via the Agent result status, breaker targets kept until replaced): ONE resolver turns a tier name, an alias or a full id into an AVAILABLE canonical id (tier → list → skip down → id; an exhausted tier → the global chain; a bare alias or id passes through even when down); ids are canonical everywhere (`[1m]` stripped for comparison and carried on the replacement, alias → id, two-way prefix); ONE decision `decideMain(cur, wanted, ctx)` in the binding order (off → unknown cur: no switch → cur down: wanted or next available, windowOk → same alias: keep → better: `mainUpgrade` and `upgradeMaxTokens` → cheaper: `mainModelSwitch` and windowOk), used by `mainPlan` AND by every text; the breaker is fed only by `classic.StopFailure` errors `rate_limit | overloaded | billing_error | model_not_found` (main → the last main plan's model, agent → `agentModels`) and by `PostModelSwitch` `source: 'auto'` (the model the engine left, one strike, idempotent while down); `turn.complete` reasons never mark; a user `/model` (`PostModelSwitch` command|picker|sdk) clears the target's mark; backoff 15 → 30 → 60 → 120 → 300 min per id, `model_not_found` until reload; the breaker survives `/clear` and is cleared by `/route reload` before the config read; the derived `orchestrate` push/pop tracks this turn's spawns and never overwrites a route the model declared after the dispatch; default prompt rules (two passes, absent `mode` = floor, `iu` flags, Unicode guards) write `turnMain` (source 'prompt') and are skipped for a leading `/`, for a prompt carrying a floor or a typed slash, and mid-turn; floor rules write `turnFloor`; no classifier; explicit Agent params still win; agent model fixed at spawn; every B1/1-A criterion still holds EXCEPT the three clauses R15 names (turnMain sources, the switch clause, the window-guard scope); no function over 25 logic lines; truthful texts come from `decideMain` and say `upgrade`, `fallback`, `switch off` or `unchanged`.
|
|
|
|
## FILE SCOPE
|
|
mods/model-router/hooks/register.ts · mods/model-router/hooks/register.test.ts
|