chore(tasks): model-router W1-C plan r2 + contract amendments after the FATAL round
This commit is contained in:
@@ -13,25 +13,29 @@ Q: main-loop model moves / A: UPGRADE (phase tier ranks above the current model)
|
||||
Q: no `/route` typed by the user / A: phases come from (1) prompt rules at turn start (keyword → phase as the turn's DEFAULT route, source 'prompt', overridable; `ultrathink` stays a FLOOR), (2) the model's own `route` calls and the skills table, (3) derived: an Agent dispatch from main pushes `orchestrate` and the previous route is restored when the last live agent ends, (4) optional classifier (`classifier: false` by default) that asks the engine's small model for a phase label when no rule matched. [user: "ne pas lancer les /route moi-même"]
|
||||
Q: default prompt rules / A: floor: `\bultrathink\b` → escalate. Defaults: `\b(plan|planifie|planning|brainstorm|architecture|con[cç]ois|design)\b` → plan; `\b(pourquoi|why|explique|explain|analyse|analyze|comprendre|understand|review|audit)\b` → reflect. No default rule lowers a turn (no `mechanical` rule): lowering is explicit (route tool, skills). [orchestrator — conservative defaults, user-editable in the override]
|
||||
Q: typed `/effort-<l>` / A: unchanged (floor + default of the turn, B1).
|
||||
Q (r2): breaker signal / A: `classic.StopFailure` error kinds `rate_limit | overloaded | billing_error | model_not_found` (+ the engine's own fallback detected at the step), NOT `turn.complete` reasons (a context-limit or network error is not unavailability); backoff 15 → 300 min; `/clear` keeps the marks (account-wide), `/route reload` and a user `/model` clear them. [orchestrator — challenge r1]
|
||||
Q (r2): classifier / A: deferred to wave 2 (dead code by default, no test can reach it, no timeout on the call). [orchestrator — challenge r1]
|
||||
Q (r2): upgrade cost / A: an upgrade is skipped above `upgradeMaxTokens` (default 200000 context tokens): one cold read of the whole context per switch into a model. [orchestrator — LRN-204]
|
||||
Q (r2): superseded clauses / A: floor AC4 (`turnMain` sources gain 'derived' and 'prompt'), W1-A AC6 (the switch gates downgrades only), BDR-115 (6) (window guard on every switch). [orchestrator]
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. Suite green: `claude plugin test` passes with at least 40 `test(` calls; `claude plugin validate` passes with no warning; no line over 80 chars; no `any` type.
|
||||
CHECK: cd mods/model-router && out=$(claude plugin test . 2>&1); rc=$?; echo "$out" | tail -n 3; [ $rc -eq 0 ] && [ "$(grep -cE '^\s*test\(' hooks/register.test.ts)" -ge 40 ] && v=$(claude plugin validate . 2>&1) && echo "$v" | grep -q 'Validation passed' && ! echo "$v" | grep -qi 'warning' && ! grep -nE '.{81,}' hooks/register.ts hooks/register.test.ts && ! grep -nE ':\s*any\b|<any>|as any\b' hooks/register.ts && echo TIERS-SUITE-OK
|
||||
1. Suite green: `claude plugin test` passes with at least 43 `test(` calls; `claude plugin validate` passes with no warning; no line over 80 chars; no `any` type.
|
||||
CHECK: cd mods/model-router && out=$(claude plugin test . 2>&1); rc=$?; echo "$out" | tail -n 3; [ $rc -eq 0 ] && [ "$(grep -cE '^\s*test\(' hooks/register.test.ts)" -ge 43 ] && v=$(claude plugin validate . 2>&1) && echo "$v" | grep -q 'Validation passed' && ! echo "$v" | grep -qi 'warning' && ! grep -nE '.{81,}' hooks/register.ts hooks/register.test.ts && ! grep -nE ':\s*any\b|<any>|as any\b' hooks/register.ts && echo TIERS-SUITE-OK
|
||||
EXPECT: TIERS-SUITE-OK
|
||||
EVIDENCE: pending
|
||||
2. Type-check clean against this build's declarations.
|
||||
CHECK: T=/Users/b.chanot/Documents/claude/mods/model-router/.claude-plugin/types; W=$(mktemp -d) && printf '{"compilerOptions":{"target":"es2023","lib":["es2023"],"types":[],"module":"esnext","moduleResolution":"bundler","strict":true,"noUncheckedIndexedAccess":true,"noEmit":true,"skipLibCheck":true,"jsx":"react","jsxFactory":"h","jsxFragmentFactory":"Fragment"},"include":["%s/claude-code/index.d.ts","%s/claude-code-tools/index.d.ts","%s/hooks"]}' "$T" "$T" "$PWD/mods/model-router" > "$W/tsconfig.json" && (cd "$W" && npx --yes -p typescript@5 tsc -p tsconfig.json) && echo TSC-OK
|
||||
EXPECT: TSC-OK
|
||||
EVIDENCE: pending
|
||||
3. Tiers in the config: `tiers` (best/big/work/cheap), `fallback`, `cooldownMinutes`, `mainUpgrade`, `classifier` exist in DEFAULT_CONFIG; every default phase names a tier, none a bare model; `/route show` prints the resolved model of each phase and a `down:` line.
|
||||
CHECK: cd mods/model-router/hooks && grep -q "tiers:" register.ts && grep -q "fallback:" register.ts && grep -q "cooldownMinutes" register.ts && grep -q "mainUpgrade" register.ts && grep -q "classifier" register.ts && [ "$(awk '/^const DEFAULT_CONFIG/,/^}/' register.ts | grep -cE "tier: '(best|big|work|cheap)'")" -ge 10 ] && ! awk '/^const DEFAULT_CONFIG/,/^}/' register.ts | grep -qE "^\s+[a-z]+: \{ model: '" && grep -q "down:" register.ts && echo TIERS-CONFIG-OK
|
||||
3. Tiers in the config: `tiers` (best/big/work/cheap), `fallback`, `cooldownMinutes`, `mainUpgrade`, `upgradeMaxTokens` exist in DEFAULT_CONFIG; every default phase names a tier, none a bare model; `/route show` prints the resolved model of each phase and a `down:` line; no `classifier` (deferred to wave 2); no `Loop.model` / `spawnModel` / `loop.model` identifier (W1-A AC8).
|
||||
CHECK: cd mods/model-router/hooks && grep -q "tiers:" register.ts && grep -q "fallback:" register.ts && grep -q "cooldownMinutes" register.ts && grep -q "mainUpgrade" register.ts && grep -q "upgradeMaxTokens" register.ts && ! grep -q "classifier" register.ts && [ "$(awk '/^const DEFAULT_CONFIG/,/^}/' register.ts | grep -cE "tier: '(best|big|work|cheap)'")" -ge 10 ] && ! awk '/^const DEFAULT_CONFIG/,/^}/' register.ts | grep -qE "model: '(haiku|sonnet|opus|fable)'" && ! grep -qE "loop\.model|explicitModel|spawnModel" register.ts && grep -q "down:" register.ts && echo TIERS-CONFIG-OK
|
||||
EXPECT: TIERS-CONFIG-OK
|
||||
EVIDENCE: pending
|
||||
4. Tests prove (names contain the quoted word): `tier` — a `plan` route on a session model `claude-haiku-4-5-20251001` makes the main step run on `claude-fable-5-1` at xhigh (upgrade, default on); `downgrade` — a `mechanical` route on a fable session leaves the model unchanged while `mainModelSwitch` is off; `fallback` — after a main `turn.complete` with `reason: 'error'` on fable, the next main step runs on `claude-opus-5-5` with its effort unchanged, and after `/route reload` fable is used again; `breaker` — a `classic.PostModelSwitch` with `source: 'auto'` from fable marks fable down and `/route show` lists it; `spawn` — `Explore` spawns on `claude-opus-5-5` while sonnet is down; `derived` — an Agent tool call from main sets `orchestrate` and the previous route (`plan`) is back after the last agent's `turn.complete`; `default rule` — a prompt "planifie la migration" sets the `plan` route as the turn default and a later `route` tool call overrides it; `floor` tests from B1 still pass.
|
||||
CHECK: cd mods/model-router/hooks && for w in tier downgrade fallback breaker spawn derived "default rule"; do grep -qE "test\('[^']*$w" register.test.ts || { echo "missing test: $w"; exit 1; }; done && echo TIERS-TESTS-OK
|
||||
4. Tests prove (names contain the quoted word; plan r2 R14 lists them): `tier` — a `plan` route on a session model `claude-haiku-4-5-20251001` makes the main step run on `claude-fable-5-1` at xhigh (upgrade, default on); `downgrade` — a `mechanical` route on a fable session leaves the model unchanged while `mainModelSwitch` is off; `fallback` — with a `plan` route, a `classic.StopFailure` `rate_limit` on main after a fable step makes the next main step run on `claude-opus-5-5` at xhigh, and after `/route reload` fable is used again; `breaker` — an `invalid_request` failure never marks a model down, backoff expiry restores it, a `/model` command (`PostModelSwitch` source `command`) clears it; `engine fallback` — a step arriving on a model other than `$.session.model()` is never upgraded back and the session model is marked down; `unknown` — a session model absent from the table is never switched; `spawn` — `Explore` spawns on `claude-opus-5-5` while sonnet is down (agent StopFailure with `agent_id`); `derived` — a main Agent call sets `orchestrate` and the previous `plan` route is back when the spawned agent ends; a route declared after the dispatch is not overwritten by the pop; `default rule` — "planifie la migration" sets `plan` as the turn default and a later route call overrides it; a typed `/analyze …` and a `/effort-low pourquoi …` prompt get no default rule; `per axis` — a model-less sticky never hides a turn route's tier; `floor` tests from B1 still pass.
|
||||
CHECK: cd mods/model-router/hooks && for w in tier downgrade fallback breaker "engine fallback" unknown spawn derived "default rule" "per axis"; do grep -qE "test\('[^']*$w" register.test.ts || { echo "missing test: $w"; exit 1; }; done && [ "$(grep -cE "test\('[^']*(breaker|derived|default rule)" register.test.ts)" -ge 7 ] && echo TIERS-TESTS-OK
|
||||
EXPECT: TIERS-TESTS-OK
|
||||
EVIDENCE: pending
|
||||
5. Judged by reading: ONE resolver (`resolveModel`) turns a tier name, an alias or a full id into the first AVAILABLE full id (tier → list → skip down → alias → id; a bare alias or id passes through even when down, since explicit means explicit); rank = position in `fallback`; the main-loop decision is: route model wanted → upgrade allowed by `mainUpgrade`, downgrade gated by `mainModelSwitch` + window, same rank → keep; no route and current model down → next available in `fallback`; the breaker is fed only by `turn.complete` `reason` in `error|refusal` (main: the last main step's model; agent: the loop's spawn model) and by `classic.PostModelSwitch` `source: 'auto'` (from_model), never by an aborted turn; the derived `orchestrate` push/pop never overrides a route the model declared after the dispatch; prompt default rules write `turnMain` (source 'prompt'), floor rules write `turnFloor`; the classifier runs only when `classifier` is true and no rule matched, through `$.model.classify`, labels limited to the phase names plus `other`, any failure → no route; explicit Agent params still win; agent model fixed at spawn; every B1/1-A criterion still holds; no function over 25 logic lines; truthful texts name the resolved model and say "fallback" when the breaker chose it.
|
||||
5. Judged by reading (plan r2 R1-R15 are binding): ONE resolver turns a tier name, an alias or a full id into an AVAILABLE canonical id (tier → list → skip down → id; an exhausted tier → the global chain; a bare alias or id passes through even when down); ids are canonical everywhere (`[1m]` stripped for comparison and carried on the replacement, alias → id, two-way prefix); ONE decision `decideMain(cur, wanted, ctx)` in the binding order (off → unknown cur: no switch → cur down: wanted or next available, windowOk → same alias: keep → better: `mainUpgrade` and `upgradeMaxTokens` → cheaper: `mainModelSwitch` and windowOk), used by `mainPlan` AND by every text; the engine's own fallback is respected (a step arriving off the session model marks the session model down and is never upgraded back); the breaker is fed only by `classic.StopFailure` errors `rate_limit | overloaded | billing_error | model_not_found` (main → the last main plan's model, agent → `agentModels`) and by the engine-fallback detection; `turn.complete` reasons and `PostModelSwitch` `auto` never mark (auto is logged); a user `/model` (`PostModelSwitch` command|picker|sdk) clears the target's mark; backoff 15 → 30 → 60 → 120 → 300 min per id, `model_not_found` until reload; the breaker survives `/clear` and is cleared by `/route reload` before the config read; the derived `orchestrate` push/pop tracks this turn's spawns and never overwrites a route the model declared after the dispatch; default prompt rules (two passes, absent `mode` = floor, `iu` flags, Unicode guards) write `turnMain` (source 'prompt') and are skipped for a leading `/`, for a prompt carrying a floor or a typed slash, and mid-turn; floor rules write `turnFloor`; no classifier; explicit Agent params still win; agent model fixed at spawn; every B1/1-A criterion still holds EXCEPT the three clauses R15 names (turnMain sources, the switch clause, the window-guard scope); no function over 25 logic lines; truthful texts come from `decideMain` and say `upgrade`, `fallback`, `switch off` or `unchanged`.
|
||||
|
||||
## FILE SCOPE
|
||||
mods/model-router/hooks/register.ts · mods/model-router/hooks/register.test.ts
|
||||
|
||||
Reference in New Issue
Block a user