Merge bugfix/router-probe-session-model into develop
This commit is contained in:
commit
6f0deb489d
9 files changed
+112
-24
No files matched your search
@@ -50,6 +50,7 @@ rules:
|
||||
| BLK-028 | 2026-10-06 | notify-attention on a VS Code client: bell + toast silent-degradation faults (merge of BLK-019 + BLK-020): terminalBell sound default off, ext hooks only terminals born after activation, Code muted in Windows mixer | resolved |
|
||||
| BLK-029 | 2026-10-08 | Claude Code mods (2.1.294): alias → id resolution for a model set by a hook lags the Agent tool's (`sonnet` → `claude-sonnet-5`, 404); Agent tool schema refuses full ids | upstream |
|
||||
| BLK-030 | 2026-10-15 | model-router kept routing best-tier phases to Fable out of usage credits: each new session + each hold lapse lost a turn to 429 `model_requires_usage_credits`; breaker reactive only | resolved |
|
||||
| BLK-031 | 2026-10-16 | model-router: session starting on Fable w/o credits lost 1st turn + each hold lapse; probe skipped kept cur | resolved |
|
||||
|
||||
---
|
||||
|
||||
@@ -321,3 +322,10 @@ rules:
|
||||
- **Solution**: 2cccc43 (bugfix/router-probe-before-route). `decideFor` decides first, then probes decided move target once: `$.model.complete` raced vs `PROBE_MS` 8000 via shared `raceTimeout`; timeout/reject → `markDown(..., 'probe')` + ONE re-decision, no further probe. `verified` set kept across `/clear`, cleared by `markDown`. Never probes cur or held id. Config `probe` (default true). Stand-in effort one notch up (reflect xhigh, plan max, orchestrate high) when lower model replaces held tier head where head would have run; explicit/floor effort never bumped. 497 tests.
|
||||
- **Status**: resolved. Residuals in TODO (explicit `Agent(model:"fable")` pins, spawn path, session model itself out of credits, ≤8 s stall per new session).
|
||||
- **Reference**: [[LRN-218]], [[BDR-116]], [[BDR-115]], [[LRN-207]], contract `.claude/tasks/contracts/2026-10-14-router-probe-before-route-0005.md`, plan r3 `.claude/tasks/plans/2026-10-14-router-probe-before-route-0005.md`.
|
||||
|
||||
## BLK-031 — model-router: session ON Fable w/o credits still loses 1st turn; probe skipped kept model — 2026-10-16
|
||||
- **Friction**: session model fable, credits out. Log `main: fable/high → fable/medium (implement: classifier)` then "You're out of usage credits", then `claude-fable-5-1 unavailable (rate_limit) for 15 min`. User suspected bad redeploy; not cause.
|
||||
- **Real cause**: 2cccc43 `unprobed()` required `call.moved`: only move target probed. Session model = cur, kept by `decideMain` → never probed. New process (empty `verified`) + hold lapse (`markDown` drops `verified`, cur = session model again) → real request to dead model. Residual of [[BLK-030]], accepted then.
|
||||
- **Solution**: c11cd3b drops `!call.moved` guard: model decision lands on (moved or kept) probed once if table id, unverified, not held. Fail → markDown 'probe', one re-decision → leaveDown → opus, stand-in effort +1. Cost ~1.3 s healthy / ≤8 s dead, 1st decision per process. Test `probe: a session model that never answers leaves for opus on the first step`.
|
||||
- **Status**: resolved. Open: leaveDown follows classifier `wanted` (implement → sonnet), not opus; "context size unknown" likely document attachment → `promptSize` unknown.
|
||||
- **Reference**: [[BLK-030]], [[LRN-218]], contract `.claude/tasks/contracts/2026-10-16-router-probe-session-model-1850.md`.
|
||||
@@ -618,3 +618,4 @@ rules:
|
||||
|
||||
## 2026-10-16
|
||||
- User asked if impeccable replaces frontend-design (install prompt in other project). Audit: yes for project UI, no for one-off visual. BDR-119: impeccable = Build path, GATE-BLOCK swap, doctrine Build line, advisor rows, rules 45 -> 61. frontend-design stays installed, light path. Branch chore/impeccable-build-path.
|
||||
- /bugfix model-router: session starting on dead Fable (no credits) lost 1st turn; probe skipped kept cur. c11cd3b `unprobed()` drops `call.moved`. 499 tests, verifier+security PASS. Merge on user signal.
|
||||
@@ -19,7 +19,7 @@ User validated: frontend-design overlaps impeccable 4.6; gate still blocked on f
|
||||
## 2026-10-15 — model-router: probe a move target before routing main to it (bugfix/router-probe-before-route)
|
||||
Contract `.claude/tasks/contracts/2026-10-14-router-probe-before-route-0005.md`, plan r3 `.claude/tasks/plans/2026-10-14-router-probe-before-route-0005.md`.
|
||||
- [x] /bugfix run: Fable out of credits routed (BLK-030, LRN-218); r1 → r3; 497 tests, verifier CONFORME 11/11, security PASS; commit 2cccc43 + docs.
|
||||
- [ ] Residuals (accepted): explicit `Agent(model:"fable")` pins (client-handover) still fail w/o credits (BDR-115 kept); spawn path not probed; session model itself out of credits → first turn fails reactively (cur never probed); no cross-session persistence → ≤8 s stall per new session; timed-out probe keeps retrying in engine (no cancel); `[1m]` gating unverified (bare id probed); no Esc regression test (kit cannot abort signal); probe sleep eats ~8 of 10 s `turn.step` budget.
|
||||
- [ ] Residuals (accepted): explicit `Agent(model:"fable")` pins (client-handover) still fail w/o credits (BDR-115 kept); spawn path not probed; session model itself out of credits → CLOSED 2026-10-16 c11cd3b (kept cur probed too); no cross-session persistence → ≤8 s stall per new session; timed-out probe keeps retrying in engine (no cancel); `[1m]` gating unverified (bare id probed); no Esc regression test (kit cannot abort signal); probe sleep eats ~8 of 10 s `turn.step` budget.
|
||||
- [ ] LIVE CHECK after `/reload-plugins`: best-tier phase w/ Fable out of credits → `/route show` lists fable down (probe), main on opus, effort bumped.
|
||||
|
||||
## 2026-10-13 — model-router: cold-read cap, switch on by default, classifier routes down (feature/router-downgrade-cap)
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# CONTRACT — router-probe-session-model
|
||||
- date: 2026-10-16 | flow: bugfix | branch: bugfix/router-probe-session-model
|
||||
- status: active
|
||||
|
||||
## REQUEST (verbatim — IMMUTABLE)
|
||||
pareil on a vait fait en sorte que si on avait plus fable, cela le detectait et il ne reuotait pas vers fable. La ca fait plusierus fois qu'il essaie de router vers fable et comme ya plus de credit il dit que les usages sont depasse blabla alors qu'il devrait ocntinuer apre en allant sur opus max ou xhigh
|
||||
|
||||
(pasted log)
|
||||
⏺ model-router: model-router: downgrade skipped: context size unknown
|
||||
⏺ model-router: main: fable/high → fable/medium (implement: classifier)
|
||||
You're out of usage credits. Run /usage-credits to keep using Fable 5.1 or /model to switch models.
|
||||
⏺ model-router: model-router: claude-fable-5-1 unavailable (rate_limit) for 15 min; routing falls back
|
||||
|
||||
(ca vient de le faire) peut etre que j'ai pas bien redeployer le claude
|
||||
|
||||
## CLARIFICATIONS
|
||||
Q: Root cause? / A: 2cccc43 probes only a MOVE target (`unprobed()` requires `call.moved`). A session whose own model is fable (cur) keeps fable unprobed: every new process and every hold lapse (cur = session model, unverified after markDown) loses a turn to the 429. Accepted residual of BLK-030, now reported by the user as the bug. [orchestrator 2026-10-16]
|
||||
Q: Why was cur excluded originally? / A: plan r1 probed in a loop BEFORE deciding (spin BLOCKER). Current design decides first, probes once, re-decides once without probe: probing the kept model reuses that bounded path. [orchestrator 2026-10-16]
|
||||
Q: Cost accepted? / A: one tiny completion per model per process (verified kept across /clear); ~1.3 s on a healthy session model at first decision, ≤8 s (PROBE_MS) when it is out of credits, instead of a lost turn. [orchestrator 2026-10-16, reported to user]
|
||||
Q: Fallback model when the classifier guessed `implement` (work tier → sonnet) while fable is down? / A: out of scope, leaveDown unchanged; reported to the user as an open choice. [orchestrator 2026-10-16]
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. The model-router mod test suite passes with zero failures.
|
||||
CHECK: cd /Users/b.chanot/Documents/claude && out=$(claude plugin test mods/model-router 2>&1); rc=$?; printf '%s\n' "$out" | tail -4; [ "$rc" -eq 0 ] && printf '%s\n' "$out" | grep -Eq '^ *0 fail$' && ! printf '%s\n' "$out" | grep -q '^(fail)' && echo SUITE_GREEN
|
||||
EXPECT: SUITE_GREEN
|
||||
EVIDENCE: MET exit=0 marker-found :: 499 pass 0 fail Ran 499 tests across 1 file. [21.41s] SUITE_GREEN
|
||||
2. Regression test present and passing: a test named starting with `probe: a session model that never answers` boots the session on fable with fable's completion hanging, runs the first main step of a plan route on fable, and asserts the step runs on opus, exactly one model.complete call (fable), and `/route show` lists fable down with reason `(probe)` — with no StopFailure fired.
|
||||
CHECK: cd /Users/b.chanot/Documents/claude && out=$(claude plugin test mods/model-router 2>&1); printf '%s\n' "$out" | grep -E '^\((pass|fail)\) probe: a session model that never answers'; printf '%s\n' "$out" | grep -q '^(pass) probe: a session model that never answers' && ! printf '%s\n' "$out" | grep -q '^(fail) probe:' && echo REGRESSION_PASS
|
||||
EXPECT: REGRESSION_PASS
|
||||
EVIDENCE: MET exit=0 marker-found :: (pass) probe: a session model that never answers leaves for opus on the first step [51.95ms] REGRESSION_PASS
|
||||
3. A test asserts a healthy session model is probed once and kept: session on fable answering, two turns of steps on fable → every step on fable, model.complete called exactly once (fable).
|
||||
4. The old test `probe: the current model is never probed` is gone (its expectation is now wrong); no other probe test deleted: tests whose `completions` expectation changes because cur is now probed are updated so they still assert their original intent (e.g. fable never probed when the classifier guess is above the session / upgrade switch off).
|
||||
CHECK: cd /Users/b.chanot/Documents/claude && f=mods/model-router/hooks/register.test.ts; ! grep -q "the current model is never probed" $f && grep -q "no probe when the classifier guess is above the session" $f && grep -q "no probe when the upgrade switch refuses the move" $f && grep -q "a lapsed hold probes again and holds longer" $f && echo OLD_TEST_REPLACED
|
||||
EXPECT: OLD_TEST_REPLACED
|
||||
EVIDENCE: MET exit=0 marker-found :: OLD_TEST_REPLACED
|
||||
5. Probe semantics: the only behavior change is that `unprobed()` (register.ts) no longer requires `call.moved`: the model the decision LANDS on (moved or kept) is probed when it is a table model, not verified, not held down, probe on, router on. Unchanged: probe after decideMain, one probe, failure → markDown(id,'probe') then ONE re-decision without a further probe, held ids never probed, `probe: false` → no completion, spawn path untouched. Doc comments that state "the current model is never probed" / "move target" are updated to match.
|
||||
6. The mods census passes.
|
||||
CHECK: cd /Users/b.chanot/Documents/claude && out=$(make test suite=lib/tests/mods.test.sh 2>&1); printf '%s\n' "$out" | tail -3; printf '%s\n' "$out" | grep -Eq '^mods: [0-9]+ pass, 0 fail$' && ! printf '%s\n' "$out" | grep -q '^FAIL ' && echo MODS_GREEN
|
||||
EXPECT: MODS_GREEN
|
||||
EVIDENCE: MET exit=0 marker-found :: PASS model-router: plugin test passed mods: 4 pass, 0 fail all suites green MODS_GREEN
|
||||
7. No edit outside FILE SCOPE.
|
||||
|
||||
## FILE SCOPE
|
||||
mods/model-router/hooks/register.ts
|
||||
mods/model-router/hooks/register.test.ts
|
||||
+1
-1
@@ -30,7 +30,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and this project
|
||||
- The five `effort-low` … `effort-max` skills, `lib/effort-pins.txt`, `lib/effort-pins.sh` and `lib/model-check.sh`, with their tests and the effort-pin re-apply steps of `install-plugins.sh` and `update-all.sh`. The model-router rows replace them. A typed `/effort-<level>` no longer exists: use `ultrathink` or `/route effort=max`. Breaking: the next release is 3.0.0.
|
||||
|
||||
### Fixed
|
||||
- model-router probes a move target before routing main to it: with a model out of usage credits, every new session and hold lapse used to route main there and lose the turn to a 429. The model main moves to is tried once with a small completion bounded to 8 s; a timeout or error marks it down (reason `probe`) and the decision is taken again once. A model that answered is not probed again until marked down (kept across `/clear`). `"probe": false` in `~/.claude/model-router.json` turns it off; `/route show` prints `probe:`. When a lower model stands in for its tier's held head, the phase's own effort goes one notch up (reflect xhigh, plan max, orchestrate high on Opus); explicit and floor efforts are never raised. Tests: `mods/model-router/hooks/register.test.ts`.
|
||||
- model-router probes the model main is about to run on, moved to or kept (the session's own model included), before routing main to it: with a model out of usage credits, every new session and hold lapse used to route main there, or keep it there, and lose the turn to a 429. That model is tried once with a small completion bounded to 8 s; a timeout or error marks it down (reason `probe`) and the decision is taken again once. A model that answered is not probed again until marked down (kept across `/clear`). `"probe": false` in `~/.claude/model-router.json` turns it off; `/route show` prints `probe:`. When a lower model stands in for its tier's held head, the phase's own effort goes one notch up (reflect xhigh, plan max, orchestrate high on Opus); explicit and floor efforts are never raised. Tests: `mods/model-router/hooks/register.test.ts`.
|
||||
- model-router residuals of waves 1-4 (2026-10-11): the kill switch fails closed (a `~/.claude/model-router.json` present but unreadable at first load, or an `enabled` that is not a boolean, turns the router off and says so; `/route reload` answers `config unreadable; previous config kept` instead of `config reloaded`); a prompt rule whose pattern nests quantifiers (`(a+)+$`) is dropped; a config file that is not a regular file reads as failed; every log of a caught error prints its kind and code only, never the message (a JSON parse error quotes the file); rejected table keys are masked in logs and model alias keys must be short printable names; `__proto__`, `constructor` and `prototype` are never read or written as keys; the agent offers map is capped like the descriptions; a `models` alias added without a `fallback` list is ranked at the end of the default chain (the upgrade gates now apply to it); a dated or variant id maps to its table id only at a `-` boundary (`claude-sonnet-5-50` no longer reads as sonnet); the status refresh and the main-decision log on the step path can no longer unroute a step; the route tool's failure answer says the route may be partly applied. `lib/tests/mods.test.sh` proves `claude plugin test` exists from its usage line (an unknown subcommand exits 0), `doctor.sh` bounds `claude plugin list --json` to 60 s, `update-all.sh` skips `@skills-dir` mods. Kit 428 → 431.
|
||||
- model-router residuals of the confirmation waves: a failed `/route forget` or `/route ask` write logs its cause and answers a generic text; forget answers clean every file-sourced name, phase and key; a restore names a frontmatter file only for a valid skill name and refuses a `__proto__` row; `/route forget` never restores `changed.phases`; the dialog's Change offers and writes shipped phases only (a machine-only phase never reaches routing.json, and an alias no shipped phase uses says so); a failure after the user answered keeps the key asked instead of asking again at every use.
|
||||
- `gitflow delete` (and `finish`) land on the base that contains the branch and drop the branch's upstream before `git branch -d`, so a branch whose upstream lags (manual-push mode) is deleted instead of refused by git (T18k).
|
||||
|
||||
@@ -132,7 +132,7 @@ effort, never version. Transcript audit `python3 lib/effort-audit.py`.
|
||||
|
||||
`mods/model-router/` is a Claude Code mod (a function-hooks plugin) that applies this table per request. It loads in every session through the tracked symlink `skills/model-router`, as `model-router@skills-dir`.
|
||||
|
||||
- Main loop: every request gets the route in force. A typed skill with a row routes the main loop to it; a best-tier row (`plan`, `reflect`, `orchestrate`, `escalate`) holds across turns in a run slot until `/route clear`, `/route off`, a user `/model` or a typed skill on a non-best row. A skill without a row leaves the route as it is. A rowed skill the model loads inside a best-tier run moves effort only when its row ranks below the run's model. Main routed onto a model whose context window the conversation outgrew returns to the session model. Before main moves to another model, the mod probes it with one small completion bounded to 8 seconds; the step (and `/route show`, the route tool's answer) waits for it. A timeout or an error marks the model down (reason `probe`, held for `cooldownMinutes` like any breaker hold) and the decision is taken again once. The current model and a model already marked down are never probed, and a model that answered is not probed again, `/clear` included, until it is marked down. When a lower model stands in for its tier's head held down (Opus for Fable on a best-tier phase), the phase's own effort goes one notch up: `reflect` xhigh, `plan` max, `orchestrate` high, `escalate` stays max. An effort asked explicitly (`/route ... effort=`, the `route` tool's `effort`) or set by a floor is never raised. `/route show` and the route tool's answer print the level with `(opus stands in for fable)`; the verbose transition line names `stand-in` as a cause.
|
||||
- Main loop: every request gets the route in force. A typed skill with a row routes the main loop to it; a best-tier row (`plan`, `reflect`, `orchestrate`, `escalate`) holds across turns in a run slot until `/route clear`, `/route off`, a user `/model` or a typed skill on a non-best row. A skill without a row leaves the route as it is. A rowed skill the model loads inside a best-tier run moves effort only when its row ranks below the run's model. Main routed onto a model whose context window the conversation outgrew returns to the session model. Before main runs on a model, the one it moves to or the one it keeps (the session's own model included), the mod probes it with one small completion bounded to 8 seconds; the step (and `/route show`, the route tool's answer) waits for it. A timeout or an error marks the model down (reason `probe`, held for `cooldownMinutes` like any breaker hold) and the decision is taken again once. A model already marked down is never probed, and a model that answered is not probed again, `/clear` included, until it is marked down. When a lower model stands in for its tier's head held down (Opus for Fable on a best-tier phase), the phase's own effort goes one notch up: `reflect` xhigh, `plan` max, `orchestrate` high, `escalate` stays max. An effort asked explicitly (`/route ... effort=`, the `route` tool's `effort`) or set by a floor is never raised. `/route show` and the route tool's answer print the level with `(opus stands in for fable)`; the verbose transition line names `stand-in` as a cause.
|
||||
- Sub-agents: a routed agent gets its row's model at spawn, within its tier and only upward from its frontmatter model, and the row's effort on every step. Explicit `model` / `effort` params on the Agent call win; a project-defined agent of the same name keeps its own definition. Built-ins: Explore runs on sonnet/medium, Plan on opus/xhigh. A built-in `general-purpose` agent that main spawns with no `model` param, while main runs below the session model (a downgrade or a breaker fallback), starts on the session model instead of inheriting the lower one; it is not pinned when the session model is marked down, for a fork, a workflow or a nested spawn, or when a project or user definition shadows the built-in. Other agent types are never pinned this way: the hook cannot see their frontmatter model.
|
||||
- Levers inside a run: `ultrathink` in a prompt sets the turn's minimum effort; `/route effort=max` holds until `/route clear`. The built-in `/effort` is not a lever inside a run, rows and routes outrank it.
|
||||
- Prompt classifier: a plain prompt typed in the composer (12 characters or more, not a slash command) that no keyword rule, skill, run slot, `/route` or floor covers is sent to the engine's small fast model, one call, bounded to 3 seconds (the prompt waits for it, typically under a second). The label (`design`, `analysis`, `coding`, `bookkeeping`, `chat`) maps to plan, reflect, implement or apply (`chat` routes nothing), and the turn takes that phase's route: its effort becomes the turn's default, and its model may move the main loop, never above the session model. A `coding` or `bookkeeping` guess moves main down only when `mainModelSwitch` is on and the cold read fits `downgradeMaxTokens`; an `ultrathink` floor in the same turn keeps its own model. No dialog opens and no file is written. The route is gone at the end of the turn, a later route call or typed skill replaces it, and a `/effort` you type between turns drops the guess for that turn only. A typed best-tier skill holds its run slot across turns, so nothing is classified there until `/route clear`. Long prompts are sampled (first 500 and last 1500 code points), so pasted text at the end can steer one turn's route. Three failures in a row pause it for `cooldownMinutes` (one toast, `classifier: paused` in the status line). Never in a sub-agent, mid-turn or under `-p`. Off per session with `/route classify off`, per machine with `"classify": false`. The status line reads `route: classifier <phase>`; `/route verbose on` adds the transition line.
|
||||
@@ -141,9 +141,9 @@ effort, never version. Transcript audit `python3 lib/effort-audit.py`.
|
||||
- Only a dialog answer, `/route ask on|off` or `/route forget` writes `routing.json`, never the model. Writes are serialized, capped at 64 KiB, and refused when the file is missing (it is never created). Each write leaves the config repo dirty; a toast reminds you to commit it from there.
|
||||
- The spinner suffix and the status line under the prompt show the route in force; the status line adds ` · switch off` when the downgrade switch is off.
|
||||
|
||||
Config layers: `mods/model-router/routing.json` (tracked: phases with their tier, effort and `about` line, rows, decisions, `ask`), then the optional per-machine `~/.claude/model-router.json`, which wins. A `.claude/model-router.json` inside a project is never read. Machine keys: `models` (alias → full id), `windows` (context window per full id), `tiers` (ordered alias lists per tier: `best` fable>opus>sonnet, `big` opus>fable>sonnet, `work` sonnet>opus, `cheap` haiku>sonnet; the first alias not down is used), `fallback` (the rank order of the aliases, best first, used by the breaker), `cooldownMinutes` (how long a model stays marked down after an availability error, 15 by default, doubling per episode up to 300), `mainUpgrade` (default `true`: the main loop may move up to a phase's tier), `upgradeMaxTokens` (default 200000: the largest cold read a main-loop upgrade may cost), `downgradeMaxTokens` (default 120000: the same cap for a downgrade; the cold read is the context minus what the target model's prompt cache already holds, taken from main's last step on that model within the prompt-cache TTL, forgotten by `/clear` and `/route reload`. The TTL follows the engine's `cache_ttl` (5 or 60 minutes) carried by main's model-switch events, is 60 minutes until the first one and survives `/clear`. The context also counts the prompt just submitted, which the engine's usage may not include yet: code points / 3 of its text, its context entries and a typed skill's body, summed and added at the turn's first main step unless usage already grew by half of it. A `document` attachment makes the size unknown, and an unknown context size blocks the switch; the step, `/route show`, the first-use dialog and the toast read the same adjusted size), `phases`, `agents`, `skills`, `prompt` (rules), `mainModelSwitch` (default `true`: the main loop may move down to a cheaper model; a value that is not a boolean turns it off and says so), `classify` (default `true`; `false` stops the prompt classifier on that machine), `probe` (default `true`: a model main moves to is tried once, 8 seconds at most, before the move; `false` skips the probe on that machine; a non-boolean value is ignored and said; `/route show` prints `probe:`), `verbose` (default `false`; `true` logs transitions only, a non-boolean value falls back to `false`), `spinner` (default `true`), `enabled` (default `true`; `false` turns the mod off on that machine), `ask` (overrides routing.json's `ask` on that machine). A `null` value in `agents` or `skills` drops a row. Edit phases and rows by hand in routing.json (a phase's `about`, 120 characters at most, is read from there only and shown in the dialog; a model and effort pair no phase offers needs a new phase there); without it the mod runs the code's default phases, with no rows and no dialog. `/route reload` re-reads both files.
|
||||
Config layers: `mods/model-router/routing.json` (tracked: phases with their tier, effort and `about` line, rows, decisions, `ask`), then the optional per-machine `~/.claude/model-router.json`, which wins. A `.claude/model-router.json` inside a project is never read. Machine keys: `models` (alias → full id), `windows` (context window per full id), `tiers` (ordered alias lists per tier: `best` fable>opus>sonnet, `big` opus>fable>sonnet, `work` sonnet>opus, `cheap` haiku>sonnet; the first alias not down is used), `fallback` (the rank order of the aliases, best first, used by the breaker), `cooldownMinutes` (how long a model stays marked down after an availability error, 15 by default, doubling per episode up to 300), `mainUpgrade` (default `true`: the main loop may move up to a phase's tier), `upgradeMaxTokens` (default 200000: the largest cold read a main-loop upgrade may cost), `downgradeMaxTokens` (default 120000: the same cap for a downgrade; the cold read is the context minus what the target model's prompt cache already holds, taken from main's last step on that model within the prompt-cache TTL, forgotten by `/clear` and `/route reload`. The TTL follows the engine's `cache_ttl` (5 or 60 minutes) carried by main's model-switch events, is 60 minutes until the first one and survives `/clear`. The context also counts the prompt just submitted, which the engine's usage may not include yet: code points / 3 of its text, its context entries and a typed skill's body, summed and added at the turn's first main step unless usage already grew by half of it. A `document` attachment makes the size unknown, and an unknown context size blocks the switch; the step, `/route show`, the first-use dialog and the toast read the same adjusted size), `phases`, `agents`, `skills`, `prompt` (rules), `mainModelSwitch` (default `true`: the main loop may move down to a cheaper model; a value that is not a boolean turns it off and says so), `classify` (default `true`; `false` stops the prompt classifier on that machine), `probe` (default `true`: the model main runs on, moved to or kept, is tried once, 8 seconds at most, before main is sent to it; `false` skips the probe on that machine; a non-boolean value is ignored and said; `/route show` prints `probe:`), `verbose` (default `false`; `true` logs transitions only, a non-boolean value falls back to `false`), `spinner` (default `true`), `enabled` (default `true`; `false` turns the mod off on that machine), `ask` (overrides routing.json's `ask` on that machine). A `null` value in `agents` or `skills` drops a row. Edit phases and rows by hand in routing.json (a phase's `about`, 120 characters at most, is read from there only and shown in the dialog; a model and effort pair no phase offers needs a new phase there); without it the mod runs the code's default phases, with no rows and no dialog. `/route reload` re-reads both files.
|
||||
|
||||
Limits: a main-loop switch re-reads cold the part of the context the target model's cache does not hold; a switch whose cold read is over its cap (`upgradeMaxTokens` up, `downgradeMaxTokens` down) is skipped and main keeps its model, and a downgrade also needs `mainModelSwitch`. The prompt estimate is conservative (3 code points per token), so a long pasted prompt can hold back a switch that would have fit. The hooks send full model ids, so the `models` table has to follow new model versions. A probe costs one small completion per model main moves to, and holds that step up to 8 seconds when the model does not answer.
|
||||
Limits: a main-loop switch re-reads cold the part of the context the target model's cache does not hold; a switch whose cold read is over its cap (`upgradeMaxTokens` up, `downgradeMaxTokens` down) is skipped and main keeps its model, and a downgrade also needs `mainModelSwitch`. The prompt estimate is conservative (3 code points per token), so a long pasted prompt can hold back a switch that would have fit. The hooks send full model ids, so the `models` table has to follow new model versions. A probe costs one small completion per model main runs on, the session's own model included, repeated only after that model is marked down, and holds that step up to 8 seconds when the model does not answer.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -187,7 +187,7 @@ l'outil `mcp__model-router__route` (`lib/effort-shift.md`). Les skills
|
||||
externes vendorés (pile design, superpowers, agent-skills, skills scroll
|
||||
MengTo, 21st) ont aussi leur ligne.
|
||||
|
||||
Taper un skill qui a une ligne route la boucle principale dessus. Une ligne du tier best (plan, reflect, orchestrate, escalate) tient d'un tour à l'autre pendant tout le run, jusqu'à `/route clear`, `/route off`, un `/model` tapé ou un skill d'un autre tier tapé. Pour relancer un tour bloqué : `ultrathink` dans le prompt (plancher du tour) ou `/route effort=max` (tient jusqu'à `/route clear`). Le `/effort` intégré n'a pas d'effet dans un run : les lignes et les routes passent devant. Un prompt tapé sans règle, skill ni route en cours passe par un classifieur (un appel au petit modèle, 3 secondes au plus) : le tour prend la route de la phase devinée : son niveau d'effort, et son modèle, jamais au-dessus de celui de la session. Une devinette `coding` ou `bookkeeping` fait descendre la boucle principale vers un modèle moins cher quand `mainModelSwitch` est actif (par défaut) et que la relecture à froid tient sous `downgradeMaxTokens` (120000 tokens par défaut). Cette relecture compte aussi le prompt qui vient d'être envoyé (un token pour 3 caractères), et un document joint bloque la descente pour cette étape. Le cache compte comme chaud pendant le TTL du moteur (`cache_ttl`, 5 ou 60 minutes), 60 minutes par défaut. `/route classify off` ou `"classify": false` dans la config machine le coupe, et un `/effort` tapé entre deux tours annule la devinette pour ce tour seulement. Avant de déplacer la boucle principale vers un autre modèle, le mod l'essaie une fois (un appel minimal, 8 secondes au plus) : sans réponse, ce modèle est marqué en panne (raison `probe`) et la décision est reprise une fois ; `"probe": false` dans la config machine coupe cet essai, et `/route show` affiche `probe:`. Quand un modèle inférieur remplace la tête de son tier en panne (Opus pour Fable), le niveau propre de la phase monte d'un cran (reflect xhigh, plan max, orchestrate high) ; un niveau demandé explicitement ou un plancher ne bouge jamais. Les sous-agents reçoivent le modèle de leur ligne au lancement (dans leur tier, jamais en dessous de leur frontmatter) et son niveau à chaque étape ; un `model` ou `effort` explicite sur l'appel gagne. Explore tourne en sonnet/medium, Plan en opus/xhigh. Un agent intégré `general-purpose` lancé par la boucle principale sans `model`, pendant que celle-ci tourne sous le modèle de la session (descente, ou repli quand un modèle est en panne), part sur le modèle de la session au lieu d'hériter du modèle inférieur ; les autres types d'agents ne sont jamais épinglés ainsi. `/route show` affiche la route en cours, slot de run compris (`main: run <phase>`), et le switch avec son plafond et ce TTL (`switch: on (downgrade, cap 120000 tokens cold, ttl 60 min)`). Première utilisation : la première fois qu'un skill avec ligne est tapé, qu'un agent avec ligne est lancé ou qu'une phase est déclarée sur la boucle principale via l'outil `route`, le mod demande une fois si la route convient. La question donne le contexte : la description du skill (première phrase du frontmatter de son `SKILL.md`) ou celle de l'agent, la phase avec sa ligne `about` (champ de la phase dans `routing.json`), puis le modèle et le niveau réels de l'étape suivante ; une phase déclarée dit aussi qui l'utilise. Pour une ligne : Later, Keep ou Change ; pour une phase déclarée : Later ou Keep. Change demande le modèle (fable, opus, sonnet, haiku, chacun avec son tier), puis le niveau parmi ceux des phases de ce modèle (fable : medium, high, xhigh ou max ; sonnet : low, medium, high ou xhigh ; opus et haiku n'en ont qu'un, la question est sautée), puis Everywhere ou This project only (exception rangée sous le remote origin du dépôt réduit à `host/path`, jamais d'identifiants). Une ligne reste un nom de phase : le couple modèle et niveau désigne une phase existante, et la même phase qu'avant vaut Keep. Un couple qu'aucune phase n'offre s'ajoute à la main comme nouvelle phase dans `routing.json`. Après un changement sur un skill, un toast donne le modèle et le niveau réels, et rappelle `/route switch on` quand le switch est coupé, ou nomme le plafond quand la relecture à froid bloque la descente. La réponse est écrite dans `routing.json` ; Later, un texte libre (Other) ou un dialogue fermé redemande à une session suivante. Un seul dialogue à la fois, jamais en headless (`-p`) ni dans un sous-agent, et le modèle n'écrit jamais ce fichier. Chaque réponse laisse le repo de config modifié : commite-le depuis ce repo. `/route pending` liste ce qui reste à confirmer, `/route ask off|on` coupe ou rallume le dialogue, `/route forget <nom|all|projects>` efface une décision (une ligne changée revient à la phase livrée ; `all` et `projects` demandent d'abord).
|
||||
Taper un skill qui a une ligne route la boucle principale dessus. Une ligne du tier best (plan, reflect, orchestrate, escalate) tient d'un tour à l'autre pendant tout le run, jusqu'à `/route clear`, `/route off`, un `/model` tapé ou un skill d'un autre tier tapé. Pour relancer un tour bloqué : `ultrathink` dans le prompt (plancher du tour) ou `/route effort=max` (tient jusqu'à `/route clear`). Le `/effort` intégré n'a pas d'effet dans un run : les lignes et les routes passent devant. Un prompt tapé sans règle, skill ni route en cours passe par un classifieur (un appel au petit modèle, 3 secondes au plus) : le tour prend la route de la phase devinée : son niveau d'effort, et son modèle, jamais au-dessus de celui de la session. Une devinette `coding` ou `bookkeeping` fait descendre la boucle principale vers un modèle moins cher quand `mainModelSwitch` est actif (par défaut) et que la relecture à froid tient sous `downgradeMaxTokens` (120000 tokens par défaut). Cette relecture compte aussi le prompt qui vient d'être envoyé (un token pour 3 caractères), et un document joint bloque la descente pour cette étape. Le cache compte comme chaud pendant le TTL du moteur (`cache_ttl`, 5 ou 60 minutes), 60 minutes par défaut. `/route classify off` ou `"classify": false` dans la config machine le coupe, et un `/effort` tapé entre deux tours annule la devinette pour ce tour seulement. Avant d'envoyer la boucle principale sur un modèle, celui vers lequel elle se déplace comme celui qu'elle garde (le modèle de la session compris), le mod l'essaie une fois (un appel minimal, 8 secondes au plus) : sans réponse, ce modèle est marqué en panne (raison `probe`) et la décision est reprise une fois ; `"probe": false` dans la config machine coupe cet essai, et `/route show` affiche `probe:`. Quand un modèle inférieur remplace la tête de son tier en panne (Opus pour Fable), le niveau propre de la phase monte d'un cran (reflect xhigh, plan max, orchestrate high) ; un niveau demandé explicitement ou un plancher ne bouge jamais. Les sous-agents reçoivent le modèle de leur ligne au lancement (dans leur tier, jamais en dessous de leur frontmatter) et son niveau à chaque étape ; un `model` ou `effort` explicite sur l'appel gagne. Explore tourne en sonnet/medium, Plan en opus/xhigh. Un agent intégré `general-purpose` lancé par la boucle principale sans `model`, pendant que celle-ci tourne sous le modèle de la session (descente, ou repli quand un modèle est en panne), part sur le modèle de la session au lieu d'hériter du modèle inférieur ; les autres types d'agents ne sont jamais épinglés ainsi. `/route show` affiche la route en cours, slot de run compris (`main: run <phase>`), et le switch avec son plafond et ce TTL (`switch: on (downgrade, cap 120000 tokens cold, ttl 60 min)`). Première utilisation : la première fois qu'un skill avec ligne est tapé, qu'un agent avec ligne est lancé ou qu'une phase est déclarée sur la boucle principale via l'outil `route`, le mod demande une fois si la route convient. La question donne le contexte : la description du skill (première phrase du frontmatter de son `SKILL.md`) ou celle de l'agent, la phase avec sa ligne `about` (champ de la phase dans `routing.json`), puis le modèle et le niveau réels de l'étape suivante ; une phase déclarée dit aussi qui l'utilise. Pour une ligne : Later, Keep ou Change ; pour une phase déclarée : Later ou Keep. Change demande le modèle (fable, opus, sonnet, haiku, chacun avec son tier), puis le niveau parmi ceux des phases de ce modèle (fable : medium, high, xhigh ou max ; sonnet : low, medium, high ou xhigh ; opus et haiku n'en ont qu'un, la question est sautée), puis Everywhere ou This project only (exception rangée sous le remote origin du dépôt réduit à `host/path`, jamais d'identifiants). Une ligne reste un nom de phase : le couple modèle et niveau désigne une phase existante, et la même phase qu'avant vaut Keep. Un couple qu'aucune phase n'offre s'ajoute à la main comme nouvelle phase dans `routing.json`. Après un changement sur un skill, un toast donne le modèle et le niveau réels, et rappelle `/route switch on` quand le switch est coupé, ou nomme le plafond quand la relecture à froid bloque la descente. La réponse est écrite dans `routing.json` ; Later, un texte libre (Other) ou un dialogue fermé redemande à une session suivante. Un seul dialogue à la fois, jamais en headless (`-p`) ni dans un sous-agent, et le modèle n'écrit jamais ce fichier. Chaque réponse laisse le repo de config modifié : commite-le depuis ce repo. `/route pending` liste ce qui reste à confirmer, `/route ask off|on` coupe ou rallume le dialogue, `/route forget <nom|all|projects>` efface une décision (une ligne changée revient à la phase livrée ; `all` et `projects` demandent d'abord).
|
||||
|
||||
La config par machine, optionnelle, vit dans `~/.claude/model-router.json` et passe devant `routing.json` ; `"enabled": false` y coupe le mod sur cette machine, `"ask": false` y coupe le dialogue, une ligne à `null` y retire une ligne. Un `.claude/model-router.json` dans un projet n'est jamais lu. Les phases et les lignes se modifient à la main dans `routing.json` ; `/route reload` relit les deux fichiers.
|
||||
|
||||
|
||||
@@ -236,7 +236,7 @@ async function boot(
|
||||
on('classic.PostModelSwitch', () => ({}))
|
||||
on('skill.prompt', ($, e) => ({ text: e.text }))
|
||||
on('agent.offer', () => ({ isOffered: true }))
|
||||
// the probe of a move target: every model answers unless a test says not
|
||||
// the probe of the model main runs on: every model answers unless a test says not
|
||||
on('model.complete', async (_$, e) => {
|
||||
world.completions.push(e.model)
|
||||
return { value: (await world.complete(e.model)) as string }
|
||||
@@ -5504,7 +5504,7 @@ test('ttl: show says 60 min by default and 5 min after a 5m event', async (
|
||||
expect(await route($, 'show')).toContain('ttl 5 min)')
|
||||
})
|
||||
|
||||
// ---- probe: a move target answers before main is sent to it --------------
|
||||
// ---- probe: the model main runs on answers before main is sent -----------
|
||||
|
||||
const PROBE_MS = 8000
|
||||
const FABLE_PLAN = 'plan the billing migration'
|
||||
@@ -5578,7 +5578,8 @@ test('probe: a lapsed hold probes again and holds longer', async ($, on) => {
|
||||
await endTurn($)
|
||||
await rig.clock.advance(wait * 60000)
|
||||
}
|
||||
expect(world.completions).toEqual([FABLE, FABLE])
|
||||
// the fallback opus is verified by its own probe the first time only
|
||||
expect(world.completions).toEqual([FABLE, OPUS, FABLE])
|
||||
expect(rig.seen.map(s => s.model)).toEqual([OPUS, OPUS])
|
||||
})
|
||||
|
||||
@@ -5588,7 +5589,7 @@ test('probe: a rejection marks the target down like a timeout', async (
|
||||
const rig = await bootRig($, on, OPUS, 1000, world)
|
||||
await planRoute($)
|
||||
expect((await stepOn($, rig, OPUS)).model).toBe(OPUS)
|
||||
expect(world.completions).toEqual([FABLE])
|
||||
expect(world.completions).toEqual([FABLE, OPUS])
|
||||
expect(downLine(await route($, 'show'))).toContain('(probe)')
|
||||
})
|
||||
|
||||
@@ -5646,7 +5647,7 @@ test('probe: no probe when the classifier guess is above the session', async (
|
||||
})
|
||||
await typed($, PLAIN)
|
||||
await runStep($, { ...highStep(), model: SONNET })
|
||||
expect(world.completions).toEqual([])
|
||||
expect(world.completions).toEqual([SONNET])
|
||||
expect(c.seen[c.seen.length - 1]?.model).toBe(SONNET)
|
||||
})
|
||||
|
||||
@@ -5656,15 +5657,47 @@ test('probe: no probe when the upgrade switch refuses the move', async (
|
||||
const rig = await bootRig($, on, OPUS, 1000, world)
|
||||
await planRoute($)
|
||||
expect((await stepOn($, rig, OPUS)).model).toBe(OPUS)
|
||||
expect(world.completions).toEqual([])
|
||||
expect(world.completions).toEqual([OPUS])
|
||||
})
|
||||
|
||||
test('probe: the current model is never probed', async ($, on) => {
|
||||
test('probe: a session model that never answers leaves for opus on the first step', async (
|
||||
$, on) => {
|
||||
const world = probeWorld(hangsOn(FABLE))
|
||||
const rig = await bootRig($, on, FABLE, 1000, world)
|
||||
await typed($, FABLE_PLAN)
|
||||
await pastProbe(rig, runStep($, { ...highStep(), model: FABLE }))
|
||||
expect(rig.seen).toEqual([{ model: OPUS, effort: 'max' }])
|
||||
expect(world.completions).toEqual([FABLE])
|
||||
expect(downLine(await route($, 'show'))).toContain(
|
||||
`${FABLE} for 15 min (probe)`)
|
||||
})
|
||||
|
||||
test('probe: a healthy session model is probed once and kept', async (
|
||||
$, on) => {
|
||||
const world = newWorld()
|
||||
const rig = await bootRig($, on, FABLE, 1000, world)
|
||||
await planRoute($)
|
||||
expect((await stepOn($, rig, FABLE)).model).toBe(FABLE)
|
||||
expect(world.completions).toEqual([])
|
||||
await endTurn($)
|
||||
await planRoute($)
|
||||
expect((await stepOn($, rig, FABLE)).model).toBe(FABLE)
|
||||
expect(world.completions).toEqual([FABLE])
|
||||
})
|
||||
|
||||
test('probe: a lapsed hold on the session model probes it again', async (
|
||||
$, on) => {
|
||||
const world = probeWorld(hangsOn(FABLE))
|
||||
const rig = await bootRig($, on, FABLE, 1000, world)
|
||||
for (const wait of [15, 30]) {
|
||||
await typed($, FABLE_PLAN)
|
||||
await pastProbe(rig, runStep($, { ...highStep(), model: FABLE }))
|
||||
expect(downLine(await route($, 'show'))).toContain(`for ${wait} min`)
|
||||
await endTurn($)
|
||||
await rig.clock.advance(wait * 60000)
|
||||
}
|
||||
// opus, the fallback, answers its own probe once
|
||||
expect(world.completions).toEqual([FABLE, OPUS, FABLE])
|
||||
expect(rig.seen.map(s => s.model)).toEqual([OPUS, OPUS])
|
||||
})
|
||||
|
||||
test('probe: false in the override makes no completion at all', async (
|
||||
|
||||
@@ -34,7 +34,7 @@ type Config = {
|
||||
mainModelSwitch: boolean // gates DOWNGRADES only
|
||||
verbose: boolean
|
||||
classify: boolean // a small-model guess of the typed prompt's phase
|
||||
probe: boolean // a move target is tried once before main is sent to it
|
||||
probe: boolean // the model main would run on is tried once first
|
||||
spinner: boolean
|
||||
enabled: boolean // false: every hook passes through (per machine)
|
||||
}
|
||||
@@ -234,7 +234,7 @@ const LABEL_PHASE: Readonly<Record<string, string>> = {
|
||||
}
|
||||
const CLASSIFY_MIN = 12 // code points of a typed prompt worth a call
|
||||
const CLASSIFY_MS = 3000 // the call is abandoned after this
|
||||
const PROBE_MS = 8000 // a move target that has not answered by then is down
|
||||
const PROBE_MS = 8000 // a probed model that has not answered by then is down
|
||||
const PROBE_TOKENS = 16 // the probe's reply cap
|
||||
const CLASSIFY_MAX = 2000 // code points sent whole; longer: head + tail
|
||||
const CLASSIFY_HEAD = 500
|
||||
@@ -1522,9 +1522,9 @@ async function probeAnswers($: Api, id: string): Promise<boolean> {
|
||||
}
|
||||
|
||||
/**
|
||||
* Tries a move target before main is sent to it. An answer verifies it,
|
||||
* unless a failure landed meanwhile (the hold wins); a timeout or a
|
||||
* rejection marks it down. True when the move may go ahead.
|
||||
* Tries the model main would run on before main is sent to it. An answer
|
||||
* verifies it, unless a failure landed meanwhile (the hold wins); a timeout
|
||||
* or a rejection marks it down. True when main may go ahead on it.
|
||||
*/
|
||||
async function probeModel($: Api, st: State, id: string): Promise<boolean> {
|
||||
if (await probeAnswers($, id)) {
|
||||
@@ -1535,9 +1535,9 @@ async function probeModel($: Api, st: State, id: string): Promise<boolean> {
|
||||
return !st.down.has(id)
|
||||
}
|
||||
|
||||
/** The table id a move lands on that still wants a probe, else undefined. */
|
||||
/** The table id the decision lands on (moved or kept) that wants a probe. */
|
||||
function unprobed(st: State, call: Call): string | undefined {
|
||||
if (!call.moved || !st.cfg.probe || st.off) return undefined
|
||||
if (!st.cfg.probe || st.off) return undefined
|
||||
const id = canonical(st.cfg, call.model)
|
||||
const fresh = !st.verified.has(id) && !st.down.has(id)
|
||||
return aliasOf(st.cfg, id) !== undefined && fresh ? id : undefined
|
||||
@@ -1545,7 +1545,7 @@ function unprobed(st: State, call: Call): string | undefined {
|
||||
|
||||
type Decided = { call: Call; wanted: string | undefined }
|
||||
|
||||
/** The decision, probing a move target once; a failed probe: decide again. */
|
||||
/** The decision, probing the model it lands on once; failure: decide again. */
|
||||
async function decideProbed(
|
||||
$: Api,
|
||||
st: State,
|
||||
@@ -1609,8 +1609,9 @@ function isStandIn(
|
||||
|
||||
/**
|
||||
* What the main loop would run on from `cur`: wanted model + decision. The
|
||||
* step passes the `known` reading it took; a text reads for itself. A move
|
||||
* target is probed here, after the decision (see `decideProbed`).
|
||||
* step passes the `known` reading it took; a text reads for itself. The
|
||||
* model main would run on is probed here, after the decision (see
|
||||
* `decideProbed`).
|
||||
*/
|
||||
async function decideFor(
|
||||
$: Api,
|
||||
|
||||
Reference in new issue
Block a user