feat(model-router): cold-read cap on main-loop downgrades, switch on by default, classifier may route down

The main loop may now move down to a cheaper model by default
(mainModelSwitch true, fail-closed picker). Both caps measure the COLD
READ a switch costs: the context minus what the target model's prompt
cache already holds, from a per-model served record written on every
main step (1 h TTL, cleared by /clear and /route reload; a smaller
context than the record counts as cold). downgradeMaxTokens defaults to
120000, upgradeMaxTokens keeps 200000; no return bypass is needed since
a warm return is cheap by construction.

The prompt classifier writes the full phase route: coding and
bookkeeping guesses move main down under the switch and the cap; a guess
never ranks above the session model; a mid-turn ultrathink floor keeps
its model; a user /effort typed between turns drops the guess for that
turn only (engine baseline is now per turn). A model-loaded helper skill
inside a best-tier run moves effort only. Main routed onto a model whose
window the context outgrew returns to the session model. The skip line
is said once per cap crossing; the toast names a cap hold; the status
line shows the switch only when off.
This commit is contained in:
bchanot committed 2026-10-13 01:13:06 +02:00
1 parent f3b47287f9
commit 2ce203a2da
3 files changed
+552 -107

No files matched your search

+6 -2
View File
@@ -47,8 +47,12 @@ Builtin `/effort` is NOT a lever inside a run: rows and routes outrank it.
- A skill typed while a background agent is live routes only through the
typed marker (unverified live 2026-10-10).
- Headless (`-p`, SDK) runs the hooks, so routing works there too.
- A plain typed prompt with no route in force gets its turn effort from a
small-model classifier (effort only; `/route classify off`; never `-p`).
- A plain typed prompt with no route in force is classified by a small
model: the guessed phase sets the turn's effort and, under
`mainModelSwitch` and `downgradeMaxTokens` (max cold read, default
120000), may move main DOWN, never up. A user `/model` only moves the
baseline the guess starts from (`/route classify off` to stop guessing;
never `-p`).
- Mod off: typed agents fall back to their `model:`/`effort:` frontmatter.
- First use of a row asks once, with context (Later, Keep or Change: model,
then effort, then Everywhere or this project only); the answer is kept in