chore(memory): BDR-116 amendment, LRN-216, EVAL-047, TODO + journal, router-downgrade-cap contract + plan
This commit is contained in:
1 parent
e8186409df
commit
ab963e97dc
7 files changed
+304
No files matched your search
@@ -1422,6 +1422,8 @@ Branch feature/user-writing-web-rules, UNMERGED (human gate).
|
||||
- **Gates**: plan r1 → r3 (simplicity CONCERNS(4), robustness FATAL(6) BLOCKER unbounded await, correctness CONCERNS(4); confirmation robustness CONCERNS(4)); feater DONE (423) + 1 round (trim, argumentHint, repeated undefined: 428, each mutation-proven); GATE 0 MET; verifier ECARTS(2) coverage → CONFORME on #2; security PASS (0 findings; LOW accepted: pasted text can steer one turn's effort up, README says so). Abort paths judged by reading (`[gated]` clarification). Residual: `smallModelBusy` checks `cfg.models.haiku` while classify uses the engine's default small model (mismatch only when the two differ). Live AC9 open.
|
||||
- **Refs**: contract `.claude/tasks/contracts/2026-10-11-model-router-w4b-classifier-1858.md`, plan r3 same slug. Links [[EVAL-045]], [[LRN-210]].
|
||||
|
||||
- **Amendment (2026-10-13, feature/router-downgrade-cap, commit 2ce203a, user decision)**: W4-B "EFFORT ONLY" clause superseded. Classifier writes full phase route; main may move DOWN on a guess, never above session model (`wantedModel` ranks vs `sessionModel`, not `cur`: breaker fallback sonnet→opus, never haiku); `ultrathink` floor model beats guess. `mainModelSwitch` default **true**, fail-closed picker (non-boolean → off + log). Caps measure COLD READ (LRN-216): `coldRead(M) = tokens − served[M].tokens` within `CACHE_TTL_MS` 1 h, else whole context; `downgradeMaxTokens` 120000 (user delegated value), `upgradeMaxTokens` 200000 same measure; served record written EVERY main step, dropped by `/clear` + `/route reload`, stale (context < record) = cold. No return bypass (warm return cheap by construction). Model-loaded helper skill inside best-tier run → effort only (`belowRun`). Window outgrown (haiku 200k) → back to session model. `/effort` typed between turns drops guess for that turn only (`engineBase` per turn). Rejected: user `/model` suspends classifier (baseline moves, guess follows; `/route classify off` pins) — trade-off accepted; raw context-size cap (r1: refused cheap re-down to warm sonnet, forced cold fable read); engine-model return bypass (premise false after N sonnet turns). Residuals TODO 2026-10-13: unrowed agent `parentModel` inheritance on demoted turn, `session.usage` lag vs new prompt, TTL const. Gates: 3 lenses CONCERNS (10 MAJOR) → r2; confirmation FATAL(7) BLOCKER → r3; feater DONE 450 tests; verifier CONFORME 10/10 first pass; security PASS. Links [[LRN-216]], [[EVAL-047]], [[LRN-204]].
|
||||
|
||||
## BDR-117 — shared gstack helpers live in lib/gstack-removed.sh, not in a new lib [accepted] (2026-10-12)
|
||||
- **Decision**: C5 /refactor 2026-10-12. profile.sh + toggle-external.sh duplicated gstack_skills + park/unpark moves. Host gstack_skills/gstack_parked_names/gstack_park/gstack_unpark in lib/gstack-removed.sh; each takes dirs as args, no caller globals; plain-statement call only (errexit, [[LRN-215]]).
|
||||
- **Why**: only sourced file 5 fixture suites copy next to both scripts; new lib = every fixture breaks.
|
||||
|
||||
@@ -425,3 +425,8 @@ Dogfood: 3 blind lenses attacked the v1 plan for the plan-challenge feature itse
|
||||
- **Method**: correctness FATAL(6), robustness FATAL(5), simplicity CONCERNS(5), confirmation CONCERNS(2). Independent re-check of the executor: `make test` 47 green, shellcheck 0, hook `$pattern` sha256 identical old/new, old vs new google_seo.py no-arg run byte-identical (stdout, stderr, rc).
|
||||
- **Anomaly**: r1 wrong on 5 points (directive placement, logic-line counter ignored the error-handling exclusion → 1 false positive + 3 reclassified, B18 half false: 2 of 4 sites already `local`, C1 move would silently void T19, `|| return $?` rule inverted). Executor denied once (`rm -rf` on a variable path), used a fresh dir, reported it. Scope file's BUG TO LOG block came out empty (regex cut at the heading) — executor fell back to the plan. Two golden gaps left open by executor (gstack-absent warning, pack_hints with stubbed CLI).
|
||||
- **Action**: keep the confirmation pass after a material rewrite (it caught 2 MAJOR the first round missed). Scope writer: assert every section non-empty before dispatch. Links [[LRN-213]], [[LRN-214]], [[LRN-215]].
|
||||
|
||||
## EVAL-047 — model-router downgrade cap: 3 lenses CONCERNS, confirmation pass FATAL(7); the 4th blind read found the BLOCKER the author + 3 lenses missed
|
||||
- **Method**: /feat. r1 (author): context-size cap + engine-model return bypass + classifier full route. 3 blind lenses on r1: simplicity CONCERNS(3), correctness CONCERNS(4), robustness CONCERNS(3), 10 MAJOR, 0 BLOCKER, convergent (cold-read measure, classifier upgrade leak, bypass useless, test list incomplete, helper-skill demotion, haiku window). r2 written. ONE confirmation challenger on r2: FATAL(7) — BLOCKER served record only on routed steps (session model's cache invisible), + 6 MAJOR (test (d) `turnModel` survives `/route clear`, guess vs `cur` + breaker → haiku, floor hidden by guess tier, `pickSwitch` fails OPEN, compaction floor-at-0, `engineBase` process-wide kill). r3 → feater DONE first pass, 450 tests, verifier CONFORME 10/10 first pass, security PASS.
|
||||
- **Anomalies**: author's r2 introduced 1 BLOCKER + 6 MAJOR while fixing r1 — a material rewrite is a NEW plan, the confirmation pass is not optional. `pickSwitch` "fail closed" was wrong by name: falls back to seeded default (now true). Verifier noticed 2 stale texts outside criteria (label comment, route tool description) — fixed by hand pre-commit (minor, justified). Doc-sync SIGNIFICANT: 13 patches, 3 docs stated the old default as a promise.
|
||||
- **Action**: keep the single confirmation pass mandatory after any r2 that changes the mechanism (not just wording); when a plan says "fail closed", name the picker and its fallback value in the plan; TODO residuals (parentModel inheritance, usage lag, TTL const) carried, not dropped. Links [[BDR-116]], [[LRN-216]], [[EVAL-045]].
|
||||
@@ -607,3 +607,6 @@ rules:
|
||||
- /code-clean whole repo on chore/code-clean (gitflow start, pushed). Plan r1→r3: 3 lenses + confirmation, C1 (gitflow-test.sh move) deferred ×2 lenses, B18 half false, C3 needed stderr oracle, errexit rule inverted. User approved all 14 non-deferred items. code-cleaner DONE: A1 import, B1-B4 lint/hook regex (sha identical), B5/B7/B9/B11 splits, B18-B20, C2 suite wired, C3 hoist. make test 47 green, shellcheck 0, verified independently. 1 bug logged (usage() 3-26 truncation). No commit. LRN-213/214/215, EVAL-046. Next (user: "capitalize all then correct bugs with what has been skipped"): usage() fix inline, C4/C5 via /refactor; C1 + B13-B17 left (blocked / no-change by design).
|
||||
- Follow-up (same branch, no commit): usage() bug fixed inline (`sed -n '3,/^# ====/p' | sed '$d'`, --help 36 lines, 14 tools). /refactor → refactorer: C4 `tool_kind`/`tool_source` single roster (exact-match loop, not the `*" $x "*` idiom: multi-word arg stayed unknown), `disable_*` split mirrors `enable_*`; C5 shared `gstack_skills/parked_names/park/unpark` in lib/gstack-removed.sh (already copied by all 5 fixtures; new sourced lib would break them). 496-run golden capture identical except `${name:?}` diagnostic line ref for an empty `gstack__` dir. Verified: make test 47 green, shellcheck 0. Left: C1 (T19 lock), B13-B17 (no change), `name` leak in enable_pack. Commit split suggested: cleanup / usage fix / C4-C5.
|
||||
- Rust default-language rule (user ask). `rules/default-language.md` always-on (no `paths:`), CLAUDE.global.md at 317/320 → BDR-085 exception path. Hooks: interviewer FAILURE MODES row, init-project BDR table, project CLAUDE.md exception list. Branch chore/rust-default, doctrine-citers green. BDR-118 pending user approval.
|
||||
|
||||
## 2026-10-13
|
||||
- model-router /feat `downgradeMaxTokens` (user: switch on by default, classifier may switch model early, back up to fable on reflection). Plan r1 context-size cap → challenge: cap must measure COLD READ (LRN-204), bypass dropped, classifier down-only vs session model; confirmation pass FATAL(7), BLOCKER served-record-on-routed-steps-only → r3. feater DONE 450 tests, verifier CONFORME 10/10, security PASS, commit 2ce203a on feature/router-downgrade-cap. Doc-sync + BDR/LRN/EVAL pending.
|
||||
@@ -1801,3 +1801,7 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
|
||||
## LRN-215 — helper extracted from a `set -e` function must be called as a plain statement; `helper || …` / `if helper` switches errexit OFF inside it
|
||||
- **Context**: code-clean B5/B7/B9 split enable_tool / enable_skill / cmd_show out of `set -euo pipefail` scripts. r2 executor rule said `|| return $?`; confirmation lens: that form disables errexit inside the helper, so a failed `mv` (toggle-external.sh:273) would print `ok …` rc 0 instead of aborting — a path no golden capture sees. Plain call keeps `return 1` → errexit in caller → exit 1 as today.
|
||||
- **Apply**: split rule for -e scripts: plain calls only, never `||`, `&&`, `if`, `!` around an extracted helper. Golden capture (stdout + stderr + rc + fixture tree) before/after for every moved arm when the suite discards output. Links [[LRN-214]], [[EVAL-046]].
|
||||
|
||||
## LRN-216 — Capping a main-loop model switch = cap the COLD READ (context − target's warm prefix), never the context size; the warm record must be written on EVERY main step
|
||||
- **Context**: model-router downgrade cap 2026-10-13. r1 compared context size to cap → after N coding turns sonnet warm, fable cold; context crosses cap → next coding turn starts on fable (cold) and cheap re-down refused = worst outcome. r2 fixed measure, but wrote `served` only on routed steps → unrouted steps on session model invisible → return to fable refused above 200k (confirmation pass BLOCKER). Per-model prefix cache (LRN-204): cost of switch INTO M = tokens − tokens at M's last step if within TTL, else all; context smaller than record = history rewritten (compaction) = cold.
|
||||
- **Apply**: any "switch model when cheap" logic: (1) keep `{at, tokens}` per model id, written at every step of the loop whatever the routing; (2) cost = delta vs that record within TTL, whole context otherwise or when record > current; (3) unknown size = blocked; (4) no special-case "return to session model" bypass, the measure covers it. A guess-sourced route ranks vs the SESSION model, not the current one, or a breaker fallback walks down to the cheapest tier. Links [[BDR-116]], [[LRN-204]], [[EVAL-047]].
|
||||
@@ -1,5 +1,11 @@
|
||||
# TODO
|
||||
|
||||
## 2026-10-13 — model-router: cold-read cap, switch on by default, classifier routes down (feature/router-downgrade-cap)
|
||||
Contract `.claude/tasks/contracts/2026-10-13-router-downgrade-cap-0013.md`, plan r3 `.claude/tasks/plans/2026-10-13-router-downgrade-cap-0013.md`.
|
||||
- [x] /feat run: 3 lenses (10 MAJOR) → r2; confirmation pass FATAL(7) (BLOCKER: served record only on routed steps) → r3; feater DONE (19 tests + 3 extra, 450 pass), GATE 0 MET 4/4, verifier CONFORME 10/10 first pass, security PASS. Commit 2ce203a.
|
||||
- [ ] Residuals (accepted, from the challenge): (1) unrowed sub-agent with no explicit model spawned during a demoted turn may inherit the demoted `parentModel` (robustness #10): all repo agents rowed, built-ins carry `model=`; settle with a spawn test if a built-in ever drops its pin. (2) `session.usage().context.tokens` may lag the new prompt (robustness #11): a big paste can slip under the cap; economic guard, not a hard limit. (3) `CACHE_TTL_MS` = 60 min is this account's TTL assumed as a const; the engine's `PostModelSwitch.cache_ttl` field suggests it varies — adopt it when observed. (4) Observed during the run: `route` tool answered "model session model (model unknown to the table)" after settings.json `model` turned to `opus` (written by another session) — check `/route show` in a fresh session.
|
||||
- [ ] Live check (next sessions): a coding prompt on fable → spinner `sonnet-5-5/medium`; a reflect turn after → fable; above 120k cold read the skip line once; `/route show` cap text.
|
||||
|
||||
## 2026-10-12 — code-clean whole repo (chore/code-clean)
|
||||
- [x] audit r1→r3 (3 lenses + confirmation), user approved all 14 non-deferred items, code-cleaner DONE, make test 47 green — plan `.claude/tasks/plans/2026-10-12-code-clean-0103.md`, scope `.claude/audits/CODE-CLEAN-SCOPE.md`
|
||||
- [x] fix BUGS-FOUND: toggle-external.sh usage() prints header 3-26, list ends at 32
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# CONTRACT — router-downgrade-cap
|
||||
- date: 2026-10-13 | flow: feat | branch: feature/router-downgrade-cap (to start off develop)
|
||||
- status: active
|
||||
|
||||
## REQUEST (verbatim — IMMUTABLE)
|
||||
User (2026-10-13, after the explanation of why `mainModelSwitch` is off):
|
||||
"on va partir sur 2 on va fiare un feat maintenant comme ca c'est bon. JE te laisse juger le seuil de token a metre en place, sachant que capourrait switcher plusieurs fois dnas la meme conversation. Dison que si je suis sur fable et que l'interprete du prompt estime que c'est un autre model il switch de model tot. Et si il y a de nouveau de la reflexion alors on switch plus haut genre fable sans forcemnt re down si on ateint la limite de token. bref je te laisse faire ca"
|
||||
|
||||
Option 2 as proposed by Claude: "Un `/feat` qui ajoute `downgradeMaxTokens` au mod et passe le switch à on par défaut sous ce plafond."
|
||||
|
||||
/feat arguments: model-router: downgrade cap. Add `downgradeMaxTokens` to the mod (symmetric to `upgradeMaxTokens`): a main-loop downgrade is allowed only while the context is under the cap; above it main keeps its model (no re-down once the limit is reached, upgrades to best still allowed). Flip `mainModelSwitch` default to on under that cap. User also wants the prompt classifier to be allowed to move the main model down (today effort-only by design, W4-B), with the cap as the guard. Threshold value left to Claude's judgment (several switches per conversation expected).
|
||||
|
||||
## CLARIFICATIONS
|
||||
Q: default value of `downgradeMaxTokens` / A: delegated — 120000 tokens of COLD READ (LRN-204; r2: measured against the target model's warm prefix, not the context size); per-machine override in `~/.claude/model-router.json`
|
||||
Q: does a return to the engine's own model obey the upgrade cap? / A: delegated — r1 said "bypass"; r2 (challenge, correctness #1): no bypass, both caps measure the COLD READ of the target (LRN-204), so a warm return passes by itself and a cold one is capped like any switch
|
||||
Q: does the classifier also move main UP (design/analysis → best tier)? / A: delegated — no, downgrades only; prompt rules keep upgrading as today [r2, correctness #2]
|
||||
Q: does a user `/model` suspend the classifier's model axis? / A: delegated — no; the baseline moves and the guess routes from it; `/route classify off` or `/route off` pins [r2, correctness #3, trade-off to the BDR]
|
||||
Q: when a user `/effort` beats the classifier's effort guess (`userMovedEffort`), does the model guess still apply? / A: delegated — no: the whole classifier route yields for that step, both axes
|
||||
Q: status line display once the switch defaults to on / A: [gated 2026-10-13] show ` · switch off` only when the switch is off; the ` · switch on` fragment is dropped (on is the norm)
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. `Config` carries `downgradeMaxTokens` (positive number, default 120000), merged like `upgradeMaxTokens` (`pickPositive`, user override wins, invalid value logged and ignored).
|
||||
CHECK: cd mods/model-router && grep -q "downgradeMaxTokens: 120000" hooks/register.ts && grep -q "user.downgradeMaxTokens, base.downgradeMaxTokens, 'downgradeMaxTokens'" hooks/register.ts && echo CRIT1-OK
|
||||
EXPECT: CRIT1-OK
|
||||
EVIDENCE: MET exit=0 marker-found :: CRIT1-OK
|
||||
2. `mainModelSwitch` defaults to `true` in `DEFAULT_CONFIG` and is merged FAIL-CLOSED (a non-boolean override is logged and reads as `false`; it never enables anything).
|
||||
CHECK: cd mods/model-router && grep -q "mainModelSwitch: true," hooks/register.ts && grep -q 'config "mainModelSwitch" is not a boolean' hooks/register.ts && echo CRIT2-OK
|
||||
EXPECT: CRIT2-OK
|
||||
EVIDENCE: MET exit=0 marker-found :: CRIT2-OK
|
||||
3. Both caps measure the COLD READ of the target model (context tokens minus the tokens at that model's last main step when it served within `CACHE_TTL_MS` = 1 h; the whole context when never served, expired, or when the context is SMALLER than the record (compaction: entry dropped); unknown size = blocked): `upgradeCall` and `leaveDown` against `upgradeMaxTokens`, `downgradeCall` against `downgradeMaxTokens`, through ONE helper (`coldBlock`). The served record is written on EVERY main step (routed or not), kept across turn ends, NOT kept across `/clear`, reset by `/route reload`. A downgrade refusal keeps `switch off` first, then logs `downgrade skipped: cold read N tokens over M` / `downgrade skipped: context size unknown` once per cap crossing (until a downgrade next succeeds). No "return bypass" exists.
|
||||
4. In `decideMain`, main routed onto a model whose known window (`windows`) the context has outgrown returns to the session model (once-log) when the size is known and the session model is a known, ranked, not-down, non-haiku id; otherwise the normal decision runs. `moveTo` carries `[1m]` when `cur` OR the session model has it (haiku excepted).
|
||||
5. The prompt classifier writes the full phase route (`tier` + `effort`); the model name comes from the first slot carrying one, and when that slot is the classifier's: a floor with a model wins over it, and a wanted that ranks above the SESSION model is dropped (a guess never moves main above the user's baseline; a breaker fallback from a guessed sonnet reaches opus, never haiku). So `coding`/`bookkeeping` move main down under the switch and the cold-read cap while `design`/`analysis` only set effort; `chat`/unknown route nothing. `userMovedEffort` compares this turn's first engine effort with the previous turn's; when true, `mainPlan` drops a classifier route for THIS turn only (both axes, `turnModel` untouched, texts agree); the next classified turn routes again. A user `/model` is NOT a guard (baseline moves). In-file comments at the classifier (~l.3438-3441, 3535, 3546) state the new rule.
|
||||
6. A model-loaded (not typed) skill row whose resolved model ranks below the run slot's resolved model moves effort only while `runMain` is set; typed skills and best-tier rows unchanged.
|
||||
7. `/route show` prints `switch: on (downgrade, cap 120000 tokens cold)` / `switch: off (downgrade)`; the status line shows ` · switch off` only when the switch is off [gated 2026-10-13]; the toast hint names a cap hold (`, held: <why>`) as well as the switch.
|
||||
8. Tests in `hooks/register.test.ts`: (a) downgrade under the cold-read cap by default; (b) above the cap keeps main, one log; (c) unknown usage blocks; (d) warm model cheap to return to, with a control (cold opus blocked at the same size); (e) classifier coding → sonnet/medium; (f) classifier coding above the cap → fable/medium; (g) `switch off` restores today's behavior; (h) user `/effort` drops the model guess, `/route show` agrees; (i) design guess never moves a sonnet session up; (j) model-loaded helper skill in a run moves effort only; (k) haiku outgrown returns to the session model; (l) quoted "false" switch reads as off with a warning; (m) cap hold toast hint; (n) TTL expiry; (o) reload forgets the record + skip line once per cap crossing across turns; (p) smaller context than the record is cold; (q) mid-turn ultrathink floor beats a guess; (r) guessed sonnet rate-limited falls to opus never haiku; (s) a user /effort drops the guess for one turn only. The tests the default flip invalidates (plan § Tests list, incl. l.305-315, 605-614, 747, 754, 951-958, 1665-1672, 2934-2943, 4576, 4783-4799, status-line `switch on` assertions) are re-expressed with an explicit `switch off`, never weakened or deleted; a mutable-usage rig uses ONE bottom `session.usage` hook.
|
||||
9. The whole mod test suite passes.
|
||||
CHECK: make test suite=lib/tests/mods.test.sh 2>&1 | tail -1
|
||||
EXPECT: all suites green
|
||||
EVIDENCE: MET exit=0 marker-found :: all suites green
|
||||
10. `lib/effort-shift.md` Limits bullet about the classifier says the guess may move the model DOWN under `mainModelSwitch` and `downgradeMaxTokens`, never up, and that a user `/model` moves the baseline; no longer "effort only".
|
||||
CHECK: ! grep -q "classifier (effort only;" lib/effort-shift.md && grep -q "downgradeMaxTokens" lib/effort-shift.md && echo CRIT10-OK
|
||||
EXPECT: CRIT10-OK
|
||||
EVIDENCE: MET exit=0 marker-found :: CRIT10-OK
|
||||
|
||||
## FILE SCOPE
|
||||
- mods/model-router/hooks/register.ts
|
||||
- mods/model-router/hooks/register.test.ts
|
||||
- lib/effort-shift.md
|
||||
(README.md l.144/146 and USAGE.md l.190 handled by STEP 6 doc-sync, out of the executor's scope)
|
||||
@@ -0,0 +1,235 @@
|
||||
# PLAN — router-downgrade-cap (feat, r3: r2 + confirmation pass FATAL(7) → 1 BLOCKER + 6 MAJOR closed by the named changes below)
|
||||
Contract: .claude/tasks/contracts/2026-10-13-router-downgrade-cap-0013.md
|
||||
|
||||
## Files
|
||||
1. mods/model-router/hooks/register.ts
|
||||
2. mods/model-router/hooks/register.test.ts
|
||||
3. lib/effort-shift.md
|
||||
|
||||
## Core change of r2 — the cap bounds the COLD READ, not the context size
|
||||
LRN-204: a switch INTO a model costs the tokens that model's prompt cache does not
|
||||
hold; within the 1 h TTL its cache holds the prefix up to its last main step. So the
|
||||
cost of moving main to model M at a context of T tokens is
|
||||
`coldRead(M) = T − servedTokens(M)` when M served main within the TTL, else `T`.
|
||||
r1 compared T itself to the cap, which (correctness #1) refused the cheap re-down to a
|
||||
warm Sonnet and forced a cold read on Fable instead. r2 compares `coldRead`.
|
||||
|
||||
## Approach
|
||||
- Config (`register.ts` Config type ~l.30-37, DEFAULT_CONFIG ~l.217-275, mergeRouting
|
||||
~l.512-537): `downgradeMaxTokens: number` next to `upgradeMaxTokens` in `Config`,
|
||||
`Routing` and `mergeRouting` (`pickPositive(user.downgradeMaxTokens,
|
||||
base.downgradeMaxTokens, 'downgradeMaxTokens', log)`); default `120000`, comment
|
||||
"max cold read a downgrade may cost (LRN-204)". `upgradeMaxTokens` comment becomes
|
||||
"max cold read an upgrade may cost". `mainModelSwitch: true`, merged FAIL-CLOSED like `pickEnabled` (~l.505-510): a
|
||||
non-boolean override logs `model-router: config "mainModelSwitch" is not a boolean;
|
||||
switch off` and reads as `false` [r3 #5: `pickSwitch` falls back to the seeded
|
||||
default, which is now `true`].
|
||||
- Served record (State ~l.80-100, `mainPlan` ~l.3052) [r3, BLOCKER #1 / #6 / #12]:
|
||||
`served: Map<string, { at: number; tokens: number }>` keyed by the canonical id main
|
||||
was sent (`plan.model`). `mainPlan` reads `tokens` (`readTokens`) and `now`
|
||||
(`$.clock.now()`) on EVERY main step, routed or not, and records
|
||||
`served.set(canonical(plan.model), { at: now, tokens })` after the decision, whatever
|
||||
the routing — an unrouted step on the session model is exactly the one that keeps
|
||||
that model warm. Texts (`snapshot`, `toastChange`) keep the lazy read. `served` is
|
||||
NOT in `resetSession`'s kept list (`/clear` starts cold) and is cleared by `/route
|
||||
reload` next to the breaker marks (~l.1099-1108). Const `CACHE_TTL_MS = 60 * 60000`
|
||||
next to `CLASSIFY_*`, comment: "the account's prompt-cache TTL (1 h here; an
|
||||
engine-reported `cache_ttl` would supersede it — TODO)". `decideFor($, st, cur,
|
||||
tokens?, now?)` takes the values when the caller has them and reads them itself
|
||||
otherwise; it returns `{ call, wanted }` as today. `capSaid` is set/cleared in
|
||||
`mainPlan` from the returned call (a text run never mutates it).
|
||||
Compaction / history rewrite [#6]: in `coldRead`, when `tokens < entry.tokens` the
|
||||
entry is stale → drop it and return `tokens` (cold).
|
||||
- One cap helper replaces `capBlock` (~l.1143-1147): `coldBlock(st, id, tokens, max,
|
||||
now)` → `undefined` when `tokens !== undefined && coldRead ≤ max`; else the reason
|
||||
`'context size unknown'` or `` `cold read ${cold} tokens over ${max}` ``.
|
||||
`coldRead(st, id, tokens, now)` = `tokens − entry.tokens` when `served.get(canonical
|
||||
id)` exists and `now − entry.at ≤ CACHE_TTL_MS` (floored at 0), else `tokens`.
|
||||
`decideFor` receives `tokens`/`now` from `mainPlan` (texts read them lazily, as today).
|
||||
- `upgradeCall` (~l.1177-1192): `coldBlock(…, st.cfg.upgradeMaxTokens)` in place of
|
||||
`capBlock`; same `keep(cur, why, 'model-router: upgrade skipped: …')`, `logKey
|
||||
'upgrade-skipped'`. `leaveDown` (~l.1160-1175) uses the same helper for `upOk`.
|
||||
NO return bypass (simplicity #1, correctness #1/#5, robustness #4-#6): a return to a
|
||||
warm model has a small cold read and passes by itself; turn end restores `e.model`.
|
||||
- `downgradeCall` (~l.1196-1205): `switch off` refusal first (text unchanged), then
|
||||
`coldBlock(…, st.cfg.downgradeMaxTokens)` → `skipped(cur, 'downgrade', block)` with
|
||||
`logKey 'downgrade-skipped'`, then `fits`, then `moveTo`. One shared `skipped(cur,
|
||||
kind, block)` builds both skip calls (upgrade too).
|
||||
- Log spam (robustness #8): `downgrade-skipped` is said once per session until a
|
||||
downgrade next succeeds: `st.capSaid: boolean`, set in `mainPlan` when the returned
|
||||
call is a downgrade skip (log suppressed while set), cleared in `mainPlan` when the
|
||||
returned call is `why === 'downgrade'`; the per-turn `turnLogged` dedupe stays for
|
||||
the upgrade.
|
||||
- `[1m]` carry (robustness #6): `moveTo` keeps `[1m]` from `cur`; after a haiku hop
|
||||
`cur` has none. Change `moveTo(cur, target, why, session)` to carry `[1m]` when
|
||||
EITHER `cur` or `st.sessionModel` has it (and the target is not haiku); all callers
|
||||
pass `st.sessionModel`.
|
||||
- Window re-check (robustness #2) [r3 #10]: in `decideMain`, after the `unknownCall`
|
||||
check: when `st.turnModel !== undefined`, `tokens !== undefined`, `!fits(st, id,
|
||||
tokens)`, and the session model is a known ranked id, not down and not haiku → `moveTo
|
||||
(cur, canonical(st.sessionModel), 'window', st.sessionModel)` with a once-log
|
||||
`model-router: context outgrew <id>: back to <session model>`; any guard failing →
|
||||
fall through to the normal decision. Only `windows` entries (haiku) can trigger it.
|
||||
- Classifier model axis, DOWNGRADES ONLY under the switch (correctness #2, simplicity
|
||||
#3, robustness #3) [r3 #3 / #4 / #9]: `routeFromLabel` (~l.3531-3541) writes `route:
|
||||
{ ...phaseRoute(st.cfg, phase) }` (tier + effort). `mainModel` becomes
|
||||
`mainModelSlot(st): Routed | null` = the FIRST slot (userMain, turnMain, runMain,
|
||||
turnFloor) whose route carries a model name; `decideFor` resolves the name from it.
|
||||
Two rules on that slot when `slot.source === 'classifier'`: (i) a floor with a model
|
||||
name wins over it (`turnFloor` route name used instead: `ultrathink` mid-turn still
|
||||
reaches fable/max); (ii) the resolved `wanted` is dropped (`undefined`) when it
|
||||
`ranksAbove` the SESSION model (`canonical(st.sessionModel)`), never compared to
|
||||
`cur` — so a guess never moves main above the user's baseline, while a breaker
|
||||
fallback from a guessed sonnet to opus (below fable) still works and never falls to
|
||||
haiku. Texts and step agree (both go through `decideFor`). Existing test l.4568
|
||||
(sonnet session, design guess → sonnet) holds; rename it `…never moves UP`.
|
||||
- Classifier vs a user `/effort` (simplicity #2, correctness #6/#7, robustness #9) [r3
|
||||
#7]: `engineBase` becomes per turn: `noteEngineBase` records the engine effort of
|
||||
THIS turn's first main step, compared against the previous turn's value
|
||||
(`st.engineBase` holds the last turn's, `st.turnEngine` this turn's; at turn end
|
||||
`engineBase = turnEngine`). `userMovedEffort` = this turn's engine effort differs
|
||||
from the previous turn's: a `/effort` typed between turns. In `mainPlan`, before
|
||||
`mainEffort`: `if (userMovedEffort(st, e)) st.turnMain = null` when `turnMain.source
|
||||
=== 'classifier'` — the guess is dropped on both axes for THIS turn only; `turnModel`
|
||||
is NOT touched (no return without a cold-read check; turn end restores `e.model`).
|
||||
The `skipTurn` parameter of `mainEffort` (~l.1302) goes away. The next classified
|
||||
turn routes again from the new baseline effort.
|
||||
- User `/model` (correctness #3, robustness #3): NO suspension. A user `/model` sets
|
||||
the new baseline (`sessionModel`) and the classifier keeps routing from it; `/route
|
||||
classify off` or `/route off` pins the model. The r1 disposition claim is struck; the
|
||||
trade-off goes into the BDR amendment and the doctrine line.
|
||||
- Helper skills inside a run (robustness #1): `routeMainBySkill(st, skill, row,
|
||||
typed)`: when `!typed && st.runMain !== null` and the row's tier ranks below the
|
||||
run's (rank through `resolveRoute` → `modelRank`, so a phase naming a model is ranked
|
||||
too [r3 #15]), write
|
||||
`route: { effort: row.route.effort }` only — a model-loaded helper (e.g.
|
||||
`using-git-worktrees` inside `/feat`) moves effort, never the orchestrator's model.
|
||||
Typed skills and best-tier rows unchanged. Test l.1130 keeps `skill mechanical` in
|
||||
the status and gains `expect((await stepOn(…)).model).toBe(FABLE)`.
|
||||
- Toast hint (correctness #6) [r3 #11]: `skipped(cur, kind, block)` sets `call.why =
|
||||
`${kind} skipped: ${block}`` (the bare block text is never the `why`) and `call.log =
|
||||
'model-router: ' + why`; `downgradeHeld` matches `why === 'switch off'` OR
|
||||
`why.startsWith('downgrade skipped')`; hint: switch → unchanged text; cap → `, held:
|
||||
<why>`. Test (m) covers the cap hint.
|
||||
- Texts (`show` ~l.1463, `statusLine` ~l.1475): `switch: on (downgrade, cap 120000
|
||||
tokens cold)` / `switch: off (downgrade)`; status line [gated 2026-10-13]: ` · switch
|
||||
off` only when off, the ` · switch on` fragment is dropped.
|
||||
- In-file comments (correctness #9): rewrite l.3438-3441 ("never a model move"),
|
||||
l.3535 and l.3546 to the new rule (effort + a downward tier under the switch and the
|
||||
cold-read cap).
|
||||
- Doctrine (`lib/effort-shift.md` Limits bullet): "A plain typed prompt with no route
|
||||
in force is classified by a small model: the guessed phase sets the turn's effort
|
||||
and, under `mainModelSwitch` and `downgradeMaxTokens` (max cold read, default
|
||||
120000), may move main DOWN, never up; a user `/model` only moves the baseline the
|
||||
guess starts from (`/route classify off` to stop guessing; never `-p`)."
|
||||
|
||||
## Edge cases
|
||||
- Context size unknown: both caps block, logged once (upgrade per turn, downgrade per
|
||||
cap crossing); the window re-check does not fire on an unknown size.
|
||||
- Compaction (`tokens` below a served entry): that entry is stale, the move is cold.
|
||||
- TTL (60 min const) is an assumption of this account's cache; residual in TODO.
|
||||
- Target down (breaker): `resolveName` yields the first available alias; a guessed route
|
||||
ranks against the session model, so its fallback stays between sonnet and fable.
|
||||
- Main already down, context grows past the downgrade cap mid-turn: `wanted` unchanged
|
||||
→ keep; a best route later moves up (cold read of Fable = delta since its last step,
|
||||
usually under 200k); no re-down unless Sonnet's cold read fits — which it does while
|
||||
Sonnet is warm. Turn end resets `turnModel`; next turn starts on `e.model`.
|
||||
- Haiku on main and the context passes 200k: window re-check returns main to the
|
||||
session model.
|
||||
- `/route switch off`: identical to today (refusal text `switch off`).
|
||||
- Classifier `chat`/unknown: nothing written.
|
||||
- Non-best session (`/model opus`): coding guess → sonnet (downgrade); design guess →
|
||||
stays opus (never up by a guess); prompt RULES still upgrade as today.
|
||||
- Unrowed sub-agent with no explicit model spawned during a demoted turn may inherit
|
||||
the demoted `parentModel` (robustness #10): accepted residual, all repo agents are
|
||||
rowed and built-ins carry `model=`; logged in TODO.
|
||||
- `session.usage().context.tokens` may lag the new prompt (robustness #11): accepted,
|
||||
the cap is an economic guard, not a hard limit; logged in TODO.
|
||||
|
||||
## Tests (register.test.ts) — contract letters
|
||||
- Rig: add `usageVar(on, initial)` returning `{ set(n) }` over ONE bottom
|
||||
`session.usage` hook with a mutable count (never a second `on('session.usage')`);
|
||||
`bootRig` gains an optional `usage` handle or a sibling `bootRigVar`.
|
||||
- Re-expressed (default flip), never weakened: l.305-315 (`/route model: alias
|
||||
resolved…` → `switch off` first), l.605-614 and l.1665-1672 (toggle survival →
|
||||
`switch off` / `switch: off`), l.747 (`switch off` first), l.754 (drop the `switch on`
|
||||
line: default proves it), l.951-958 (`show reflects /model opus` → `switch off`
|
||||
first), l.2934-2943 (toast: `switch off` first, hint text unchanged), l.4576 (`route
|
||||
carries no tier` → `carries the tier`), l.4783-4799 (`classify guard: a route landing
|
||||
during the call wins` → `switch off` first [r3 #8]), every status-line assertion on
|
||||
` · switch on` (grep) → asserts the fragment is ABSENT by default and present as
|
||||
` · switch off` after `switch off`. `bootCls` gains a `tokens` option (ClsOpts) for (f)
|
||||
[r3 #13].
|
||||
- (a) `cap: a downgrade under the cold-read cap moves main` (fable, 1000 tokens, route
|
||||
mechanical → haiku, no `switch on`).
|
||||
- (b) `cap: a downgrade above the cold-read cap keeps main` (fable, 300000, route
|
||||
mechanical → `{ model: FABLE, effort: 'low' }`, log contains `downgrade skipped: cold
|
||||
read 300000 tokens over 120000`); second step same turn: no second log line.
|
||||
- (c) `cap: unknown usage blocks the downgrade` (tokens null).
|
||||
- (d) `cap: a warm model is cheap to return to, a cold one is capped` [r3 #2] — rig
|
||||
with `usageVar` at 100000: route implement, step → SONNET (served sonnet@100000);
|
||||
`endTurn`; step on FABLE unrouted → FABLE (served fable@100000, the unrouted write);
|
||||
`endTurn`; `set(210000)`; route implement, step → SONNET (moved from FABLE: cold read
|
||||
110000 ≤ 120000 although the context is 210000); `endTurn`; CONTROL: route judge
|
||||
(opus, never served, ranks below fable), step → FABLE kept, log contains `downgrade
|
||||
skipped: cold read 210000 tokens over 120000`.
|
||||
- (e) `classify: coding moves main to sonnet medium under the cap` (fable, 1000).
|
||||
- (f) `classify: coding above the cap keeps fable at medium` (fable, 300000).
|
||||
- (g) `downgrade: switch off keeps today's behavior` (= l.747 re-expressed).
|
||||
- (h) `classify: a user /effort drops the model guess too` (step effort ≠ engine base →
|
||||
FABLE at the step's effort, `/route show` agrees: `session defaults`).
|
||||
- (i) `classify: a design guess never moves a sonnet session up` (= l.4568 renamed).
|
||||
- (j) `run slot: a model-loaded helper skill moves effort only` (= l.1130 extended).
|
||||
- (k) `window: haiku outgrown returns main to the session model` (route mechanical on
|
||||
fable at 1000 → haiku; `set(250000)`; next step → FABLE, log once).
|
||||
- (l) `config: a quoted "false" mainModelSwitch reads as off with a warning` (`/route
|
||||
show` contains `switch: off`) [r3 #5].
|
||||
- (m) `toast: a downgrade held by the cap says so` (`, held: downgrade skipped: cold
|
||||
read …`) [r3 #11].
|
||||
- (n) `cap: the served record expires after the TTL` (clock advanced 61 min: cold =
|
||||
whole context, downgrade above cap refused) [r3 #13].
|
||||
- (o) `cap: /route reload forgets the served record`; `cap: the skip line is said once
|
||||
until a downgrade succeeds` (two turns above cap → one line; a downgrade under cap on
|
||||
a later turn; above cap again → one new line) [r3 #13].
|
||||
- (p) `cap: a smaller context than the record is cold` (set(50000) after served@100000
|
||||
→ downgrade to a never-served model refused only if 50000 > cap; use cap override
|
||||
`downgradeMaxTokens: 40000` in `home` → refused, entry dropped) [r3 #6].
|
||||
- (q) `classify: a mid-turn ultrathink floor still reaches fable max` (classifier coding
|
||||
→ sonnet; mid-turn prompt `ultrathink` → next step FABLE/max) [r3 #4].
|
||||
- (r) `classify: a guessed sonnet rate-limited falls to opus, never haiku` (coding →
|
||||
sonnet; `failWith('rate_limit')` → next step OPUS) [r3 #3].
|
||||
- (s) `classify: a user /effort drops the guess for one turn only` (turn 1 engine effort
|
||||
high = base; turn 2 engine effort low (user /effort) → guess dropped, FABLE; turn 3
|
||||
engine effort low → coding guess routes again → SONNET) [r3 #7].
|
||||
- Run: `make test suite=lib/tests/mods.test.sh`.
|
||||
|
||||
## Disposition (STEP 0.6)
|
||||
- honors BDR-115 (one writer per axis, full ids): cap and served record live in
|
||||
`decideMain`'s path, the only writer of main's model; full ids only.
|
||||
- honors LRN-204 (cold read per switch INTO a model, warm return): the cap measures
|
||||
exactly that quantity; no bypass needed.
|
||||
- supersedes one clause of BDR-116 amendment W4-B ("EFFORT ONLY: main model never
|
||||
moves on a guess") by user decision 2026-10-13, DOWN only, under the switch and the
|
||||
cold-read cap; the `/effort` guard stays (route dropped on both axes); the `/model`
|
||||
guard is NOT kept (baseline moves, guess follows) — stated trade-off for the BDR.
|
||||
- honors BDR-107 effort rungs: no effort value changes.
|
||||
|
||||
## CHALLENGE SUMMARY (r1 → r2 → r3)
|
||||
- correctness #1 / simplicity #1 / robustness #4-#6 → cold-read cap + served record,
|
||||
bypass dropped. correctness #2 / simplicity #3 / robustness #3 → downgrades only.
|
||||
correctness #3 → claim struck, trade-off surfaced. correctness #4 / simplicity #6 →
|
||||
full test list, letters aligned. simplicity #2 / correctness #6-#7 / robustness #9 →
|
||||
drop the guessed route on both axes in `mainPlan`. simplicity #4 → one `coldBlock`.
|
||||
simplicity #5 → moot. robustness #1 → helper-skill effort-only inside a run.
|
||||
robustness #2 → window re-check. robustness #7 → pickSwitch. robustness #8 → once per
|
||||
cap crossing. robustness #10/#11 → accepted residuals (TODO). correctness #9 →
|
||||
comments + USAGE.md/README to STEP 6.
|
||||
- Confirmation pass (r2, FATAL(7)): BLOCKER #1 → served written on EVERY main step
|
||||
(tokens + clock read per step). #2 → test (d) with `endTurn` between legs. #3 → guess
|
||||
ranked against the session model. #4 → floor model wins over a guess. #5 → fail-closed
|
||||
switch picker. #6 → stale entry = cold, `served` not kept across /clear. #7 →
|
||||
per-turn `engineBase`, no `turnModel` reset. #8/#13 → tests added. #9 → slot-sourced
|
||||
predicate. #10 → window re-check guards. #11 → `why` pinned through `skipped()`. #12 →
|
||||
`capSaid` in `mainPlan`. #14 → TTL const + TODO. #15 → rank via `resolveRoute`.
|
||||
- Deferred BLOCKERs: none. Re-challenge budget consumed (1 extra pass); verifier and
|
||||
security gates follow.
|
||||
Reference in new issue
Block a user