chore(memory): BLK-030, LRN-218, TODO + journal, router-probe-before-route contract + plan
This commit is contained in:
1 parent
da0b4a65ba
commit
304270fe5b
6 files changed
+236
No files matched your search
@@ -49,6 +49,7 @@ rules:
|
||||
| BLK-027 | 2026-10-06 | this machine never ran `make link`/`make plugin`: no global `core.hooksPath` → post-commit push never fired, branches landed ahead of upstream; 11 vendored skills + `~/.claude/.env` missing | resolved (link) / open (plugin) |
|
||||
| BLK-028 | 2026-10-06 | notify-attention on a VS Code client: bell + toast silent-degradation faults (merge of BLK-019 + BLK-020): terminalBell sound default off, ext hooks only terminals born after activation, Code muted in Windows mixer | resolved |
|
||||
| BLK-029 | 2026-10-08 | Claude Code mods (2.1.294): alias → id resolution for a model set by a hook lags the Agent tool's (`sonnet` → `claude-sonnet-5`, 404); Agent tool schema refuses full ids | upstream |
|
||||
| BLK-030 | 2026-10-15 | model-router kept routing best-tier phases to Fable out of usage credits: each new session + each hold lapse lost a turn to 429 `model_requires_usage_credits`; breaker reactive only | resolved |
|
||||
|
||||
---
|
||||
|
||||
@@ -314,3 +315,9 @@ rules:
|
||||
- **Status**: upstream (report to anthropics/claude-code with the request id `req_011Cfppp7VFt8z2Pd3zJrpUi`); workaround in model-router.
|
||||
- **Reference**: [[LRN-203]], plan `.claude/tasks/plans/2026-10-08-model-router-mod.md`.
|
||||
|
||||
## BLK-030 — model-router routes main to Fable out of usage credits; turn lost every session + hold lapse — 2026-10-15
|
||||
- **Friction**: Fable 5.1 out of credits. `route(reflect)` answered `claude-fable-5-1 (upgrade)` after hold lapse; next request died `<synthetic>` "You're out of usage credits", `apiError: model_requires_usage_credits`, `error: rate_limit`, HTTP 429. Holds 15→30→60 min, user typed "reprends" each time.
|
||||
- **Real cause**: availability learned only reactively. Breaker (`markDown`) fed by `classic.StopFailure` + `PostModelSwitch auto`, i.e. AFTER real request died. Hold in-memory w/ backoff; nothing checked move target before main sent there. 429 body generic (no credits marker) → no cheap typed signal.
|
||||
- **Solution**: 2cccc43 (bugfix/router-probe-before-route). `decideFor` decides first, then probes decided move target once: `$.model.complete` raced vs `PROBE_MS` 8000 via shared `raceTimeout`; timeout/reject → `markDown(..., 'probe')` + ONE re-decision, no further probe. `verified` set kept across `/clear`, cleared by `markDown`. Never probes cur or held id. Config `probe` (default true). Stand-in effort one notch up (reflect xhigh, plan max, orchestrate high) when lower model replaces held tier head where head would have run; explicit/floor effort never bumped. 497 tests.
|
||||
- **Status**: resolved. Residuals in TODO (explicit `Agent(model:"fable")` pins, spawn path, session model itself out of credits, ≤8 s stall per new session).
|
||||
- **Reference**: [[LRN-218]], [[BDR-116]], [[BDR-115]], [[LRN-207]], contract `.claude/tasks/contracts/2026-10-14-router-probe-before-route-0005.md`, plan r3 `.claude/tasks/plans/2026-10-14-router-probe-before-route-0005.md`.
|
||||
@@ -611,3 +611,7 @@ rules:
|
||||
## 2026-10-13
|
||||
- model-router /feat `downgradeMaxTokens` (user: switch on by default, classifier may switch model early, back up to fable on reflection). Plan r1 context-size cap → challenge: cap must measure COLD READ (LRN-204), bypass dropped, classifier down-only vs session model; confirmation pass FATAL(7), BLOCKER served-record-on-routed-steps-only → r3. feater DONE 450 tests, verifier CONFORME 10/10, security PASS, commit 2ce203a on feature/router-downgrade-cap. Doc-sync + BDR/LRN/EVAL pending.
|
||||
- Residuals /feat (user "fix les residus"): plan r1 → 3 lenses FATAL×2 (BLOCKER: `e.model===undefined && route===undefined` ≠ "inherits", frontmatter model invisible to the hook) → allowlist `INHERITING_AGENTS` + built-in source + below-session guard; estimate via shared `withEstimate`, raw served record; TTL enum lookup. Confirmation CONCERNS(3) → r3. feater DONE 472 tests, verifier CONFORME 7/7, security PASS, commit 1be57a3. Doc-sync + capitalize pending.
|
||||
|
||||
## 2026-10-15
|
||||
- /bugfix model-router: Fable out of credits still routed (BLK-030). Live check: `complete()` on credit-less model never settles → probe w/ 8 s timeout (LRN-218). Plan r1 BLOCKERs (loop, cur probed) → r3 decide-then-probe. User: stand-in effort one notch up, `probe` key, explicit fable pins out of scope. bugfixer 497 tests, GATE 0 MET, verifier CONFORME 11/11, security PASS. Commit 2cccc43, docs P1-P5, merged to develop on user signal.
|
||||
|
||||
@@ -1809,3 +1809,8 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
|
||||
## LRN-217 — In an `agent.spawn` hook, "no explicit `model` and no routing row" does NOT mean "inherits the parent": the agent's frontmatter `model:` is invisible to the hook; only an allowlist of types known to inherit is safe, and `agent.offer.source` tells a built-in from a shadowing definition
|
||||
- **Context**: model-router residuals 2026-10-13, plan r1 pinned every unrowed, model-less spawn to the session model during a demoted turn. Two lenses BLOCKER: engine 2.1.296 builds the spawn input as `model: e.model` (Agent tool param only); frontmatter model never reaches `agent.spawn` nor `agent.offer`; `route === undefined` also covers frozen forks/workflows (hook model ignored) and project-source skips; nested spawns carry the SUB-AGENT's `parentModel`; `turnModel` is set on upgrades too. A repo/user `general-purpose.md` can shadow the built-in with its own `model:` → `offers.get(type) === 'built-in'` is the discriminator.
|
||||
- **Apply**: before rewriting a spawn's model, prove the agent would inherit: allowlist of built-in types with no frontmatter (`general-purpose`), source `built-in`, not fork/workflow, `parentAgentId` undefined, and the parent really sits below the baseline. Record what the hook cannot see as a `shortcut:` at the allowlist. Links [[BDR-116]], [[BDR-115]], [[LRN-203]].
|
||||
|
||||
## LRN-218 — `$.model.complete` as availability probe: credits-exhausted model NEVER settles (timeout = only failure signal); answer is an object; probe the decided target only, after the decision
|
||||
- **Context**: model-router bugfix 2026-10-14/15, live scratch mod (`claude -p --plugin-dir`). `complete({model:'claude-opus-5-5', prompt:'ok', maxTokens:1})` resolved 1308 ms with OBJECT `{ isAnswered, text, usage }` (d.ts says string; maxTokens not honored, 43 output tokens). Same call on `claude-fable-5-1` without credits: no settle, no rejection after 900 s (client retries the 429). Plan r1 probed in a loop before deciding → BLOCKERs: unbounded spin on explicit alias, cur probed.
|
||||
- **Apply**: availability check via `complete()` = race vs own timeout, treat timeout + rejection as down, any resolved shape as up; never read `isAnswered:false` as failure (unobserved). Decide first, probe only the move target, once, skip cur and held ids, one re-decision without probe. No hook signal on the probe (Esc must not mark model down); probe sleep counts against 10 s `turn.step` budget. Links [[BLK-030]], [[LRN-207]].
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# TODO
|
||||
|
||||
## 2026-10-15 — model-router: probe a move target before routing main to it (bugfix/router-probe-before-route)
|
||||
Contract `.claude/tasks/contracts/2026-10-14-router-probe-before-route-0005.md`, plan r3 `.claude/tasks/plans/2026-10-14-router-probe-before-route-0005.md`.
|
||||
- [x] /bugfix run: Fable out of credits routed (BLK-030, LRN-218); r1 → r3; 497 tests, verifier CONFORME 11/11, security PASS; commit 2cccc43 + docs.
|
||||
- [ ] Residuals (accepted): explicit `Agent(model:"fable")` pins (client-handover) still fail w/o credits (BDR-115 kept); spawn path not probed; session model itself out of credits → first turn fails reactively (cur never probed); no cross-session persistence → ≤8 s stall per new session; timed-out probe keeps retrying in engine (no cancel); `[1m]` gating unverified (bare id probed); no Esc regression test (kit cannot abort signal); probe sleep eats ~8 of 10 s `turn.step` budget.
|
||||
- [ ] LIVE CHECK after `/reload-plugins`: best-tier phase w/ Fable out of credits → `/route show` lists fable down (probe), main on opus, effort bumped.
|
||||
|
||||
## 2026-10-13 — model-router: cold-read cap, switch on by default, classifier routes down (feature/router-downgrade-cap)
|
||||
Contract `.claude/tasks/contracts/2026-10-13-router-downgrade-cap-0013.md`, plan r3 `.claude/tasks/plans/2026-10-13-router-downgrade-cap-0013.md`.
|
||||
- [x] /feat run: 3 lenses (10 MAJOR) → r2; confirmation pass FATAL(7) (BLOCKER: served record only on routed steps) → r3; feater DONE (19 tests + 3 extra, 450 pass), GATE 0 MET 4/4, verifier CONFORME 10/10 first pass, security PASS. Commit 2ce203a.
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# CONTRACT — router-probe-before-route
|
||||
- date: 2026-10-14 | flow: bugfix | branch: bugfix/router-probe-before-route
|
||||
- status: active
|
||||
- plan: .claude/tasks/plans/2026-10-14-router-probe-before-route-0005.md (r3)
|
||||
|
||||
## REQUEST (verbatim — IMMUTABLE)
|
||||
dans le model-router, quand on a plus d'usage fable 5, comme maintenant, il essaie quand meme de router vers fable 5, il faudrait detecter quand il n'y a plus de credit fable, alors il faut utiliser opus 5.5 xhigh ou max a la place. et donc verifier si fable est dispo avant de router vers fable
|
||||
|
||||
## CLARIFICATIONS
|
||||
Q: Opus effort when it stands in for Fable? / A: one notch up (orchestrate medium→high, reflect high→xhigh, plan xhigh→max, escalate max) [gated 2026-10-14]
|
||||
Q: Explicit Agent(model:"fable") pins (client-handover)? / A: leave them (BDR-115 kept), log a TODO residual [gated 2026-10-14]
|
||||
Q: Approve plan r3? / A: yes [gated 2026-10-14]
|
||||
Q: Config key `probe` (bool, default true) in ~/.claude/model-router.json to turn the probe off alone? / A: yes [gated 2026-10-14]
|
||||
Q: plan test 8 records FABLE then OPUS probes (leaveDown leaves the held explicit fable for opus, an unverified move target)? / A: accepted (orchestrator, internal) — fable probed once, no spin; criterion = no re-probe of a held id [gated 2026-10-14]
|
||||
Q: criterion 9 (Esc) has no regression test: claude-code/testing cannot abort the dispatch signal? / A: accepted (orchestrator, internal) — verifier judges the code path; residual noted [gated 2026-10-14]
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. The model-router mod test suite passes with zero failures.
|
||||
CHECK: cd /home/bchanot/Documents/claude && out=$(claude plugin test mods/model-router 2>&1); rc=$?; printf '%s\n' "$out" | tail -4; [ "$rc" -eq 0 ] && printf '%s\n' "$out" | grep -Eq '^ *0 fail$' && ! printf '%s\n' "$out" | grep -q '^(fail)' && echo SUITE_GREEN
|
||||
EXPECT: SUITE_GREEN
|
||||
EVIDENCE: MET exit=0 marker-found :: 497 pass 0 fail Ran 497 tests across 1 file. [23.89s] SUITE_GREEN
|
||||
2. Regression tests for the probe exist and pass: at least 9 passing tests whose names start with `probe:` (plan §3 tests 1-4, 6-11), among them one where the fable probe times out and the first step of a plan route lands on opus with exactly one model.complete call and no StopFailure.
|
||||
CHECK: cd /home/bchanot/Documents/claude && out=$(claude plugin test mods/model-router 2>&1); n=$(printf '%s\n' "$out" | grep -c '^(pass) probe:'); f=$(printf '%s\n' "$out" | grep -c '^(fail) probe:'); echo "probe pass=$n fail=$f"; [ "$n" -ge 9 ] && [ "$f" -eq 0 ] && echo PROBE_TESTS_PASS
|
||||
EXPECT: PROBE_TESTS_PASS
|
||||
EVIDENCE: MET exit=0 marker-found :: probe pass=16 fail=0 PROBE_TESTS_PASS
|
||||
3. Regression tests for the stand-in effort exist and pass: at least 5 passing tests whose names start with `stand-in:` (plan §3 test 5 cases).
|
||||
CHECK: cd /home/bchanot/Documents/claude && out=$(claude plugin test mods/model-router 2>&1); n=$(printf '%s\n' "$out" | grep -c '^(pass) stand-in:'); f=$(printf '%s\n' "$out" | grep -c '^(fail) stand-in:'); echo "stand-in pass=$n fail=$f"; [ "$n" -ge 5 ] && [ "$f" -eq 0 ] && echo STANDIN_TESTS_PASS
|
||||
EXPECT: STANDIN_TESTS_PASS
|
||||
EVIDENCE: MET exit=0 marker-found :: stand-in pass=9 fail=0 STANDIN_TESTS_PASS
|
||||
4. The mods census (manifest, link, `claude plugin validate`, `claude plugin test`) passes.
|
||||
CHECK: cd /home/bchanot/Documents/claude && out=$(make test suite=lib/tests/mods.test.sh 2>&1); printf '%s\n' "$out" | tail -3; printf '%s\n' "$out" | grep -Eq '^mods: [0-9]+ pass, 0 fail$' && ! printf '%s\n' "$out" | grep -q '^FAIL ' && echo MODS_GREEN
|
||||
EXPECT: MODS_GREEN
|
||||
EVIDENCE: MET exit=0 marker-found :: PASS model-router: plugin test passed mods: 4 pass, 0 fail all suites green MODS_GREEN
|
||||
5. Probe semantics (plan §1): the probe fires only in `decideFor`, AFTER `decideMain`, only when the call moves main to a table model that is neither verified nor held down; it probes the bare canonical id via `$.model.complete` raced against PROBE_MS (8000) through one `raceTimeout` helper shared with the classifier; no hook abort signal on the probe; timeout or rejection → `markDown(id, 'probe')` then ONE re-decision without a further probe; success (any resolved shape) → verified only if not held at resolution; `markDown` drops the id from `verified` and logs through `tryLog`; `verified` survives `/clear`; the current model is never probed; the spawn path is unchanged.
|
||||
6. Stand-in effort (plan §2): main only; one notch up on LEVELS (max stays max) exactly when the counterfactual rule holds (tier route, head held down, decided model = tier resolution ranked below head, and with the head available the router would have run it: classifier ceiling, upgrade switch, cold-read cap, window); never when the effort came from a different slot than the tier route, an explicit effort (`explicitEffort` flag via parseRoute and Picked→applyRoute), a prompt floor, or a non-Level engine effort; computed once and carried on the Verdict; `effortWord`, `mainRoutedText` and the verbose transition line report it ("stands in", "stand-in").
|
||||
7. Truthful route answer: with the fable probe failing, `route(phase=reflect)` answers an opus id, never `claude-fable-5-1`.
|
||||
8. Config key `probe` (boolean, default true) parsed and validated like `classify`, settable in `~/.claude/model-router.json`, shown in `/route show`; `probe: false` → no `model.complete` call at all.
|
||||
9. Esc during a probe: the mark stands but `mainPlan` writes no turn state (`turnModel`, served record, prompt estimate, `lastPlan`) when the dispatch signal has aborted.
|
||||
10. Existing tests whose expectations change under the bump are updated, not deleted (e.g. 'fallback: a rate limit on fable moves plan to opus xhigh' now expects max); `boot` installs a default successful `model.complete` answer.
|
||||
11. No edit outside FILE SCOPE; the user's uncommitted `mods/model-router/routing.json` and `settings.json` changes are left untouched.
|
||||
|
||||
## FILE SCOPE
|
||||
mods/model-router/hooks/register.ts
|
||||
mods/model-router/hooks/register.test.ts
|
||||
@@ -0,0 +1,170 @@
|
||||
# Bugfix plan — model-router: probe a move target before routing to it (Fable out of credits)
|
||||
|
||||
Branch: bugfix/router-probe-before-route (off develop). Skill: /bugfix.
|
||||
Revision r3 (r1: correctness FATAL(4), simplicity FATAL(3), robustness
|
||||
CONCERNS(3) → r2 rebuild; r2 confirmation correctness CONCERNS(3), 0 BLOCKER
|
||||
→ r3 folds its findings; live engine checks).
|
||||
|
||||
## Bug
|
||||
Fable 5.1 out of usage credits. Router still routes best-tier phases (plan,
|
||||
reflect, orchestrate, escalate) to `claude-fable-5-1`. Next request dies:
|
||||
`<synthetic>` message "You're out of usage credits. Run /usage-credits to keep
|
||||
using Fable 5.1 or /model to switch models.", `apiError:
|
||||
model_requires_usage_credits`, `error: rate_limit`, HTTP 429. Turn lost.
|
||||
|
||||
## Root cause
|
||||
Availability learned only reactively. Breaker (`markDown`, register.ts
|
||||
~l.1134) fed solely by `classic.StopFailure` (UNAVAILABLE kinds) and
|
||||
`PostModelSwitch auto`, i.e. AFTER a real user request died. Nothing checks a
|
||||
move target before main is sent to it. Hold in-memory, backoff 15→30→60…:
|
||||
each new session and each hold lapse costs one user turn on a 429.
|
||||
|
||||
## Evidence
|
||||
- Session e2ecb2c1 transcript: 21:31:29 `model_requires_usage_credits` →
|
||||
"fable unavailable (rate_limit) for 15 min"; hold lapsed; `route(reflect)`
|
||||
answered "model claude-fable-5-1 (upgrade)"; 22:04:34 same error → "for 30
|
||||
min". `errorDetails` = generic 429 body ("would exceed your account's rate
|
||||
limit"), no credits marker → no cheap typed signal to hold longer.
|
||||
- LIVE CHECK (scratch mod, `claude -p --plugin-dir`, 2026-10-14):
|
||||
`$.model.complete({model:'claude-opus-5-5', prompt:'ok', maxTokens:1})`
|
||||
RESOLVED in 1308 ms with an OBJECT `{ isAnswered: true, text, usage }` (not
|
||||
the documented string; maxTokens 1 not honored: 43 output tokens).
|
||||
Same call on `claude-fable-5-1` (no credits): NO settle after >190 s (still
|
||||
pending; the client apparently retries the 429). → for the credits case the
|
||||
probe's failure signal IS the timeout. Long run: fable never settled
|
||||
within 900 s (killed by timeout, rc 124) → never settles in useful time. `isAnswered:false` never
|
||||
observed → NOT used as a failure signal.
|
||||
|
||||
## Constraints (in force)
|
||||
- LRN-207: `rateLimits` = account windows, not per model → no pre-check
|
||||
there. Breaker per episode, idempotent while held; `/clear` keeps it.
|
||||
- BDR-115: explicit Agent params never rewritten (user kept it 2026-10-14);
|
||||
explicit `Agent(model:"fable")` pins (client-handover) → TODO residual.
|
||||
- BDR-116 amendments: `decideMain` = THE single main decision, step and every
|
||||
text agree; caps; agents move UP within tier only (`inTier`).
|
||||
- User 2026-10-14: stand-in effort ONE NOTCH UP; no cross-session persistence.
|
||||
|
||||
## Fix plan
|
||||
|
||||
### 1. Probe the decided move target, once — `hooks/register.ts`
|
||||
- Decide FIRST, probe after (closes r1 BLOCKERs: no loop, no explicit-alias
|
||||
spin, cur never probed, no probe for a refused move). In `decideFor`:
|
||||
`call = decideMain(...)`; if `call.moved` AND `id = canonical(call.model)`
|
||||
is a table id AND not in `st.verified` AND NOT `st.down.has(id)` (a hold
|
||||
already answers it: an explicit route onto a held id is sent as today,
|
||||
never re-probed each step) → `ok = await probeModel($, st, id)`. The BARE
|
||||
canonical id is probed (`[1m]` acceptance by complete() unverified; a
|
||||
separately gated `[1m]` stays the reactive breaker's job).
|
||||
- ok → `verified.add(canonical id)` only if `!st.down.has(id)` at
|
||||
resolution (a StopFailure during flight wins).
|
||||
- not ok → `markDown($, st, id, 'probe', now)`, then re-run `wantedModel`
|
||||
+ `decideMain` ONCE, no further probe (a double outage stays the reactive
|
||||
breaker's job; an explicit alias/id route re-decides to the same id, as
|
||||
"explicit, never skipped" already means).
|
||||
- Covers every move kind decideMain makes (upgrade, downgrade, leaveDown
|
||||
fallback, windowReturn) and every caller of decideFor: the step
|
||||
(`mainPlan`), the route tool's answer (`snapshot`/`mainAnswer`, read by
|
||||
the model gate), `/route show`, first-use dialog text, `toastChange`.
|
||||
- `probeModel`: `$.model.complete({ model, prompt: 'ok', maxTokens: 16 })`
|
||||
raced against `PROBE_MS` (8000, next to CLASSIFY_MS) through ONE extracted
|
||||
helper `raceTimeout($, promise, ms)` shared with `raceClassify` (classify
|
||||
keeps its abort signal; the probe passes NO hook signal: a user Esc must not
|
||||
mark a model down). Result:
|
||||
- resolved, whatever the shape (documented string or observed object) → ok;
|
||||
- TIMEOUT → not ok (the live credits case);
|
||||
- rejection → not ok (no typed kinds documented for `complete`).
|
||||
- Esc during the probe: the probe completes (no signal), the mark stands,
|
||||
but `mainPlan` skips its turn-state writes (`turnModel`, `noteServed`,
|
||||
`promptEstimate`, `lastPlan`) when the dispatch signal has aborted.
|
||||
- State: `verified: Set<string>` in State, kept by `resetSession` (account-
|
||||
wide like `down`). `markDown` deletes the id from `verified`. No `probing`
|
||||
map, `clearBreaker` leaves `verified` alone (a user `/model` target becomes
|
||||
cur, never probed).
|
||||
- `markDown` logs through `tryLog` (it now runs on the step path; a throwing
|
||||
`ui.log` must not unroute the step).
|
||||
- A lapsed hold leaves the id unverified → the next move to it probes again
|
||||
instead of spending a user turn.
|
||||
- Config key `probe` (bool, default true) in DEFAULT_CONFIG + override
|
||||
validation like `classify`, and `/route show` lists it: lets a machine turn
|
||||
the probe off without `/route off`. [gated: user approval at the gate]
|
||||
|
||||
### 2. Stand-in effort, one notch up — `hooks/register.ts`
|
||||
- One pure helper computes `standIn` and decideFor returns it on the Verdict
|
||||
(step and texts read the same value). COUNTERFACTUAL rule: the bump tracks
|
||||
a real stand-in, not the head's availability. True when ALL hold:
|
||||
(a) the main model slot's route has a `tier` and the head alias's id is
|
||||
held down; (b) `canonical(call.model)` equals the tier's `resolveName` id
|
||||
and ranks BELOW the head (fallback rank); (c) with the head treated as
|
||||
available, the router WOULD have run it: the classifier ceiling of
|
||||
`wantedModel` does not refuse the head, and `decideMain(st, cur, headId,
|
||||
read)` lands on the head (upgrade on, cap not blocking, window fits).
|
||||
Today: best with fable down → opus, only where fable would have run.
|
||||
- `mainPlan`: effort = `floored(standIn && !explicit ? notchUp(base) :
|
||||
base, floor)`, notchUp on LEVELS, max stays max. Precedence of
|
||||
`mainEffort` unchanged.
|
||||
- Only the phase's own effort is bumped: bump iff the slot that SUPPLIED the
|
||||
effort (per `mainEffort`'s `by`: sticky→userMain, turn→turnMain or
|
||||
runMain) is the same slot whose route names the tier, that slot carries no
|
||||
explicit effort, and the base is a Level (never an engine number). A
|
||||
floor-supplied base (`by: 'floor'`) is user-explicit → never bumped. Flag
|
||||
`explicitEffort` on `Routed`, set when an effort was given: `parseRoute`
|
||||
(`/route`), and the route tool through `Picked` → `applyRoute`. Case to
|
||||
keep: `/route effort=low` sticky + plan turn route with fable down → low.
|
||||
- Main only: agents move up within tier, never get a below-head stand-in.
|
||||
- Texts: `effortWord` and `mainRoutedText` report the bumped level and why,
|
||||
e.g. "effort xhigh (opus stands in for fable)"; the bump is recorded on
|
||||
`Step.why` so the verbose transition line names "stand-in" as the effort
|
||||
cause. `phasesText` unchanged.
|
||||
|
||||
### 3. Tests — `hooks/register.test.ts`
|
||||
- `boot`: default bottom `on('model.complete', …)` answering success so the
|
||||
472 existing tests keep every model available; helpers: `probeHangs(on, id)`
|
||||
(never settles; advance the mock clock past PROBE_MS), `probeRejects(on,
|
||||
id)`, a call counter.
|
||||
- New tests (boot on OPUS, so a best-tier route is a move):
|
||||
1. plan route, fable probe times out → the step lands on opus/max, exactly
|
||||
ONE model.complete call (on fable), no StopFailure, `show` lists fable
|
||||
down (probe).
|
||||
2. route tool answer with the fable probe failing names opus, never fable.
|
||||
3. probe success cached: move to fable, end turn, re-route next turn from
|
||||
opus → no second call.
|
||||
4. hold lapse (clock +15 min) → re-probe; still failing → fable held 30 min,
|
||||
main on opus; no step sent on fable.
|
||||
5. bump: reflect stand-in → xhigh; orchestrate → high; `route(phase=reflect,
|
||||
effort=high)` → high (explicit); fable available → no bump; classifier
|
||||
reflect on an opus session with fable down → no bump (ceiling); upgrade
|
||||
off → no bump; `/route effort=low` sticky + plan route, fable down → low;
|
||||
an ultrathink floor → max, not bumped past.
|
||||
6. a probe rejection → marked down like a timeout; a resolved object or
|
||||
string answer → ok.
|
||||
7. a StopFailure on fable while its probe is in flight → probe success does
|
||||
NOT re-verify it.
|
||||
8. `/route fable` (explicit) with the probe failing → ONE probe in total
|
||||
across `/route` itself, two consecutive steps and a `/route show`.
|
||||
11. Esc during a probe: fable marked down, no stale `turnModel` next turn.
|
||||
9. no probe when the move is refused (classifier guess above the session
|
||||
model; upgrade off) and none for cur.
|
||||
10. `probe: false` in the override → no model.complete call. [if gated in]
|
||||
- Update the existing tests whose expectations change ('fallback: a rate
|
||||
limit on fable moves plan to opus xhigh' → opus max under the bump).
|
||||
- Suite: `claude plugin test mods/model-router` (472 pass baseline) and
|
||||
`make test suite=lib/tests/mods.test.sh`.
|
||||
|
||||
## Out of scope / residuals (TODO)
|
||||
- Explicit `Agent(model:"fable")` pins (client-handover) still fail without
|
||||
credits (BDR-115 kept).
|
||||
- Spawn path not probed: no agent row maps to best; `inTier` only reaches
|
||||
fable when opus is also down (double outage → reactive breaker).
|
||||
- Session model itself out of credits (settings `model: fable`): cur never
|
||||
probed, first turn fails reactively.
|
||||
- No cross-session persistence: with fable out of credits each new session
|
||||
pays one PROBE_MS stall (≤8 s) at its first best-tier move.
|
||||
- A timed-out probe's request keeps retrying in the engine (no cancel param).
|
||||
- `[1m]` gating: the bare id is probed; a separately gated `[1m]` would pass
|
||||
the probe and fail reactively, as today.
|
||||
|
||||
## Risk
|
||||
Medium. ≤8 s stall once per model per session on a failing target; ~1.3 s on
|
||||
a healthy one. A network blip marks the move target down 15 min (main stays
|
||||
on its current model: errs toward a working one). Probe spends a tiny
|
||||
completion when the target IS available.
|
||||
Reference in new issue
Block a user