chore(memory): BDR-150 Fable advisor-only routing
This commit is contained in:
1 parent
42385d2936
commit
67f27e65ae
6 files changed
+335
No files matched your search
@@ -1386,6 +1386,7 @@ Branch feature/user-writing-web-rules, UNMERGED (human gate).
|
||||
- **Amendment (2026-10-23, feat C 786ee13 + docs d2d590f)**: keyword-lift gap CLOSED: prompt rules (keywords, `ultrathink`, user rules) + derived `orchestrate` on main Agent dispatch (user pick 2026-10-23) write effort-only routes; model axis falls through to held run or stays on cur. Amends [[BDR-115]] amendment 2026-10-13 clause "ultrathink floor model beats guess": mid-turn ultrathink after classifier demotion keeps guessed model that turn (floor raises effort only). Accepted edges: haiku session → keywords no-op; session model held → keyword turn = plain-turn fallback (sonnet), not fable; no stand-in notch on keyword/dispatch turns; dispatch after demotion orchestrates on demoted model. Rejected: cap in `wantedModel` (r1) → shadowed held run slot + fail-open on `sessionModel` '' after /reload-plugins. Contract `.claude/tasks/contracts/2026-10-23-router-keyword-cap-0019.md`, plan r2.1 `.claude/tasks/plans/2026-10-23-router-keyword-cap-0019.md`. Links [[LRN-226]], [[EVAL-048]].
|
||||
- **Amendment (2026-10-25, bugfix 8e93507)**: accepted edge "haiku session → keywords no-op" REVERSED (user pick: Haiku 5.x like any model). No-effort rule now Haiku 4.x only (`NO_EFFORT = 'claude-haiku-4'`); on Haiku 5.x keyword rules, `ultrathink`, sticky effort and derived orchestrate apply, unrouted step passes engine effort. Links [[BLK-035]], [[LRN-230]].
|
||||
- **Status (2026-11-16)**: accepted, amended by BDR-136: ceiling model-agnostic, main loop writes no code outside a skill run on any model, held by router write guard (fail-closed since 859e504); Fable-only /hotfix-scope ceiling gone (0bf4620)
|
||||
- **Status (2026-11-16)**: accepted, amended by BDR-150: no Fable via rows, opus heads best and big; session opus, Fable is the advisor
|
||||
|
||||
## BDR-124 — Foreign code keeps its own style; limits strict in our code only [accepted] (2026-10-23)
|
||||
- **Decision**: CLAUDE.global.md header: strict limits in our code, "Foreign code: keep its style" section wins on foreign code (granularity, layout, naming, comments, idioms; limits = guidelines; no refactor toward our norms unless asked). Origin = `lib/code-origin.sh`: `git config claude.origin` (ours|foreign, per clone, invalid → foreign) > root commit author vs `user.email` + global `claude.identity` list; undecidable (no git, shallow, no commit) → foreign, said. Session banner line `code : ours|foreign (reason)`. refactorer, /refactor, /audit-delta conformity follow it; refactorer cites CLAUDE.md limits, no copy.
|
||||
@@ -1490,6 +1491,7 @@ Supersedes [[BDR-030]].
|
||||
- **Residual (user-accepted, F)**: Fable held or cap exceeded at the route event → no question, slot unstamped; hold lapse / compaction → next step upgrades unasked (breaker behaviour today). `/model <x>` while any dialog holds `asking` falls to native (pre-existing, BDR-126).
|
||||
- **Gates**: r1 correctness FATAL(6) robustness FATAL(6) simplicity CONCERNS(4) → r2 → confirmation CONCERNS(6) → r3; feater DONE 28 TIER tests (590 → 618); GATE 0 MET 5/5; verifier CONFORME 11/11 first pass; security PASS (1 LOW fixed: `clean(skill)`). Commits 9a24a62, docs 789fd00.
|
||||
- **Refs**: [[BDR-116]], [[BDR-123]], [[BLK-032]], [[LRN-231]], [[LRN-242]], [[LRN-254]].
|
||||
- **Status (2026-11-16)**: accepted, amended by BDR-150: ask-before-Fable inert unless an override heads best with fable; stored answers gate on fableHeadsBest
|
||||
|
||||
## BDR-136 — Phase 2 guards: native ask merge gate, bench-proven shell rules, rtk rc 3 passthrough, router write guard with a session skill latch, code-limits signal [accepted] (2026-11-02)
|
||||
- **Context**: BDR-125 phase 2, feature/phase2-guards, contract `.claude/tasks/contracts/2026-11-02-phase2-guards-1545.md`, bench `.claude/tasks/plans/2026-11-02-phase2-bench/`.
|
||||
@@ -1596,3 +1598,12 @@ Supersedes [[BDR-030]].
|
||||
- **Alternatives rejected**: drop every routing line (loses the gstack counterweight + pack-off remedy); "never gstack ship" text in ship-feature description (ship retired from every pack, lib/gstack-removed.list: dead text); trigger "open a PR" (no skill opens a PR).
|
||||
- **Accepted**: description triggers not pinned by a test (darwin rewrite risk, MINOR); listing budget +~60 chars.
|
||||
- **Amends**: BDR-130 Deferred item done; BDR-125 phase 3 "Skill routing table lines → descriptions" done.
|
||||
|
||||
## BDR-150 — Fable = advisor only, never auto-routed; Opus heads best and big [accepted] (2026-11-16)
|
||||
- **Context**: user 2026-11-16: Fable set as advisor (`/advisor`, settings `advisorModel: fable`, bdbf1cd); "no auto-route on Fable, Opus = strongest model to auto-route; advisor called whenever a smarter agent is needed". feature/router-no-fable, contract `.claude/tasks/contracts/2026-11-16-router-no-fable-1100.md`, plan r3 after 2 challenge rounds.
|
||||
- **Decision**: DEFAULT_CONFIG tiers `best` and `big` = ['opus','sonnet']; fable kept in `models` and at `fallback[0]` (capability rank: hand pick of fable ranks above opus, breaker never moves up to it). Row Change dialog (CHOICE_ALIASES) drops fable; bare /model keeps it ("fable (custom)"). Ask-before-Fable (askTier) fires only when alias heading best = fable (override); stored "instead" answers gate on fableHeadsBest; `/route ask-tier on` replies inert. "tiers.best does not lead the fallback chain" log deleted. routing.json untouched (askTier "best"): mod runs live from tree, old-code sessions would route Fable unasked on `askTier: false`. Agents floor = tier head (opus) for every tier.
|
||||
- **Why**: Fable reasoning now comes via advisor tool (server tool, hooks cannot see/refuse it); routing main/agents to Fable duplicated cost (BLK-030 credits) and ceiling logic.
|
||||
- **Alternatives rejected**: tiers in tracked routing.json (new layer code, breaks machine-key doctrine); per-machine `~/.claude/model-router.json` (not portable to work machine, near hard_deny mod clause); removing askTier feature (churn; kept dormant, `shortcut:` at TIER_HEAD); fable out of `fallback` (appended last → hand-picked fable ranks below haiku); routing.json `askTier: false` (live-install consent gap).
|
||||
- **Residuals**: opus outage → best/big agents spawn on held opus (gates fail, verify-secure retries then escalates; was move-up to Fable); single opus outage → sonnet stands in with effort notch; fable-baseline session (native Everywhere) demoted to opus per downgrade caps, sessionPin pins general-purpose to fable; skill row answering opus+xhigh in Change lands on `plan`. Test rig: shipped tiers default, `legacyTiers: true` on 47 fable-mechanics tests.
|
||||
- **Gates**: r1 3 lenses CONCERNS → r2 → re-challenge CONCERNS → r3; feater DONE 661 tests; GATE 0 MET; verifier CONFORME; security PASS (MEDIUM fixed: stored answer survives ask-tier off); commits 24b0f35, docs 42385d2.
|
||||
- **Refs**: amends [[BDR-123]], [[BDR-135]]; supersedes [[LRN-224]]; [[BLK-030]], [[BDR-077]] (Fable dispatch pins: separate run).
|
||||
@@ -769,3 +769,4 @@ rules:
|
||||
- A6 router write guard fail-closed: .catch uses next.called (called → replay next(e); not called → hold main-loop out-of-run non-doc write with user-chosen deny); Skill latch ignores isError answers (BDR-137 shape, was one-call bypass) (859e504). BDR-136 (4) amended. 3 plan rounds: r1/r2 latches unreachable, engine-documented pattern won. gates.sh 120s timeout < make test ~135s: GATES_TIMEOUT=400. Next A7.
|
||||
- C7 hard_deny mod off-switch clause: entry 6 names enabled:false in ~/.claude/<mod>.json, enabledPlugins false, profile.sh disable of mod (8c21c4b); off switches only, no source paths (2026-10-25 challenge). Staged via crafted index blob so user's dirty settings.json /model change stayed out. Residuals (closed list, /route off in run, over-refusal risk) BUGS-FOUND e22893d.
|
||||
- C1-C3 BDR-125 gap: 11 status lines (BDR-125 summary; BDR-079/017 + LRN-155 superseded, 7 amended by BDR-134/136; amended entries still recalled with old title, only superseded/deprecated leave recall); C3 deferrals → 7 TODO lines + 3 dated drops; C6 dropped (throwaway repos, user).
|
||||
- Router no-Fable (BDR-150, 24b0f35, docs 42385d2): best/big = opus>sonnet, fable ranked + hand pick only, Change dialog drops fable, askTier inert unless override heads best with fable; routing.json untouched (live install). 2 challenge rounds caught dialog labels, live-install consent gap, legacy rig masking. BDR-123/135 amended, LRN-224 superseded. Next: Fable pins → opus, advisor doctrine.
|
||||
@@ -1771,6 +1771,7 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
|
||||
- **Date**: 2026-10-21
|
||||
- **Context**: feat A (routing policy BLK-032) moved verifier + security-auditor to new `audit` phase (best = fable>opus>sonnet). Plan r1 set frontmatter `model: fable` (census rule floor = tier head) and claimed "Fable down → falls to opus". False: `inTier` (register.ts) writes a model only if it is the head or ranks above it, else logs "tier … down, frontmatter model kept" → fable frontmatter = gates spawn on dead Fable every credit outage (BLK-030/031). Lower tiers never hit it (head sonnet/opus always has a higher alias). Caught by correctness + robustness lenses r1, not by plan author.
|
||||
- **Apply**: an agent on a best-tier row keeps opus frontmatter; router lifts to Fable when up. Census `tier_floor` (effort-routing.test.sh) locks 2nd alias for best. Residual: spawn path unprobed → first gate of an outage window still dies once, retry lands on floor (register.test.ts audit-row case b). Links [[BLK-030]], [[BLK-032]], [[LRN-204]].
|
||||
- **Status (2026-11-16)**: superseded by BDR-150: best and big headed by opus, agents floor on tier head for every tier (2nd-alias rule gone)
|
||||
|
||||
## LRN-225 — Gate on the tool's own exit-code contract, never on its prose; control through the gate's own helper on a clean/dirty pair
|
||||
- **Date**: 2026-10-22
|
||||
|
||||
@@ -144,6 +144,7 @@ Scope (user 2026-11-11) [gated]: D1-D6 + the 6 dead deny entries + MIGRATION/CHA
|
||||
- [ ] deferred (BDR-125 4c, re-challenge 2026-10-25): Read path triggers, Bash command triggers, memory_search tool for skills (analyze-before-plan pass 1); reopen when a /memory-feedback gate shows misses the prompt and error channels cannot reach (no triggers= field: 4a shipped without Meta, BDR-125 Am.7)
|
||||
- [ ] conditional (BDR-146 residual): a noisy entry on the tool channel survives the /prune-memory title + lesson rewrite (the 4-gram matcher reads body backtick spans, SKILL.md STEP 1.D limit); act when a tool-gate FEEDBACK.md lists one (gate 1 2026-11-12: 0 tool records)
|
||||
- [ ] deferred (BDR-125 Am.7, phase 2): commit-msg hook for the 50-word cap; revisit when overruns grow (4/78 on 2026-10-25; 17/530 ≈ 3 % on 2026-11-16 recount, the gap audit's 21/526 not reproducible)
|
||||
- [ ] shortcut (register.ts TIER_HEAD, BDR-150): the ask-before-Fable dialog (askTier, BDR-135) is inert while no tier is headed by fable; remove the feature (TIER_HEAD, INSTEAD_TIER, tier ask paths and their LEGACY tests) if Fable never returns to a tier
|
||||
|
||||
### Known limits (accepted, no action)
|
||||
- model-router spawn path unprobed: 1st gate of an outage window dies once (seen live 2026-10-23, retry lands on the opus floor); tour/onboard/audit-delta no retry; onAutoSwitch with main on opus marks nothing if the engine auto-falls a gate off Fable
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
# CONTRACT — router-no-fable (user 2026-11-16: Fable = advisor, never auto-routed)
|
||||
- date: 2026-11-16 | flow: feat | branch: feature/router-no-fable (off develop)
|
||||
- status: active
|
||||
|
||||
## REQUEST (verbatim — IMMUTABLE)
|
||||
Router: Fable leaves auto-routing (user 2026-11-16: Fable is now the advisor via /advisor; Opus becomes the strongest model the router routes to). In mods/model-router/hooks/register.ts DEFAULT_CONFIG: tiers.best and tiers.big become ['opus', 'sonnet']; fable stays in `models` and at the head of `fallback` (capability rank, so a hand-picked Fable session never ranks below opus); the mergeRouting "tiers.best does not lead the fallback chain" warning must not fire on these defaults. mods/model-router/routing.json: "askTier": false (the ask-before-head dialog was a Fable credit guard, BLK-030). Fable stays selectable by hand in the /model dialog (user choice). lib/tests/effort-routing.test.sh tier_floor: the best-tier "2nd alias" floor special case goes (floor = tier head for every tier). Tests in mods/model-router/hooks/register.test.ts: the shipped defaults route plan/judge/audit to opus, best and big never pick fable even with opus down, a hand-picked fable session is not treated as below opus, no "does not lead" log.
|
||||
|
||||
## CLARIFICATIONS
|
||||
- [gated 2026-11-16] user: Fable is the advisor (/advisor, settings advisorModel fable, bdbf1cd); "on ne doit plus autorouter sur fable, opus devient le model le plus puissant a autorouter"; Fable stays selectable by hand in the /model dialog.
|
||||
- [internal 2026-11-16] plan .claude/tasks/plans/2026-11-16-router-no-fable-1100.md, section r2 wins. Explicit `model: "fable"` sub-agent pins and the advisor doctrine are separate runs (file cap).
|
||||
- [gated 2026-11-16] user: Fable leaves the row Change dialog (CHOICE_ALIASES); the bare /model dialog keeps Fable, labelled "fable (custom)".
|
||||
- [internal 2026-11-16, challenge r1] deviation from REQUEST: routing.json is NOT edited (askTier stays "best"). The mod runs live from this tree; old-code sessions reading `askTier: false` would route to Fable without consent. Instead tierHead asks nothing when best's head equals big's head (true on shipped tiers), so the dialog is inert whatever askTier says, and comes back if an override puts Fable at the head of best. The "does not lead" check is deleted, not reworded.
|
||||
- [internal 2026-11-16, re-challenge r2] plan section r3 amends r2 (final): ask-before-Fable fires only when the alias at the head of `best` is `fable` (stored "instead" answers ignored otherwise; `/route ask-tier` replies say `inert`); targetPhase step dropped (already shipped); test buckets SHIPPED / LEGACY / REWRITTEN with test names reported; one-pass edit order (live install).
|
||||
|
||||
- [gated 2026-11-16, after verify] user: keep routing.json as is (askTier stays "best", inert on shipped tiers). Security MEDIUM fixed: stored "Opus instead" answers gate on fableHeadsBest, not tierHead (survive `/route ask-tier off`), test added.
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. model-router suite green.
|
||||
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH claude plugin test mods/model-router </dev/null 2>&1 | grep -cE '(^| )0 fail'
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
2. effort-routing suite green.
|
||||
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/effort-routing.test.sh </dev/null 2>&1 | tail -1
|
||||
EXPECT: all suites green
|
||||
EVIDENCE: MET exit=0 marker-found :: all suites green
|
||||
3. Default best and big tiers are opus then sonnet.
|
||||
CHECK: cd /home/bchanot/Documents/claude && grep -cE "^ (best|big): \['opus', 'sonnet'\],$" mods/model-router/hooks/register.ts
|
||||
EXPECT: 2
|
||||
EVIDENCE: MET exit=0 marker-found :: 2
|
||||
4. The "does not lead the fallback chain" check is gone.
|
||||
CHECK: cd /home/bchanot/Documents/claude && { grep -c 'does not lead the fallback chain' mods/model-router/hooks/register.ts || true; }
|
||||
EXPECT: 0
|
||||
EVIDENCE: MET exit=0 marker-found :: 0
|
||||
5. Full make test green (run gates with GATES_TIMEOUT=400: the full suite takes ~135 s).
|
||||
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test </dev/null 2>&1 | tail -1
|
||||
EXPECT: all suites green
|
||||
EVIDENCE: MET exit=0 marker-found :: all suites green
|
||||
6. New tests on the SHIPPED defaults (plan r2 step 9 as amended by r3 step 6) exist; (i) plan/audit, (ii), (v) and (vi) options are red on the pre-change register.ts, (v) also red without r3 step 2; LEGACY-bucket tests keep their assertions unchanged; test names reported per bucket. `claude plugin validate --strict mods/model-router` passes; touched lines ≤ 80 columns (manual).
|
||||
|
||||
## FILE SCOPE
|
||||
mods/model-router/hooks/register.ts (DEFAULT_CONFIG tiers ~:298-303, mergeRouting check ~:618-621, CHOICE_ALIASES ~:226, tierHead ~:1177, nextAvailable ~:1147, wantedModel ~:1608, spawnTier ~:3247, flagCommand/askTierWord, comment ~:2802), mods/model-router/hooks/register.test.ts (rig fixture defaults and new tests), lib/tests/effort-routing.test.sh (tier_floor and its flip-test fixture ~:50-110).
|
||||
@@ -0,0 +1,280 @@
|
||||
# PLAN — router-no-fable (Fable = advisor, Opus = strongest auto-route)
|
||||
|
||||
## CONTEXT
|
||||
- register.ts DEFAULT_CONFIG (~:290-305): models {haiku, sonnet, opus,
|
||||
fable}; tiers best ['fable','opus','sonnet'], big ['opus','fable',
|
||||
'sonnet'], work ['sonnet','opus'], cheap ['haiku','sonnet']; fallback
|
||||
['fable','opus','sonnet','haiku'] (rank = index, best first).
|
||||
- Tiers and fallback are machine keys: the tracked routing.json contributes
|
||||
only phases, skills and agents (routingTables ~:887); the override
|
||||
`~/.claude/model-router.json` is absent on this machine. So the shipped
|
||||
policy lives in DEFAULT_CONFIG.
|
||||
- mergeRouting (~:608-625) logs "tiers.best does not lead the fallback
|
||||
chain; rank comes from the fallback list alone" when tiers.best[0] !==
|
||||
fallback[0].
|
||||
- pickFallback/withEveryAlias (~:473-502): every `models` alias ends up
|
||||
ranked; one missing from the chain is appended LAST with a log. So fable
|
||||
must stay in `fallback`, at its head (true capability rank).
|
||||
- askTier (BDR-135): tierHead = head of `best`; with best headed by opus
|
||||
the dialog ("Fable" / "Opus instead" / stop, TIER_HEAD hardcoded) would
|
||||
ask before every opus route. routing.json sets askTier.
|
||||
- inTier (~:3857): an agent moves UP from its frontmatter alias to the
|
||||
first available model of its tier only if that model is the head or ranks
|
||||
above it; else frontmatter kept + "tier … down" log. With best/big =
|
||||
[opus, sonnet] and opus down: sonnet ranks below opus → frontmatter
|
||||
(opus) kept. Today big-tier agents with opus down move UP to fable; after
|
||||
this change they stay on opus frontmatter (by design: never below the
|
||||
floor; the engine's own fallback applies). Main loop with opus down:
|
||||
breaker fallback goes down the chain to sonnet (unchanged mechanism).
|
||||
- effort-routing.test.sh tier_floor (~:56-60): for `best` it reads the
|
||||
tier's 2nd alias as the agents' frontmatter floor (LRN-224, Fable credit
|
||||
outage); for other tiers the head. Its flip-test fixture (~:94-110)
|
||||
carries the old fable tiers.
|
||||
- Test rig (register.test.ts): synthetic routing.json from baseRouting()
|
||||
(phases only, ~:87); tiers come from DEFAULT_CONFIG; the machine override
|
||||
comes only from `fx.home` (HOME_FILE). 381 FABLE references; many tests
|
||||
assume best is headed by fable.
|
||||
|
||||
## APPROACH
|
||||
1. register.ts DEFAULT_CONFIG: `best: ['opus', 'sonnet']`, `big: ['opus',
|
||||
'sonnet']`; models unchanged (fable kept); fallback unchanged
|
||||
(['fable', 'opus', 'sonnet', 'haiku']). Comment on the tiers block: Fable
|
||||
is ranked (hand picks, the advisor model) but never an auto-route target
|
||||
(user 2026-11-16); a machine override may add it back to a tier.
|
||||
2. register.ts mergeRouting warning: fire only when the head of `best`
|
||||
ranks BELOW the head of another tier in the fallback chain (a tier
|
||||
"better" than best: a real misconfiguration), not merely when it is not
|
||||
fallback[0]. Wording: `model-router: tiers.best head <alias> ranks below
|
||||
tiers.<name> head <alias> in the fallback chain`. Use keyWord() for the
|
||||
names. Defaults: best opus, big opus, work sonnet, cheap haiku → no log.
|
||||
3. routing.json: `"askTier": false` (replaces "best"). The askTier code
|
||||
stays (a machine override with `askTier: "best"` and fable back in best
|
||||
still works); TIER_HEAD 'Fable' label hardcoded = residual (dormant).
|
||||
4. effort-routing.test.sh: tier_floor drops the best-tier special case
|
||||
(floor = tier head for every tier); its comment says why (best is headed
|
||||
by opus since 2026-11-16, LRN-224's credit floor no longer applies). The
|
||||
flip-test fixture moves to the new defaults (best ['opus','sonnet'], big
|
||||
['opus','sonnet']) and still proves the readers reject the wrong
|
||||
verdicts.
|
||||
5. register.test.ts:
|
||||
a. Measure first: apply 1-3, run `claude plugin test mods/model-router`,
|
||||
count red tests.
|
||||
b. If more than 25 are red: add a rig default so existing mechanics
|
||||
tests keep a 3-deep tier set: when a fixture gives no `home` and does
|
||||
not set `shippedTiers: true`, newWorld writes HOME_FILE with
|
||||
`{ tiers: { best: ['fable','opus','sonnet'], big: ['opus','fable',
|
||||
'sonnet'] } }` (machine override; nothing else in it). Tests that
|
||||
assert the override's absence (source word, askTier machine value,
|
||||
`fx.home` users) get `shippedTiers: true` or merge their `home` with
|
||||
these tiers, case by case, without changing what they assert. If 25
|
||||
or fewer are red: update each red test's expectation to the new
|
||||
defaults directly.
|
||||
c. New tests, all on the SHIPPED defaults (`shippedTiers: true`, no
|
||||
override): (i) typed plan, judge and audit rows route to opus (main
|
||||
and spawn); (ii) with opus marked down, a best-tier and a big-tier
|
||||
route never choose fable (main falls to sonnet via the breaker; a
|
||||
big-tier rowed agent keeps its opus frontmatter, "tier … down" log);
|
||||
(iii) a /model hand pick of fable, then a rate limit on fable → main
|
||||
moves to opus (fable ranks above opus); (iv) boot logs no "does not
|
||||
lead" / "ranks below" line; (v) the warning still fires on a machine
|
||||
override whose best head ranks below big's head (e.g. best
|
||||
['sonnet'], big ['opus']).
|
||||
d. Red-first: (i), (ii) and (iv) fail on the pre-change register.ts.
|
||||
|
||||
## EDGE CASES
|
||||
- Machine override re-adding fable to a tier: honored (validated by
|
||||
acceptTier against models).
|
||||
- Hand pick of fable through the /model dialog: unchanged (BARE_ALIASES and
|
||||
CHOICE_ALIASES keep fable).
|
||||
- Sessions already on fable (settings model fable): breaker and probe
|
||||
unchanged; phases route down to opus only per existing downgrade rules.
|
||||
- askTier absent in a foreign routing.json: code default "best" now means
|
||||
asking before opus routes; acceptable (only this repo's routing.json ships
|
||||
the mod) and noted as residual.
|
||||
|
||||
## DISPOSITION
|
||||
Amends BDR-123 (Fable via best-tier rows → no Fable auto-route; session
|
||||
opus), BDR-135 (askTier off by config; code kept dormant), LRN-224 (2nd-alias
|
||||
floor no longer applies: head = floor = opus). BDR-077 W1 Fable dispatch
|
||||
pins handled by the next run (pins). Honors BDR-129-style single source:
|
||||
the policy lives in DEFAULT_CONFIG, tracked.
|
||||
|
||||
## FILES
|
||||
mods/model-router/hooks/register.ts, mods/model-router/hooks/register.test.ts,
|
||||
mods/model-router/routing.json, lib/tests/effort-routing.test.sh.
|
||||
|
||||
## r2 (challenge r1: correctness CONCERNS(3), robustness CONCERNS(3), simplicity CONCERNS(2)) — SUPERSEDES APPROACH 2-5, EDGE CASES and DISPOSITION above
|
||||
Landed: (a) the row Change dialog and /model labels change on shipped tiers
|
||||
(fable heads no phase → dead option; opus heads best AND big → "opus
|
||||
(best)", and targetPhase maps opus/xhigh to `plan`, judge unreachable)
|
||||
(corr 1, rob 2, simp 2); (b) askTier is not dormant: `/route ask-tier on`,
|
||||
a machine `askTier: "best"` or an absent key ask "Run it on fable?" and
|
||||
"Opus instead" runs SONNET, gates included (corr 2, rob 1, simp 6); (c) the
|
||||
mod runs live from this working tree: a routing.json `askTier: false` read
|
||||
by OLD-code sessions on /clear or reload removes the Fable consent guard
|
||||
while their old tiers still route to Fable (rob 4); (d) a rig default of
|
||||
legacy tiers would test a config nobody ships and mask (a)/(b) (rob 3,
|
||||
simp 3); (e) the warning reword is new scope (simp 1); (f) test (iv) is not
|
||||
red-first, judge in (i) already opus (corr 3, simp 5); (g) flip fixture
|
||||
must keep distinct heads (corr 8, rob 7, simp 4); (h) unranked `after` in
|
||||
nextAvailable can display fable (corr 10).
|
||||
User answers 2026-11-16: Fable leaves the row Change dialog
|
||||
(CHOICE_ALIASES); /model keeps Fable, label stays "fable (custom)" (no
|
||||
label code).
|
||||
|
||||
1. register.ts DEFAULT_CONFIG: tiers best ['opus','sonnet'], big
|
||||
['opus','sonnet']; models and fallback unchanged (fable kept, fallback[0],
|
||||
capability rank). Comment on the tiers block: Fable is ranked (hand
|
||||
picks, advisor model) but never an auto-route target (user 2026-11-16);
|
||||
a machine override may put it back in a tier.
|
||||
2. register.ts mergeRouting: DELETE the "tiers.best does not lead the
|
||||
fallback chain" check (log only, no consumer). No replacement warning.
|
||||
3. register.ts CHOICE_ALIASES: ['opus', 'sonnet', 'haiku'] (fable out of
|
||||
the row Change dialog). BARE_ALIASES unchanged (fable stays in /model).
|
||||
4. register.ts targetPhase (~:2834-2841, row Change dialog: model+effort →
|
||||
phase): tie-break when several phases match the picked model and effort
|
||||
= the row's CURRENT phase first if it matches, else the first match as
|
||||
today. So an agent row on `judge` answered "opus, xhigh" stays `judge`.
|
||||
5. register.ts tierHead (~:1177-1181): return undefined (no tier ask) also
|
||||
when the head of `best` equals the head of `big` (the "instead" option
|
||||
would not be a different model). So on shipped tiers no ask fires
|
||||
whatever askTier says (routing.json, machine override, `/route ask-tier
|
||||
on`); a machine override putting fable back at the head of best brings
|
||||
the consent dialog back unchanged. routing.json is NOT edited (askTier
|
||||
stays "best"): old-code sessions keep their Fable consent guard until
|
||||
/reload-plugins. `shortcut:` comment at TIER_HEAD: the ask-tier feature
|
||||
is inert on shipped tiers (Fable credit guard, BLK-030); remove it if
|
||||
Fable never returns to a tier; harvested into TODO.md by the closing
|
||||
step.
|
||||
6. register.ts nextAvailable (~:1147-1151): when `after` is given and not
|
||||
ranked (findIndex -1), return undefined instead of walking from the
|
||||
chain's head (display-only fable leak).
|
||||
7. effort-routing.test.sh: delete tier_floor; field_of (~:64) uses
|
||||
tier_head; drop the duplicate assertion (~:123) and make the best-tier
|
||||
floor assertion (~:122) expect the fixture's best head. KEEP the
|
||||
synthetic flip fixture's distinct heads (fable-headed best, opus-headed
|
||||
big) so a wrong-tier read still fails. Comment: best is headed by opus
|
||||
since 2026-11-16, the agents' floor is the tier head for every tier.
|
||||
8. register.test.ts rig: shipped tiers stay the default (DEFAULT_CONFIG).
|
||||
New fixture flag `legacyTiers: true` → newWorld merges `{ tiers: {
|
||||
best: ['fable','opus','sonnet'], big: ['opus','fable','sonnet'] } }`
|
||||
UNDER fx.home (`{ tiers: LEGACY, ...fx.home }`) into HOME_FILE. Measure
|
||||
the red tests after steps 1-6; for each red test:
|
||||
- the test is about a 3-deep chain or Fable mechanics (breaker or
|
||||
stand-in with fable, ask-tier dialog, hand pick of fable, Fable [1m],
|
||||
fable efforts) → add `legacyTiers: true`, assertion unchanged;
|
||||
- the test is about shipped behavior (row labels, Change dialog options,
|
||||
/model labels, rows routed on defaults, noHome / env-less routing) →
|
||||
update the expected value to the shipped tiers.
|
||||
Tests that set HOME_FILE directly mid-test (:705, :1812, :1835, :2328,
|
||||
:2791, :3392, :3735, :4410) keep what they assert: if they need legacy
|
||||
tiers, include them in the HOME_FILE they write. Report the count per
|
||||
bucket.
|
||||
9. New tests on the SHIPPED defaults (no legacyTiers), main booted on OPUS:
|
||||
(i) typed plan and audit rows route to opus (red-first: plan and audit
|
||||
went to fable); (ii) with opus marked down (switchTo(OPUS, SONNET,
|
||||
'auto') or the breaker), best and big routes never choose fable: main
|
||||
on sonnet, a big-tier rowed agent keeps its opus frontmatter with the
|
||||
"tier … down" log (assert the model only; the stand-in effort notch is
|
||||
expected) (red-first); (iii) hand pick fable via the /model dialog, then
|
||||
a rate limit on fable → main moves to opus; (iv) boot logs no "does not
|
||||
lead" line (regression guard, not red-first); (v) askTier "best" in the
|
||||
machine override on shipped tiers → no tier question, the audit gates
|
||||
are not sent to sonnet (red-first); (vi) row Change dialog options are
|
||||
opus/sonnet/haiku labels without fable, and an agent row on `judge`
|
||||
answered opus + xhigh stays `judge` (red-first); (vii) bare /model
|
||||
dialog still offers fable, labelled "fable (custom)".
|
||||
|
||||
EDGE CASES (real behavior):
|
||||
- Opus outage: best/big agents keep their opus frontmatter and spawn on the
|
||||
held opus (the gates fail, verify-secure-loop retries and escalates);
|
||||
today they moved up to Fable. Accepted residual (no engine fallback key
|
||||
in settings.json; LRN-224 hazard moves from fable to opus).
|
||||
- A single opus outage now makes sonnet stand in for the head of best and
|
||||
big (standInEffort notches the row's effort up).
|
||||
- A session whose baseline is fable (native /model Everywhere, not a hand
|
||||
pick) is routed by the tiers: best phases go to opus, or to sonnet with
|
||||
opus held. Accepted (Fable is not an auto-route target).
|
||||
- Rollout: the working tree is the live install. Open sessions keep old
|
||||
code until /reload-plugins; routing.json is untouched so they keep the
|
||||
Fable consent dialog meanwhile. Tell the user to /reload-plugins after
|
||||
the merge.
|
||||
|
||||
DISPOSITION: amends BDR-123 (no Fable via rows; session opus), BDR-135
|
||||
(ask-tier inert on shipped tiers by rule, kept for overrides), LRN-224
|
||||
(floor = tier head for every tier). BDR-077 W1 Fable pins: next run.
|
||||
FILES: register.ts, register.test.ts, lib/tests/effort-routing.test.sh
|
||||
(routing.json dropped from scope).
|
||||
|
||||
## r3 (re-challenge r2: correctness CONCERNS(3 MAJOR), robustness CONCERNS(1 MAJOR)) — amends r2; final
|
||||
1. r2 step 4 DROPPED: targetPhase already prefers the row's current phase
|
||||
(register.ts:2836-2843). Residual: on shipped tiers a SKILL row that
|
||||
answers opus + xhigh in the Change dialog lands on `plan` (run-holding)
|
||||
rather than `judge`; agents unaffected in effect (plan/judge/audit all
|
||||
route opus xhigh).
|
||||
2. r2 step 5 REPLACED: `tierHead` returns undefined unless the alias at
|
||||
the head of `best` is `fable` (the dialog text is fable-specific:
|
||||
TIER_HEAD, "Run it on fable?"). So the ask-before-Fable dialog fires
|
||||
only when an override puts Fable at the head of best, whatever `big`
|
||||
says. Stored answers: `wantedModel` (slot tierChoice ~:1608) and
|
||||
`spawnTier` (tierSpawn ~:3247) ignore a stored "instead" answer while
|
||||
tierHead(st) is undefined. `/route ask-tier on|off` keeps writing the
|
||||
flag; when tierHead is undefined its reply appends ` (inert: best is not
|
||||
headed by fable)` and `/route show`'s ask-tier word reads `inert` in that
|
||||
case. `shortcut:` comment at TIER_HEAD: the ask-before-Fable dialog only
|
||||
runs when an override heads best with fable (BLK-030 credit guard);
|
||||
remove the feature if Fable never returns to a tier.
|
||||
3. register.ts doc comment example at ~:2802 ("fable (best)") updated to a
|
||||
shipped label ("opus (best)").
|
||||
4. Edit order: apply r2 steps 1, 2, 3, 6 and r3 step 2 in ONE pass on
|
||||
register.ts before running anything (the tree is the live install: a
|
||||
session started mid-edit must never see opus-headed best with the old
|
||||
ask armed).
|
||||
5. Test buckets (replaces r2 step 8's rule), for each red test after the
|
||||
code change; report test NAMES per bucket:
|
||||
a. SHIPPED (default): update the expected value to the shipped tiers.
|
||||
Wins unless the assertion cannot hold on shipped tiers.
|
||||
b. LEGACY: `legacyTiers: true`, assertion unchanged, ONLY when the
|
||||
assertion needs a tier holding fable or a 3-deep chain (fable
|
||||
breaker/stand-in, ask-before-Fable dialog, fable [1m] routing).
|
||||
c. REWRITTEN: a test that answers the row Change dialog with
|
||||
'fable (best)' (fable efforts loop ~:3058, upgrade toasts ~:3164,
|
||||
~:3173, fableEfforts ~:4527 used at ~:4538, ~:4544): answer
|
||||
'opus (best)' on shipped tiers (same [medium, high, xhigh, max]),
|
||||
keep the test's intent (residual f: null phase ignored, file
|
||||
without phases).
|
||||
Shared constants used by both buckets (ROUTED ~:2814, STEP1 ~:6440,
|
||||
TO_JUDGE ~:31) are split per bucket (e.g. ROUTED_LEGACY) rather than
|
||||
moving a whole group. TO_JUDGE ('opus (big)') users go LEGACY.
|
||||
newWorld throws when `legacyTiers` is combined with `noHome`, an empty
|
||||
env, `home: 'broken'` or kinds that turn the router off (the override
|
||||
cannot be read there; a silent shipped run would lie).
|
||||
6. New tests (replace r2 step 9 (i) and (v); others stand):
|
||||
(i) typed plan, judge and audit rows route to opus on shipped tiers,
|
||||
main booted on OPUS (plan and audit red-first; judge a guard).
|
||||
(v) two cases on shipped tiers with the machine override's askTier
|
||||
"best": (v-a) an audit-row spawn (auditVerifier edit, ~:7162) with an
|
||||
"Opus instead" answer queued → no ask, the answer left unconsumed, the
|
||||
spawn on OPUS; (v-b) main booted on SONNET types a best row → no ask,
|
||||
main on OPUS. Prove (v) red against the change WITHOUT r3 step 2 (a
|
||||
scratch copy with steps 1-3, 6 only), not only against the pre-change
|
||||
file. (viii) override heading best AND big with fable: the dialog asks
|
||||
(fable alias at the head of best).
|
||||
7. Residuals restated:
|
||||
- Rollout: live from the FIRST edit on the checked-out branch (new
|
||||
sessions load the tree); open old-code sessions keep their dialog for
|
||||
interactive best-tier routes only (old big = [opus, fable, sonnet]
|
||||
still sends judge spawns to Fable unasked while opus is held;
|
||||
headless runs too). After the merge: /reload-plugins in every open
|
||||
session.
|
||||
- Fable-baseline session: best phases now demote it to opus (or sonnet
|
||||
with opus held) only through downgradeCall, so `mainModelSwitch` and
|
||||
the 120k cold-read cap apply (past either, main stays on Fable); on a
|
||||
demotion, sessionPin pins general-purpose spawns to the Fable session
|
||||
model.
|
||||
- Opus outage: best/big agents spawn on the held opus (frontmatter
|
||||
floor), the gates fail, verify-secure-loop retries once then
|
||||
escalates. Main: sonnet stands in with the effort notch.
|
||||
FILES (final): mods/model-router/hooks/register.ts,
|
||||
mods/model-router/hooks/register.test.ts, lib/tests/effort-routing.test.sh.
|
||||
README.md, USAGE.md and CHANGELOG through the doc-sync step.
|
||||
Reference in new issue
Block a user