forked from bchanot/claude
chore(memory): feat C BLK-032 resolved, BDR-123 amendment, LRN-226, EVAL-048, plan and contract
This commit is contained in:
1 parent
d2d590f834
commit
b097c894a1
8 files changed
+334
-4
No files matched your search
@@ -7,7 +7,7 @@
|
||||
- [~] routing policy /feat (user 2026-10-19, BLK-032): settings model → Opus 5.5; `judge` split (security-auditor + verifier → Fable, plan-challenger/analyzer/code review → opus); executors stay sonnet, "what to write" on opus; ceiling rule: main on Fable never writes code beyond a hotfix edit → dispatched executor; BDR superseding part of BDR-066 / BDR-115.
|
||||
- [x] feat A audit phase: d3635b6 (dialog writes, session model opus), a7d747c (audit row, opus floor, census, citers), dae142b (docs); gates 0/1/2 green, LRN-224
|
||||
- [x] feat B (feature/routing-ceiling-rule): ceiling rule CLAUDE.global.md, README doctrine, challenge citers, deploy `effort: xhigh`, routing.json `changed` cleared; ef13713 + 887cf89, gates 0/1/2 green, BDR-123; merge = user go
|
||||
- [ ] feat C: cap the default keyword prompt rules (plan|design|architecture, why|review|audit|analyze, ultrathink) at the session model, effort only like the classifier; close BLK-032 after
|
||||
- [x] feat C: cap the default keyword prompt rules (plan|design|architecture, why|review|audit|analyze, ultrathink) at the session model, effort only like the classifier; close BLK-032 after — DONE 2026-10-23 feature/router-keyword-cap: 786ee13 effort-only routes (keywords, ultrathink, derived orchestrate), d2d590f docs, gates green, BLK-032 resolved; merge = user go
|
||||
- [x] bugfix lib/tests/mods.test.sh (bugfix/mods-validate-advisory): verdict = `--strict` rc, positive control warnfix/cleanfix; 801992a, gates green, BLK-034 LRN-225; merge = user go. Residual: 3 historical contracts keep the old scrape oracle (BLK-034)
|
||||
- [ ] deferred (BDR-123): remap inline-writing best-tier skills (design Build stack, writing-skills, deprecation-and-migration, /tour single-project, /audit-delta 3d, /harden STEP 3); stale citers skills/feat/SKILL.md:6, skills/code-clean/SKILL.md:36
|
||||
- [ ] residuals (accepted): spawn path unprobed, 1st gate of an outage window dies once (tour/onboard/audit-delta no retry); onAutoSwitch with main on opus marks nothing if engine auto-falls a gate off Fable
|
||||
@@ -90,7 +90,7 @@ migration of shifters/pins/model-gate in wave 2 after proof; names model-router
|
||||
- [x] W1-B2 residuals CLOSED 2026-10-11: mods suite probe greps the `claude plugin test` usage line; doctor `plugin list --json` under `timeout 60`; update-all skips `@skills-dir` mods. Left (accepted): doctor `echo -e` helpers (tracked names only); fallback timeout guard orphans grandchildren without coreutils timeout.
|
||||
- [x] W1 close-out (2026-10-09): doc-sync opus audit SIGNIFICANT → user go all 10 → patched (README effort routing + Explore row + /route, USAGE, ARCHITECTURE mods/, CHANGELOG Unreleased) b22f894; hot-reload link removed from `~/.claude/dev-mods/<session>/`; BDR-115 amended (a6e2003). Pending user: `/reload-plugins` in this session (new sessions load the skills-dir copy by themselves); merge decision on feature/model-router-mod (`gitflow finish`, human signal)
|
||||
- [x] W1-C adaptive tiers (user 2026-10-09, contract `2026-10-09-model-router-tiers-1237`, plan r4 after 3 lenses + 2 confirmations: 6 BLOCKER + 29 MAJOR closed by named changes): phases name absolute tiers (best fable>opus>sonnet · big opus>fable>sonnet · work sonnet>opus · cheap haiku>sonnet); breaker fed by `classic.StopFailure` kinds rate_limit|overloaded|billing_error|model_not_found + `PostModelSwitch` auto (episode backoff 15→300 min, `/model` clears, `/clear` keeps, `/route reload` clears); fallback chain fable→opus→sonnet→haiku; main UPGRADE by default under `upgradeMaxTokens` 200k (fails closed on unknown usage), DOWNGRADE gated by `mainModelSwitch`; sticky `turnModel` per turn; derived `orchestrate` on background dispatches; prompt default rules (plan/reflect FR+EN, Unicode guards, skipped on `/…`, floor match, mid-turn). 58 tests; verifier CONFORME then 3 gap/hardening rounds; security PASS
|
||||
- [ ] W1-C accepted-by-design (security 2026-10-09, MEDIUM, not coded around; 2026-10-11 closed: unranked `models` alias → `withEveryAlias` on the default chain too, `canonical` segment boundary `known + '-'`): (1) the model itself can raise the main loop to fable for the rest of a turn through the `route` tool (plan/reflect/escalate/judge) or a background dispatch (derived orchestrate = best tier) — bounded by the turn and `upgradeMaxTokens`, no sticky route is model-callable; (2) `ultrathink` and the default keyword rules now mean "best tier" (fable) at the phase's effort, so an incidental keyword in a pasted composer prompt costs a fable turn (origin composer only). Residuals: `PostModelSwitch` `auto` covers "other programmatic change" (a healthy model could be marked 15 min); strikes never decay inside a session; a `[1m]` variant's `model_not_found` marks the base model until reload; a `models` alias added by the override without a `fallback` key stays unranked (`withEveryAlias` not applied to the default chain) so `leaveDown` skips the upgrade gates for it; `canonical` prefix match has no segment boundary (`claude-sonnet-5-50` would map to sonnet); classifier deferred to W2
|
||||
- [ ] W1-C accepted-by-design (security 2026-10-09, MEDIUM, not coded around; 2026-10-11 closed: unranked `models` alias → `withEveryAlias` on the default chain too, `canonical` segment boundary `known + '-'`): (1) the model itself can raise the main loop to fable for the rest of a turn through the `route` tool (plan/reflect/escalate/judge) or a background dispatch (derived orchestrate = best tier) — bounded by the turn and `upgradeMaxTokens`, no sticky route is model-callable; (2) `ultrathink` and the default keyword rules now mean "best tier" (fable) at the phase's effort, so an incidental keyword in a pasted composer prompt costs a fable turn (origin composer only). Residuals: `PostModelSwitch` `auto` covers "other programmatic change" (a healthy model could be marked 15 min); strikes never decay inside a session; a `[1m]` variant's `model_not_found` marks the base model until reload; a `models` alias added by the override without a `fallback` key stays unranked (`withEveryAlias` not applied to the default chain) so `leaveDown` skips the upgrade gates for it; `canonical` prefix match has no segment boundary (`claude-sonnet-5-50` would map to sonnet); classifier deferred to W2. 2026-10-23 feat C 786ee13: (1) derived-orchestrate part and (2) keyword/ultrathink part CLOSED (effort-only routes); (1) route-tool lift stays by design (sanctioned path)
|
||||
- [x] W1-C live verification part 1 (2026-10-09 after `/reload-plugins`, skills-dir copy loaded, 12 hooks): derived orchestrate on a real background Explore (main high → medium while it ran → high after its end); Explore on sonnet/medium; `route plan` → next request xhigh (engine record); `Skill(effort-low)` bridged → next request low (engine record); `/route show` resolves all 10 phases to full ids, `down: none`; steps arrive as bare `claude-fable-5-1` (no `[1m]`) so the suffix carry never fires on this session
|
||||
- [ ] W1-C live verification part 2 (needs a real incident): StopFailure vs turn.complete order; `PostModelSwitch` `auto` semantics and its `from_model` after a router upgrade; whether a router rewrite raises `auto`; `[1m]` carry validity on opus (only on a session whose steps carry it)
|
||||
- [x] W2-A mod (2026-10-09, bb56f3e on feature/model-router-w2, contract `2026-10-09-model-router-w2a-1546`, plan r4 `2026-10-09-model-router-w2-1546`): user decisions — effort-* skills deleted (W2-B), pins REWORKED not deleted (rows = phases by role; frontmatter `model:`/`effort:` kept as census-locked off-state floor after a robustness BLOCKER), orchestrators declare phases, slim model gate. Phases `write` work/high + `apply` work/low; 56 skill rows, 21 agent rows; agents' model at spawn within tier, upward only, project-defined agents skipped (agent.offer); typed slash → name-bound marker (composer|sdk|bridge) + pending slot + idle fallback; best-tier rows in a `runMain` slot surviving turn end; unrowed skill leaves the route; route answer always names the id; `null` override rows. 3 lenses + 2 confirmations (1 BLOCKER each round closed), feater + 4 gap rounds, GATE 0 MET, verifier 3× ECARTS on coverage clauses only → user accepted at the cap, security PASS (2 MEDIUM pre-existing: first-load kill switch fails open, ReDoS size-bounded). Kit 58 → 88 tests. Doc-sync skipped for A (mod-only, docs at B).
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# CONTRACT — router-keyword-cap (feat C of the routing policy, BLK-032)
|
||||
- date: 2026-10-23 | flow: feat | branch: feature/router-keyword-cap
|
||||
- status: active
|
||||
|
||||
## REQUEST (verbatim — IMMUTABLE)
|
||||
push tout ce qui doit etre push et suprprime ce qui doit etre supprime et apres passe au feat C !
|
||||
lance le /feat il ne cdevrait pas etre bloque je comprend pas
|
||||
|
||||
/feat args: feat C: model-router keyword rules capped at the session model (BDR-123 deferred item: keyword-lift gap), closes BLK-032. Context: .claude/memory/decisions.md BDR-123, blockers.md BLK-032, journal 2026-10-19..22. Uncommitted mods/model-router/routing.json adds "audit": "audit" in the phases map (dialog write from feat A).
|
||||
|
||||
TODO.md:10 (the feat C definition the request points at): "feat C: cap the default keyword prompt rules (plan|design|architecture, why|review|audit|analyze, ultrathink) at the session model, effort only like the classifier; close BLK-032 after"
|
||||
|
||||
## CLARIFICATIONS
|
||||
- Internal: the cap reuses the classifier's ceiling (`wantedModel` + `wouldRunHead` in register.ts): a route whose source is `prompt` (default keyword rules AND prompt floor rules such as `ultrathink`, user-added rules from the machine config included) never names a model ranked above the session model; above it → no model move (main keeps its current model), the phase's effort still applies. Same semantics as the classifier, one shared predicate.
|
||||
- Internal: the route tool (`source: model`), skill rows (`skill`/`run`), sticky `/route` (`user`) and derived dispatches stay uncapped: they are the sanctioned paths to Fable (BDR-123).
|
||||
- Internal: an unknown or unranked session model leaves the cap inert, like the classifier; harmless because main is never switched off an unranked model (test `unknown: a session model absent from the table is never switched`).
|
||||
- Internal: the mods/model-router/routing.json `"audit": "audit"` line (feat A dialog write, flagged out of scope by the BLK-034 verifier) is committed on this branch as its own commit by the orchestrator before dispatch; it is not part of the executor's diff.
|
||||
|
||||
- [gated 2026-10-23, after challenge r1] MECHANISM CHANGED (supersedes the first 3 internal bullets above): prompt rules (default keyword rules, `ultrathink` floor, user-added rules) and the derived `orchestrate` route on a main Agent dispatch write EFFORT-ONLY routes (`belowRun` pattern), so they never name a model; the model axis falls through to a held run slot or stays on the current model. `wantedModel`/`wouldRunHead` (classifier ceiling) are not edited. Reason: the r1 cap shadowed a held best-tier run slot (keyword reply inside /feat dropped Fable) and failed open on an unknown session model.
|
||||
- [gated 2026-10-23] Derived orchestrate (background agent dispatch) capped too, same mechanism (user pick: "Oui, effort seul aussi").
|
||||
- [gated 2026-10-23] Accepted edges (plan r2 §Edge cases): haiku session → keywords are no-ops; session model held down → keyword turn follows the plain-turn fallback; mid-turn ultrathink after a classifier demotion keeps the guessed model for that turn (amends BDR-115 2026-10-13 clause "floor model beats guess"); no stand-in notch on keyword turns.
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. The mods suite is green (manifest, link, validate --strict, `claude plugin test` running register.test.ts).
|
||||
CHECK: cd /home/bchanot/Documents/claude && make test suite=lib/tests/mods.test.sh 2>&1 | tail -3
|
||||
EXPECT: all suites green
|
||||
EVIDENCE: MET exit=0 marker-found :: PASS model-router: plugin test passed mods: 6 pass, 0 fail all suites green
|
||||
2. On an opus session, a typed default keyword prompt of the `plan` rule (e.g. `plan the billing migration`) runs the next main step on opus, never fable, at the plan phase's effort (xhigh). A dedicated test named `keyword cap: a plan keyword on an opus session stays on opus at xhigh` exists in register.test.ts.
|
||||
CHECK: grep -c "test('keyword cap: a plan keyword on an opus session stays on opus at xhigh'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
3. On an opus session, a typed `reflect` keyword prompt (e.g. `why does the cache miss`) runs on opus at the reflect phase's effort (high). Test `keyword cap: a reflect keyword on an opus session stays on opus at high`.
|
||||
CHECK: grep -c "test('keyword cap: a reflect keyword on an opus session stays on opus at high'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
4. On an opus session, `ultrathink` runs on opus at effort max (floor kept, model capped). Test `keyword cap: ultrathink on an opus session stays on opus at max`.
|
||||
CHECK: grep -c "test('keyword cap: ultrathink on an opus session stays on opus at max'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
5. The sanctioned lifts still reach fable on an opus session: a keyword turn followed by a route tool call `phase: 'plan'` runs on fable. Test `keyword cap: a route tool call after a keyword still lifts to fable`.
|
||||
CHECK: grep -c "test('keyword cap: a route tool call after a keyword still lifts to fable'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
6. A capped keyword is never a stand-in: on an opus session with fable held down, a `plan` keyword runs on opus at xhigh (no +1 stand-in notch), and no probe of fable is made. Test `keyword cap: a held fable gives no stand-in notch to a capped keyword`.
|
||||
CHECK: grep -c "test('keyword cap: a held fable gives no stand-in notch to a capped keyword'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
7. [gated 2026-10-23] Fable-session tests keep their expectations, except the ones plan r2 §Tests B names (l.935 rebased, l.5300 renamed, l.5690 effort max→xhigh), each edit carrying a one-line comment saying why (verifier reads the diff).
|
||||
8. [gated 2026-10-23] The probe tests l.5564, l.5597, l.5636 lift through `planRoute($)` as plan r2 §Tests B spells out; their subject (probe of fable, hold lengths, in-flight failure leaves the target unverified) is still asserted; expectation edits limited to what plan r2 names, each commented.
|
||||
9. [gated 2026-10-23] `defaultFromPrompt`, the floor branch of `routeFromPrompt` and `pushOrchestrate` write routes with an `effort` and no `tier`/`model` (one shared helper); `wantedModel` and `wouldRunHead` are byte-identical to develop; `mainModelSlot` no longer reads `turnFloor`.
|
||||
CHECK: cd /home/bchanot/Documents/claude && f=mods/model-router/hooks/register.ts && a=$(git show HEAD:$f | sed -n '/^function wantedModel/,/^}/p;/^function wouldRunHead/,/^}/p' | md5sum) && b=$(sed -n '/^function wantedModel/,/^}/p;/^function wouldRunHead/,/^}/p' $f | md5sum) && c=$(sed -n '/^function mainModelSlot/,/^}/p' $f | grep -c turnFloor || true) && d=$(sed -n '/^function defaultFromPrompt/,/^}/p;/^function routeFromPrompt/,/^}/p;/^function pushOrchestrate/,/^}/p' $f | grep -c 'phaseRoute(' || true) && e=$(grep -c '^function mainModelSlot' $f || true) && [ "$a" = "$b" ] && [ "$e" = 1 ] && echo "ceiling-same slot-floor=$c phaseRoute-in-writers=$d"
|
||||
EXPECT: ceiling-same slot-floor=0 phaseRoute-in-writers=0
|
||||
EVIDENCE: MET exit=0 marker-found :: ceiling-same slot-floor=0 phaseRoute-in-writers=0
|
||||
10. lib/effort-shift.md "Limits" states that keyword prompt rules, `ultrathink` and a background agent dispatch set the turn's effort only, never a model above the session model, Fable on main coming only from a skill row, the `route` tool or a typed `/route`; lines ≤ 80 columns.
|
||||
CHECK: f=/home/bchanot/Documents/claude/lib/effort-shift.md; a=$(tr '\n' ' ' < $f | tr -s ' ' | grep -c 'never a model above the session model'); b=$(awk 'length($0)>80' $f | wc -l); echo "cap=$a/long=$b"
|
||||
EXPECT: cap=1/long=0
|
||||
EVIDENCE: MET exit=0 marker-found :: cap=1/long=0
|
||||
11. The executor's diff touches only the FILE SCOPE.
|
||||
CHECK: cd /home/bchanot/Documents/claude && git diff --name-only HEAD | grep -v -x -e mods/model-router/hooks/register.ts -e mods/model-router/hooks/register.test.ts -e lib/effort-shift.md | wc -l | sed 's/^/outside=/'
|
||||
EXPECT: outside=0
|
||||
EVIDENCE: MET exit=0 marker-found :: outside=0
|
||||
12. [gated 2026-10-23] A keyword reply inside a held best-tier run keeps main on fable (opus session). Test `keyword cap: a keyword inside a held best-tier run keeps the run on fable`.
|
||||
CHECK: grep -c "test('keyword cap: a keyword inside a held best-tier run keeps the run on fable'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
13. [gated 2026-10-23] An unknown session model ('') never lets a keyword lift main. Test `keyword cap: an unknown session model never lets a keyword lift main`.
|
||||
CHECK: grep -c "test('keyword cap: an unknown session model never lets a keyword lift main'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
14. [gated 2026-10-23] A main background dispatch outside a run keeps main on the opus session model at orchestrate's effort. Test `keyword cap: a dispatch outside a run keeps main on the session model`.
|
||||
CHECK: grep -c "test('keyword cap: a dispatch outside a run keeps main on the session model'" /home/bchanot/Documents/claude/mods/model-router/hooks/register.test.ts
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
|
||||
## FILE SCOPE
|
||||
- mods/model-router/hooks/register.ts
|
||||
- mods/model-router/hooks/register.test.ts
|
||||
- lib/effort-shift.md
|
||||
@@ -0,0 +1,231 @@
|
||||
# PLAN r2.1 — router-keyword-cap (feat C, BLK-032)
|
||||
|
||||
Contract: `.claude/tasks/contracts/2026-10-23-router-keyword-cap-0019.md`
|
||||
Branch: feature/router-keyword-cap (off develop 6168d90)
|
||||
|
||||
## r1 → r2 (challenge round 1: correctness FATAL(4), robustness FATAL(5), simplicity CONCERNS(1))
|
||||
|
||||
- BLOCKER (correctness 1, robustness 1): r1 capped the prompt slot in
|
||||
`wantedModel`; a capped `turnMain` then shadowed a held best-tier
|
||||
`runMain` (mainModelSlot order) → a keyword reply inside a /feat run
|
||||
dropped main from Fable to opus. CLOSED by changing the mechanism:
|
||||
prompt rules write EFFORT-ONLY routes (the `belowRun` pattern,
|
||||
register.ts:3060), so the model axis falls through to `runMain` by the
|
||||
existing per-axis rule (test `per axis: a model-less sticky never hides
|
||||
a turn route tier`). `wantedModel` / `wouldRunHead` are NOT edited.
|
||||
- MAJOR (robustness 2, correctness 7): r1 failed open when `sessionModel`
|
||||
is '' (after /reload-plugins). CLOSED by the same change: an
|
||||
effort-only route never names a model, whatever the session model.
|
||||
- BLOCKER (correctness 2) / MAJOR (robustness 3): r1's one-line swap of
|
||||
the probe tests hangs or goes red (the route tool's answer runs the
|
||||
probe). CLOSED: per-test rewrites spelled out below.
|
||||
- MAJOR (all three): tests l.935 and l.1011 (HAIKU sessions) missed.
|
||||
CLOSED: named below with their new form.
|
||||
- MAJOR (robustness 5, correctness 6): derived `orchestrate` on a main
|
||||
Agent dispatch still lifted main to Fable. User decision 2026-10-23:
|
||||
cap it too, same mechanism (effort-only).
|
||||
- MINOR (test 5 setup, all three): hold fable with
|
||||
`switchTo($, FABLE, OPUS, 'auto')`, as the classifier stand-in test.
|
||||
- MINOR (correctness 5, 8; robustness 6, 7): edge notes corrected below;
|
||||
robustness 7 (auto fallback rewrites sessionModel) is moot, nothing
|
||||
reads sessionModel any more for prompt routes.
|
||||
|
||||
## r2 → r2.1 (confirmation pass: correctness SOLID, 5 MINOR, all applied)
|
||||
|
||||
- l.5636 completions spelled out: `[FABLE, OPUS, FABLE]`.
|
||||
- `effortRoute` returns `{}` (not undefined) for a phase with no effort,
|
||||
so a matched rule still claims the turn and the classifier stays off.
|
||||
- Derived-dispatch edges added below.
|
||||
- `mainModelSlot` stays a `function` declaration (oracle reads it).
|
||||
- A6 types a PLAN keyword and expects `{ model: FABLE, effort: 'xhigh' }`;
|
||||
every new test name stays on ONE line even past 80 columns (oracles
|
||||
grep it; l.5690 is the precedent).
|
||||
|
||||
## Problem
|
||||
|
||||
`register.ts` DEFAULTS.prompt (l.295-309): `ultrathink` (floor, phase
|
||||
escalate), plan keywords (default, phase plan), reflect keywords
|
||||
(default, phase reflect); all three phases are `tier: best` → fable.
|
||||
`defaultFromPrompt` (l.3924) writes `turnMain` and `routeFromPrompt`
|
||||
(l.3937-3943) writes `turnFloor` with the phase's FULL route (tier +
|
||||
effort), `source: 'prompt'`. `pushOrchestrate` (l.3251) writes the full
|
||||
`orchestrate` route (`tier: best`), `source: 'derived'`, on every main
|
||||
Agent dispatch not under a model/skill route. On the `opus` session
|
||||
(BDR-123) each lifts main to Fable with no skill row involved.
|
||||
|
||||
## Approach
|
||||
|
||||
1. register.ts — one helper next to `phaseRoute` (l.352):
|
||||
```ts
|
||||
/** A phase's effort alone: a route that never moves main's model. */
|
||||
function effortRoute(cfg: Config, phase: string): Route | undefined {
|
||||
const route = phaseRoute(cfg, phase)
|
||||
if (route === undefined) return undefined
|
||||
return route.effort === undefined ? {} : { effort: route.effort }
|
||||
}
|
||||
```
|
||||
2. Use it at the three writers, replacing `phaseRoute(...)` +
|
||||
`{ ...route }`:
|
||||
- `defaultFromPrompt` (l.3926-3928): `effortRoute(st.cfg, rule.phase)`;
|
||||
- `routeFromPrompt` floor (l.3940-3942): `effortRoute(st.cfg,
|
||||
floor.phase)`;
|
||||
- `pushOrchestrate` (l.3253-3257): `effortRoute(st.cfg,
|
||||
'orchestrate')`.
|
||||
Phase names, sources, labels, precedence: unchanged. The effort path
|
||||
(`mainEffort` l.1663) reads `route.effort` only, so efforts are
|
||||
unchanged; only the model axis changes.
|
||||
3. Dead code: no floor route carries a model any more (`turnFloor` is
|
||||
written only by `floorFromPrompt`/`endMainTurn` from prompt rules;
|
||||
`PromptRule` has no `model` field). In `mainModelSlot` (l.1385-1397)
|
||||
drop `st.turnFloor` from the slot list and the classifier-vs-floor
|
||||
branch; the function becomes `[userMain, turnMain, runMain].find(...)`,
|
||||
kept as a `function` declaration.
|
||||
Update its doc comment (no "floor's own model").
|
||||
4. Comments: DEFAULTS comment above `prompt:` (l.293-294): "Rules set the
|
||||
turn's effort only, never a model." `pushOrchestrate` doc: "the turn's
|
||||
effort becomes orchestrate's; the model stays where it is (a held run
|
||||
keeps its model)." Keep 80 columns.
|
||||
5. lib/effort-shift.md "Limits", new bullet after the classifier one:
|
||||
```
|
||||
- Keyword prompt rules (`plan|design|architecture…`,
|
||||
`why|review|audit…`), `ultrathink` and a background agent
|
||||
dispatch (derived `orchestrate`) set the turn's effort only, never
|
||||
a model above the session model: Fable on main comes only from a
|
||||
skill row, the `route` tool or a typed `/route`.
|
||||
```
|
||||
(wording: "never a model above the session model" is the contract
|
||||
oracle string; effort-only routes name no model at all, which
|
||||
satisfies it.)
|
||||
|
||||
## Out of scope (unchanged on purpose)
|
||||
|
||||
- Route tool (`model`), skill rows (`skill`/`run`), sticky `/route`
|
||||
(`user`): the sanctioned Fable paths (BDR-123).
|
||||
- Classifier: unchanged (its ceiling in `wantedModel`/`wouldRunHead`
|
||||
stays as is; it never runs under a held route).
|
||||
- `userMovedEffort` (l.3501): classifier-only, unchanged.
|
||||
- README.md / USAGE.md / CHANGELOG: STEP 6 doc-sync (README l.142 says
|
||||
"an `ultrathink` floor in the same turn keeps its own model": stale
|
||||
after this change, doc-sync must catch it).
|
||||
|
||||
## Edge cases (decided)
|
||||
|
||||
- Keyword reply inside a best-tier run: effort from the keyword, model
|
||||
from `runMain` (Fable). New test.
|
||||
- `sessionModel` '' (after /reload-plugins): no lift, the route names no
|
||||
model. New test.
|
||||
- Fable session (old setting): keyword turns start on fable and stay
|
||||
there (cur = session); efforts unchanged.
|
||||
- Haiku session: a keyword or `ultrathink` never moves main off haiku;
|
||||
`planStep` strips effort on haiku (l.3572-3574), so they are no-ops.
|
||||
Accepted (never above the session model).
|
||||
- Session model held down (e.g. opus outage on an opus session): a
|
||||
keyword turn follows `leaveDown(cur, undefined)` like any plain turn
|
||||
(→ sonnet), where it used to reach fable. Accepted: same as a plain
|
||||
prompt; the route tool or a skill row still reach fable.
|
||||
- Fable held + keyword: the prompt route names no tier, `heldHead`
|
||||
(l.1565) is undefined → no stand-in notch. On a fable session whose
|
||||
fable is dead (`probe: a session model that never answers…`, l.5690)
|
||||
the keyword turn lands on opus at xhigh, no longer max.
|
||||
- Mid-turn `ultrathink` after a classifier demotion in the same turn:
|
||||
the floor names no model, so the guess's model holds for the rest of
|
||||
that turn (sonnet, at max); the pending floor carries to the next turn,
|
||||
which starts on the session model. This AMENDS BDR-115 amendment
|
||||
2026-10-13 "ultrathink floor model beats guess" (recorded at
|
||||
CAPITALIZE).
|
||||
- Derived orchestrate outside a run on the opus session: main stays on
|
||||
opus at orchestrate's effort (medium). Inside a held run: the run's
|
||||
model (Fable). After the pop nothing to restore on the model axis.
|
||||
- Derived dispatch after a classifier demotion in the same turn: main
|
||||
orchestrates on the demoted model (`turnModel`, e.g. sonnet) for the
|
||||
rest of the turn, where develop lifted it to fable; spawned agents are
|
||||
still pinned to the session model by `sessionPin` (l.3184). Accepted.
|
||||
- Derived dispatch with fable held: no stand-in notch (no tier named).
|
||||
Accepted.
|
||||
- A phase with no effort (machine config): the rule writes an empty
|
||||
route `{}`, still claiming the turn (classifier stays off).
|
||||
- `turnModel` (sticky per turn) is only set when the router moved main;
|
||||
effort-only routes never move it, so no sticky fable after a dispatch.
|
||||
|
||||
## Tests (mods/model-router/hooks/register.test.ts)
|
||||
|
||||
A. New block `// ---- keyword cap: prompt rules and dispatches set effort only`,
|
||||
each test asserting the step's `{ model, effort }`:
|
||||
1. `keyword cap: a plan keyword on an opus session stays on opus at xhigh`
|
||||
— `bootRig($, on, OPUS)`, typed `plan the billing migration`,
|
||||
`stepOn($, rig, OPUS)` → `{ model: OPUS, effort: 'xhigh' }`.
|
||||
2. `keyword cap: a reflect keyword on an opus session stays on opus at high`
|
||||
— typed `why does the cache miss` → `{ model: OPUS, effort: 'high' }`.
|
||||
3. `keyword cap: ultrathink on an opus session stays on opus at max`
|
||||
— `ultrathink($)` → `{ model: OPUS, effort: 'max' }`.
|
||||
4. `keyword cap: a route tool call after a keyword still lifts to fable`
|
||||
— typed plan keyword, `planRoute($)`, step → model FABLE.
|
||||
5. `keyword cap: a held fable gives no stand-in notch to a capped keyword`
|
||||
— `bootRig($, on, OPUS, 1000, world)` with `newWorld()`,
|
||||
`switchTo($, FABLE, OPUS, 'auto')` (marks fable down, no probe),
|
||||
typed plan keyword, step on OPUS → `{ model: OPUS, effort: 'xhigh' }`,
|
||||
`world.completions` contains no FABLE.
|
||||
6. `keyword cap: a keyword inside a held best-tier run keeps the run on fable`
|
||||
— opus session; hold a best-tier run slot the way the existing
|
||||
`run slot:` tests do (l.1159-1230: typed rowed best-tier skill +
|
||||
Skill call), `endTurn($)`, typed `plan the next step`, step on
|
||||
OPUS → `{ model: FABLE, effort: 'xhigh' }`.
|
||||
7. `keyword cap: an unknown session model never lets a keyword lift main`
|
||||
— `bootRig($, on, '')` (session.model answers ''), typed plan
|
||||
keyword, `stepOn($, rig, OPUS)` → model OPUS.
|
||||
8. `keyword cap: a dispatch outside a run keeps main on the session model`
|
||||
— opus session, a main background Agent dispatch the way the
|
||||
existing `derived:` tests do (l.914), step during the dispatch on
|
||||
OPUS → `{ model: OPUS, effort: 'medium' }`.
|
||||
|
||||
B. Existing tests to rewrite (expectation edits allowed ONLY here, each
|
||||
with a one-line comment saying why):
|
||||
- l.935 `default rule: planifie sets plan, a later route overrides`:
|
||||
boot on FABLE (`bootRig($, on)`), `stepOn($, rig, FABLE)`, expect
|
||||
`{ model: FABLE, effort: 'xhigh' }`; rest unchanged.
|
||||
- l.1011 `text: show names the model a floor upgrade moves to`:
|
||||
the behaviour is gone; replace with `text: show names no model move
|
||||
for an ultrathink floor` — `bootRig($, on, OPUS)`, ultrathink, the
|
||||
main line contains `floor max` and not `(upgrade)` nor FABLE.
|
||||
- l.5300 `classify: a mid-turn ultrathink floor still reaches fable max`:
|
||||
rename `classify: a mid-turn ultrathink floor raises effort only`,
|
||||
expect `{ model: SONNET, effort: 'max' }`.
|
||||
- l.5564 `probe: a fable that never answers lands the plan step on opus`:
|
||||
`await pastProbe(rig, planRoute($))` in place of the typed keyword,
|
||||
then `await runStep($, { ...highStep(), model: OPUS })`; `seen`
|
||||
and the `down:` line unchanged; `completions` → `[FABLE, OPUS]`
|
||||
(fable probed by the route call, opus by the step).
|
||||
- l.5597 `probe: a lapsed hold probes again and holds longer`:
|
||||
`await pastProbe(rig, planRoute($))` per loop turn in place of the
|
||||
typed keyword; expectations unchanged if they hold (derive: route
|
||||
probes FABLE, first step probes OPUS, second turn probes FABLE).
|
||||
- l.5636 `probe: a failure during the flight keeps the target unverified`:
|
||||
start `const call = planRoute($)` (in flight on the gated probe),
|
||||
`await rig.clock.settle()`, `switchTo($, FABLE, OPUS, 'auto')`,
|
||||
`open()`, `await call`, then the step; after the 15 min advance a
|
||||
second `planRoute($)` + step. Invariant asserted: FABLE appears
|
||||
twice in `completions` (still unverified after the in-flight
|
||||
failure): `completions` → `[FABLE, OPUS, FABLE]` (route probe
|
||||
fails in flight, re-decision lands on opus unprobed, the step
|
||||
probes opus, the second route probes fable); `seen[0]` stays
|
||||
`{ model: OPUS, effort: 'max' }`.
|
||||
- l.5690 `probe: a session model that never answers leaves for opus on the first step`:
|
||||
effort `max` → `xhigh` (a keyword names no tier: no stand-in notch).
|
||||
- Any OTHER test the suite turns red: if its subject is a model lift
|
||||
by a keyword, `ultrathink` or a derived dispatch, rewrite it to the
|
||||
effort-only result with a comment; for any other subject → STOP,
|
||||
report BLOCKED with the test names and failures.
|
||||
|
||||
Suite: `make test suite=lib/tests/mods.test.sh`.
|
||||
|
||||
## Disposition (memory)
|
||||
|
||||
- honors BDR-123: Fable on main only through rows, route tool, `/route`;
|
||||
closes its keyword-lift known gap and the derived-orchestrate leak.
|
||||
- honors BDR-115 rule 5 "per axis" and the `belowRun` precedent
|
||||
(effort-only route under a run); AMENDS BDR-115 amendment 2026-10-13
|
||||
clause "ultrathink floor model beats guess" (edge above).
|
||||
- honors LRN-218 / BLK-031: probe lands on the decided model, untouched.
|
||||
- LRN-224: agent rows untouched.
|
||||
- BLK-032 → resolved at CAPITALIZE; TODO W1-C accepted-by-design (1)
|
||||
derived-orchestrate part and (2) keyword part closed.
|
||||
Reference in new issue
Block a user