15 KiB
PLAN r3 — routing-audit-phase (feat A, BLK-032 routing policy)
r2 after the 2026-10-21 challenge round 1 (outage floor, test fixture, citers, no stash). r3 after round 2: single about string, no confirmed.phases.audit (first-use dialog = tripwire for main), unprobed-spawn residual + its test, explicit branch step, oracles 1/3/10/11 tightened, tier_floor flip test.
Contract: .claude/tasks/contracts/2026-10-21-routing-audit-phase-1506.md
Why
User policy 2026-10-19 (BLK-032): Fable only for orchestrator reflection/plans and the important audits (security, functional); other judges opus; executors sonnet. Today verifier routes to verify (work tier = sonnet xhigh) and security-auditor to judge (big tier = opus xhigh). Both are the merge-blocking gates of verify-secure-loop.md (GATE 1 contract verdict, GATE 2 security). A new best-tier phase audit carries them; judge keeps plan-challenger, analyzer, Plan, plugin-advisor, seo/geo-analyzer on opus.
Router facts that bind the plan
- Rows are phase names; a phase names a tier; a tier is an ordered alias list, first alias not down is used (
best= fable > opus > sonnet). An agent gets its row's model at spawn, upward only within the tier; explicitmodel=at a call site wins (BDR-115, BDR-077). No dispatch of verifier / security-auditor carries an explicitmodel=(grep 2026-10-21: tour, onboard, audit-delta dispatch them bare). lib/tests/effort-routing.test.shlocks: every routing.json phase equals the code copy inregister.tsDEFAULT_CONFIG (code_phase); every rowed agent's frontmattermodel:equals its phase's tier head andeffort:equals the phase effort, unless achangedentry explains the drift (WARN). Neither verifier nor security-auditor has achangedentry, so the frontmatter MUST move with the row or the census FAILs.lib/tests/model-routing.test.sh:23-24greps the literal old pins.register.test.tsbuilds every world frombaseRouting()(11 phases, :65-78).logMissingPhases(register.ts:862-869) diffsDEFAULT_CONFIG.phasesagainst the file, so a DEFAULT_CONFIG phase absent from the fixture is logged in every test: :1592-1601 expects exactlylacks phases reflect, and/route pendingcountsst.cfg.phases(:2555pending confirmation: 19, :256717). The fixture gainsauditand the two counts become 20 and 18.inTier(register.ts:3163-3174) writes a model only when it is the tier head or ranks above it; otherwise "tier … down, frontmatter model kept" and the frontmatter runs (test :1278-1291). A best-tier row with afablefrontmatter therefore has NO fallback: every Fable credit outage (BLK-030/031) would spawn both gates on a dead model. Decision (user 2026-10-21): the gates keepmodel: opusas the outage floor; the router lifts them to Fable when it is up; Fable down or router off = opus. The census gets a named carve-out: a best-tier agent row's model floor is the tier's SECOND alias.- No stash.
~/.claude/settings.jsonand~/.claude/skills/model-routerare symlinks into the repo: stashing settings.json puts every new session back on Fable 1M (the BLK-032 condition) and stashing routing.json re-arms the router's dialogs. The dirty copies carry over the branch start unchanged (git diff develop..HEADis empty for both files). They are committed FIRST on the feature branch as one dedicated commit (the router's own dialog writes:deployapply→verify +changedentry,release-executorKeep; the session modelopus= BLK-032 policy point 1), before the executor runs. Feat B then carries CLAUDE.global.md, the BDR and the docs only. confirmed.agentsrecords first-use dialog answers; an unconfirmed row re-opens the dialog at first use. This feat is the user's decision → setauditby hand for both.
Branch (main loop, before the executor)
bash ~/.claude/lib/gitflow.sh start feature routing-audit-phasefrom the currentbugfix/guardrails-residuals(not merged): the lib branches off develop; the dirtyrouting.json/settings.jsoncarry over (develop and HEAD hold the same blobs).- One dedicated commit of those two files:
chore(model-router): dialog decisions (deploy verify, release-executor) and session model opus— the router's own writes plus BLK-032 policy point 1. - Dispatch the executor on the clean tree.
- Memory IDs for this run's capitalize start above the highest on BOTH develop and the unmerged
bugfix/guardrails-residuals(8555bff: BLK-033, LRN-222/223): next BDR-123, LRN-224, BLK-034. The TODO line for this run goes under the 2026-10-19 routing bullet, away from the bugfix hunk at TODO.md:5-6.
Checklist (dispatch-ready)
[ ] mods/model-router/routing.json — phases: insert after judge (line 23-27):
"audit": { "tier": "best", "effort": "xhigh", "about": "the fresh gates: contract verdict and security scan; dispatched agents, not a main-loop phase" }
(exact string, 93 chars; about is shown in the first-use dialog only, register.ts:154, so this text is what main's first route(phase="audit") would print). Reword judge.about → "independent judgment of work: challenges, analyses and reviews"; verify.about → "checking a result against its spec: tests and suites". Rows: agents.verifier "verify" → "audit"; agents.security-auditor "judge" → "audit". confirmed.agents.verifier → "audit", confirmed.agents.security-auditor → "audit" (the two spawns the user decided here). Do NOT write confirmed.phases.audit: an unconfirmed phase keeps the first-use dialog (confirmPhase register.ts:2580-2585, suppressed by isDecided :2081 once confirmed) as the tripwire against a main-loop route(phase="audit"); /route pending listing phases:audit is the price. Nothing else changes: the changed block keeps the committed deploy entry, no entry is forged for the hand-moved rows.
[ ] mods/model-router/hooks/register.ts — DEFAULT_CONFIG.phases (line ~280): insert audit: { tier: 'best', effort: 'xhigh' }, right after the judge line. Same indentation, trailing comma. No other code change (inTier untouched).
[ ] mods/model-router/hooks/register.test.ts — baseRouting() phases: insert audit: { tier: 'best', effort: 'xhigh' }, after judge (:70-71); pending confirmation: 19 → 20 (:2555), 17 → 18 (:2567). Leave the fixture's verifier: 'verify' row (:89) as is, other tests use it. Add two tests next to :1278-1291, built with bootRun($, on, FABLE, newWorld({ edit: f => { (f.agents as Rec)['security-auditor'] = 'audit' } })) (bootRun's 4th param, :1035-1041) and bare spawnOf('security-auditor') (no model field: an explicit model skips routing, :3199):
(a) test('audit row: fable down, no write, frontmatter runs', …): switchTo($, FABLE, SONNET, 'auto'), spawn once, expect rig.specs [undefined] and the one log line 'model-router: security-auditor tier best down, frontmatter model kept' (recordLogs filter on frontmatter model kept).
(b) test('audit row: first spawn dies on fable, the retry lands on the floor', …): spawn, await failWith($, 'rate_limit', 'a1'), spawn again, expect rig.specs [FABLE, undefined] (same shape as :886-891); no log assertion in (b). This is the path verify-secure-loop's single fresh retry now depends on.
[ ] agents/verifier.md — frontmatter model: sonnet → model: opus (effort stays xhigh). Body untouched. Rationale in the census comment, not here.
[ ] agents/security-auditor.md — frontmatter stays model: opus, effort: xhigh: no edit.
[ ] lib/tests/effort-routing.test.sh — add tier_floor <file> <tier> right after tier_head: second alias when $2 = best, else tier_head. ONE comment above it (the only place citing BLK-030 in the file): # a best-tier row floors on its 2nd alias: inTier keeps the frontmatter when the head is down (Fable credits, BLK-030); a fable floor would spawn the gates on a dead model. field_of calls tier_floor "$REG" "$tier" for the model field. Section (c) header becomes # ── (c) tracked agents with a row: tier floor == model:, effort == effort: ─ (no BLK cite). Flip test: add best: ['fable', 'opus', 'sonnet'], to the fixture's reg.ts tiers block (:88-92) and two checks: [ "$(tier_floor "$FIX/reg.ts" best)" = opus ] && ok || ko "flip: best floors on its 2nd alias" and [ "$(tier_floor "$FIX/reg.ts" big)" = opus ] && ok || ko "flip: big floors on its head".
[ ] lib/tests/model-routing.test.sh:23-24 — has "agents/verifier.md" 'model: opus' # audit row (best tier, opus = outage floor; BLK-032 policy 2026-10-21) and has "agents/security-auditor.md" 'model: opus' # audit row since 2026-10-21 (was judge since 2026-10-11). Keep the column alignment of the block.
[ ] agents/plan-challenger.md:112-113 — replace "(Contrast the verifier on its verify row and the executors on their implement row, both sonnet, oracle-anchored to a contract.)" with "(Contrast the executors on their implement row, sonnet, oracle-anchored to a contract; the verifier and security-auditor sit on the audit row, Fable with an opus floor.)". Reflow the paragraph to 80 columns.
[ ] lib/challenge-plan.md:50 — "(The executor gates stay sonnet.)" → "(The executors stay sonnet; the two merge gates sit on the audit row, Fable with an opus floor.)". Reflow to 80 columns.
[ ] skills/tour/SKILL.md:83 — "security-auditor (sonnet frontmatter)" → "security-auditor (audit row: Fable, opus floor)". Reflow if the line exceeds 80 columns.
[ ] README.md:86-87 — verifier row: | verifier | Fable (audit row: best/xhigh, opus frontmatter floor) | fresh gate (≤3×/loop) |; security-auditor row: | security-auditor | Fable (audit row: best/xhigh, opus frontmatter floor) | fresh gate (≤3×/loop) |. Also README.md:111-113 phase list: insert `audit` best/xhigh after `judge` big/xhigh; :114-116 "kept equal to the rows by the census" → "kept equal to the rows by the census lib/tests/effort-routing.test.sh (a best-tier row floors on the tier's second alias, opus, since Fable is credit-gated; a row changed through the first-use dialog passes with a WARN floor drift line"; :135 best-tier row list → "(plan, reflect, orchestrate, escalate; audit is an agent row, never typed)". Every citer edit above carries the token `audit` row (oracle 11).
[ ] run: the four suites first (make test suite=lib/tests/effort-routing.test.sh, model-routing, mods, doctrine-citers), then the full make test (plan-challenger.test.sh locks strings in plan-challenger.md / challenge-plan.md; contract-verifier and gates suites read verifier.md); then the contract oracles.
Edge cases
- Fable down (credits, BLK-030/031):
inTierwrites nothing, theopusfrontmatter runs: the gates degrade to opus, one log linetier best down, frontmatter model kept. Router off (/route off,enabled:false, unreadable routing.json): opus too. Locked by the new register.test.ts case and the census carve-out. auditappears in the main loop'sroutetool enum as a bare name (register.ts:2912-2916;aboutis never in the tool text). A main-looproute(phase="audit")opens the first-use phase dialog (phase left unconfirmed), which prints the about: that dialog is the tripwire. Same capability asplan; no orchestrator skill calls it; feat B's ceiling rule covers inline work on Fable.- Unprobed spawn path (TODO.md:56 residual, unchanged): the router learns Fable is down from a failure, never from a probe before a spawn. In an outage window the FIRST gate spawn dies on Fable (429),
StopFailuremarks it down, the fresh retry lands on the opus floor. verify-secure-loop's single retry covers /feat, /bugfix, /ship-feature, /init-project, /onboard, /guardrails, /a11y; tour Phase A, onboard and audit-delta dispatch the auditor with no retry rule: one dead dispatch per outage window there. Accepted, recorded in TODO; locked by test (b). onAutoSwitch(register.ts:3662-3674) charges a sub-agent's engine fallback to main's last plan (TODO 2026-10-13 residual): a gate falling off Fable while main's last plan is sonnet marks sonnet down 15 min; with main's last plan on opus (the new session model) the early return at :3670 marks nothing, so an engine-side fallback would never mark Fable down and every gate spawn would pay the 429 again. Which event the engine emits for a sub-agent credit 429 is not established (BLK-030 saw StopFailure on main). Accepted, no code change here; test (b) locks the StopFailure path only.auditduplicatesplan's (best, xhigh) pair: the Change dialog maps a pair to the first phase that fits in file order (targetPhase:2311-2317), so a gate row Changed away and back to "fable + xhigh" lands onplan;auditis reachable by hand edit only. Accepted.register.tsvalidates a phase's tier againsttierskeys:bestexists. A phase named like an alias would be rejected:auditis not an alias.model-gate.md(names a fable or opus id | proceed) is about the MAIN loop, not the gates: unaffected.lib/challenge-plan.md:46says challengers are on the judge row (opus): still true, no edit.- The
verifyphase keeps one consumer once the dialog writes are committed first (skill rowdeploy= verify); noroute(phase="verify")caller exists in skills/agents/lib. Only itsaboutchanges. - The dirty
routing.json/settings.jsonare committed first on the feature branch (see Router facts); the executor starts from a clean tree and edits the committed routing.json, which already holds thedeployrow andchangedentry.
Tests
register.test.ts: fixture + counts + the new outage-floor case.effort-routing.test.sh: best-tier floor carve-out.model-routing.test.sh: literal pins.mods.test.shrunsclaude plugin test(CLI present on this machine).doctrine-citers.test.sh: citer census.
Disposition (STEP 0.6, decisions-first)
- BDR-066 [in force, partially superseded by this feat]: "verifier + security-auditor STAY sonnet" → overturned by user policy 2026-10-19; feat B writes the superseding BDR. Executors stay sonnet (honored).
- BDR-077 [in force]: fail-safe pin rule (a forgotten override over-tiers, never downgrades judgment); no call site overrides the gates → honored: the frontmatter floor (opus) never sits below today's security-auditor pin, and the verifier floor rises sonnet → opus.
- BDR-115 [in force]: rows in routing.json are the route, frontmatter = off-state floor, "an agent moves UP from its frontmatter alias, never below" (inTier) → honored: opus floor, router lifts to Fable;
confirmedset because the user decided here, nochangedentry forged. - BLK-030/031 [resolved, residual TODO.md:56 "spawn path not probed"]: the opus floor gives the retry a landing spot once the first failure marks Fable down; it does not save the first spawn of an outage window (Edge cases). An explicit
model:"fable"pin would have neither. - LRN-203 [in force]: hooks write full ids from
config.models;fablealias maps toclaude-fable-5-1→ honored, no id typed. - BLK-030/031 [resolved]: Fable credit outages → the
besttier falls back; gates still run (edge case above).