Files
claude/.claude/tasks/plans/2026-10-21-routing-audit-phase-1506.md
T

15 KiB
Raw Blame History

PLAN r3 — routing-audit-phase (feat A, BLK-032 routing policy)

r2 after the 2026-10-21 challenge round 1 (outage floor, test fixture, citers, no stash). r3 after round 2: single about string, no confirmed.phases.audit (first-use dialog = tripwire for main), unprobed-spawn residual + its test, explicit branch step, oracles 1/3/10/11 tightened, tier_floor flip test.

Contract: .claude/tasks/contracts/2026-10-21-routing-audit-phase-1506.md

Why

User policy 2026-10-19 (BLK-032): Fable only for orchestrator reflection/plans and the important audits (security, functional); other judges opus; executors sonnet. Today verifier routes to verify (work tier = sonnet xhigh) and security-auditor to judge (big tier = opus xhigh). Both are the merge-blocking gates of verify-secure-loop.md (GATE 1 contract verdict, GATE 2 security). A new best-tier phase audit carries them; judge keeps plan-challenger, analyzer, Plan, plugin-advisor, seo/geo-analyzer on opus.

Router facts that bind the plan

  • Rows are phase names; a phase names a tier; a tier is an ordered alias list, first alias not down is used (best = fable > opus > sonnet). An agent gets its row's model at spawn, upward only within the tier; explicit model= at a call site wins (BDR-115, BDR-077). No dispatch of verifier / security-auditor carries an explicit model= (grep 2026-10-21: tour, onboard, audit-delta dispatch them bare).
  • lib/tests/effort-routing.test.sh locks: every routing.json phase equals the code copy in register.ts DEFAULT_CONFIG (code_phase); every rowed agent's frontmatter model: equals its phase's tier head and effort: equals the phase effort, unless a changed entry explains the drift (WARN). Neither verifier nor security-auditor has a changed entry, so the frontmatter MUST move with the row or the census FAILs.
  • lib/tests/model-routing.test.sh:23-24 greps the literal old pins.
  • register.test.ts builds every world from baseRouting() (11 phases, :65-78). logMissingPhases (register.ts:862-869) diffs DEFAULT_CONFIG.phases against the file, so a DEFAULT_CONFIG phase absent from the fixture is logged in every test: :1592-1601 expects exactly lacks phases reflect, and /route pending counts st.cfg.phases (:2555 pending confirmation: 19, :2567 17). The fixture gains audit and the two counts become 20 and 18.
  • inTier (register.ts:3163-3174) writes a model only when it is the tier head or ranks above it; otherwise "tier … down, frontmatter model kept" and the frontmatter runs (test :1278-1291). A best-tier row with a fable frontmatter therefore has NO fallback: every Fable credit outage (BLK-030/031) would spawn both gates on a dead model. Decision (user 2026-10-21): the gates keep model: opus as the outage floor; the router lifts them to Fable when it is up; Fable down or router off = opus. The census gets a named carve-out: a best-tier agent row's model floor is the tier's SECOND alias.
  • No stash. ~/.claude/settings.json and ~/.claude/skills/model-router are symlinks into the repo: stashing settings.json puts every new session back on Fable 1M (the BLK-032 condition) and stashing routing.json re-arms the router's dialogs. The dirty copies carry over the branch start unchanged (git diff develop..HEAD is empty for both files). They are committed FIRST on the feature branch as one dedicated commit (the router's own dialog writes: deploy apply→verify + changed entry, release-executor Keep; the session model opus = BLK-032 policy point 1), before the executor runs. Feat B then carries CLAUDE.global.md, the BDR and the docs only.
  • confirmed.agents records first-use dialog answers; an unconfirmed row re-opens the dialog at first use. This feat is the user's decision → set audit by hand for both.

Branch (main loop, before the executor)

  1. bash ~/.claude/lib/gitflow.sh start feature routing-audit-phase from the current bugfix/guardrails-residuals (not merged): the lib branches off develop; the dirty routing.json / settings.json carry over (develop and HEAD hold the same blobs).
  2. One dedicated commit of those two files: chore(model-router): dialog decisions (deploy verify, release-executor) and session model opus — the router's own writes plus BLK-032 policy point 1.
  3. Dispatch the executor on the clean tree.
  4. Memory IDs for this run's capitalize start above the highest on BOTH develop and the unmerged bugfix/guardrails-residuals (8555bff: BLK-033, LRN-222/223): next BDR-123, LRN-224, BLK-034. The TODO line for this run goes under the 2026-10-19 routing bullet, away from the bugfix hunk at TODO.md:5-6.

Checklist (dispatch-ready)

[ ] mods/model-router/routing.json — phases: insert after judge (line 23-27): "audit": { "tier": "best", "effort": "xhigh", "about": "the fresh gates: contract verdict and security scan; dispatched agents, not a main-loop phase" } (exact string, 93 chars; about is shown in the first-use dialog only, register.ts:154, so this text is what main's first route(phase="audit") would print). Reword judge.about → "independent judgment of work: challenges, analyses and reviews"; verify.about → "checking a result against its spec: tests and suites". Rows: agents.verifier "verify" → "audit"; agents.security-auditor "judge" → "audit". confirmed.agents.verifier → "audit", confirmed.agents.security-auditor → "audit" (the two spawns the user decided here). Do NOT write confirmed.phases.audit: an unconfirmed phase keeps the first-use dialog (confirmPhase register.ts:2580-2585, suppressed by isDecided :2081 once confirmed) as the tripwire against a main-loop route(phase="audit"); /route pending listing phases:audit is the price. Nothing else changes: the changed block keeps the committed deploy entry, no entry is forged for the hand-moved rows. [ ] mods/model-router/hooks/register.ts — DEFAULT_CONFIG.phases (line ~280): insert audit: { tier: 'best', effort: 'xhigh' }, right after the judge line. Same indentation, trailing comma. No other code change (inTier untouched). [ ] mods/model-router/hooks/register.test.ts — baseRouting() phases: insert audit: { tier: 'best', effort: 'xhigh' }, after judge (:70-71); pending confirmation: 19 → 20 (:2555), 17 → 18 (:2567). Leave the fixture's verifier: 'verify' row (:89) as is, other tests use it. Add two tests next to :1278-1291, built with bootRun($, on, FABLE, newWorld({ edit: f => { (f.agents as Rec)['security-auditor'] = 'audit' } })) (bootRun's 4th param, :1035-1041) and bare spawnOf('security-auditor') (no model field: an explicit model skips routing, :3199): (a) test('audit row: fable down, no write, frontmatter runs', …): switchTo($, FABLE, SONNET, 'auto'), spawn once, expect rig.specs [undefined] and the one log line 'model-router: security-auditor tier best down, frontmatter model kept' (recordLogs filter on frontmatter model kept). (b) test('audit row: first spawn dies on fable, the retry lands on the floor', …): spawn, await failWith($, 'rate_limit', 'a1'), spawn again, expect rig.specs [FABLE, undefined] (same shape as :886-891); no log assertion in (b). This is the path verify-secure-loop's single fresh retry now depends on. [ ] agents/verifier.md — frontmatter model: sonnet → model: opus (effort stays xhigh). Body untouched. Rationale in the census comment, not here. [ ] agents/security-auditor.md — frontmatter stays model: opus, effort: xhigh: no edit. [ ] lib/tests/effort-routing.test.sh — add tier_floor <file> <tier> right after tier_head: second alias when $2 = best, else tier_head. ONE comment above it (the only place citing BLK-030 in the file): # a best-tier row floors on its 2nd alias: inTier keeps the frontmatter when the head is down (Fable credits, BLK-030); a fable floor would spawn the gates on a dead model. field_of calls tier_floor "$REG" "$tier" for the model field. Section (c) header becomes # ── (c) tracked agents with a row: tier floor == model:, effort == effort: ─ (no BLK cite). Flip test: add best: ['fable', 'opus', 'sonnet'], to the fixture's reg.ts tiers block (:88-92) and two checks: [ "$(tier_floor "$FIX/reg.ts" best)" = opus ] && ok || ko "flip: best floors on its 2nd alias" and [ "$(tier_floor "$FIX/reg.ts" big)" = opus ] && ok || ko "flip: big floors on its head". [ ] lib/tests/model-routing.test.sh:23-24 — has "agents/verifier.md" 'model: opus' # audit row (best tier, opus = outage floor; BLK-032 policy 2026-10-21) and has "agents/security-auditor.md" 'model: opus' # audit row since 2026-10-21 (was judge since 2026-10-11). Keep the column alignment of the block. [ ] agents/plan-challenger.md:112-113 — replace "(Contrast the verifier on its verify row and the executors on their implement row, both sonnet, oracle-anchored to a contract.)" with "(Contrast the executors on their implement row, sonnet, oracle-anchored to a contract; the verifier and security-auditor sit on the audit row, Fable with an opus floor.)". Reflow the paragraph to 80 columns. [ ] lib/challenge-plan.md:50 — "(The executor gates stay sonnet.)" → "(The executors stay sonnet; the two merge gates sit on the audit row, Fable with an opus floor.)". Reflow to 80 columns. [ ] skills/tour/SKILL.md:83 — "security-auditor (sonnet frontmatter)" → "security-auditor (audit row: Fable, opus floor)". Reflow if the line exceeds 80 columns. [ ] README.md:86-87 — verifier row: | verifier | Fable (audit row: best/xhigh, opus frontmatter floor) | fresh gate (≤3×/loop) |; security-auditor row: | security-auditor | Fable (audit row: best/xhigh, opus frontmatter floor) | fresh gate (≤3×/loop) |. Also README.md:111-113 phase list: insert `audit` best/xhigh after `judge` big/xhigh; :114-116 "kept equal to the rows by the census" → "kept equal to the rows by the census lib/tests/effort-routing.test.sh (a best-tier row floors on the tier's second alias, opus, since Fable is credit-gated; a row changed through the first-use dialog passes with a WARN floor drift line"; :135 best-tier row list → "(plan, reflect, orchestrate, escalate; audit is an agent row, never typed)". Every citer edit above carries the token `audit` row (oracle 11). [ ] run: the four suites first (make test suite=lib/tests/effort-routing.test.sh, model-routing, mods, doctrine-citers), then the full make test (plan-challenger.test.sh locks strings in plan-challenger.md / challenge-plan.md; contract-verifier and gates suites read verifier.md); then the contract oracles.

Edge cases

  • Fable down (credits, BLK-030/031): inTier writes nothing, the opus frontmatter runs: the gates degrade to opus, one log line tier best down, frontmatter model kept. Router off (/route off, enabled:false, unreadable routing.json): opus too. Locked by the new register.test.ts case and the census carve-out.
  • audit appears in the main loop's route tool enum as a bare name (register.ts:2912-2916; about is never in the tool text). A main-loop route(phase="audit") opens the first-use phase dialog (phase left unconfirmed), which prints the about: that dialog is the tripwire. Same capability as plan; no orchestrator skill calls it; feat B's ceiling rule covers inline work on Fable.
  • Unprobed spawn path (TODO.md:56 residual, unchanged): the router learns Fable is down from a failure, never from a probe before a spawn. In an outage window the FIRST gate spawn dies on Fable (429), StopFailure marks it down, the fresh retry lands on the opus floor. verify-secure-loop's single retry covers /feat, /bugfix, /ship-feature, /init-project, /onboard, /guardrails, /a11y; tour Phase A, onboard and audit-delta dispatch the auditor with no retry rule: one dead dispatch per outage window there. Accepted, recorded in TODO; locked by test (b).
  • onAutoSwitch (register.ts:3662-3674) charges a sub-agent's engine fallback to main's last plan (TODO 2026-10-13 residual): a gate falling off Fable while main's last plan is sonnet marks sonnet down 15 min; with main's last plan on opus (the new session model) the early return at :3670 marks nothing, so an engine-side fallback would never mark Fable down and every gate spawn would pay the 429 again. Which event the engine emits for a sub-agent credit 429 is not established (BLK-030 saw StopFailure on main). Accepted, no code change here; test (b) locks the StopFailure path only.
  • audit duplicates plan's (best, xhigh) pair: the Change dialog maps a pair to the first phase that fits in file order (targetPhase :2311-2317), so a gate row Changed away and back to "fable + xhigh" lands on plan; audit is reachable by hand edit only. Accepted.
  • register.ts validates a phase's tier against tiers keys: best exists. A phase named like an alias would be rejected: audit is not an alias.
  • model-gate.md (names a fable or opus id | proceed) is about the MAIN loop, not the gates: unaffected.
  • lib/challenge-plan.md:46 says challengers are on the judge row (opus): still true, no edit.
  • The verify phase keeps one consumer once the dialog writes are committed first (skill row deploy = verify); no route(phase="verify") caller exists in skills/agents/lib. Only its about changes.
  • The dirty routing.json / settings.json are committed first on the feature branch (see Router facts); the executor starts from a clean tree and edits the committed routing.json, which already holds the deploy row and changed entry.

Tests

  • register.test.ts: fixture + counts + the new outage-floor case. effort-routing.test.sh: best-tier floor carve-out. model-routing.test.sh: literal pins. mods.test.sh runs claude plugin test (CLI present on this machine). doctrine-citers.test.sh: citer census.

Disposition (STEP 0.6, decisions-first)

  • BDR-066 [in force, partially superseded by this feat]: "verifier + security-auditor STAY sonnet" → overturned by user policy 2026-10-19; feat B writes the superseding BDR. Executors stay sonnet (honored).
  • BDR-077 [in force]: fail-safe pin rule (a forgotten override over-tiers, never downgrades judgment); no call site overrides the gates → honored: the frontmatter floor (opus) never sits below today's security-auditor pin, and the verifier floor rises sonnet → opus.
  • BDR-115 [in force]: rows in routing.json are the route, frontmatter = off-state floor, "an agent moves UP from its frontmatter alias, never below" (inTier) → honored: opus floor, router lifts to Fable; confirmed set because the user decided here, no changed entry forged.
  • BLK-030/031 [resolved, residual TODO.md:56 "spawn path not probed"]: the opus floor gives the retry a landing spot once the first failure marks Fable down; it does not save the first spawn of an outage window (Edge cases). An explicit model:"fable" pin would have neither.
  • LRN-203 [in force]: hooks write full ids from config.models; fable alias maps to claude-fable-5-1 → honored, no id typed.
  • BLK-030/031 [resolved]: Fable credit outages → the best tier falls back; gates still run (edge case above).