chore(memory): feat A audit phase, LRN-224, plan and contract
This commit is contained in:
1 parent
dae142ba44
commit
c03ad8c949
5 files changed
+165
-1
No files matched your search
@@ -629,3 +629,6 @@ rules:
|
||||
|
||||
## 2026-10-19
|
||||
- Guardrails split shipped via /ship-feature (feature/a11y-split-inj-exp): /a11y skill on the shared engine, injection + exposure domains, probe exposure block, secure-by-default six domains. SDD 9 tasks, GATE 1 ECARTS(9)→CONFORME (fields CLOSED route, suppressions removed), GATE 2 PASS (7 LOW), opus review 0 Critical / 6 Important fixed. Routing finding: Fable pinned in long sessions (BLK-032), user policy recorded for a later /feat. Merge on user go.
|
||||
|
||||
## 2026-10-21
|
||||
- /feat routing policy A (feature/routing-audit-phase, BLK-032): `audit` phase (best/xhigh) for verifier + security-auditor, opus frontmatter = outage floor (LRN-224), census tier_floor, citers. 2 challenge rounds (r1 2 BLOCKERs: dead-Fable floor, test fixture), GATE 0 15/15, verifier CONFORME, auditor PASS. d3635b6 dialog writes + session model opus, a7d747c, dae142b docs. Feat B next: ceiling rule, BDR, README doctrine.
|
||||
@@ -213,6 +213,7 @@ rules:
|
||||
| LRN-219 | 2026-10-19 | Scanner regex over untrusted trees: bound every repeat, cap comment blankers, timing lock on adversarial file | any whole-file regex scanner |
|
||||
| LRN-220 | 2026-10-19 | Live probe verdicts fail closed by construction: fixture route per negative branch, 3xx = SKIPPED, depth/size limit = SKIPPED never CLOSED | writing probes and their suites |
|
||||
| LRN-221 | 2026-10-19 | Floor-guard suppressions: remove, never waive; contract CHECK asserts intent (symbol/behaviour), not one code spelling | suppressions, writing CHECK: lines |
|
||||
| LRN-224 | 2026-10-21 | Best-tier agent row floors on tier 2nd alias: inTier writes head-or-above only, fable frontmatter = no fallback in credit outage | pinning agents on best tier, census floors |
|
||||
|
||||
---
|
||||
|
||||
@@ -1828,3 +1829,7 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
|
||||
## LRN-221 — Floor-guard suppressions: remove, never waive; contract oracles assert intent, not syntax
|
||||
- **Context**: GATE 1 counted 6 floor findings: `# noqa: E402` ×2 (import after `sys.path.insert`), a `# shellcheck disable=SC2016` copied from another suite (dead: deleting it raised nothing), 3 plan quotes of the same lines. `importlib.import_module` after the insert needs no noqa. Then criterion 16's CHECK grepped `^import scanner` and went red on the legit fix.
|
||||
- **Apply**: a suppression = first try the construct that needs none; test a copied `disable` by deleting it; a CHECK greps the symbol or behaviour (`scanner.Catalogue` loaded), never one spelling of the code. Links [[BDR-102]].
|
||||
|
||||
## LRN-224 — Best-tier agent row floors on tier's 2nd alias: inTier writes head-or-above only
|
||||
- **Context**: feat A (routing policy BLK-032) moved verifier + security-auditor to new `audit` phase (best = fable>opus>sonnet). Plan r1 set frontmatter `model: fable` (census rule floor = tier head) and claimed "Fable down → falls to opus". False: `inTier` (register.ts) writes a model only if it is the head or ranks above it, else logs "tier … down, frontmatter model kept" → fable frontmatter = gates spawn on dead Fable every credit outage (BLK-030/031). Lower tiers never hit it (head sonnet/opus always has a higher alias). Caught by correctness + robustness lenses r1, not by plan author.
|
||||
- **Apply**: an agent on a best-tier row keeps opus frontmatter; router lifts to Fable when up. Census `tier_floor` (effort-routing.test.sh) locks 2nd alias for best. Residual: spawn path unprobed → first gate of an outage window still dies once, retry lands on floor (register.test.ts audit-row case b). Links [[BLK-030]], [[BLK-032]], [[LRN-204]].
|
||||
@@ -4,7 +4,10 @@
|
||||
- [x] run — SDD 9/9, gates 0/1/2 green, opus review Importants fixed (c0fdbec, 9275e0d, 63a88bc), capitalize BDR-122 LRN-219..221 BLK-032; merge = user go
|
||||
- [ ] parked (review Minors): SENSITIVE_KEY matches metadata names (`token_type`, `has_password`, `next_page_token` locked as sensitive); 403 on first map candidate → SKIPPED before header/comment candidates (S3/CloudFront); EXP-HYDRATED-SENSITIVE scans whole route file (Remix/SvelteKit actions); `webpack.dev.js` WARN; `data-password-toggle` WARN; GraphQL `$where: users_bool_exp` → INJ-NOSQL-OPERATOR; misses: Go raw-string Sprintf, Drizzle `sql.raw`, `db.raw`, `rf""`, Supabase `.select()` no arg, multi-line JSX hidden input, triple-quoted f-strings; `IN (${placeholders})` = FAIL (name in judging step); timing lock skips CONCAT (~2.6 s/1 MB); per-language variants without mutant; SKILL.md:96 fields bearer JWT2; large `.map` head misses esbuild/rollup (`"version":3`); URL-encoded `data:` maps.
|
||||
- [ ] parked (GATE 2 non-blocking, pre-existing on develop): probe `supabase.url` not validated by `_http_url` nor shown in the `--authorized` refusal (MEDIUM, probe.py:168,194: planted SUPABASE_URL would receive the test JWT; `file://` reads CLOSED); same-host redirects not counted in `session.budget` (LOW); `_nonempty_rows` catches ValueError only, deep JSON → RecursionError exit 1 (LOW); scanner text output prints `f.path` / `project.root` without `_printable` (LOW).
|
||||
- [ ] routing policy /feat (user 2026-10-19, BLK-032): settings model → Opus 5.5; `judge` split (security-auditor + verifier → Fable, plan-challenger/analyzer/code review → opus); executors stay sonnet, "what to write" on opus; ceiling rule: main on Fable never writes code beyond a hotfix edit → dispatched executor; BDR superseding part of BDR-066 / BDR-115.
|
||||
- [~] routing policy /feat (user 2026-10-19, BLK-032): settings model → Opus 5.5; `judge` split (security-auditor + verifier → Fable, plan-challenger/analyzer/code review → opus); executors stay sonnet, "what to write" on opus; ceiling rule: main on Fable never writes code beyond a hotfix edit → dispatched executor; BDR superseding part of BDR-066 / BDR-115.
|
||||
- [x] feat A audit phase: d3635b6 (dialog writes, session model opus), a7d747c (audit row, opus floor, census, citers), dae142b (docs); gates 0/1/2 green, LRN-224
|
||||
- [ ] feat B: ceiling rule CLAUDE.global.md, BDR-123 superseding parts of BDR-066/115, README l.17 + l.72-73 doctrine (P2/P3 deferred), `skills/deploy/SKILL.md` effort low→xhigh (WARN floor drift)
|
||||
- [ ] residuals (accepted): spawn path unprobed, 1st gate of an outage window dies once (tour/onboard/audit-delta no retry); onAutoSwitch with main on opus marks nothing if engine auto-falls a gate off Fable
|
||||
|
||||
## 2026-10-18 — secure-by-default guardrails: 5 domains + /guardrails skill (feature/secure-by-default)
|
||||
User ask: hosting, a11y, Supabase, SMS/email consent, cloud functions must be baseline on every project, API and access secured systematically, plus a skill that tests all of it.
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
# CONTRACT — routing-audit-phase (feat A of the routing policy, BLK-032)
|
||||
|
||||
## REQUEST (verbatim, feat A scope after the 2026-10-21 scope split)
|
||||
model-router `judge` level split: security-auditor and verifier gates → Fable, plan-challenger / analyzer / code review → opus; executors stay sonnet. Citers census: skills/agents/lib rows that name the judge level must be patched in the same commit (make test runs the census). Feat B (settings model, CLAUDE.global.md ceiling rule, BDR) is a separate run.
|
||||
|
||||
## CLARIFICATIONS
|
||||
- [gated 2026-10-21] New phase name: `audit` (user pick over `gate`), tier `best`, effort `xhigh`.
|
||||
- [gated 2026-10-21, amended after the challenge] Scope split: feat A = router + agent floors + census + citers, preceded on the feature branch by one dedicated commit of the pre-existing routing.json (router dialog writes) and settings.json (session model `opus`) diffs; feat B = CLAUDE.global.md ceiling rule, BDR, docs.
|
||||
- Internal: `confirmed.agents` for verifier and security-auditor set to `audit` by hand (this feat is the user's confirmation; no dialog re-fire). The `about` lines of `judge` and `verify` drop the words that now describe `audit`.
|
||||
- [gated 2026-10-21] Outage floor: the gates keep `model: opus` in frontmatter (inTier never writes below the tier head; a fable floor would spawn on a dead Fable during a credit outage). The census gets a named carve-out: a best-tier agent row floors on the tier's second alias.
|
||||
- [gated 2026-10-21] Scope: 11 files in one /feat run (user call over /ship-feature after the challenge).
|
||||
- Internal: no stash. The pre-existing diffs on `mods/model-router/routing.json` (router dialog writes) and `settings.json` (session model `opus`) are committed first on the feature branch as one dedicated commit, before the executor runs.
|
||||
- Internal: `confirmed.phases.audit` is NOT written: the first-use dialog stays as the tripwire against a main-loop `route(phase="audit")`.
|
||||
- Internal: feature branch `feature/routing-audit-phase` started from the lib off develop before any commit.
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. routing.json has the `audit` phase: tier best, effort xhigh, the exact about "the fresh gates: contract verdict and security scan; dispatched agents, not a main-loop phase", placed right after `judge`.
|
||||
CHECK: python3 -I -c "import json;d=json.load(open('mods/model-router/routing.json'));p=d['phases'];k=list(p);a=p['audit'];assert a['tier']=='best' and a['effort']=='xhigh' and a['about']=='the fresh gates: contract verdict and security scan; dispatched agents, not a main-loop phase' and k[k.index('judge')+1]=='audit';print('AUDIT-PHASE ok')"
|
||||
EXPECT: AUDIT-PHASE ok
|
||||
EVIDENCE: MET exit=0 marker-found :: AUDIT-PHASE ok
|
||||
2. register.ts DEFAULT_CONFIG phases carry the same `audit` line, right after `judge`.
|
||||
CHECK: grep -n -A1 "^ judge: { tier: 'big', effort: 'xhigh' }," mods/model-router/hooks/register.ts | grep -c "audit: { tier: 'best', effort: 'xhigh' },"
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
3. routing.json rows `agents.verifier` and `agents.security-auditor` are `audit`, so are their `confirmed.agents` entries; `confirmed.phases` has NO `audit` key (first-use dialog kept for main).
|
||||
CHECK: python3 -I -c "import json;d=json.load(open('mods/model-router/routing.json'));assert all(t[n]=='audit' for t in (d['agents'],d['confirmed']['agents']) for n in ('verifier','security-auditor'));assert 'audit' not in d['confirmed']['phases'];print('ROWS ok')"
|
||||
EXPECT: ROWS ok
|
||||
EVIDENCE: MET exit=0 marker-found :: ROWS ok
|
||||
4. `agents/verifier.md` and `agents/security-auditor.md` frontmatter read `model: opus` and `effort: xhigh` (opus = outage floor of the best tier).
|
||||
CHECK: for f in agents/verifier.md agents/security-auditor.md; do awk 'NR==1&&/^---$/{p=1;next} p&&/^---$/{exit} p' "$f" | grep -cE '^(model: opus|effort: xhigh)$'; done | tr '\n' ' '
|
||||
EXPECT: 2 2
|
||||
EVIDENCE: MET exit=0 marker-found :: 2 2
|
||||
5. The `judge` and `verify` `about` lines no longer claim audits or security review: judge = "independent judgment of work: challenges, analyses and reviews", verify = "checking a result against its spec: tests and suites".
|
||||
CHECK: grep -c '"about": "independent judgment of work: challenges, analyses and reviews"\|"about": "checking a result against its spec: tests and suites"' mods/model-router/routing.json
|
||||
EXPECT: 2
|
||||
EVIDENCE: MET exit=0 marker-found :: 2
|
||||
6. `lib/tests/model-routing.test.sh` locks the new floors (`agents/verifier.md` `model: opus`, `agents/security-auditor.md` `model: opus`, both commented as the audit row) and no longer locks `verifier.md` on sonnet.
|
||||
CHECK: a=$(grep -c "has \"agents/verifier.md\" *'model: opus'.*audit\|has \"agents/security-auditor.md\" *'model: opus'.*audit" lib/tests/model-routing.test.sh); b=$(grep -c "verifier.md\" *'model: sonnet'" lib/tests/model-routing.test.sh); echo "$a/$b"
|
||||
EXPECT: 2/0
|
||||
EVIDENCE: MET exit=0 marker-found :: 2/0
|
||||
7. Census suites green: `make test suite=lib/tests/effort-routing.test.sh` and `make test suite=lib/tests/model-routing.test.sh` exit 0.
|
||||
CHECK: make test suite=lib/tests/effort-routing.test.sh >/dev/null 2>&1 && make test suite=lib/tests/model-routing.test.sh >/dev/null 2>&1 && echo CENSUS-GREEN
|
||||
EXPECT: CENSUS-GREEN
|
||||
EVIDENCE: MET exit=0 marker-found :: CENSUS-GREEN
|
||||
8. Mod suite green and not skipped: `make test suite=lib/tests/mods.test.sh` exits 0 and reports `plugin test passed` for model-router (the CLI is present on this machine).
|
||||
CHECK: make test suite=lib/tests/mods.test.sh 2>&1 | grep -c 'model-router: plugin test passed'
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
9. `register.test.ts` carries the `audit` phase in `baseRouting()`, the `pending confirmation` counts 20 and 18, and two audit-row cases: fable already down → no write + log `tier best down, frontmatter model kept`; first spawn on fable then `rate_limit` → the retry writes nothing (`[FABLE, undefined]`).
|
||||
CHECK: f=mods/model-router/hooks/register.test.ts; a=$(grep -c "^ audit: { tier: 'best', effort: 'xhigh' }," $f); b=$(grep -c "pending confirmation: 20'\|pending confirmation: 18'" $f); c=$(grep -c "tier best down, frontmatter model kept" $f); d=$(grep -c "toEqual(\[FABLE, undefined\])" $f); echo "$a/$b/$([ "$c" -ge 1 ] && echo ok)/$d"
|
||||
EXPECT: 1/2/ok/1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1/2/ok/1
|
||||
10. `lib/tests/effort-routing.test.sh` names the best-tier carve-out: a `tier_floor` helper returning the second alias for `best`, used by the model check, a comment naming BLK-030, and a flip test of the helper on the fixture (best → opus, big → opus).
|
||||
CHECK: f=lib/tests/effort-routing.test.sh; a=$(grep -c '^tier_floor()' $f); b=$(grep -c 'tier_floor "\$REG" "\$tier"' $f); c=$(grep -c 'BLK-030' $f); d=$(grep -c 'tier_floor "\$FIX/reg.ts" best\|tier_floor "\$FIX/reg.ts" big' $f); echo "$a/$b/$([ "$c" -ge 1 ] && echo ok)/$d"
|
||||
EXPECT: 1/1/ok/2
|
||||
EVIDENCE: MET exit=0 marker-found :: 1/1/ok/2
|
||||
11. The four citers no longer claim sonnet gates: `agents/plan-challenger.md`, `lib/challenge-plan.md`, `skills/tour/SKILL.md`, `README.md` each carry the token `` `audit` row `` (whitespace-normalised, survives a reflow), the old sentences are gone, and README's phase list names `` `audit` best/xhigh `` (checked by the verifier, no oracle).
|
||||
CHECK: a=0; for f in agents/plan-challenger.md lib/challenge-plan.md skills/tour/SKILL.md README.md; do tr '\n' ' ' < "$f" | tr -s ' ' | grep -q '`audit` row' && a=$((a+1)); done; b=$(cat agents/plan-challenger.md lib/challenge-plan.md skills/tour/SKILL.md README.md | tr '\n' ' ' | tr -s ' ' | grep -o 'both sonnet, oracle-anchored\|The executor gates stay sonnet\|security-auditor (sonnet frontmatter)\|| verifier | sonnet (pinned)\|| security-auditor | opus (judge row' | wc -l); echo "$a/$b"
|
||||
EXPECT: 4/0
|
||||
EVIDENCE: MET exit=0 marker-found :: 4/0
|
||||
12. Doctrine citers census green: `make test suite=lib/tests/doctrine-citers.test.sh` exits 0.
|
||||
CHECK: make test suite=lib/tests/doctrine-citers.test.sh >/dev/null 2>&1 && echo CITERS-GREEN
|
||||
EXPECT: CITERS-GREEN
|
||||
EVIDENCE: MET exit=0 marker-found :: CITERS-GREEN
|
||||
13. `lib/challenge-plan.md` still states the challengers run on the `judge` row (opus): no edit needed, the statement stays true.
|
||||
CHECK: grep -c 'routed to the `judge` row (opus)' lib/challenge-plan.md
|
||||
EXPECT: 1
|
||||
EVIDENCE: MET exit=0 marker-found :: 1
|
||||
14. No new lint or test suppression in the diff (floor-guard clean).
|
||||
CHECK: git diff -U0 -- . ':!.claude' | grep -E '^\+' | grep -cE 'shellcheck disable|noqa|eslint-disable|@ts-ignore|@ts-expect-error' | sed 's/^/n=/'
|
||||
EXPECT: n=0
|
||||
EVIDENCE: MET exit=0 marker-found :: n=0
|
||||
15. The diff touches only the FILE SCOPE: `git status --short` lists nothing outside it (the dialog-write and session-model diffs are committed before the executor runs).
|
||||
CHECK: git status --short -- . ':!.claude' | awk '{print $2}' | grep -vcE '^(mods/model-router/routing.json|mods/model-router/hooks/register.ts|mods/model-router/hooks/register.test.ts|agents/verifier.md|agents/security-auditor.md|lib/tests/model-routing.test.sh|lib/tests/effort-routing.test.sh|agents/plan-challenger.md|lib/challenge-plan.md|skills/tour/SKILL.md|README.md)$' | sed 's/^/n=/'
|
||||
EXPECT: n=0
|
||||
EVIDENCE: MET exit=0 marker-found :: n=0
|
||||
|
||||
## FILE SCOPE
|
||||
- mods/model-router/routing.json
|
||||
- mods/model-router/hooks/register.ts
|
||||
- mods/model-router/hooks/register.test.ts
|
||||
- agents/verifier.md
|
||||
- agents/security-auditor.md (no edit expected: frontmatter already opus/xhigh)
|
||||
- lib/tests/model-routing.test.sh
|
||||
- lib/tests/effort-routing.test.sh
|
||||
- agents/plan-challenger.md
|
||||
- lib/challenge-plan.md
|
||||
- skills/tour/SKILL.md
|
||||
- README.md
|
||||
@@ -0,0 +1,64 @@
|
||||
# PLAN r3 — routing-audit-phase (feat A, BLK-032 routing policy)
|
||||
|
||||
r2 after the 2026-10-21 challenge round 1 (outage floor, test fixture, citers, no stash). r3 after round 2: single about string, no `confirmed.phases.audit` (first-use dialog = tripwire for main), unprobed-spawn residual + its test, explicit branch step, oracles 1/3/10/11 tightened, tier_floor flip test.
|
||||
|
||||
Contract: `.claude/tasks/contracts/2026-10-21-routing-audit-phase-1506.md`
|
||||
|
||||
## Why
|
||||
User policy 2026-10-19 (BLK-032): Fable only for orchestrator reflection/plans and the important audits (security, functional); other judges opus; executors sonnet. Today `verifier` routes to `verify` (work tier = sonnet xhigh) and `security-auditor` to `judge` (big tier = opus xhigh). Both are the merge-blocking gates of verify-secure-loop.md (GATE 1 contract verdict, GATE 2 security). A new best-tier phase `audit` carries them; `judge` keeps plan-challenger, analyzer, Plan, plugin-advisor, seo/geo-analyzer on opus.
|
||||
|
||||
## Router facts that bind the plan
|
||||
- Rows are phase names; a phase names a tier; a tier is an ordered alias list, first alias not down is used (`best` = fable > opus > sonnet). An agent gets its row's model at spawn, upward only within the tier; explicit `model=` at a call site wins (BDR-115, BDR-077). No dispatch of verifier / security-auditor carries an explicit `model=` (grep 2026-10-21: tour, onboard, audit-delta dispatch them bare).
|
||||
- `lib/tests/effort-routing.test.sh` locks: every routing.json phase equals the code copy in `register.ts` DEFAULT_CONFIG (`code_phase`); every rowed agent's frontmatter `model:` equals its phase's tier head and `effort:` equals the phase effort, unless a `changed` entry explains the drift (WARN). Neither verifier nor security-auditor has a `changed` entry, so the frontmatter MUST move with the row or the census FAILs.
|
||||
- `lib/tests/model-routing.test.sh:23-24` greps the literal old pins.
|
||||
- `register.test.ts` builds every world from `baseRouting()` (11 phases, :65-78). `logMissingPhases` (register.ts:862-869) diffs `DEFAULT_CONFIG.phases` against the file, so a DEFAULT_CONFIG phase absent from the fixture is logged in every test: :1592-1601 expects exactly `lacks phases reflect`, and `/route pending` counts `st.cfg.phases` (:2555 `pending confirmation: 19`, :2567 `17`). The fixture gains `audit` and the two counts become 20 and 18.
|
||||
- `inTier` (register.ts:3163-3174) writes a model only when it is the tier head or ranks above it; otherwise "tier … down, frontmatter model kept" and the frontmatter runs (test :1278-1291). A best-tier row with a `fable` frontmatter therefore has NO fallback: every Fable credit outage (BLK-030/031) would spawn both gates on a dead model. Decision (user 2026-10-21): the gates keep `model: opus` as the outage floor; the router lifts them to Fable when it is up; Fable down or router off = opus. The census gets a named carve-out: a best-tier agent row's model floor is the tier's SECOND alias.
|
||||
- No stash. `~/.claude/settings.json` and `~/.claude/skills/model-router` are symlinks into the repo: stashing settings.json puts every new session back on Fable 1M (the BLK-032 condition) and stashing routing.json re-arms the router's dialogs. The dirty copies carry over the branch start unchanged (`git diff develop..HEAD` is empty for both files). They are committed FIRST on the feature branch as one dedicated commit (the router's own dialog writes: `deploy` apply→verify + `changed` entry, `release-executor` Keep; the session model `opus` = BLK-032 policy point 1), before the executor runs. Feat B then carries CLAUDE.global.md, the BDR and the docs only.
|
||||
- `confirmed.agents` records first-use dialog answers; an unconfirmed row re-opens the dialog at first use. This feat is the user's decision → set `audit` by hand for both.
|
||||
|
||||
## Branch (main loop, before the executor)
|
||||
1. `bash ~/.claude/lib/gitflow.sh start feature routing-audit-phase` from the current `bugfix/guardrails-residuals` (not merged): the lib branches off develop; the dirty `routing.json` / `settings.json` carry over (develop and HEAD hold the same blobs).
|
||||
2. One dedicated commit of those two files: `chore(model-router): dialog decisions (deploy verify, release-executor) and session model opus` — the router's own writes plus BLK-032 policy point 1.
|
||||
3. Dispatch the executor on the clean tree.
|
||||
4. Memory IDs for this run's capitalize start above the highest on BOTH develop and the unmerged `bugfix/guardrails-residuals` (8555bff: BLK-033, LRN-222/223): next BDR-123, LRN-224, BLK-034. The TODO line for this run goes under the 2026-10-19 routing bullet, away from the bugfix hunk at TODO.md:5-6.
|
||||
|
||||
## Checklist (dispatch-ready)
|
||||
[ ] mods/model-router/routing.json — phases: insert after `judge` (line 23-27):
|
||||
"audit": { "tier": "best", "effort": "xhigh", "about": "the fresh gates: contract verdict and security scan; dispatched agents, not a main-loop phase" }
|
||||
(exact string, 93 chars; `about` is shown in the first-use dialog only, register.ts:154, so this text is what main's first `route(phase="audit")` would print). Reword `judge.about` → "independent judgment of work: challenges, analyses and reviews"; `verify.about` → "checking a result against its spec: tests and suites". Rows: `agents.verifier` "verify" → "audit"; `agents.security-auditor` "judge" → "audit". `confirmed.agents.verifier` → "audit", `confirmed.agents.security-auditor` → "audit" (the two spawns the user decided here). Do NOT write `confirmed.phases.audit`: an unconfirmed phase keeps the first-use dialog (`confirmPhase` register.ts:2580-2585, suppressed by `isDecided` :2081 once confirmed) as the tripwire against a main-loop `route(phase="audit")`; `/route pending` listing `phases:audit` is the price. Nothing else changes: the `changed` block keeps the committed `deploy` entry, no entry is forged for the hand-moved rows.
|
||||
[ ] mods/model-router/hooks/register.ts — DEFAULT_CONFIG.phases (line ~280): insert `audit: { tier: 'best', effort: 'xhigh' },` right after the `judge` line. Same indentation, trailing comma. No other code change (inTier untouched).
|
||||
[ ] mods/model-router/hooks/register.test.ts — `baseRouting()` phases: insert `audit: { tier: 'best', effort: 'xhigh' },` after `judge` (:70-71); `pending confirmation: 19` → `20` (:2555), `17` → `18` (:2567). Leave the fixture's `verifier: 'verify'` row (:89) as is, other tests use it. Add two tests next to :1278-1291, built with `bootRun($, on, FABLE, newWorld({ edit: f => { (f.agents as Rec)['security-auditor'] = 'audit' } }))` (bootRun's 4th param, :1035-1041) and bare `spawnOf('security-auditor')` (no `model` field: an explicit model skips routing, :3199):
|
||||
(a) `test('audit row: fable down, no write, frontmatter runs', …)`: `switchTo($, FABLE, SONNET, 'auto')`, spawn once, expect `rig.specs` `[undefined]` and the one log line `'model-router: security-auditor tier best down, frontmatter model kept'` (recordLogs filter on `frontmatter model kept`).
|
||||
(b) `test('audit row: first spawn dies on fable, the retry lands on the floor', …)`: spawn, `await failWith($, 'rate_limit', 'a1')`, spawn again, expect `rig.specs` `[FABLE, undefined]` (same shape as :886-891); no log assertion in (b). This is the path verify-secure-loop's single fresh retry now depends on.
|
||||
[ ] agents/verifier.md — frontmatter `model: sonnet` → `model: opus` (effort stays xhigh). Body untouched. Rationale in the census comment, not here.
|
||||
[ ] agents/security-auditor.md — frontmatter stays `model: opus`, `effort: xhigh`: no edit.
|
||||
[ ] lib/tests/effort-routing.test.sh — add `tier_floor <file> <tier>` right after `tier_head`: second alias when `$2` = best, else `tier_head`. ONE comment above it (the only place citing BLK-030 in the file): `# a best-tier row floors on its 2nd alias: inTier keeps the frontmatter when the head is down (Fable credits, BLK-030); a fable floor would spawn the gates on a dead model`. `field_of` calls `tier_floor "$REG" "$tier"` for the model field. Section (c) header becomes `# ── (c) tracked agents with a row: tier floor == model:, effort == effort: ─` (no BLK cite). Flip test: add `best: ['fable', 'opus', 'sonnet'],` to the fixture's `reg.ts` tiers block (:88-92) and two checks: `[ "$(tier_floor "$FIX/reg.ts" best)" = opus ] && ok || ko "flip: best floors on its 2nd alias"` and `[ "$(tier_floor "$FIX/reg.ts" big)" = opus ] && ok || ko "flip: big floors on its head"`.
|
||||
[ ] lib/tests/model-routing.test.sh:23-24 — `has "agents/verifier.md" 'model: opus' # audit row (best tier, opus = outage floor; BLK-032 policy 2026-10-21)` and `has "agents/security-auditor.md" 'model: opus' # audit row since 2026-10-21 (was judge since 2026-10-11)`. Keep the column alignment of the block.
|
||||
[ ] agents/plan-challenger.md:112-113 — replace "(Contrast the verifier on its `verify` row and the executors on their `implement` row, both sonnet, oracle-anchored to a contract.)" with "(Contrast the executors on their `implement` row, sonnet, oracle-anchored to a contract; the verifier and security-auditor sit on the `audit` row, Fable with an opus floor.)". Reflow the paragraph to 80 columns.
|
||||
[ ] lib/challenge-plan.md:50 — "(The executor gates stay sonnet.)" → "(The executors stay sonnet; the two merge gates sit on the `audit` row, Fable with an opus floor.)". Reflow to 80 columns.
|
||||
[ ] skills/tour/SKILL.md:83 — "security-auditor (sonnet frontmatter)" → "security-auditor (`audit` row: Fable, opus floor)". Reflow if the line exceeds 80 columns.
|
||||
[ ] README.md:86-87 — verifier row: `| verifier | Fable (`audit` row: best/xhigh, opus frontmatter floor) | fresh gate (≤3×/loop) |`; security-auditor row: `| security-auditor | Fable (`audit` row: best/xhigh, opus frontmatter floor) | fresh gate (≤3×/loop) |`. Also README.md:111-113 phase list: insert `` `audit` best/xhigh `` after `` `judge` big/xhigh ``; :114-116 "kept equal to the rows by the census" → "kept equal to the rows by the census `lib/tests/effort-routing.test.sh` (a best-tier row floors on the tier's second alias, opus, since Fable is credit-gated; a row changed through the first-use dialog passes with a `WARN floor drift` line"; :135 best-tier row list → "(`plan`, `reflect`, `orchestrate`, `escalate`; `audit` is an agent row, never typed)". Every citer edit above carries the token `` `audit` row `` (oracle 11).
|
||||
[ ] run: the four suites first (`make test suite=lib/tests/effort-routing.test.sh`, `model-routing`, `mods`, `doctrine-citers`), then the full `make test` (plan-challenger.test.sh locks strings in plan-challenger.md / challenge-plan.md; contract-verifier and gates suites read verifier.md); then the contract oracles.
|
||||
|
||||
## Edge cases
|
||||
- Fable down (credits, BLK-030/031): `inTier` writes nothing, the `opus` frontmatter runs: the gates degrade to opus, one log line `tier best down, frontmatter model kept`. Router off (`/route off`, `enabled:false`, unreadable routing.json): opus too. Locked by the new register.test.ts case and the census carve-out.
|
||||
- `audit` appears in the main loop's `route` tool enum as a bare name (register.ts:2912-2916; `about` is never in the tool text). A main-loop `route(phase="audit")` opens the first-use phase dialog (phase left unconfirmed), which prints the about: that dialog is the tripwire. Same capability as `plan`; no orchestrator skill calls it; feat B's ceiling rule covers inline work on Fable.
|
||||
- Unprobed spawn path (TODO.md:56 residual, unchanged): the router learns Fable is down from a failure, never from a probe before a spawn. In an outage window the FIRST gate spawn dies on Fable (429), `StopFailure` marks it down, the fresh retry lands on the opus floor. verify-secure-loop's single retry covers /feat, /bugfix, /ship-feature, /init-project, /onboard, /guardrails, /a11y; tour Phase A, onboard and audit-delta dispatch the auditor with no retry rule: one dead dispatch per outage window there. Accepted, recorded in TODO; locked by test (b).
|
||||
- `onAutoSwitch` (register.ts:3662-3674) charges a sub-agent's engine fallback to main's last plan (TODO 2026-10-13 residual): a gate falling off Fable while main's last plan is sonnet marks sonnet down 15 min; with main's last plan on opus (the new session model) the early return at :3670 marks nothing, so an engine-side fallback would never mark Fable down and every gate spawn would pay the 429 again. Which event the engine emits for a sub-agent credit 429 is not established (BLK-030 saw StopFailure on main). Accepted, no code change here; test (b) locks the StopFailure path only.
|
||||
- `audit` duplicates `plan`'s (best, xhigh) pair: the Change dialog maps a pair to the first phase that fits in file order (`targetPhase` :2311-2317), so a gate row Changed away and back to "fable + xhigh" lands on `plan`; `audit` is reachable by hand edit only. Accepted.
|
||||
- `register.ts` validates a phase's tier against `tiers` keys: `best` exists. A phase named like an alias would be rejected: `audit` is not an alias.
|
||||
- `model-gate.md` (`names a fable or opus id | proceed`) is about the MAIN loop, not the gates: unaffected.
|
||||
- `lib/challenge-plan.md:46` says challengers are on the judge row (opus): still true, no edit.
|
||||
- The `verify` phase keeps one consumer once the dialog writes are committed first (skill row `deploy` = verify); no `route(phase="verify")` caller exists in skills/agents/lib. Only its `about` changes.
|
||||
- The dirty `routing.json` / `settings.json` are committed first on the feature branch (see Router facts); the executor starts from a clean tree and edits the committed routing.json, which already holds the `deploy` row and `changed` entry.
|
||||
|
||||
## Tests
|
||||
- `register.test.ts`: fixture + counts + the new outage-floor case. `effort-routing.test.sh`: best-tier floor carve-out. `model-routing.test.sh`: literal pins. `mods.test.sh` runs `claude plugin test` (CLI present on this machine). `doctrine-citers.test.sh`: citer census.
|
||||
|
||||
## Disposition (STEP 0.6, decisions-first)
|
||||
- BDR-066 [in force, partially superseded by this feat]: "verifier + security-auditor STAY sonnet" → overturned by user policy 2026-10-19; feat B writes the superseding BDR. Executors stay sonnet (honored).
|
||||
- BDR-077 [in force]: fail-safe pin rule (a forgotten override over-tiers, never downgrades judgment); no call site overrides the gates → honored: the frontmatter floor (opus) never sits below today's security-auditor pin, and the verifier floor rises sonnet → opus.
|
||||
- BDR-115 [in force]: rows in routing.json are the route, frontmatter = off-state floor, "an agent moves UP from its frontmatter alias, never below" (inTier) → honored: opus floor, router lifts to Fable; `confirmed` set because the user decided here, no `changed` entry forged.
|
||||
- BLK-030/031 [resolved, residual TODO.md:56 "spawn path not probed"]: the opus floor gives the retry a landing spot once the first failure marks Fable down; it does not save the first spawn of an outage window (Edge cases). An explicit `model:"fable"` pin would have neither.
|
||||
- LRN-203 [in force]: hooks write full ids from `config.models`; `fable` alias maps to `claude-fable-5-1` → honored, no id typed.
|
||||
- BLK-030/031 [resolved]: Fable credit outages → the `best` tier falls back; gates still run (edge case above).
|
||||
Reference in new issue
Block a user