Files
claude/.claude/tasks/plans/2026-07-19-model-tiering-v2-plan.md
T
Bastien Chanot 1c2d30dbf0 chore(tasks): model-tiering v2 — analysis + challenged plan v3 + TODO reconcile
Plan challenged by 3 blind lenses + 1 confirmation pass (1 BLOCKER closed
by fable-dispatch spike, 6 MAJORs + 8 MINORs fixed by named changes, 0
deferred). TODO: seo-geo-integrity 'UNMERGED' note was stale (92301fe
already in develop) — corrected.
2026-07-19 19:51:06 +02:00

276 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# PLAN: model-tiering v2 — full framework re-tier + splits
Input: `.claude/tasks/plans/2026-07-19-model-tiering-v2-analysis.md` (read it
first — consumer map, test locks, LRN/BDR constraints live there).
User arbitrage (2026-07-19): 7 recos approved + no-inherit/fable-pin amendment
+ Fable scope = REFLECTION / ORCHESTRATION / PLANNING / LOGIC only.
## D0 — DOCTRINE (end state)
1. Main loop (session model, gated big by model-gate) keeps ONLY: brainstorm,
plan, contract, loop decisions, gate arbitration, human interaction,
conversation-context work (capitalize), trivial glue bash (<~1k tokens).
Retention criteria (any suffices): interactive | needs conversation context
| orchestration decision | dispatch overhead > step cost.
2. NOTHING dispatched inherits. Typed agents: frontmatter pin. Built-ins
(general-purpose/Explore/Plan): explicit `model=` at EVERY call site.
VERIFIED (2026-07-19 spike, closes robustness BLOCKER): `model: "fable"`
on a dispatch resolves to claude-fable-5 at runtime (echo spike via
general-purpose); the harness enum-validates the `model` param — an
invalid value fails LOUDLY (InputValidationError), no silent fallback.
Call-site `model=` takes precedence over a typed agent's frontmatter pin
(documented Agent-tool contract); fallback direction if a call site omits
it = the frontmatter pin, i.e. today's behavior — fail-safe, never worse.
3. Tiers: fable = dispatched reflection-on-behalf-of-main-loop (skill-runner
children ONLY); opus = deep judgment (audit scoring, plan critique, drift
semantics, review, synthesis); sonnet = standard execution from closed
instructions + collectors AND probes (wave-1 prudence — robustness MAJOR:
plugin PHASE 1 is a ~26-call branching bash chain, not a short probe);
haiku = status-reporter ONLY in wave 1; haiku expansion = wave 2 after
reliability proven per candidate.
4. Grammars/sentinels/valves survive VERBATIM (list in analysis §CONSTRAINTS).
Loops/gates stay in main loop (BDR-050/LRN-083). Fix-bundle → L1 apply
(BDR-061) preserved: audit agents never get the Agent tool.
5. Every split: LRN-126 data-flow pass (enumerate child-read fields vs
parent-set; explicit handoff contract on disk or in prompt) PLUS an
IN-WAVE planted-input smoke proving the fields cross the dispatch boundary
at runtime — the smoke GATES that wave's merge (confirmation MAJOR:
enumeration is design-time reading; census can't catch severed wires; a
split must never reach develop empirically unproven). Every change:
LRN-113 whole-surface sweep + census lock + flip-test (LRN-096, no `\n` in
patterns LRN-093, strict YAML G3).
## D1 — AGENT END STATE
Pins (frontmatter):
- opus: analyzer, plan-challenger, seo-judge*, geo-judge*, doc-auditor*,
plugin-reasoner*, handover-synthesizer* (*new, from splits)
- sonnet: feater, bugfixer, hotfixer, code-cleaner, refactorer, verifier,
security-auditor, scaffolder (effort high), onboarder, release-executor,
commit-changer, doc-syncer (patcher half), validator-analyzer (TIER-DOWN
from opus), seo-worker*, geo-worker* (2-way split per domain — simplicity
MAJOR: collector+templater both sonnet in wave 1 → one worker file with
`MODE: collect | template`, no cross-domain share: domain bodies genuinely
diverge), handover-renderer* (renamed handover-doc-writer render half),
plugin-probe* (wave-1 prudence; haiku candidate wave 2)
- haiku: status-reporter (only)
- none (inline-only, main loop, gate-protected): interviewer,
client-handover-writer
Per-dispatch `model=` overrides (no new file): commit-changer propose=opus /
apply=sonnet (2 sites in /commit-change — precedence over the sonnet
frontmatter pin is the documented Agent-tool contract, verified direction
D0.2; the pin stays as the no-inherit fallback = today's behavior; both
call-site strings census-locked + W3 behavioral smoke); SDD
implementers+reviewers
sonnet (already prose-mandated → make it a census lock); code-review steps
(ship-feature 6, init-project 10) = opus explicit; client-handover-writer's 8
general-purpose skill-runners = fable; onboard's 7 general-purpose = opus
(keep); any Explore/Plan ad-hoc reflection dispatch = fable (doctrine line in
model-gate.md + CLAUDE.global routing note).
Splits (each = new agent file(s) + handoff contract + census + consumers):
S1 plugin-advisor → plugin-probe (SONNET wave 1; PHASE 1 CLI probes → PROBE
REPORT) + plugin-reasoner (opus; PHASE 2/2.5 scoring + reco → PLUGIN CHECK
block). PHASE 3-4 report+apply-gate HOISTED into ONE shared include
`lib/plugin-gate.md` (simplicity MINOR — doc-commit.md ×6 pattern, never
4 hand-copies), referenced by the 4 consumers (plugin-check, onboard
STEP 0, init-project STEP 0, ship-feature STEP 0) — main loop. The
pre-recommendation validation checkpoint (advisor :201-212, straddles the
seam, can skip PHASE 4) runs IN THE CONSUMER between the two dispatches
(correctness MINOR); its inputs (toggle-external availability,
project-signal presence) are PROBE REPORT fields. Handoff: PROBE REPORT
fields = plugin list, toggle state, profile, CLI/anim/monorepo/embedded
signals + checkpoint inputs (enumerate ALL PHASE-2-read fields).
S2 doc-syncer → doc-auditor (opus; STEP 3-4 drift + semantic analysis + A3
MINOR/SIGNIFICANT call w/ doc-shape.sh oracle → DRIFT REPORT [AUTO]/
[HUMAN] items) + doc-syncer (sonnet; render/patch half, keeps
PATCHED_FILES: grammar + BDR-022 bans). Validation gate stays in
DISPATCHER (/doc skill, orchestrator steps) — auto-mode flows: auditor →
dispatcher applies AUTO via doc-syncer → SIGNIFICANT escalates inline.
Consumers rerouted: /doc, onboard, + doc-commit steps in bugfix/hotfix/
feat/init-project(×2)/ship-feature (inline→dispatch conversion) +
scaffolder PHASE 6 (scaffolder DISPATCHES nothing — it has no Agent tool:
README bootstrap moves to init-project STEP 5b dispatch of doc-syncer).
PLUS (robustness MAJOR): rework `lib/doc-commit.md`'s in-thread contract
BEFORE converting any doc-commit site — it requires the orchestrator to
"hold the patch context" to compose the rc-0 CHANGE SUMMARY (the review
surface that replaced the removed MINOR gate). Dispatched doc-syncer adds
a `CHANGE SUMMARY` block to its report grammar (per patched file: what
changed and why, ≤1 line each); doc-commit.md's composer consumes THAT
instead of in-thread context; census-locks the new field + a planted-input
smoke proves the summary crosses the dispatch boundary.
S3 seo-analyzer → 2-WAY (simplicity MAJOR — 3-way was YAGNI while collector
and templater share the sonnet tier; commit-changer mode-precedent):
seo-worker (sonnet; `MODE: collect` = STEP 2-5 signals → SIGNALS file;
`MODE: template` = STEP 12-14 FIX BUNDLE + sentinel + SEO.md + envelope)
+ seo-judge (opus; STEP 6-11 sampling judgment, competitive, scoring /20,
trajectory, triage → FINDINGS+PLAN). Orchestrated by /seo at L1 (BDR-061
conserved: no Agent tool in either). Wave-2 option: carve `MODE: collect`
into a haiku file once proven — the mode boundary IS the future cut line.
HANDOFF (robustness MAJOR — freshness/atomicity): run-scoped paths
`.audit/seo-signals-<RUNID>.md` / `.audit/geo-signals-<RUNID>.md` —
`.audit/` is the GITIGNORED derived-artifact tree (confirmation MINOR,
LRN-124: a crash-stranded transient with scraped GSC/competitor content
must never be committable; `.claude/audits/` keeps only the SEO.md/GEO.md
deliverables). RUNID minted by the dispatcher per run, passed to every
stage; the file ENDS with `COLLECTION COMPLETE — RUNID: <id>` and the
judge FAILS CLOSED (report ERROR, never score) if the file is absent,
RUNID mismatches, or the completeness sentinel is missing; dispatcher
cleans the file post-run.
DISPATCHER CONTRACT (confirmation MAJOR — fail-closed at the judge must
not fail OPEN at the pipeline): on a judge ERROR the orchestrator
(/seo /geo /harden /onboard) STOPS — no template dispatch, no L1 apply —
surfaces the ERROR verbatim, retries ONCE with a fresh collect+judge,
then escalates to the human. A mute or ERROR judge is NEVER carried into
templating (verify-secure-loop discipline). This handler is part of the
W5 skill rewrites, census-locked.
Explicit field list per LRN-126 (STEP 1-2 business+tech context consumed
by ALL later steps — full enumeration REQUIRED before cutting).
seo-data.test.sh locks (fetch.sh wiring) move with the worker body —
update suite same commit.
S4 geo-analyzer → geo-worker (sonnet, 2 modes) + geo-judge (opus) — mirror of
S3 incl. run-scoped `.audit/geo-signals-<RUNID>.md` + the same dispatcher
ERROR contract. No cross-domain file share:
seo vs geo bodies genuinely diverge (different checks, scoring blocks,
envelopes) — that divergence, not LRN-125, is the reason.
S5 handover-doc-writer → handover-synthesizer (opus; STEP 9 memory-registry
load + STEP 10 phase clustering + STEP 12 6-chapter synthesis — STEP 9
allocated here, it feeds the synthesis; correctness MINOR) +
handover-renderer (sonnet; STEP 13-16 annex render, precheck apply,
deterministic gates, HTML/PDF). client-handover-writer dispatches
synthesizer then renderer; PACKAGE contract split per LRN-126
(re-enumerate DEPLOY_HINTS/--skip-seo class fields — the EXACT prior
failure). W4 MUST same-commit relock model-routing.test.sh:52-55 (the
handover-doc-writer name + dispatch-string locks break on the rename;
"make test green per wave" D4 invariant — correctness MINOR).
Tier-downs (no split): validator-analyzer opus→sonnet (deterministic
validators+tables). onboarder stays sonnet wave 1 (haiku candidate wave 2).
release-executor stays sonnet (NEED-DECISION valve).
Verifier: STAYS sonnet (approved — oracle-anchored gate).
## D2 — SKILL MAP (main loop = session model; every dispatch tier explicit)
Gated reflection skills (model-gate kept, 15):
- ship-feature / init-project: per the two example maps in the analysis file
(amendment section) + S1 gate hoist at STEP 0 + doc-commit conversions.
- feat: scope/plan/contract/loop = main; challenge 3× plan-challenger opus;
feater sonnet; verifier+security sonnet; doc-commit → doc-auditor opus +
doc-syncer sonnet dispatch; commit via /commit-change (propose opus / apply
sonnet).
- bugfix: investigation/diagnosis/contract = main (reflection); challenge
opus (3b); bugfixer sonnet; verifier+security sonnet; doc-commit as feat.
- hotfix: LOCATE + guard = main (logic); challenge opus when guard fires;
hotfixer sonnet; security gate sonnet (revert-not-loop conserved);
doc-commit as feat.
- analyze: analyzer INLINE = main loop (it IS the reflection) — unchanged.
- code-clean: PHASE 1 audit inline = main (audit judgment feeding a human
gate); code-cleaner sonnet PHASE 2 (hosts refactorer inline at SAME tier —
LRN-125 OK); re-audit sonnet inside executor.
- seo / geo: skill = orchestration + GATED arbitrage (main); pipeline
collector sonnet → judge opus → templater sonnet (L1 serial); appliers
hotfixer/feater sonnet at L1; build-verify inline.
- web-validate: validator-analyzer sonnet; hotfixer applier sonnet; loop main.
- harden: audit dispatch follows S3 narrow-scope path (seo-judge opus on
harden axes w/ collector reuse); direct-Edit apply stays inline (tiny
scope, BDR-061 carve-out conserved).
- audit-delta: axis audits dispatched opus (delta judgment); security-auditor
sonnet; fix gate + markers = main.
- tour: orchestration main; security-auditor sonnet; cleanup audit = analyzer
opus (or general-purpose model="opus"); fixes via sonnet appliers; doc axis
→ S2 pipeline; reconcile axis = deterministic bash (main).
- onboard: onboarder DISPATCHED sonnet (was inline); plugin S1 pipeline;
analyzer opus; general-purpose audits model="opus" (kept); seo/geo → S3/S4
pipelines; security-auditor + doc pipeline as above; synthesis
general-purpose model="opus"; backlog arbitration = main.
- client-handover: writer INLINE (orchestrator, main); its 8 skill-runner
children model="fable"; handover S5 split (synth opus → render sonnet);
gates all main.
Excluded-from-gate skills (5, stay ungated): commit-change (propose opus /
apply sonnet via model=; approval gates main); doc (S2: auditor opus →
gate main → patcher sonnet); status (haiku); release-candidate (executor
sonnet; version/when/push decisions main); refactor (refactorer sonnet).
Memory/util skills (capitalize, close, prune-memory, reconcile, learn,
profile, skills-perso, gitflow, deploy, plugin-check(S1), status): main
loop by nature (conversation context, human gates, deterministic bash) —
no dispatch changes except plugin-check S1.
External/gstack skills (graphify, design-*, review, qa, ship, investigate…):
NOT edited (external ownership, BDR-015 class) — covered by doctrine line;
local wrapper skills only if locally owned. Verify ownership per file
before touching (symlink → skip).
## D3 — WAVES (each = gitflow feature branch, tests green, census extended)
W0 SEQUENCING: merge `feature/opus-pin-audit-agents` → develop (baseline,
human gate). `bugfix/seo-geo-integrity` is ALREADY MERGED (correctness
MAJOR — the TODO.md "UNMERGED" note was stale; verified `92301fe` is an
ancestor of develop AND this branch): no arbitrage, no W5 wait — one-line
ancestry re-check in W0 + fix the stale TODO.md entry (reconcile-class
correction). Absorb the analysis+plan files into the new feature branch.
W1 NO-INHERIT ENFORCEMENT (small, high-value): code-review model= opus
(ship-feature 6, init-project 10); client-handover-writer 8× model="fable";
doctrine line in model-gate.md + census locks (`model="fable"`,
`model=` presence per site); SDD sonnet prose → census lock. Prose sweep
of stale BDR-066/076 claims touched by W1.
W2 INLINE→DISPATCH CONVERSIONS: scaffolder (init 5 — liveness pings move to
orchestrator; scaffolder loses PHASE 6 inline-load → init 5b owns README
via S2), onboarder (onboard), doc-commit steps ×5 flows → S2 pipeline
(gate hoist FIRST: /doc + flows own the validation gate; doc-syncer body
loses its inline gate → census re-lock), S1 plugin split + gate hoist ×4
consumers. Data-flow pass per LRN-126 on each (fields enumerated in the
wave's contract file before edits).
W3 TIER MOVES: validator-analyzer → sonnet (pin + prose + census flip);
commit-changer per-mode model= (2 sites + prose + census).
W4 S5 handover split (synth opus / render sonnet) + PACKAGE re-enumeration.
W5 S3/S4 seo/geo pipelines: worker(2-mode)/judge ×2, /seo /geo /harden
/onboard rerouted, seo-data.test.sh moved locks, run-scoped signals
handoff (RUNID + completeness sentinel + fail-closed judge),
envelope/sentinel/score grammars verbatim, COVERAGE lines preserved.
W6 DOCTRINE + CLOSE-OUT: model-gate.md rewrite (protects main loop; tier
table; fable-dispatch doctrine), challenge-plan.md + plan-challenger
ORCHESTRATOR PROTOCOL text (keep BDR-066+BDR-076 tokens per census, add
BDR-077), census consolidation (model-routing new sections; every new
agent: YAML G3, pin lock, dispatch-string lock, AskUserQuestion/Agent
bans), LRN-113 whole-surface prose sweep (~30 refs list in analysis),
BDR-077 + LRN entries + journal, EVAL-023-style post-merge ronde.
Per-split planted-input smokes run IN their own waves (W2/W4/W5, merge
gates) — W6 is the consolidated ronde only, never the first empirical
proof of a split.
## D4 — ZERO-REGRESSION PROTOCOL (every wave)
- Before edits: wave contract file (.claude/tasks/contracts/) with FILE SCOPE
+ acceptance criteria; challenge-plan on THIS plan (done once, below);
verify-secure-loop on each wave's diff (verifier sonnet + security sonnet).
- Grammar diff-guard: `grep -F` each verbatim marker (analysis §CONSTRAINTS
list) pre/post per wave — zero drift.
- Census: flip-test every NEW lock (plant violation → RED) before trusting.
- `make test` green per wave; no wave merges without human signal (gitflow).
- Rollback story (robustness MINOR — waves are textually interdependent, an
early wave is NOT independently revertible after later merges): revert in
REVERSE merge order, or revert the whole stack; never a mid-stack single
revert. Pre-merge, the rollback unit is the wave branch.
## CHALLENGE LOG (2026-07-19 — 3 blind lenses on plan v1)
- correctness: CONCERNS(2) — seo-geo-integrity phantom sequencing (fixed W0);
commit-changer precedence ambiguity (fixed D1 + D0.2 citation + W3 smoke);
3 MINORs (S5 STEP 9 + W4 relock; plugin checkpoint seam; templater label)
— all fixed in place.
- robustness: FATAL(4) — BLOCKER fable-dispatch unverified → CLOSED by spike
(D0.2: resolves to claude-fable-5, enum-validated, loud failure); doc-commit
in-thread contract (fixed S2: CHANGE SUMMARY crosses the report grammar);
plugin-probe haiku contradiction (fixed: sonnet wave 1); signals handoff
freshness (fixed S3: RUNID + sentinel + fail-closed); rollback claim
(fixed D4).
- simplicity: CONCERNS(1) — 3-way seo/geo YAGNI → 2-way worker/judge (fixed
S3/S4); twin-templater share (dissolved by 2-way; divergence stated);
plugin gate ×4 copies → lib/plugin-gate.md include (fixed S1).
- Confirmation pass (fresh robustness challenger on v2): CONCERNS(2) — v1
fixes HOLD (doc-commit CHANGE SUMMARY, plugin-probe sonnet, rollback order,
fable spike, RUNID); 2 new MAJORs + 1 MINOR opened by the revisions, all
fixed in v3: (a) per-split planted-input smokes moved IN-WAVE as merge
gates (W6 = ronde only); (b) dispatcher ERROR contract on judge failure
(STOP, no templating/apply, retry once, escalate — pipeline never fails
open); (c) transient signals files relocated to gitignored `.audit/`
(LRN-124). Protocol cap reached (1 re-challenge) → to the human gate.