chore(memory): BDR-081 + LRN-139 + journal + CHANGELOG + plan (opus5 tuning)
Capitalizes the Claude-5-family config recalibration: decision record, trait-inversion learning (LRN-030 superseded premise, #80988 injections, no-effort-hold trap), journal line, CHANGELOG Unreleased entries, and the challenged plan (3 blind Opus 5 lenses, synthesis in §5bis).
This commit is contained in:
Bastien Chanot
@@ -1,5 +1,30 @@
|
||||
# TODO
|
||||
|
||||
## 2026-07-30 — adapt config for Claude 5 family / Opus 5 (feature/opus5-config-tuning)
|
||||
User: Opus 5 "needs more freedom" → research (official migration guide +
|
||||
web + registres) confirms: over-delegates (inverts LRN-030 Opus 4.8 trait),
|
||||
over-verifies if told to verify, literal instruction following, scope
|
||||
expansion named regression, harness already injects anti-delegation on
|
||||
Opus 5 (#80988). Plan: .claude/tasks/plans/2026-07-30-opus5-config-tuning-1238.md
|
||||
— to be challenged by 3 blind plan-challengers (opus pins → Opus 5), then
|
||||
executed on feature branch. NO merge (human gate).
|
||||
Challenged 2026-07-30: correctness CONCERNS(4) · robustness FATAL(5, 1
|
||||
BLOCKER: symlink-live deployment) · simplicity CONCERNS(4) — all fixes
|
||||
adopted as prescribed (plan §5bis, v2 items below).
|
||||
- [x] W0 branch first (eab2a10 parent); hook regex validated on scratch copy
|
||||
(bash -n + shellcheck + 5 replays, HOME sandboxed) before live write
|
||||
- [x] W1 delegation block v2 (when-guidance + gates carve-out + scoped don't-redo) — 0f7b565
|
||||
- [x] W2 "staff engineer" bar line deleted — 0f7b565
|
||||
- [x] W3 finish-whole-task folded into Deviations (+ gone-WRONG→STOP) — 0f7b565
|
||||
- [x] W4 deliverable-length rule — 0f7b565
|
||||
- [x] W5 line budget: 308/320
|
||||
- [x] W6 hook \bux\b dropped, \bui\b kept + F10 must-fire lock, D11 quiet row
|
||||
flip-tested (fire before/quiet after) — eab2a10, suite 22/0
|
||||
- [x] W7 plan-challenger :82-83 reworded → [MINOR] routing, census row — c3d3f4d, 44/0
|
||||
- [x] W8 BDR-081 + LRN-139 + journal + CHANGELOG
|
||||
- [ ] W9 final gate: make test full suite
|
||||
- [ ] W10 no gitflow finish (human gate) — merge only on explicit user signal
|
||||
|
||||
## 2026-07-22 — auto-purge transient superpowers artifacts at finish (feature/gitflow-auto-purge-transient)
|
||||
User: transient planning artifacts (`docs/superpowers/{specs,plans}`) leak into
|
||||
develop; BDR-065 "post-merge cleanup" is DOCTRINE ONLY (no code) — manual chore,
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
# PLAN — Adapt claude-config for the Claude 5 family (Opus 5 focus)
|
||||
|
||||
Date: 2026-07-30 · Branch (planned): feature/opus5-config-tuning (off develop)
|
||||
KIND: build-plan · Author: main-loop session (Fable 5)
|
||||
|
||||
## 1. Context & evidence
|
||||
|
||||
Opus 5 (`claude-opus-5`, released 2026-07-24) now backs every `model: opus`
|
||||
agent pin in this repo (analyzer, plan-challenger, seo/geo-analyzer,
|
||||
plugin-advisor — BDR-076/077) and any session the user switches to via
|
||||
`/model opus`. Its documented behavioral profile differs from Opus 4.8 in
|
||||
ways that make parts of this config counterproductive:
|
||||
|
||||
- E1 **Over-delegation**: Opus 5 "delegates to subagents more readily than
|
||||
prior models" (official prompting guide). Opus 4.8 had the OPPOSITE trait
|
||||
(LRN-030), and `CLAUDE.global.md:43-47` was written to counter it
|
||||
("Counters model tendency to under-delegate"). The premise is inverted.
|
||||
- E2 **Anti-delegation already injected by the harness**: Claude Code
|
||||
v2.1.219 server-gates an Opus-5-only prompt section (`heron_brook` +
|
||||
`subagent_steer_delegation`, GitHub issue #80988) that says "Do not call
|
||||
the AgentTool unless the user requested it" and "Subagents multiply cost
|
||||
and time…". Stacking our own hard cap on top would triple-constrain;
|
||||
keeping a pro-delegation nudge would fight the injection. Model-neutral
|
||||
when-guidance is the stable middle.
|
||||
- E3 **Over-verification**: official guidance — "If your prompt contains
|
||||
explicit verification instructions … remove them: instructions like these
|
||||
cause over-verification on Claude Opus 5, and removing them reduces wasted
|
||||
tokens with no loss in quality." Also true of per-prompt "double-check"
|
||||
phrasing. Targets PROSE told to the model, not harness-level gates.
|
||||
- E4 **Scope expansion**: named Opus 5 regression ("can expand the scope of
|
||||
a task, adding steps that weren't requested"). Anthropic ships a literal
|
||||
counter-block; tested to reduce scope changes "to nearly zero".
|
||||
- E5 **Literal instruction following** (since 4.7, stronger now): aggressive
|
||||
MUST/CRITICAL language over-triggers; conservative-reporting instructions
|
||||
("only report high-severity") measurably depress recall in review/challenge
|
||||
harnesses.
|
||||
- E6 **Longer written deliverables**: files written to disk run ~30-40%
|
||||
longer; `effort` does NOT control visible/deliverable length — only prose
|
||||
instructions do.
|
||||
- E7 **Overconstraint costs reasoning**: Anthropic removed >80% of Claude
|
||||
Code's system prompt for Claude-5-generation models "with no measurable
|
||||
loss"; named mechanism = tokens burned resolving conflicting rules.
|
||||
- E8 **Hook false positive (today)**: `\bux\b` in
|
||||
`hooks/design-toolchain-reminder.sh:47` fired on French prose ("changement
|
||||
ux vu" — matches after apostrophe/slash/space); 2nd `ux` FP in the log,
|
||||
both French. Continues the LRN-1005/1007 false-positive series. No test
|
||||
row covers `\bui\b`/`\bux\b`.
|
||||
- E9 **Effort carry-over trap**: Opus 5 has no model-default effort hold in
|
||||
Claude Code — a persisted `xhigh` (our `settings.json:333`) silently
|
||||
carries onto Opus 5 sessions, against Anthropic's "start at high, sweep
|
||||
low/medium" guidance for that model.
|
||||
|
||||
## 2. Design decisions
|
||||
|
||||
- D1 The global instruction layer must be MODEL-NEUTRAL across the Claude 5
|
||||
family (sessions run Fable 5 by default; dispatched judgment agents run
|
||||
Opus 5; executors Sonnet). Fixes therefore express WHEN-guidance and
|
||||
outcome bars, not directional compensation for one model's trait.
|
||||
- D2 Harness-level quality gates (fresh blind verifier + security-auditor,
|
||||
BDR-049/050; plan-challenge, BDR-075) are architecture, not model
|
||||
self-check prompting. They stay. E3 applies only to prose that tells the
|
||||
MODEL to verify its own work.
|
||||
- D3 Per BDR-021, the Security and Architecture-decisions sections of
|
||||
CLAUDE.global.md stay verbatim (deliberate policy). No softening there.
|
||||
- D4 Registries are append-only: LRN-030 is not edited; a new LRN records
|
||||
the trait inversion and points back to it.
|
||||
- D5 Deterministic backstops (gitflow pre-commit, Gitea protection,
|
||||
permissions.deny, rtk pinning) are explicitly out of "more freedom" scope
|
||||
— community reports show Opus 5 working AROUND soft controls, which argues
|
||||
for keeping hard ones.
|
||||
|
||||
## 3. Work items
|
||||
|
||||
### W1 — CLAUDE.global.md: rewrite the delegation block (:43-47)
|
||||
Replace the 5-line block (incl. "Default to delegation for multi-file
|
||||
exploration. Counters model tendency to under-delegate.") with model-neutral
|
||||
when-guidance, same footprint (≤5 lines):
|
||||
|
||||
```
|
||||
- Sub-agents: one task per sub-agent, main context stays clean.
|
||||
Delegate genuinely independent, sizeable tracks (wide multi-file
|
||||
exploration, parallel audits) — not work doable in a few tool
|
||||
calls, and not self-verification (harness gates own that). Brief
|
||||
precisely, then commit to the delegation — don't redo its work.
|
||||
```
|
||||
Rationale: E1+E2. No hard spawn cap in prose (harness already injects one on
|
||||
Opus 5; Fable benefits from delegation).
|
||||
|
||||
### W2 — CLAUDE.global.md: reframe "After code changes" (:75-83)
|
||||
Keep the concrete quality bar; drop the proof-mandate/self-check phrasing
|
||||
(E3). Replace steps 2-4 with faithful-outcome reporting:
|
||||
|
||||
```
|
||||
## After code changes
|
||||
1. Run tests, lint, build, type-check if available.
|
||||
2. Report outcomes faithfully: what passed, what wasn't run,
|
||||
remaining risks, surviving deviations. Completion claims only
|
||||
for verified work.
|
||||
3. Correction or notable event → capitalize to right registry.
|
||||
```
|
||||
Net: -2 lines. "Would staff engineer approve?" bar and "Don't mark complete
|
||||
without proof" are removed as self-check choreography; honest-reporting
|
||||
line preserves the intent (grounded completion claims) without mandating an
|
||||
extra verification pass.
|
||||
|
||||
### W3 — CLAUDE.global.md: add scope fence (Workflow section)
|
||||
Append (adapted from Anthropic's tested block, caveman-compressed, ~5 lines):
|
||||
|
||||
```
|
||||
- Scope: deliver what was asked, at the scope intended. Routine
|
||||
judgment calls → decide alone; materially different readings →
|
||||
ask. Better approach spotted → say so in one line, still do the
|
||||
task as asked. Finish the whole task; genuinely blocked → do the
|
||||
rest, state plainly what's missing.
|
||||
```
|
||||
Rationale: E4. Complements existing "Scope changes to task — no unrelated
|
||||
edits" (line ~15) without contradicting it.
|
||||
|
||||
### W4 — CLAUDE.global.md: add deliverable-length rule (Code style / Comments area)
|
||||
~2 lines:
|
||||
|
||||
```
|
||||
- Written deliverables (docs, reports, .md): length matched to what
|
||||
the task needs — no filler sections, no boilerplate summaries.
|
||||
```
|
||||
Rationale: E6. Registries already covered by caveman rule.
|
||||
|
||||
### W5 — Line budget
|
||||
After W1-W4: expected ~309 lines. Hard check: `wc -l CLAUDE.global.md` ≤ 320
|
||||
(session-start.sh warning threshold at :202-213).
|
||||
|
||||
### W6 — hooks/design-toolchain-reminder.sh: drop `\bui\b` and `\bux\b`
|
||||
- Remove the two 2-char alternatives from the pattern at :47. Keep
|
||||
`ui/ux|ux/ui|ui kit` and all other tokens.
|
||||
- Add a dated header comment (3rd tightening pass, 2026-07-30, cites the
|
||||
two French-prose `ux` FPs; series LRN-1005/1007).
|
||||
- Trade-off accepted: a bare "améliore l'ux" prompt with no other design
|
||||
token goes quiet — the CLAUDE.global.md "Design work" section still
|
||||
routes it (the hook is a belt, self-described soft nudge).
|
||||
- Update `lib/tests/design-toolchain-reminder.test.sh`: add 2 quiet rows
|
||||
(the real FP prompt excerpt; a bare "l'ui" French sentence) — flip-tested
|
||||
per LRN-096. Existing 9 must-fire rows unaffected (none uses ui/ux).
|
||||
|
||||
### W7 — agents/plan-challenger.md: coverage-first reporting line
|
||||
Add one clause to the findings rules (add-only, no removal): uncertain or
|
||||
low-severity findings are REPORTED with an explicit confidence + severity
|
||||
tag rather than self-censored — severity filtering happens in the
|
||||
orchestrator's synthesis, not in the challenger. Rationale: E5 (literal
|
||||
Opus 5 + "manufactured concern is a failure" wording risks suppressing real
|
||||
low-confidence findings). Must not touch: verdict grammar, MANDATORY PROOF
|
||||
clause, blind-dispatch rules (test-locked in plan-challenger.test.sh).
|
||||
|
||||
### W8 — Memory + docs capitalization (same branch, follows the work)
|
||||
- decisions.md: new BDR (config adapted for Claude 5 family — scope,
|
||||
rationale, alternatives incl. "leave config as-is" and "hard spawn caps"
|
||||
rejected).
|
||||
- learnings.md: new LRN — Opus 5 behavioral profile (over-delegation
|
||||
inverts LRN-030's Opus 4.8 trait; over-verification; literal following;
|
||||
no effort hold on Opus 5 in Claude Code; heron_brook/#80988 injection).
|
||||
- journal.md: one line.
|
||||
- CHANGELOG.md: entry under Unreleased.
|
||||
|
||||
### W9 — Gates (before commit)
|
||||
- `shellcheck hooks/design-toolchain-reminder.sh` clean.
|
||||
- Manual flip-test of the hook: FP prompt → quiet; "redesign the navbar" →
|
||||
fires.
|
||||
- `make test` full suite green (design-toolchain-reminder.test.sh,
|
||||
plan-challenger.test.sh, model-routing.test.sh untouched-but-must-pass,
|
||||
curated-config-guard, loops-light…).
|
||||
- `wc -l CLAUDE.global.md` ≤ 320.
|
||||
|
||||
### W10 — Gitflow
|
||||
`bash ~/.claude/lib/gitflow.sh start feature opus5-config-tuning` off
|
||||
develop; atomic commits (hook+test / CLAUDE.global.md / agent / memory+docs);
|
||||
NO `gitflow finish` — merge only on explicit human signal.
|
||||
|
||||
## 4. Explicitly NOT doing (considered, rejected)
|
||||
|
||||
- N1 Touching lib/verify-secure-loop.md or the fresh-verifier/security
|
||||
gates: harness architecture (BDR-049/050, D2), verifies SONNET executor
|
||||
output — not Opus 5 self-check prose.
|
||||
- N2 Softening the Security / Architecture sections (BDR-021, D3).
|
||||
- N3 Editing the superpowers plugin's "1% chance → MUST invoke" language:
|
||||
external upstream code; flagged as residual over-triggering risk in the
|
||||
new LRN, revisit as its own decision if observed.
|
||||
- N4 Changing `settings.json` `effortLevel: "xhigh"`: user preference,
|
||||
optimal for the Fable 5 session default; the Opus 5 carry-over trap (E9)
|
||||
is documented in the LRN + surfaced to the user for a manual decision.
|
||||
- N5 De-prescribing seo-analyzer.md / geo-analyzer.md (1528/1106 lines,
|
||||
heavy MUST density): separate project, backlog note in TODO.md.
|
||||
- N6 Removing or session-gating the design/ctx7 reminder hooks: soft
|
||||
nudges, cheap, deliberately built; tightened only (W6).
|
||||
- N7 Any model pin change: `model: opus` pins now resolve to Opus 5 —
|
||||
desired outcome, census (model-routing.test.sh) untouched.
|
||||
- N8 Committing settings.json for any reason (LRN-098/1049 /model-churn
|
||||
trap): file is currently clean; keep it out of every commit.
|
||||
|
||||
## 5bis. CHALLENGE SYNTHESIS (2026-07-30) — FINAL amendments (v2)
|
||||
|
||||
Verdicts: correctness CONCERNS(4) · robustness FATAL(5, 1 BLOCKER) ·
|
||||
simplicity CONCERNS(4). Every fix below is the challenger's own named FIX,
|
||||
adopted as written. No re-challenge pass: scope narrowed, no new dependency;
|
||||
W0 is an execution-time safety procedure, not a new config mechanism.
|
||||
|
||||
- **W0 (NEW — robustness BLOCKER)**: all edited surfaces are symlink-deployed
|
||||
LIVE (~/.claude/CLAUDE.md, hooks/, agents/ → this repo); edits take effect
|
||||
machine-wide at save time, before any W9 gate. Mitigations:
|
||||
(a) `gitflow start` BEFORE any live-file edit; never checkout develop
|
||||
mid-work; (b) hook regex change validated on a SCRATCH copy first
|
||||
(bash -n + shellcheck + pattern replay), then written to the live file in
|
||||
ONE atomic Edit; (c) named reverts: `git show develop:<file> > <file>`;
|
||||
escape hatch = remove the hook registration block from settings.json.
|
||||
- **W1 v2** (robustness#3, correctness#2): replacement text carves out the
|
||||
mandated gates explicitly and scopes "don't redo":
|
||||
"Skill-mandated gates (fresh verifier/security/challenge) always dispatch
|
||||
as written. Don't redo delegated work by hand — failed gates re-dispatch
|
||||
fresh executors instead."
|
||||
- **W2 v2** (simplicity#2): minimal diff — delete ONLY the line
|
||||
`Bar: "would staff engineer approve?"`. Steps 1-4 + capitalize step stay.
|
||||
- **W3 v2** (simplicity#1, robustness#4): no new bullet. Fold the only new
|
||||
clause into the existing Deviations bullet: "Finish the whole task:
|
||||
blocked on an independent sub-part → do the rest, state what's missing.
|
||||
Gone WRONG → still STOP, re-plan." (net +2 lines, no conflict with :53).
|
||||
- **W4**: unchanged (+2 lines). Budget v2: 304 +1 −1 +2 +2 = 308 ≤ 320.
|
||||
- **W6 v2** (all lenses): drop `\bux\b` ONLY — keep `\bui\b` (zero evidenced
|
||||
FP; one logged true positive). Accepted trade-off: the 2026-07-21 "ameliore
|
||||
le tutoriel…gamifier" ux row (plausible TP) goes quiet; CLAUDE.global.md
|
||||
design-routing section remains the router. Header comment notes the log
|
||||
records `head -1` only → per-token FP rate not fully derivable. Tests:
|
||||
quiet row = synthetic "changement ux vu…" (verified matches pre-change →
|
||||
flips); must-fire row = "revois l'ui du panneau admin" (locks `\bui\b`;
|
||||
apostrophe escaped correctly, doubles as JSON-path control per
|
||||
robustness#7). No log-excerpt rows (vacuous — 100-char truncation).
|
||||
- **W7 v2** (all lenses): in-place reword of the `:82-83` sentence (NOT
|
||||
test-locked; plan v1 misstated that) instead of an add-only clause:
|
||||
"No invention — ungrounded is noise. Silently dropping a grounded doubt is
|
||||
equally a failure: file it as `[MINOR]` with the uncertainty stated in
|
||||
`WHY:`. Nothing real at all → `SOLID` with `FINDINGS: none`."
|
||||
OUTPUT grammar byte-identical; no confidence axis; no consumer change.
|
||||
Census: add `has "$A" "grounded doubt"` row to plan-challenger.test.sh in
|
||||
the same commit.
|
||||
- **W9 v2**: adds the W0 scratch-validation step; rest unchanged.
|
||||
- **W10 v2**: branch creation moves FIRST in execution order.
|
||||
|
||||
## 5. Constraints for challengers
|
||||
|
||||
- Registries append-only; curation only via /prune-memory.
|
||||
- Census tests lock behavior: any hook/agent edit must land with its test
|
||||
update in the same commit; `make test` must stay green.
|
||||
- CLAUDE.global.md ≤ 320 lines (runtime warning threshold).
|
||||
- BDR-021: Security + Architecture sections verbatim.
|
||||
- Gitflow: feature branch off develop, no merge without human signal.
|
||||
- The global file serves ALL models (Fable sessions, Opus 5 dispatches,
|
||||
Sonnet executors read skill/agent prompts instead) — no Opus-5-only
|
||||
wording in CLAUDE.global.md.
|
||||
Reference in New Issue
Block a user