chore(memory): LRN-296 simulate checks, BDR-150 amended, run B1 journal

This commit is contained in:
bchanot committed 2026-11-17 14:07:02 +01:00
1 parent 9033b4a944
commit d55f4911ce
6 files changed
+239 -1

No files matched your search

+1
View File
@@ -1610,3 +1610,4 @@ Supersedes [[BDR-030]].
- **Refs**: amends [[BDR-123]], [[BDR-135]]; supersedes [[LRN-224]]; [[BLK-030]], [[BDR-077]] (Fable dispatch pins: separate run).
- **Status (2026-11-17)**: accepted, completed: pins opus beebfe2 (gate row 'fixed at spawn' passes a pinned sub-agent), doctrine c3507ea (main loop calls advisor, sub-agents do not, absent advisor -> proceed, advice waives no gate)
- **Status (2026-11-17)**: accepted, amended 6ef3069: main loop named call points (before a skill's plan, once per verify-secure run at first gate loop failing twice, before merge); plan-challenger/verifier/security-auditor at most one call before verdict, advice add-only (security: STEP 3 class severity, no class -> REPORTED LOW); other sub-agents do not
- **Status (2026-11-17)**: accepted, amended 58a5273: plan point in lib/challenge-plan.md (on written plan before challengers, once per skill run, Advisor: row in CHALLENGE SUMMARY, no attribution in plan); stuck lines inline in verify-secure-loop (main-loop calls only since GATE 0); dispatched analyzer joins; trigger before recommending doctrine/architecture/public-interface change
+1
View File
@@ -775,3 +775,4 @@ rules:
## 2026-11-17
- Advisor doctrine (c3507ea, docs): CLAUDE.global § Workflow + effort-shift § Advisor: main loop calls Fable advisor (approach, stuck, before done), sub-agents do not (they have tool listed: prose-only, residual), absent → proceed, advice waives no gate/cap/guard. Stale Fable prose → opus (effort-shift, challenge-plan, plan-challenger, tour). BDR-150 change complete: router 24b0f35, pins beebfe2, doctrine c3507ea.
- Advisor call points run A (6ef3069): transcript count showed main loop 2 calls on Opus (409k/680k uncached) vs 18/20 opus sub-agents on Fable (~70k); doctrine now names main-loop points (plan, first gate loop failing twice, merge) + judgment agents once, advice add-only; 2 challenge rounds, gates found block E double count, fixed r5. Runs B, C next.
- Advisor run B1 (58a5273): plan point moved into lib/challenge-plan.md (once per skill run, Advisor: row), stuck lines inline in verify-secure-loop (main-loop calls only), analyzer joins, trigger for doctrine/architecture/public-interface recommendations. 2 challenge rounds (2 BLOCKERs were my own oracle markers), scratch simulation then MET first run; gates sequential, CONFORME + PASS.
+6
View File
@@ -2134,3 +2134,9 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
- **Context**: 2026-11-17, user asked if advisor called when needed. Since /advisor (2026-11-16 16:19Z): main loop 2 calls, served by Opus (conversation kept old advisor), 409k + 680k input tokens, cache_read 0; 18/20 opus sub-agents 19 calls on Fable, median 70k (43k-119k), total 1.46M; 8 sonnet sub-agents 0. Data reversed "main loop only" (c3507ea) → 6ef3069 (judgment agents once, main loop named points).
- **Apply**: before scoping who calls advisor, or judging cost: count per caller + model + uncached input; main-loop call in long session costs ~6-10 sub-agent calls. Run C (/memory-feedback advisor section) automates it.
- **Reference**: lib/effort-shift.md § Advisor, BDR-150, LRN-294
## LRN-296 — Simulate contract CHECKs on a scratch copy of the planned edit before dispatch (2026-11-17)
- **Pattern**: before dispatching a prose/doctrine run, apply the planned blocks to a scratch copy (deterministic applier asserting each anchor once) and run every contract CHECK against it. Grepping markers against the block text alone misses cross-block effects.
- **Context**: 2026-11-17 advisor runs A and B1. Markers broke three times: phrase wrapped across two lines (grep per line, count 0); awk `/X/{a=NR}` kept LAST match after a later block mentioned "X"; census extractor read word after `§ Label` as part of heading ("§ Workflow names"). Each would have cost a floor loop. Also `Nb.` criterion label absorbed by gates.sh (LRN-284). Simulation of B1 (apply.py on mktemp copy) passed 5/5 CHECKs; executor then got MET first run.
- **Apply**: any /feat or /bugfix whose oracles are grep/awk markers on prose: write blocks to files, applier script with exact-once asserts, run CHECKs with `cd` swapped to the scratch copy; anchor awk on line start (`/^\*\*Label/`); keep each marker and each `§ Label` citation on one line followed by `)`/`;`. Give the executor the applier.
- **Reference**: lib/gates.sh, lib/tests/doctrine-citers.test.sh, .claude/tasks/plans/2026-11-17-advisor-points-b1-1500.md, LRN-284
+3 -1
View File
@@ -5,7 +5,9 @@ Done history before 2026-10-08 removed 2026-10-23 (git keeps it: `git log -p --
### Advisor call points (user 2026-11-17: "oui 1 et 2, et le 3 aussi"; amends BDR-150)
- [x] Run A DONE 6ef3069, merged fc2d84d (docs 09277f5, audits c1a0c32, memory 78a6942) /feat feature/advisor-call-points: CLAUDE.global § Workflow + lib/effort-shift.md § Advisor + plan-challenger/verifier/security-auditor one-call rule (contract .claude/tasks/contracts/2026-11-17-advisor-call-points-1130.md)
- [ ] Run B /feat: inline advisor call lines at the step sites (plan step of skills/feat, bugfix, ship-feature; second-failure + cap sites in lib/verify-secure-loop.md; merge proposal), citing `lib/effort-shift.md § Advisor`; + agents/security-auditor.md advisor bullet: severity applies only when higher, closing sentence "Advice never removes…" (BUGS-FOUND LOW, c1a0c32) = 5 files. Plan notes: test each citation line against the census regex (per line; word after `§ Label` joins the heading); fast oracle `bash lib/tests/doctrine-citers.test.sh` before full make test; settle the merge-proposal site first (/feat and /bugfix never finish); read run B challengers' notes for one late advisor call (first live test of block C)
- [x] Run B1 DONE 58a5273 (docs, memory below) /feat feature/advisor-points-b1 (user 2026-11-17 "oui a, b et c"): (a) main-loop plan point in lib/challenge-plan.md (before first dispatch), (b) analyzer joins judgment agents (CLAUDE.global, effort-shift, agents/analyzer.md), (c) trigger "before recommending a change to the doctrine, the architecture or a public interface", stuck call lines in lib/verify-secure-loop.md (contract .claude/tasks/contracts/2026-11-17-advisor-points-b1-1500.md)
- [ ] Run B2 /feat: stuck trigger line in the gate bullets of skills/feat, skills/bugfix, skills/ship-feature (they restate the loop inline); merge-proposal lines in skills/ship-feature STEP 9 + skills/release-candidate STEP 4; agents/security-auditor.md severity applies only when higher + "Advice never removes…" (BUGS-FOUND LOW c1a0c32). Plan notes: test citation lines against the census regex; fast oracle doctrine-citers; no `Nb.` criterion labels (LRN-284)
- [x] After B1 DONE (doc commit + BDR-150 status + LRN-296): README/USAGE/CHANGELOG say "before writing a skill's plan (/feat, /bugfix, /ship-feature)" and list 3 agents → doc-sync; BDR-150 status line
- [ ] Run C /feat: advisor usage section in /memory-feedback (main loop vs sub-agents, model served, input tokens; dedupe usage rows by message.id, model from the `advisor_message` iteration)
- [x] Stale citers after A: README/USAGE/CHANGELOG patched, BUGS-FOUND residual superseded + LOW logged, BDR-150 status, LRN-295 (LRN-294 kept: its lesson holds)
@@ -0,0 +1,52 @@
# CONTRACT — advisor-points-b1 (BDR-150 amendment, run B1)
- date: 2026-11-17 | flow: feat | branch: feature/advisor-points-b1 (off develop)
- status: active
- plan: .claude/tasks/plans/2026-11-17-advisor-points-b1-1500.md (r4)
## REQUEST (verbatim — IMMUTABLE)
User 2026-11-17: "oui a, b et c dans la run B", answering: (a) move the main loop's "before the plan" point into lib/challenge-plan.md, shared by 9 skills: the main loop consults the advisor on the written plan before dispatching the challengers; (b) add the analyzer to the agents allowed one call; (c) add a concrete trigger to the global line: "before recommending a change to the doctrine, the architecture or a public interface". Run B (from run A) also places inline call lines in lib/verify-secure-loop.md (second failure, caps).
## CLARIFICATIONS
- [internal 2026-11-17] run B split (/feat max 5 files): B1 = CLAUDE.global.md, lib/effort-shift.md, lib/challenge-plan.md, lib/verify-secure-loop.md, agents/analyzer.md (this contract); B2 = stuck trigger lines in skills/feat, skills/bugfix, skills/ship-feature, skills/init-project gate bullets, merge lines in skills/ship-feature STEP 9 and skills/release-candidate STEP 4, agents/security-auditor.md direction fix.
- [internal 2026-11-17] plan r2: stuck unit "since GATE 0 first ran", block C once, user wording "before recommending", analyzer moved in.
- [internal 2026-11-17] re-challenge (correctness FATAL(2), robustness CONCERNS(1), simplicity CONCERNS(1)) → plan r4, no third round: criterion 3 awk anchored on `^**Advisor first`; block F `Advisor:` row added to whatever summary template the caller prints, states `called, no change` / `not called (…)`; block C dedup "no main-loop advisor call since the skill run started", re-persist PLAN on disk, route clause dropped; block H (nominal-path cost line); block I (named points cover the § Workflow triggers inside a skill run); B2 inline sentence narrowed to the analyzer; E parenthetical dropped. Residuals → BUGS-FOUND at capitalize: verifier/security-auditor advisor rules not dispatch-scoped (inline load path unused today), agents/plan-challenger.md ORCHESTRATOR PROTOCOL mirror lacks the advisor step and row.
- [internal 2026-11-17] challenge round 1 (correctness FATAL(5), robustness CONCERNS(4), simplicity SOLID) → plan r3: plan call once per skill run, reads PLAN first, route sent with it, no attribution in PLAN, CHALLENGE SUMMARY lists advisor-driven changes (block F); stuck unit counts main-loop calls only; STOP clause moved before "## Order invariant" (block G); effort-shift item 2 "says whether it predates"; analyzer rule scoped to dispatch, anchored after TASKS/PROCESS, fix advice allowed where the brief asks; `analyzer` backticked; B2 gains skills/init-project. Rejected: one-line D triggers (S1, the brief ban stays at the site), on-disk stuck marker (R10, over-built).
## RELATED MEMORY
- BDR-150 (accepted, amended 6ef3069): amended again here.
- LRN-294, LRN-295, BUGS-FOUND advisor residual: prose-only scope, cost per main-loop call.
- BDR-066 / LRN-083: decisions and advice stay in the main loop.
## ACCEPTANCE CRITERIA
1. CLAUDE.global.md carries block A (trigger c, dispatched analyzer, one-line citation).
CHECK: cd /home/bchanot/Documents/claude && echo "$({ grep -c 'before recommending a change' CLAUDE.global.md || true; })/$({ grep -c 'verifier, security-auditor and `analyzer` at most once, before their' CLAUDE.global.md || true; })/$({ grep -cF '`~/.claude/lib/effort-shift.md` § Advisor;' CLAUDE.global.md || true; })/$(wc -l < CLAUDE.global.md)"
EXPECT: 1/1/1/316
EVIDENCE: MET exit=0 marker-found :: 1/1/1/316
2. lib/effort-shift.md § Advisor: plan point on the written plan, "whether it predates", dispatched analyzer, inline loads, old wording gone.
CHECK: cd /home/bchanot/Documents/claude && sec() { sed -n '/^## Advisor/,/^## Never/p' lib/effort-shift.md; }; echo "$(sec | grep -c 'on the written plan, before the')/$(sec | grep -c 'says whether it predates')/$(sec | grep -c 'and `analyzer` call it at most once per dispatch')/$(sec | grep -c 'loaded inline by the main loop follows the')/$(sec | grep -c 'no second call for the same recommendation')/$(sec | grep -c "ahead of writing a skill's plan")/$(sec | grep -c STEP)"
EXPECT: 1/1/1/1/1/0/0
EVIDENCE: MET exit=0 marker-found :: 1/1/1/1/1/0/0
3. lib/challenge-plan.md: main-loop advisor call before the first dispatch, challengers blind, summary lists advisor-driven changes.
CHECK: cd /home/bchanot/Documents/claude && echo "$({ grep -c 'Advisor first (main loop)' lib/challenge-plan.md || true; })/$({ grep -c 'The advice never enters a challenger brief' lib/challenge-plan.md || true; })/$({ grep -c 'whatever summary template the caller prints' lib/challenge-plan.md || true; })/$({ grep -c 'revises the `PLAN` file on disk before the dispatch' lib/challenge-plan.md || true; })/$(awk '/^\*\*Advisor first/{a=NR} /^Dispatch THREE fresh/{d=NR} END{print (a>0 && a<d)}' lib/challenge-plan.md)"
EXPECT: 1/1/1/1/1
EVIDENCE: MET exit=0 marker-found :: 1/1/1/1/1
4. lib/verify-secure-loop.md: five main-loop stuck lines, the STOP clause before the order invariant.
CHECK: cd /home/bchanot/Documents/claude && echo "$({ grep -c 'with no main-loop advisor call since GATE 0' lib/verify-secure-loop.md || true; })/$({ grep -cF '`lib/effort-shift.md` § Advisor' lib/verify-secure-loop.md || true; })/$({ grep -c 'never$' lib/verify-secure-loop.md || true; })/$(awk '/the STOP carries its answer/{s=NR} /^## Order invariant/{o=NR} END{print (s>0 && s<o)}' lib/verify-secure-loop.md)/$({ grep -c 'NEVER a PASS' lib/verify-secure-loop.md || true; })"
EXPECT: 5/5/3/1/2
EVIDENCE: MET exit=0 marker-found :: 5/5/3/1/2
5. agents/analyzer.md carries its dispatch-scoped one-call rule.
CHECK: cd /home/bchanot/Documents/claude && echo "$({ grep -c 'Advisor, when dispatched as a sub-agent' agents/analyzer.md || true; })/$({ grep -c 'erroring → proceed' agents/analyzer.md || true; })/$({ grep -c 'never on the advice' agents/analyzer.md || true; })"
EXPECT: 1/1/1
EVIDENCE: MET exit=0 marker-found :: 1/1/1
6. Citation census green (fast oracle).
CHECK: cd /home/bchanot/Documents/claude && bash lib/tests/doctrine-citers.test.sh </dev/null 2>&1 | tail -1
EXPECT: FAIL=0
EVIDENCE: MET exit=0 marker-found :: PASS=23 FAIL=0
7. Full make test green (run with GATES_TIMEOUT=400).
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test </dev/null 2>&1 | tail -1
EXPECT: all suites green
EVIDENCE: MET exit=0 marker-found :: all suites green
8. Touched lines ≤ 80 columns; blocks applied verbatim from the plan; no other wording changed in the five files (manual).
## FILE SCOPE
CLAUDE.global.md (§ Workflow bullet ~:81-88), lib/effort-shift.md (§ Advisor items), lib/challenge-plan.md (§ DISPATCH head), lib/verify-secure-loop.md (GATE 0/1/2 bullets ~:38, :78, :91, :110, :115), agents/analyzer.md (§ RULES).
@@ -0,0 +1,176 @@
# PLAN — advisor call points, run B1 (r4)
Contract: .claude/tasks/contracts/2026-11-17-advisor-points-b1-1500.md
Branch: feature/advisor-points-b1 (off develop)
## Why
User 2026-11-17 "oui a, b et c dans la run B": outside the named points,
advisor use rested on judgment and under-fired. (a) the main loop's plan point
moves into lib/challenge-plan.md, wired into 12 skills (9 core + seo, geo,
web-validate packs), once per skill run; (b) the dispatched `analyzer` joins the
judgment agents; (c) trigger "before recommending a change to the doctrine, the
architecture or a public interface". Stuck point: inline at the verify-secure
loop sites, counting main-loop calls only. B2 (separate run): stuck trigger in
the gate bullets of skills/feat, bugfix, ship-feature, init-project; merge lines
(ship-feature STEP 9, release-candidate STEP 4); security-auditor severity
direction; file count settled there.
## Exact edit (deterministic)
The edits are the blocks below, applied by
/tmp/claude-1000/-home-bchanot-Documents-claude/4ba4524e-88fb-4f6c-a31d-fedd261f8925/scratchpad/b1/apply.py
(`python3 apply.py <blocks dir> <repo root>`; blocks dir = .../scratchpad/b1).
It asserts every anchor occurs exactly once and fails loudly otherwise.
Summary of what it does, by file:
1. CLAUDE.global.md § Workflow: the whole "- Stronger model needed … (BDR-150)."
bullet → block A (file 315 → 316 lines, guard 320).
2. lib/effort-shift.md § Advisor: item 1 (2 lines) → B1; the line " reuses that
advice and says it predates the last iteration. The" → B1b; after item 3
("…calling a long task\n done.") insert I; the "Sub-agents: …listed."
paragraph → B2.
3. lib/challenge-plan.md: after "— which is the point." insert H; before
"Dispatch THREE fresh" insert C; the 2-line CHALLENGE SUMMARY bullet in
§ OUTPUT → F.
4. lib/verify-secure-loop.md (inserts only): Dfloor after "then STOP + human
escalation with the rows."; Dconf after " CRITERIA table (the
contract-vs-realized diff)."; Dstruct after " A mute verifier is NEVER a
PASS."; Dsec after " `/route effort=max` (sticky, `/route clear` after).";
Dstruct after " auditor is NEVER a PASS."; G before "## Order invariant".
5. agents/analyzer.md § RULES: E after "- Do not modify files".
### Block A
```
- Stronger model needed (an approach to settle before building, a stuck
problem, a long task before calling it done, before recommending a change
to the doctrine, the architecture or a public interface) → the main loop
calls the advisor tool when it is listed (settings `advisorModel`,
Fable, set by /advisor), at least at the points of
`~/.claude/lib/effort-shift.md` § Advisor; dispatched plan-challenger,
verifier, security-auditor and `analyzer` at most once, before their
verdict or report; other sub-agents do not. Fable is never routed or
pinned: the router tops out at opus (BDR-150).
```
### Block B1
```
1. Before settling an approach: on the written plan, before the
challengers see it (`lib/challenge-plan.md`), at most once per skill run.
```
### Block B1b
```
reuses that advice and says whether it predates the last iteration. The
```
### Block I
```
Inside a skill run, the points above also cover the triggers of
CLAUDE.global.md § Workflow; no second call for the same recommendation.
```
### Block B2
```
Sub-agents, when dispatched: plan-challenger, verifier, security-auditor
and `analyzer` call it at most once per dispatch, before their verdict or
report; executors and every other sub-agent do not, even when the tool is
listed. The analyzer file loaded inline by the main loop follows the
main-loop points.
```
### Block H
```
The main loop's advisor call before the dispatch (below) is the costliest
single item: once per skill run.
```
### Block C
```
**Advisor first (main loop).** Before the first dispatch, with no main-loop
advisor call since the skill run started (so at most once per skill run, not
before a re-challenge), the main loop reads the `PLAN` file unless its full
text is already in this conversation, then calls the advisor when the tool
is listed (`lib/effort-shift.md` § Advisor). An answer that lands a real
flaw revises the `PLAN` file on disk before the dispatch, as plain plan
text: no advisor attribution, no "changes after review" section; the
summary's `Advisor:` row carries it (OUTPUT below). Absent or erroring →
dispatch as written.
The advice never enters a challenger brief: they stay blind.
```
### Block F
```
- a CHALLENGE SUMMARY: each BLOCKER raised → the named change that closed it;
anything `[deferred]`; any lens that failed to return; and an `Advisor:`
row, added to whatever summary template the caller prints: each plan
change an advisor answer drove, `called, no change`, or
`not called (absent | error | already called this run)`.
```
### Block Dfloor
```
A second UNMET with no main-loop advisor call since GATE 0 first ran →
the main loop calls it before the re-dispatch
(`lib/effort-shift.md` § Advisor); the answer stays here, never
in the dev brief.
```
### Block Dconf
```
A second ECARTS with no main-loop advisor call since GATE 0 first ran →
the main loop calls it before the re-dispatch
(`lib/effort-shift.md` § Advisor); the answer stays here, never
in the dev brief.
```
### Block Dsec
```
A second BLOCK with no main-loop advisor call since GATE 0 first ran →
the main loop calls it before the re-dispatch
(`lib/effort-shift.md` § Advisor); the answer stays here, never
in the dev brief.
```
### Block Dstruct
```
A 2nd structural failure with no main-loop advisor call since GATE 0
first ran → the main loop calls it before escalating
(`lib/effort-shift.md` § Advisor).
```
### Block G
```
Every cap or escalation STOP above: when the main loop consulted the advisor
since GATE 0 first ran, the STOP carries its answer and says whether it
predates the last iteration.
```
### Block E
```
- Advisor, when dispatched as a sub-agent: at most one call, after the last
TASKS or PROCESS step and before the OUTPUT, whatever the harness prompt
schedules; absent or erroring → proceed. Advice may point you back to
the code; every OUTPUT line rests on what you read, never on the advice
alone. Advice proposing a design or a fix counts only where the brief
asks for a fix.
```
## Edge cases
- Every new line ≤ 80 columns; every `§ Label` citation on one line followed by
`)` or `;` (census extractor: 6 × effort-shift § Advisor, 2 × CLAUDE.global.md
§ Workflow) — checked by script.
- Contract CHECKs 1-5 simulated on a scratch copy with apply.py: all equal EXPECT.
- Stuck-call unit counts main-loop calls only; plan point skipped when the main
loop already consulted the advisor in this skill run.
- lib/tests pin "NEVER a PASS" lines: unchanged.
## Tests
None added (prose). Oracles: contract greps; fast census
`bash lib/tests/doctrine-citers.test.sh`; full `make test`.
## Disposition
- amends BDR-150 (6ef3069): plan point in challenge-plan; analyzer joins; (c).
- honors LRN-294 / BUGS-FOUND residual: prose; run C measures.
- honors BDR-066 / LRN-083: advice stays in the main loop, never in an executor
or challenger brief, never attributed in the plan file.
- applies LRN-295 cost: at most one plan call per skill run, at most one stuck
call per verify-secure run, no second call for the same recommendation.