forked from bchanot/claude
chore(memory): advisor sub-agent scope, LRN-303, BDR-150 status, follow-ups
This commit is contained in:
1 parent
c62ee9c5cf
commit
1dedf9056d
6 files changed
+166
-1
No files matched your search
@@ -1613,3 +1613,4 @@ Supersedes [[BDR-030]].
|
|||||||
- **Status (2026-11-17)**: accepted, amended 58a5273: plan point in lib/challenge-plan.md (on written plan before challengers, once per skill run, Advisor: row in CHALLENGE SUMMARY, no attribution in plan); stuck lines inline in verify-secure-loop (main-loop calls only since GATE 0); dispatched analyzer joins; trigger before recommending doctrine/architecture/public-interface change
|
- **Status (2026-11-17)**: accepted, amended 58a5273: plan point in lib/challenge-plan.md (on written plan before challengers, once per skill run, Advisor: row in CHALLENGE SUMMARY, no attribution in plan); stuck lines inline in verify-secure-loop (main-loop calls only since GATE 0); dispatched analyzer joins; trigger before recommending doctrine/architecture/public-interface change
|
||||||
- **Status (2026-11-17)**: accepted, amended 6022b9d: B2 stuck trigger in 6 gate skills (points to verify-secure-loop); end-of-run call before done/merge in feat, bugfix, ship-feature, init-project, a11y, guardrails, audit-delta+tour (fix applied only), release-candidate STEP 4; own budget; gate flows print diff stat, fix back through gates; verifier/security-auditor/plan-challenger advise only when dispatched; auditor raise upward only; plan-challenger protocol mirror. Commits f48343e 24354d9 6022b9d
|
- **Status (2026-11-17)**: accepted, amended 6022b9d: B2 stuck trigger in 6 gate skills (points to verify-secure-loop); end-of-run call before done/merge in feat, bugfix, ship-feature, init-project, a11y, guardrails, audit-delta+tour (fix applied only), release-candidate STEP 4; own budget; gate flows print diff stat, fix back through gates; verifier/security-auditor/plan-challenger advise only when dispatched; auditor raise upward only; plan-challenger protocol mirror. Commits f48343e 24354d9 6022b9d
|
||||||
- **Status (2026-11-18)**: accepted, amended 51f343f: B3: challenge route rides first lib/challenge-plan.md call; summary Advisor row in 12 callers; feat/bugfix skip end call when GATE 0 MET + verifier CONFORME + security PASS (not DEGRADED, TOOL line) first attempt, ADVISOR field shows skip; item 3 reuse clause names later code commits
|
- **Status (2026-11-18)**: accepted, amended 51f343f: B3: challenge route rides first lib/challenge-plan.md call; summary Advisor row in 12 callers; feat/bugfix skip end call when GATE 0 MET + verifier CONFORME + security PASS (not DEGRADED, TOOL line) first attempt, ADVISOR field shows skip; item 3 reuse clause names later code commits
|
||||||
|
- **Status (2026-11-20)**: accepted, amended 7979b9b..b71db28: allowed set + opus code review (ship-feature STEP 6, init-project STEP 10, standalone SDD final) one call via brief; never-call lines in opus agents/briefs; SubagentStart hook hooks/subagent-advisor.sh backstop; standalone SDD workers sonnet; Advisor row reason sub-agent
|
||||||
@@ -786,3 +786,6 @@ rules:
|
|||||||
- /bugfix degraded-tool-line 9814720: 3 security readers read DEGRADED off TOOL line (BLK-051), BUGS-FOUND:134 FIXED, TODO:13 done, 2 pre-existing minors logged; CHANGELOG Fixed entry.
|
- /bugfix degraded-tool-line 9814720: 3 security readers read DEGRADED off TOOL line (BLK-051), BUGS-FOUND:134 FIXED, TODO:13 done, 2 pre-existing minors logged; CHANGELOG Fixed entry.
|
||||||
- /feat x2 merge-reuse (af846d6, 87566e2, feature/advisor-merge-reuse): 6 gate-flow end lines restate effort-shift item 3 merge-reuse clause, loops-light 97; TODO:14 done, 3 follow-ups logged; run 1 GATE 0 ERROR = contract without EVIDENCE: pending (LRN-173 again).
|
- /feat x2 merge-reuse (af846d6, 87566e2, feature/advisor-merge-reuse): 6 gate-flow end lines restate effort-shift item 3 merge-reuse clause, loops-light 97; TODO:14 done, 3 follow-ups logged; run 1 GATE 0 ERROR = contract without EVIDENCE: pending (LRN-173 again).
|
||||||
- Run C /feat advisor-usage (c6f2dad + retention fix, feature/advisor-usage): FEEDBACK.md advisor usage table per caller/model, reconciled vs probe (156 calls, 21.7M); found general-purpose sub-agent advisor leak (30 calls, 7.1M) → TODO; LRN-301 (call = served iteration), LRN-302 (scratch develop ref); mods.test.sh timeout under load → BUGS-FOUND.
|
- Run C /feat advisor-usage (c6f2dad + retention fix, feature/advisor-usage): FEEDBACK.md advisor usage table per caller/model, reconciled vs probe (156 calls, 21.7M); found general-purpose sub-agent advisor leak (30 calls, 7.1M) → TODO; LRN-301 (call = served iteration), LRN-302 (scratch develop ref); mods.test.sh timeout under load → BUGS-FOUND.
|
||||||
|
|
||||||
|
## 2026-11-20
|
||||||
|
- /ship-feature advisor-subagent-scope (7979b9b 6be0669 b71db28): leak root cause = advisor server tool on every sub-agent, no filter, opus ignores CLAUDE.md attachment; rule moved into agent defs/briefs + SubagentStart hook + SDD sonnet pin; LRN-303; behavioural proof pending next usage table.
|
||||||
@@ -2174,3 +2174,9 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
|
|||||||
- **Context**: 2026-11-19 merge-reuse run 2 simulation (LRN-296 method): criterion 3 SKIP-INTACT failed only on scratch clone; fixed by creating ref.
|
- **Context**: 2026-11-19 merge-reuse run 2 simulation (LRN-296 method): criterion 3 SKIP-INTACT failed only on scratch clone; fixed by creating ref.
|
||||||
- **Apply**: every LRN-296 scratch simulation from a non-develop branch: create local develop ref first, else false red.
|
- **Apply**: every LRN-296 scratch simulation from a non-develop branch: create local develop ref first, else false red.
|
||||||
- **Reference**: LRN-296, .claude/tasks/contracts/2026-11-19-merge-reuse-feat-bugfix-1450.md
|
- **Reference**: LRN-296, .claude/tasks/contracts/2026-11-19-merge-reuse-feat-bugfix-1450.md
|
||||||
|
|
||||||
|
## LRN-303 — Server tool scope per sub-agent: no filter removes advisor; rule must ride agent definition or brief, hook is backstop (2026-11-20)
|
||||||
|
- **Pattern**: server-side advisor tool attached to every sub-agent (sonnet too); no tool filter removes it: `tools` allowlist, agent `disallowedTools: ["advisor"]`, `--disallowedTools advisor` all probed 2026-11-19, still listed. Opus sub-agents ignored CLAUDE.global rule delivered as `instructions` attachment (18/25 called); agents carrying rule in own definition (system prompt) obeyed (≤1 call per dispatch, 89 dispatches). Sonnet never called (0/~450). SubagentStart `additionalContext` reaches sub-agent before first turn (nonce probe) = attachment channel too.
|
||||||
|
- **Context**: 2026-11-19/20 advisor leak (Run C table: general-purpose 30 calls 7.1M tok, 29 from standalone SDD on opus in another repo). Fix 7979b9b..b71db28: rule in each agent definition/brief, SDD workers sonnet, hook backstop.
|
||||||
|
- **Apply**: to scope any harness-listed server tool per sub-agent: put rule in agent definition or dispatcher brief, not CLAUDE.md; word allowance as explicit permission ("one call allowed"); hook = backstop only; behavioural proof = transcript tally, not delivery.
|
||||||
|
- **Reference**: hooks/subagent-advisor.sh, lib/effort-shift.md § Advisor, LRN-294, LRN-295, LRN-301, BDR-150
|
||||||
@@ -16,7 +16,12 @@ Done history before 2026-10-08 removed 2026-10-23 (git keeps it: `git log -p --
|
|||||||
- [ ] item 3 "names any code commit made after it" has no anchor: ADVISOR field records no HEAD SHA, commits made outside the session are invisible; candidate: record the short SHA with the answer (visible format change, user decision) (merge-reuse robustness challenger, low)
|
- [ ] item 3 "names any code commit made after it" has no anchor: ADVISOR field records no HEAD SHA, commits made outside the session are invisible; candidate: record the short SHA with the answer (visible format change, user decision) (merge-reuse robustness challenger, low)
|
||||||
- [ ] end-of-run sites restate the call but not "one end-of-run call per skill run": a fix that goes back through the gates re-enters the end step and may call again (merge-reuse robustness challenger, low)
|
- [ ] end-of-run sites restate the call but not "one end-of-run call per skill run": a fix that goes back through the gates re-enters the end step and may call again (merge-reuse robustness challenger, low)
|
||||||
- [x] DONE c6f2dad + 63f5fa4, merged 6e45536 (docs 7d09837, audits 77aac4b, memory 3ffd81f) (/feat feature/advisor-usage; user 2026-11-19: FEEDBACK.md section, per agentType, all projects no repo, since last run, retention + drift lines). Was: Run C /feat: advisor usage section in /memory-feedback (main loop vs sub-agents, model served, input tokens; dedupe usage rows by message.id, model from the `advisor_message` iteration)
|
- [x] DONE c6f2dad + 63f5fa4, merged 6e45536 (docs 7d09837, audits 77aac4b, memory 3ffd81f) (/feat feature/advisor-usage; user 2026-11-19: FEEDBACK.md section, per agentType, all projects no repo, since last run, retention + drift lines). Was: Run C /feat: advisor usage section in /memory-feedback (main loop vs sub-agents, model served, input tokens; dedupe usage rows by message.id, model from the `advisor_message` iteration)
|
||||||
- [ ] Advisor doctrine leak: general-purpose sub-agents called the advisor 30× (7.1M input tok, the largest line of the first advisor usage table; 16 on 2026-11-18/19, after 6ef3069 2026-11-17 12:59 limited sub-agents to plan-challenger/verifier/security-auditor/analyzer), doc-syncer 4× (2026-11-16/17). Likely SDD implementers/reviewers dispatched as general-purpose: find why the "executors and every other sub-agent do not" rule does not reach them (brief, agent prompt or tool listing) (Run C first table, medium)
|
- [x] FIX SHIPPED 7979b9b 6be0669 b71db28 (/ship-feature feature/advisor-subagent-scope; behavioural proof pending, next line). Was: Advisor doctrine leak: general-purpose sub-agents called the advisor 30× (7.1M input tok, the largest line of the first advisor usage table; 16 on 2026-11-18/19, after 6ef3069 2026-11-17 12:59 limited sub-agents to plan-challenger/verifier/security-auditor/analyzer), doc-syncer 4× (2026-11-16/17). Likely SDD implementers/reviewers dispatched as general-purpose: find why the "executors and every other sub-agent do not" rule does not reach them (brief, agent prompt or tool listing) (Run C first table, medium)
|
||||||
|
- [ ] 2026-11-27 or later, after a fresh session: advisor leak behavioural check. Run /memory-feedback (advisor usage table) plus a per-dispatch transcript tally since 7979b9b. Tripwires: every typed non-allowed type 0 calls; sonnet stays 0; general-purpose at most 1 call per dispatch and only in code-review dispatches; allowed four keep about 1 per dispatch (a drop to 0 = the "one call allowed" line not read); standalone SDD workers served sonnet. Revert path: drop the SubagentStart entry from settings.json. Then mark BUGS-FOUND :107 FIXED or reopen (advisor-subagent-scope, medium)
|
||||||
|
- [ ] lib/memory-feedback/advisor.ts: split general-purpose callers per dispatch (calls per dispatch, code-review vs other from a fixed description marker), so the advisor usage table can read the general-purpose tripwire without a hand tally (advisor-subagent-scope robustness challenger, low)
|
||||||
|
- [ ] census test: every agents/*.md served on opus (frontmatter or a model="opus" dispatch) carries either the never-call line or an allowance; every model="opus" general-purpose brief in skills/agents names its advisor rule (STEP 6 review minor 3, low)
|
||||||
|
- [ ] hooks/subagent-advisor.sh shortcut: SubagentStart only, a sub-agent that compacts may lose the line; re-inject on SessionStart `compact` when agent_type is set (hook header shortcut, low)
|
||||||
|
- [ ] advisor-scope review minors: lib/model-gate.md Dispatch tiers names the advisor line for opus built-ins; feat/bugfix ADVISOR fields gain `not called (sub-agent)`; client-handover-writer quoted child templates carry the line literally, not only the routing paragraph (STEP 6 review minors 1, 6, 8, low)
|
||||||
- [ ] lib/tests/mods.test.sh model-router plugin test runs ~121 s alone vs CLI_TIMEOUT=120 (load avg 26-31 on 2026-11-19): flaky red in make test, probable cause of run 1 feater's unexplained red; raise the bound or speed the test (Run C GATE 0, low)
|
- [ ] lib/tests/mods.test.sh model-router plugin test runs ~121 s alone vs CLI_TIMEOUT=120 (load avg 26-31 on 2026-11-19): flaky red in make test, probable cause of run 1 feater's unexplained red; raise the bound or speed the test (Run C GATE 0, low)
|
||||||
- [x] Stale citers after A: README/USAGE/CHANGELOG patched, BUGS-FOUND residual superseded + LOW logged, BDR-150 status, LRN-295 (LRN-294 kept: its lesson holds)
|
- [x] Stale citers after A: README/USAGE/CHANGELOG patched, BUGS-FOUND residual superseded + LOW logged, BDR-150 status, LRN-295 (LRN-294 kept: its lesson holds)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# CONTRACT — advisor-subagent-scope
|
||||||
|
- date: 2026-11-19 | flow: ship-feature | branch: feature/advisor-subagent-scope
|
||||||
|
- status: active
|
||||||
|
|
||||||
|
## REQUEST (verbatim — IMMUTABLE)
|
||||||
|
corrige la fuite advisor des sous-agents et fais les todos, et si il y en a d'autre ensuite fais les, tant que des todos apparaissent fais les sauf si tu vois que ca se repete
|
||||||
|
|
||||||
|
Scope of this run (main loop): the advisor sub-agent leak (TODO advisor-leak), escalated from /bugfix (>5 files). Diagnosis and probes: .claude/tasks/plans/2026-11-19-subagent-advisor-hook-1905.md.
|
||||||
|
|
||||||
|
## CLARIFICATIONS
|
||||||
|
Q: fix layers / A: "Both": SDD sonnet pin + never-call lines in opus-served agent prompts and opus briefs + a SubagentStart hook backstop [gated 2026-11-19]
|
||||||
|
Q: opus judgment reviewers / A: "Allow code reviews once": the opus code review (ship-feature STEP 6, init-project STEP 10, standalone SDD final review) may call the advisor at most once before its verdict; other judgment children never [gated 2026-11-19]
|
||||||
|
Q: merges in the TODO loop / A: gitflow finish per item once gates are clean [gated 2026-11-19]
|
||||||
|
Q: design / A: approved: carriers = CLAUDE.global.md § Skill routing (SDD pin) + § Workflow (allowed set, brief rule, hook) + lib/effort-shift.md § Advisor detail; six agent prompts (plugin-advisor, seo-analyzer, geo-analyzer, doc-syncer, commit-changer, handover-doc-writer); onboard's six opus briefs; client-handover-writer routing paragraph (not its top prompt: inline-loaded by the main loop); tour Phase B; ship-feature STEP 6 + init-project STEP 10 allowance; silent SubagentStart hook, timeout 5; vendored skills-external files NOT edited (update-all.sh overwrites them, BDR-106, BLK-024) [gated 2026-11-19]
|
||||||
|
Q: Advisor row of a skill run by a sub-agent / A: add `sub-agent` to the `not called (...)` reasons in lib/challenge-plan.md OUTPUT and the 9 caller template rows [gated 2026-11-19]
|
||||||
|
Q: plugin profile SET dev / A: no, keep full [gated 2026-11-19]
|
||||||
|
|
||||||
|
Q: 3-lens challenge fixes (hook line: agent definition or dispatcher's brief, the tool's own instructions never an allowance, ~/.claude path; all SDD workers sonnet incl. fix rounds, no opus carve-out; imperative code-review brief line; agent lines scoped "when you run as a sub-agent"; atomic applier; tripwire redefined for general-purpose) / A: main loop, internal; no gated choice reopened [2026-11-19]
|
||||||
|
|
||||||
|
Q: STEP 3 validation gate, plan docs/superpowers/plans/2026-11-19-advisor-subagent-scope.md (1 task, 33-check suite) / A: yes, execute [gated 2026-11-20]
|
||||||
|
|
||||||
|
Q: GATE 1 round 1 ECARTS(4): plan artifacts outside FILE SCOPE + SC2016 shellcheck disable (FLOOR) / A: main loop: the docs/superpowers plan + apply.py are transient run artifacts committed during the run and purged by gitflow finish (BDR-065), added to FILE SCOPE; the disable is removed (double quotes with escaped backticks, as loops-light), no suppression authorized [2026-11-20]
|
||||||
|
|
||||||
|
Q: STEP 6 opus review (With fixes: 2 Important) / A: main loop: fixed as plan section "Review fix round 1" (allowed agents' lines reworded "one call allowed" and locked T20; CLAUDE.global parenthetical clarified; hook prohibition locked T2c; plan-challenger mirror; effort-shift "other opus-served agents"; agent lines cite ~/.claude path; test lines ≤ 80); suite 38 checks; other minors to TODO [2026-11-20]
|
||||||
|
|
||||||
|
Q: add agents/{plan-challenger,verifier,security-auditor,analyzer}.md to FILE SCOPE (review fix: "one call allowed", locked T20) / A: yes [gated 2026-11-20]
|
||||||
|
|
||||||
|
## ACCEPTANCE CRITERIA
|
||||||
|
1. The new suite lib/tests/subagent-advisor.test.sh (hook output + wiring + carrier locks) is green.
|
||||||
|
CHECK: cd /home/bchanot/Documents/claude && bash lib/tests/subagent-advisor.test.sh 2>&1 | tail -1 | grep -qE '^subagent-advisor: [0-9]+ pass, 0 fail$' && echo SA-GREEN
|
||||||
|
EXPECT: SA-GREEN
|
||||||
|
EVIDENCE: MET exit=0 marker-found :: SA-GREEN
|
||||||
|
2. The hook emits one valid SubagentStart JSON object naming the advisor tool, whatever its stdin, and exits 0.
|
||||||
|
CHECK: cd /home/bchanot/Documents/claude && for i in '{"agent_type":"general-purpose"}' '' 'not json'; do o=$(printf '%s' "$i" | bash hooks/subagent-advisor.sh); [ $? -eq 0 ] || exit 1; printf '%s' "$o" | jq -e '.hookSpecificOutput.hookEventName == "SubagentStart" and (.hookSpecificOutput.additionalContext | test("advisor"))' >/dev/null || exit 1; done && echo HOOK-OK
|
||||||
|
EXPECT: HOOK-OK
|
||||||
|
EVIDENCE: MET exit=0 marker-found :: HOOK-OK
|
||||||
|
3. settings.json wires SubagentStart to the hook with timeout 5.
|
||||||
|
CHECK: cd /home/bchanot/Documents/claude && jq -e '.hooks.SubagentStart[0].hooks[0] | .command == "bash ~/.claude/hooks/subagent-advisor.sh" and .timeout == 5' settings.json >/dev/null && echo WIRED
|
||||||
|
EXPECT: WIRED
|
||||||
|
EVIDENCE: MET exit=0 marker-found :: WIRED
|
||||||
|
4. The 9 caller template rows and lib/challenge-plan.md carry the `sub-agent` reason; no row keeps the old closing.
|
||||||
|
CHECK: cd /home/bchanot/Documents/claude && n=$(grep -l 'already called this run | sub-agent)' skills/init-project/SKILL.md skills/code-clean/SKILL.md skills/onboard/SKILL.md skills/audit-delta/SKILL.md skills/ship-feature/SKILL.md skills/harden/SKILL.md skills/pack-seo/skills/seo/SKILL.md skills/pack-seo/skills/geo/SKILL.md skills/pack-web-validate/skills/web-validate/SKILL.md lib/challenge-plan.md | wc -l); [ "$n" -eq 10 ] && ! git grep -q 'already called this run)' -- skills lib ':!lib/tests' && echo ROWS-10
|
||||||
|
EXPECT: ROWS-10
|
||||||
|
EVIDENCE: MET exit=0 marker-found :: ROWS-10
|
||||||
|
5. CLAUDE.global.md stays within its 320-line density budget.
|
||||||
|
CHECK: cd /home/bchanot/Documents/claude && [ "$(wc -l < CLAUDE.global.md)" -le 320 ] && echo BUDGET-OK
|
||||||
|
EXPECT: BUDGET-OK
|
||||||
|
EVIDENCE: MET exit=0 marker-found :: BUDGET-OK
|
||||||
|
6. Full make test green.
|
||||||
|
CHECK: cd /home/bchanot/Documents/claude && PATH=$HOME/.bun/bin:$PATH make test </dev/null 2>&1 | tail -1
|
||||||
|
EXPECT: all suites green
|
||||||
|
EVIDENCE: MET exit=0 marker-found :: all suites green
|
||||||
|
7. Live delivery: a headless run with a scratch settings file wiring the real hooks/subagent-advisor.sh shows a sub-agent quoting the injected rule (manual, recorded by the main loop; this session may not reload settings). Behavioural proof is NOT claimed here: it is the next /memory-feedback advisor usage table plus a per-dispatch transcript check (dated TODO; tripwires: every typed non-allowed type → 0 calls, sonnet stays 0, general-purpose at most 1 call per dispatch and only in code-review dispatches, checked by hand until advisor.ts splits general-purpose by dispatch; revert = drop the settings entry).
|
||||||
|
8. No vendored skills-external file edited; no wording changed outside the plan's blocks; new prose lines ≤ 80 columns (manual).
|
||||||
|
|
||||||
|
## FILE SCOPE
|
||||||
|
hooks/subagent-advisor.sh (new), lib/tests/subagent-advisor.test.sh (new), settings.json, CLAUDE.global.md, lib/effort-shift.md, lib/challenge-plan.md, agents/{plugin-advisor,seo-analyzer,geo-analyzer,doc-syncer,commit-changer,handover-doc-writer,client-handover-writer}.md, skills/onboard/SKILL.md, skills/tour/SKILL.md, skills/ship-feature/SKILL.md, skills/init-project/SKILL.md, skills/{code-clean,audit-delta,harden}/SKILL.md, skills/pack-seo/skills/{seo,geo}/SKILL.md, skills/pack-web-validate/skills/web-validate/SKILL.md, docs/superpowers/plans/2026-11-19-advisor-subagent-scope.md + docs/superpowers/plans/2026-11-19-advisor-subagent-scope/apply.py (transient, BDR-065), agents/{plan-challenger,verifier,security-auditor,analyzer}.md (review fix round 1, gated)
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
# Bugfix plan: sub-agents call the advisor against the doctrine (TODO advisor leak)
|
||||||
|
|
||||||
|
## DIAGNOSIS
|
||||||
|
BUG: sub-agents outside the allowed four call the advisor. First advisor usage table
|
||||||
|
(2026-11-19): general-purpose 30 calls / 7.1M input tokens (29 in faunosteo's U2 SDD run, 1 the
|
||||||
|
B3 STEP 6 code review), doc-syncer 4 (opus audit). 16 general-purpose calls came after 6ef3069
|
||||||
|
(2026-11-17 12:59) restricted sub-agent calls to plan-challenger, verifier, security-auditor and
|
||||||
|
analyzer.
|
||||||
|
ROOT CAUSE: the advisor is a server tool the harness attaches to every sub-agent, whatever its
|
||||||
|
model, with its own system prompt telling it to call "before substantive work … and before
|
||||||
|
declaring done". The restriction lives only in CLAUDE.global.md § Workflow and
|
||||||
|
lib/effort-shift.md § Advisor (:140-143), text a dispatched agent does not hold in its own
|
||||||
|
prompt; the four allowed agents carry an in-prompt "at most once" line and comply. No tool
|
||||||
|
filter exists: probed 2026-11-19, the advisor stays listed under a `tools:` allowlist
|
||||||
|
(plan-challenger), under `--disallowedTools advisor` (headless main) and under an agent's
|
||||||
|
`disallowedTools: ["advisor"]` (headless sub-agent). Sonnet sub-agents also list it (probe) and
|
||||||
|
simply never used it (0 calls in ~400 dispatches); opus sub-agents act on the tool's prompt.
|
||||||
|
EVIDENCE: transcript tally per agentType × model (all 1105 transcripts): every call comes from
|
||||||
|
an opus-served agent; leakers' meta: general-purpose `model: opus` (faunosteo SDD standalone,
|
||||||
|
41 opus dispatches; ship-feature's sonnet pin did not apply), doc-syncer `model=opus` audits;
|
||||||
|
three live probes (opus gp, sonnet gp, headless with/without disallowedTools) all list advisor.
|
||||||
|
Delivery and compliance proven before planning (2026-11-19): a headless `claude -p --settings
|
||||||
|
<scratch>` run with a SubagentStart hook injecting a random nonce → the sub-agent returned the
|
||||||
|
exact nonce (additionalContext reaches the sub-agent before its first turn). In-context rules are
|
||||||
|
obeyed: since 6ef3069 the four allowed types made at most 1 call per dispatch (plan-challenger
|
||||||
|
43×1, verifier 22×1 + 3×0, security-auditor 21×1 + 1×0, analyzer 3×1); each carries its line in
|
||||||
|
its own prompt (agents/analyzer.md:46 and the other three).
|
||||||
|
PRIOR: LRN-294 (check what tools sub-agents really get; prefer a mechanical scope; prose →
|
||||||
|
residual) applied: the mechanical tool scope does not exist, so the next most mechanical layer is
|
||||||
|
used, a harness hook that delivers the rule into the sub-agent's own context. BUGS-FOUND :107
|
||||||
|
(prose-only residual, d292911, superseded line notes a verifier making 2 calls) is this bug.
|
||||||
|
honors BDR-150 (call points unchanged; this only delivers them), BDR-066 (executors sonnet).
|
||||||
|
LRN-295/301: counts from transcripts, call = served iteration.
|
||||||
|
BLAST RADIUS: every sub-agent of every project (settings.json hooks are global, symlinked to
|
||||||
|
~/.claude/settings.json). The four allowed agents get a reinforcing "at most once" line. No skill,
|
||||||
|
agent file or brief changes. Not fixed here (logged): faunosteo-style standalone SDD runs dispatch
|
||||||
|
implementers on opus (BDR-066 tier rule only lives in ship-feature STEP 4).
|
||||||
|
Rejected alternatives: (a) `disallowedTools` / deny rules: probed, no effect. (b) a "do not call
|
||||||
|
the advisor" line in each opus-dispatched agent prompt and brief template (doc-syncer,
|
||||||
|
commit-changer, plugin-advisor, seo/geo analyzers, handover writers, SDD implementer/reviewer
|
||||||
|
templates, code-reviewer template, onboard/tour audit briefs): 12+ files, misses ad-hoc
|
||||||
|
general-purpose dispatches and other projects' agents, drifts as agents are added. (c)
|
||||||
|
`CLAUDE_CODE_DISABLE_ADVISOR_TOOL` / advisorModel unset: kills the main loop's advisor too.
|
||||||
|
|
||||||
|
## FIX PLAN
|
||||||
|
1. hooks/subagent-advisor.sh (new) — SubagentStart hook. Reads stdin JSON, takes `agent_type`
|
||||||
|
(jq `.agent_type // ""`, errors → empty). Always exits 0, prints exactly one JSON object
|
||||||
|
`{"hookSpecificOutput":{"hookEventName":"SubagentStart","additionalContext":<msg>}}`:
|
||||||
|
- `agent_type` exactly one of `plan-challenger`, `verifier`, `security-auditor`, `analyzer`
|
||||||
|
→ msg: "Advisor rule (lib/effort-shift.md § Advisor): as a dispatched <type> you may call
|
||||||
|
the advisor tool at most once, before your verdict or report. Never a second call, even if
|
||||||
|
the tool's own prompt asks for one before and after the work."
|
||||||
|
- anything else (general-purpose, Explore, Plan, executors, plugin agents, empty, malformed)
|
||||||
|
→ msg: "Advisor rule (lib/effort-shift.md § Advisor): do not call the advisor tool in this
|
||||||
|
run, even though it is listed and its prompt asks for calls. Only the main loop and the
|
||||||
|
plan-challenger, verifier, security-auditor and analyzer sub-agents call it."
|
||||||
|
The JSON is built with `jq -n --arg` (the agent type reaches the text only from the fixed
|
||||||
|
allowed set); without jq, a printf of the same fixed text for the "do not call" case (no
|
||||||
|
interpolation). Style: like hooks/unpushed-guard.sh / ctx7-reminder.sh (header comment with
|
||||||
|
purpose + BDR ref, `set -u`, never blocks).
|
||||||
|
2. settings.json — add a `SubagentStart` event (no matcher: all agent types) running
|
||||||
|
`bash ~/.claude/hooks/subagent-advisor.sh`, beside the existing events, same entry shape.
|
||||||
|
3. lib/effort-shift.md:140-144 — after "executors and every other sub-agent do not, even when
|
||||||
|
the tool is listed." add: "A SubagentStart hook (hooks/subagent-advisor.sh) states each
|
||||||
|
sub-agent's rule in its own context at spawn: no tool filter removes the advisor." (wrapped
|
||||||
|
≤ 80 cols; the list sentence itself unchanged, the hook's allowed set is locked to it).
|
||||||
|
4. lib/tests/subagent-advisor.test.sh (new) — style of lib/tests/unpushed-guard.test.sh:
|
||||||
|
T1 general-purpose → valid JSON, hookEventName SubagentStart, context contains "do not call
|
||||||
|
the advisor tool"; T2-T5 each allowed type → contains "at most once" and the type, not "do not
|
||||||
|
call"; T6 empty stdin and T7 malformed JSON → "do not call", valid JSON, rc 0; T8 an
|
||||||
|
agent_type carrying `"`, newline and `$(…)` → valid JSON, the raw value absent from the
|
||||||
|
output, "do not call"; T9 PATH without jq (a temp PATH dir holding only bash/cat/printf links,
|
||||||
|
or `env PATH=` minimal) → valid JSON (checked with python3), "do not call"; T10 settings.json
|
||||||
|
wires `SubagentStart` to `hooks/subagent-advisor.sh` (jq query); T11 consistency: each of the
|
||||||
|
four allowed names appears in lib/effort-shift.md's "Sub-agents, when dispatched" sentence and
|
||||||
|
in the hook, and the effort-shift sentence names the hook. Lines ≤ 80 cols where the file
|
||||||
|
style allows.
|
||||||
|
|
||||||
|
Bookkeeping (main loop): BUGS-FOUND :107 residual → FIXED line; TODO advisor-leak line done;
|
||||||
|
new TODO: standalone SDD opus dispatches (BDR-066 pin only in ship-feature STEP 4).
|
||||||
|
|
||||||
|
Skill-runner children (onboard, tour, client-handover dispatch opus children that run whole
|
||||||
|
skills whose text has main-loop advisor points): the hook tells them not to call; proposed rule
|
||||||
|
(user decision at the STEP 3 gate): the parent main loop owns the call points, a child's own
|
||||||
|
`Advisor:` row / ADVISOR field reads `not called (sub-agent)`.
|
||||||
|
Proof of "fixed": the contract proves delivery (headless nonce through the real hook script and a
|
||||||
|
scratch settings file, since this session may not reload settings) and the hook's output per
|
||||||
|
type; the behavioural proof is the next /memory-feedback advisor usage table showing no calls
|
||||||
|
from other agent types after the hook commit (dated follow-up in TODO).
|
||||||
|
|
||||||
|
RISK: low-medium — a new global hook on every sub-agent spawn (cost ~10 ms, a few dozen context
|
||||||
|
tokens per sub-agent); a hook failure injects nothing (never blocks a spawn). Live check after
|
||||||
|
commit: a probe sub-agent reports the injected line.
|
||||||
Reference in new issue
Block a user