Commit Graph
837 Commits
Author SHA1 Message Date
Bastien Chanot 45e679ac23 docs: README model-routing v2 table (BDR-076/077) + ctx7 surfaces (BDR-078) + STEP 2b — post-release 1.2.0 doc pass 2026-07-20 14:21:24 +02:00
Bastien Chanot 2d38ffd843 Merge release/1.2.0 into develop 2026-07-20 14:07:28 +02:00
Bastien Chanot 56571805a1 chore(release): 1.2.0 — version.txt + CHANGELOG 2026-07-20 14:06:14 +02:00
Bastien Chanot 8ee7d19d70 Merge feature/ctx7-coverage into develop 2026-07-20 14:02:33 +02:00
Bastien Chanot b7026e4bda feat(ctx7): coverage extension — fast-libs single source + reminder hook + executor briefs (BDR-078)
- lib/fast-libs.sh: detect/cache-status verbs, JS+Python manifests,
  7-day cache freshness, LC_ALL=C sort — replaces 3 hardcoded lists
  (ship-feature 0c, init-project 5c, onboard 3.5)
- hooks/ctx7-reminder.sh: once-per-session UserPromptSubmit nudge when
  the project carries fast-libs and .ctx7-cache/ is missing/stale
- find-docs: before-writing-code trigger + cache-first rule; dist is
  machine-owned (gitignored) so the durable patch lives in
  install-plugins.sh STEP ctx7 (idempotent, grep-guarded)
- feater/bugfixer briefs: fast-lib docs rule (fresh cache read, else
  2-topic ctx7 fetch, else NOTES cache miss + proceed)
- tests: lib/tests/fast-libs.test.sh (11 checks); shellcheck + full
  make test green (review-guards 5/0)
2026-07-20 10:45:06 +02:00
Bastien Chanot 6838d5a8fa Merge feature/model-tiering-w6-doctrine into develop 2026-07-19 23:56:58 +02:00
Bastien Chanot 444c79acb2 fix(routing): W6 ronde — 6 findings closed (README bootstrap path, 2 census gaps, 3 stale refs)
Fresh-opus whole-chantier ronde (17fbe51..HEAD, EVAL-023 style): axes
severed-wires / gate-regressions / fail-open PROVEN CLEAN (all 7 new
handoffs traced end-to-end both sides). Fixed: init 5b now dispatches the
FULL-AUDIT path (auto-mode gated a missing README as SIGNIFICANT →
[CREATE-AUTO] unconditional restored, sole greenfield README path);
census locks added for /geo ERROR CONTRACT + never-re-derive (deleting
the fail-closed handler would have stayed green), doc-audit
model=opus override x5 flows, SYNTH REPORT grammar; doc-syncer ex-STEP-8
prose repointed to the dispatcher gate; plugin-advisor anim rows read
the PROBE REPORT ANIM field (no Bash anymore); client-handover 9.7
cleans the transient draft. Census caught one more line-wrapped lock
before it shipped vacuous. 133 pass / 0 fail, make test exit 0.
2026-07-19 23:56:58 +02:00
Bastien Chanot 07253e093c feat(doctrine): W6 — prose sweep + BDR-077 + LRN-137 + plan execution notes
LRN-113 whole-surface sweep: client-handover x2 + commit-change prose
repointed to the two-mode reality; code-cleaner/status historic notes
kept (accurate). Memory: BDR-077 (full architecture), LRN-137
(mode-based re-tiering + fail-safe pin rule), journal. Plan carries
as-built EXECUTION NOTES.
2026-07-19 23:44:24 +02:00
Bastien Chanot 6c59a424ae Merge feature/model-tiering-w5-seo-geo into develop 2026-07-19 23:23:11 +02:00
Bastien Chanot 9e4ebb4cf4 feat(agents): W5 seo/geo 3-mode pipelines — collect sonnet / judge opus pin / template sonnet (BDR-077)
seo-analyzer + geo-analyzer gain MODE: collect|judge|template around the
dispatcher (mode-based, zero body-text moves — seo-data fetch-wiring
locks survive; opus pin kept = fail-safe direction, a forgotten override
over-tiers but never downgrades judgment). Run-scoped gitignored
signals handoff (.audit/*-signals-<RUNID>.md + COLLECTION COMPLETE
sentinel), judge fails closed on absent/mismatched/unsealed signals.
/seo rewired to 3 phases (domains parallel per phase) + DISPATCHER ERROR
CONTRACT (mute/ERROR judge never carried into templating; retry once,
escalate); /geo same single-domain; legacy no-MODE single-shot kept on
the opus pin for /harden narrow-scope + /onboard report-only. Dropped
/geo's 'ask and I relay' fiction (dispatched agents cannot ask).
In-wave smokes PASSED disk-verified: collect signals+sentinel; judge
ERROR-verdict on wrong RUNID; real judge = honest N/A + deterministic
engine + full scoring grammar; template = complete envelope + verbatim
sentinel + zero re-derivation. Census §18 (125 pass — one vacuous
line-wrapped lock caught by the census itself and fixed), make test
exit 0.
2026-07-19 23:23:10 +02:00
Bastien Chanot 6886622ecf Merge feature/model-tiering-w4-handover into develop 2026-07-19 22:59:35 +02:00
Bastien Chanot d2a10de08b feat(agents): W4 handover two-mode — synthesize opus / render sonnet, run-scoped draft handoff (BDR-077)
handover-doc-writer: MODE synthesize (model="opus" call-site — STEP
9/10/12 → .audit/handover-draft-<RUNID>.md + DRAFT COMPLETE sentinel) /
MODE render (sonnet pin — STEP 13-16 from the draft, fail-closed on
absent/mismatched RUNID). Mode-based, not a file split: the §9
name+dispatch census locks survive untouched. client-handover-writer 9.6
dispatches twice with the FULL PACKAGE both times (LRN-126) + RUNID mint
+ post-run draft cleanup. In-wave smokes PASSED disk-verified: draft
written+sentinel+gitignored; render BLOCKED on wrong RUNID (no phantom
synthesis); render consumed draft + honored skip-write, report grammar
intact. Census §17 (111 pass), make test exit 0.
2026-07-19 22:59:34 +02:00
Bastien Chanot d3d5e3802c Merge feature/model-tiering-w3-tier-moves into develop 2026-07-19 22:46:19 +02:00
Bastien Chanot 5e8bb0c22e feat(routing): W3 tier moves — validator-analyzer opus→sonnet, commit-changer propose=opus override (BDR-077)
validator-analyzer tiered down (deterministic validator-runner + fixed
deduction tables — no deep judgment; supersedes its BDR-076 opus pin).
commit-changer: MODE propose dispatched model="opus" (narrative
reconstruction + capitalize routing = judgment), MODE apply on the sonnet
pin. Typed-pin precedence smoke PASSED: sonnet-pinned verifier dispatched
model="haiku" ran on claude-haiku-4-5 — call-site wins, documented +
now behaviorally proven. Census §11 flip + §16 (106 pass), make test
exit 0.
2026-07-19 22:45:00 +02:00
Bastien Chanot e4d2629c88 Merge feature/model-tiering-w2-conversions into develop 2026-07-19 22:42:39 +02:00
Bastien Chanot 18075a38db feat(agents): W2/S2 doc pipeline two-mode + last inline conversions (BDR-077)
doc-syncer: ONE agent, TWO dispatch modes around the dispatcher's gate —
MODE: audit (model="opus" call-site override, READ-ONLY, drafts + PATCH
PLAN) / MODE: patch (sonnet pin, applies the APPROVED plan, shape oracle
w/ revert-on-fail, emits CHANGE SUMMARY + PATCHED_FILES). Deviation from
plan's 2-file split, per the challenge's own commit-changer mode
precedent: zero text duplication, zero lock moves. Fixes a LATENT DEFECT:
/doc dispatched an agent whose STEP 8 gate could never fire (dispatched
agents cannot ask) — the gate now lives in the dispatcher (DISPATCHER
PROTOCOL section). doc-commit.md consumes the patcher's CHANGE SUMMARY
(the in-thread context now crosses the dispatch boundary, LRN-126).
Consumers rewired: /doc (audit→gate→patch→commit), onboard (audit
report-only, opus), doc-commit steps in bugfix/hotfix/feat/ship-feature/
init-project(5b+10c); scaffolder loses PHASE 6 (README = init 5b's job);
scaffolder + onboarder now DISPATCHED in init-project/onboard (pins live,
was inline on session model). In-wave planted-drift smoke PASSED
end-to-end, disk-verified (audit caught npm-run-dev drift → [MINOR] plan
→ patch applied → summary crossed). Census §14-15 (103 pass), make test
exit 0. Typed plugin-probe dispatch resolution verified post-restart.
2026-07-19 22:28:09 +02:00
Bastien Chanot 74528a6910 feat(agents): W2/S1 plugin split — probe (sonnet) + advisor reasoner (opus) + plugin-gate include (BDR-077)
plugin-advisor keeps its name, becomes the opus REASONER: PHASE 1 bash
extracted to new plugin-probe (sonnet, facts-only PROBE REPORT), PHASE 4
apply + checkpoint hoisted to new lib/plugin-gate.md (main-loop include,
doc-commit.md x6 pattern). Fail-closed: advisor ERRORs on missing report.
4 consumers rewired (plugin-check, onboard, init-project, ship-feature).
In-wave planted-input smoke PASSED: probe report complete w/ fallbacks;
advisor consumed every planted field (monorepo per-package note, fast-libs
ctx7 reco) with zero re-detection; ERROR verdict on absent report.
Census §13 (81 pass). Note: new subagent_type registers next session —
resolution re-check before wave merge.
2026-07-19 20:57:10 +02:00
Bastien Chanot 896d3faaf9 Merge feature/model-tiering-w1-no-inherit into develop 2026-07-19 20:00:37 +02:00
Bastien Chanot 3f7c754239 feat(routing): W1 no-inherit — fable skill-runners, opus review dispatches, dispatch-tier doctrine (BDR-077)
No dispatched agent inherits the session model anymore:
- client-handover-writer's 7 general-purpose skill-runner dispatch sites
  carry model: "fable" (+ normative rule; spike-verified alias — resolves
  claude-fable-5, enum-validated, loud failure, never silent fallback)
- ship-feature STEP 6 + init-project STEP 10 code-review dispatches carry
  model: "opus" (was: inherit — the leak the maps exposed)
- model-gate.md §4: dispatch-tier doctrine (typed = frontmatter pin,
  built-ins = explicit model= at every call site)
- census §12 (66 pass), make test green
2026-07-19 19:51:06 +02:00
Bastien Chanot 1c2d30dbf0 chore(tasks): model-tiering v2 — analysis + challenged plan v3 + TODO reconcile
Plan challenged by 3 blind lenses + 1 confirmation pass (1 BLOCKER closed
by fable-dispatch spike, 6 MAJORs + 8 MINORs fixed by named changes, 0
deferred). TODO: seo-geo-integrity 'UNMERGED' note was stale (92301fe
already in develop) — corrected.
2026-07-19 19:51:06 +02:00
Bastien Chanot 17fbe51aa1 Merge feature/opus-pin-audit-agents into develop 2026-07-19 19:46:10 +02:00
Bastien Chanot 3eaf31ca09 chore(memory): BDR-076 + journal + TODO — opus-pin dispatched judgment agents 2026-07-19 17:38:58 +02:00
Bastien Chanot 354ff2644f feat(agents): pin dispatched judgment agents to opus — Fable = inline reflection only (BDR-076)
Reverses the BDR-066 rejected alternative (opus pins on audit agents):
session default is now Fable, so inherit burned Fable quota on every
dispatched audit/challenge. analyzer, plan-challenger, seo/geo/
validator-analyzer pinned model: opus; onboard's 6 general-purpose
audit dispatches carry model="opus"; tour Phase B repointed.
interviewer + client-handover-writer stay unpinned (inline-load only,
a pin there is inert). settings.json default: claude-fable-5[1m].
Census flipped: model-routing §3 + new §11 (61 pass), loops-light 35,
full make test green.
2026-07-19 17:38:55 +02:00
Bastien Chanot 9bc6ab7e07 Merge feature/hotfix-challenge-guard into develop 2026-07-18 23:24:04 +02:00
Bastien Chanot 727a41ad71 chore(memory): BDR-075 amendment (hotfix included) + journal — Option B + behavioral smoke 2026-07-18 23:08:53 +02:00
Bastien Chanot 311ea14789 feat(skills): wire hotfix into plan-challenge under a logic-only guard
STEP 1.8 (Option B): skip purely cosmetic fixes (CSS/copy/typo), run the 3-lens
challenge only when the fix touches control flow/behaviour (off-by-one, wrong
operator, behaviour-changing config, execution-altering import); a BLOCKER means
it was never a hotfix -> escalate to /bugfix. 12th orchestrator wired.

- skills/hotfix/SKILL.md          — STEP 1.8 guarded challenge
- lib/tests/plan-challenger.test.sh — lock hotfix into the census (43 assertions)
2026-07-18 23:08:52 +02:00
Bastien Chanot a68f26ca9c Merge feature/plan-challenge-phase into develop 2026-07-17 22:57:40 +02:00
Bastien Chanot d5d1584c1c Merge feature/drop-config-protection into develop 2026-07-17 22:57:10 +02:00
Bastien Chanot 2aa95636ee chore(memory): BDR-074 BDR-075 EVAL-026 LRN-136 — config-protection removal + plan-challenge phase 2026-07-17 22:54:12 +02:00
Bastien Chanot 6bfc0543e5 feat(skills): add 3-way adversarial plan-challenge phase to reflection orchestrators
After a plan/reflection is elaborated and before it executes, three fresh blind
sub-agents (correctness / robustness / simplicity) attack it on the big model;
the main loop RE-THINKS every aspect a BLOCKER lands (a named plan change, or
[deferred]) and re-challenges once if the plan materially changed. Advisory into
each skill's existing human gate — the human stays the decider.

- lib/challenge-plan.md — reusable phase: fail-safe (never fail open),
  severity-driven (any single-lens BLOCKER = must-address), RE-THINK loop
- agents/plan-challenger.md — challenger role (read-only, big-model per BDR-066)
- lib/tests/plan-challenger.test.sh — 41-assertion structure lock
- wired into 11 orchestrators: ship-feature/init-project/feat/bugfix (build-plan),
  onboard/audit-delta/code-clean (proposals), seo/geo/harden/web-validate (fix-bundle)

Hardened by dogfooding: 3 blind challengers reviewed this feature's own v1 plan
and caught 4 BLOCKERs (fail-open, consensus-buries-lone-finding, wrong model
tier vs BDR-066, false on-disk-plan premise) — all fixed here.
2026-07-17 22:51:50 +02:00
Bastien Chanot 0e1b89c71a chore(hooks): remove config-protection edit-block guardrail
Full removal per user request: the PreToolUse hook that blocked model
Edit/Write on quality-gate files (settings.json, gitflow.sh, .githooks,
doctor.sh, hooks, lib/tests, lint configs) plus its one-shot sentinel.

- delete hooks/config-protection.sh
- delete lib/tests/config-protection.test.sh
- deregister the hook from settings.json (rtk-rewrite PreToolUse kept)
- drop the README mention

Residual protection unchanged: gitflow pre-commit guard + Gitea branch
protection still block direct code commits to main/develop.
2026-07-17 21:56:32 +02:00
Bastien Chanot 23c8c290d7 Merge feature/dns-rebinding-guard into develop 2026-07-17 20:17:45 +02:00
Bastien Chanot a391be4906 chore(memory): LRN-134 LRN-135 — capitalize 2026-07-17 20:16:50 +02:00
Bastien Chanot b00e8ef442 feat(seo-data): safe_fetch — resolve-then-pin, close DNS-rebinding + SSRF
By-principle hardening. H1's url-guard validates the NAME; urlopen then resolved
AND connected — two DNS lookups with a window a hostile authority uses to answer
PUBLIC to validation and PRIVATE (169.254.169.254 metadata, 127.0.0.1, the LAN)
to the connect. A name-level guard cannot see that rebind.

safe_fetch collapses the two lookups into one: resolve ONCE, validate every IP
(ipaddress, dual-stack v4+v6), refuse if ANY is non-public (the multi-A vector),
connect to the exact validated IP with Host+SNI+cert for the real host — no
second resolution to poison. Redirects re-validate each hop (urlopen followed
them blind). One seam: sitemap._fetch, which linkgraph/render_check/drift all
call, so every network verb inherits it.

The load-bearing property (confirmed by the security review): classification is
on the OS-resolved address (sockaddr[0]), never the URL text — so octal/hex/
decimal literals, IPv4-mapped IPv6, NAT64, 6to4 are all defeated structurally,
not by enumeration.

Better than the source idea (claude-seo url_safety.py, MIT): dual-stack (theirs
IPv4-only), no global monkeypatch so thread-safe by construction (theirs locks a
patched getaddrinfo), stdlib-only (no requests). Proven end-to-end before
writing: pinned connect keeps SNI+cert for the real host.

NOT covered, stated not silent: shell `curl` in the agent specs (separate
process, unpinnable here). Smaller surface; `curl --resolve` is a separate change.

REVIEW-SURFACED (fresh security-auditor, adversarial, VERDICT PASS) — two real
holes it found while attacking the diff, both fixed here:
- billion-laughs REOPENED in C1b: _refuse_dtd scanned only raw[:4096], so a
  >4KB leading comment pushed <!DOCTYPE past the window while ET parsed AND
  EXPANDED the entities. Proven (&lol2; → "lollollollollol"), now a full-doc
  case-insensitive scan. This is a genuine fix to already-merged C1b, not this
  feature — fixed here rather than filed, per root-cause discipline.
- 192.88.99.0/24 (6to4-relay anycast) passed is_global as public — added to an
  extra special-use deny list.

Verified: rebind-to-metadata refused BEFORE any connect (injected resolver),
multi-A public+private refused, classifier fuzzed dual-stack incl. CGNAT/6to4,
non-http scheme refused, both review fixes proven with no false positive; real
fetch still works (zenquality 86 loc, lavageangels 24) through the pinned path;
all 4 verbs work end-to-end via fetch.sh; seo-data 210 → 221 pass, 0 fail; full
suite green; shellcheck + py_compile clean.
2026-07-17 19:58:59 +02:00
Bastien Chanot 4ccfb606a8 Merge feature/seo-data-cherry-picks into develop 2026-07-17 19:10:10 +02:00
Bastien Chanot 0564afcb3c chore(memory): journal — content_quality shipped, both easy picks done 2026-07-17 19:07:10 +02:00
Bastien Chanot b271e83fb6 feat(seo-data): content_quality verb — deterministic filler/AI-slop signal
Cherry-picked from claude-seo (github.com/AgriciDaniel/claude-seo, MIT)
content_quality.py, rewritten to the lib/seo-data contract per BDR-070. The
Content Shape axis was 100% LLM judgement; this gives it a measured input.

fetch.sh content_quality (stdin or --file) → {filler_score, ai_pattern_score,
information_density, overall_quality, flags[], matches{}}. 100% deterministic:
QRG §4.6 filler list (26 phrases) + AI-pattern list (46) kept intact, regex
matching, no LLM. Stdlib only (argparse/json/re/sys/collections/typing).

Advisory, NOT a verdict — the point of the wiring. It never claims a page "is
AI-written" (LRN-131/133); flags are candidates for human review. geo-analyzer
STEP 8 Check 10 makes it a deterministic input that INFORMS checks 1-9, never
replaces them, never scored on its own. A low number is not an automatic
finding.

Detection proven both directions (a detector that always- or never-flags is
useless): filler+slop text → flags [filler, low-density], overall 34-49; clean
dense factual text (dates/EUR/percentages) → no flags, overall 90. Empty input →
degraded/empty_input, never zeros-as-a-result.

Verified: GATE 1 verifier CONFORME 10/10 (both directions exercised live, lists
diffed intact vs source, advisory language confirmed); GATE 2 self-scan clean
(only sink is read-only open() for --file); seo-data 190 → 210 pass, 0 fail;
full suite green; shellcheck + py_compile clean.
2026-07-17 19:06:58 +02:00
Bastien Chanot fb0b587240 chore(memory): journal — schema_gen shipped, gap-revisit note 2026-07-17 14:31:26 +02:00
Bastien Chanot cfdd89e73b feat(seo-data): schema_gen verb — generate JSON-LD, not just audit it
Cherry-picked from claude-seo (github.com/AgriciDaniel/claude-seo, MIT)
schema_generate.py, rewritten to the lib/seo-data contract per BDR-070 — adapt,
never copy. The system audited JSON-LD but could not generate it; geo-analyzer's
G2 batch hand-wrote markup. Now it calls the verb.

fetch.sh schema_gen {reservation|order|discussion|profile} → fail-open envelope
{"status":"ok","source":"schema_gen","type":…,"jsonld":{…}}. Types: Reservation
(7 subtypes), OrderAction, DiscussionForumPosting, ProfilePage (sameAs/knowsAbout
for the entity graph). Stdlib only (import argparse, json — zero third-party),
_strip_nones so a null is never emitted, --script-tag wraps for direct paste.

Fail-open mirrors score.py's _cli exactly (the contract's named pattern): a
flag-omitted required field → argparse exit 2 + {"status":"error","bad_usage"};
a flag-present-but-empty field → {"status":"degraded","reason":"missing required
field: …"} exit 0. Never a traceback, never empty stdout (LRN-133: the
can't-generate case stays legible).

geo-analyzer G2 wiring preserves the data-integrity rule — the verb generates
STRUCTURE, unknown values stay [À COMPLÉTER], never invented.

Verified: GATE 1 verifier CONFORME 10/10 (exercised the fail-open edge cases +
diffed field surface against the source); GATE 2 self-scan clean (no
network/shell/secret/eval sink); seo-data 167 → 190 pass, 0 fail; full suite
green; shellcheck + py_compile clean.
2026-07-17 14:30:31 +02:00
Bastien Chanot 92301fe1c8 Merge bugfix/seo-geo-integrity into develop 2026-07-17 13:48:13 +02:00
Bastien Chanot f96206ff21 chore(memory): BDR-070..073 LRN-131..133 BLK-017 EVAL-025 — capitalize 2026-07-17 13:46:17 +02:00
Bastien Chanot 4818c6116f feat(seo-data): I7 — compute the score instead of feeling it
/harden has a real scale (SKILL.md:435 — Critique -15, Haute -8, Moyenne -3,
Basse -1, clamp [0,100]). /seo had none: every axis was felt, so two runs over
identical code could disagree. That is a credibility problem on its own, and
/client-handover gates on 17/20 — a wobbling number makes the gate arbitrary.
H2 sharpened it: now that drift reports what actually changed, a score moving
on its own is visibly noise.

The split is the whole point. WHICH findings exist and how severe each is
stays the LLM's judgement — irreducible, and I am not pretending otherwise.
The arithmetic stops being judgement: same findings in, same score out. Same
principle as grouping cannibalisation rows in the engine rather than handing a
model 1000 rows to add up.

Reuses /harden's scale, /5 into /20, so the family speaks one vocabulary
instead of two.

Two things it makes real that were prose:
- **N/A is not a zero.** R2 (client-rendered on-page) and I1 (unauditable
  off-page) both mandate excluding an axis and renormalising the rest. Both
  left that arithmetic to the model. Now the engine does it and refuses to let
  N/A behave like a zero — verified: all-20 axes with two N/A still yields
  global 20.0, not a dragged-down mean.
- **Prevalence.** affected/sampled shift severity ONE step (>=50% escalates, a
  single page de-escalates). A defect on 1 of 12 pages is not the defect on
  12 of 12, and flattening the two is part of what made the old numbers move.

Malformed input is an error, never a silently wrong number — unlike the fetch
verbs, a degrade here would mean bad input, not a network fact. Unknown
severity and unknown profile both rejected, tested.

Verified: hand-checkable arithmetic (haute+moyenne = 100-11 = 89 → 17.8;
critique+haute = 77 → 15.4), identical global across repeated runs, weights
renormalised to sum 1.0 with two axes N/A. seo-data 155 -> 167 pass, 0 fail;
full suite green; shellcheck + py_compile clean.
2026-07-17 13:29:14 +02:00
Bastien Chanot f69cfc5cb4 feat(seo-data): H2 — drift baseline; regressions vs changes, not prose
seo-analyzer.md:1365 keeps history as "date + score + key changes" — prose the
LLM writes about its own previous prose. Lossy, unreproducible, and
machine-uncomparable, so "the redesign silently dropped 40 canonicals" is
invisible unless someone happens to notice.

drift snapshots title/description/canonical/robots/h1_count/jsonld_types per
URL and diffs them. Stdlib only, no auth.

The classification IS the feature: LOSING a signal is a regression, CHANGING
one is a change that may well be intended. The engine says which kind; the
agent judges. A reworded title is not an alert; an evaporated canonical is.

Runs over the WHOLE sitemap, never a sample — caught while designing: a drift
computed over a sample that changes between runs compares nothing.

NOT rank tracking. That is the common misread of this same feature elsewhere;
positions come from GSC `queries`. This is on-page regression detection.

Also caught in my own draft before testing: _capture reused
sm._mock("page.html"), the exact single-fixture flaw I had already fixed in
linkgraph — one fixture cannot express a multi-page snapshot, every URL would
read identical. Now pages.json, same convention.

Proved on a planted failure rather than a happy path — two clean sites would
look identical to a detector that always returns []:
  v1 -> v2: canonical lost on /a, h1 + jsonld lost on /, title reworded,
  /gone removed, /neuve added
  → 3 regressions, 1 change, gone/new both detected, title correctly NOT a
    regression.

Store is ~/.claude/seo-data/drift/<host>.json, 0700, written via os.replace so
a crash never leaves a half-written baseline; a corrupt store degrades to
"first run" instead of killing the audit.

Verified: seo-data 144 -> 155 pass, 0 fail; full suite green.
2026-07-17 13:25:45 +02:00
Bastien Chanot d6b8edc8ea fix(seo): B1 KILLED — Common Crawl backlinks measured, not assumed
The plan said Common Crawl was the free backlink source and the 70/100 cap
was therefore mandatory. Measured before building, and both premises die.

HEAD against data.commoncrawl.org, live:
  cc-main-2026-feb-mar-apr-domain-edges.txt.gz    17.3 GB   gzipped
  cc-main-2026-feb-mar-apr-domain-ranks.txt.gz     2.3 GB
  cc-main-2026-feb-mar-apr-domain-vertices.txt.gz  879 MB

Finding one domain's inbound links means scanning the edges file end to end,
per audit. That is not slow, it is non-viable — and abusive toward a
nonprofit serving the data free.

Worse, the reference implementation everyone points at
(claude-seo scripts/commoncrawl_graph.py:169) does this:

    max_compressed_bytes = 500 * 1024 * 1024   # 500 MiB safety cap
    if total_downloaded > max_compressed_bytes: break

500 MiB of 17.3 GB is **2.9% of the edges file**, which is sorted by source
ID — so it reads an arbitrary slice of source domains and reports whatever
backlinks happened to be in it, as a backlink profile, capped at "70/100
health". Nothing in the output says 3%. That is a random sample wearing a
measurement's clothes: the exact failure class this branch exists to remove,
and I was one step from copying it.

B2 dies with B1: nothing left to cap.

CONSEQUENCE, and it is the point: I1's narrowed Off-page axis — brand
mentions only, backlinks + authority declared unauditable in §14 — is the
FINAL state, not a placeholder waiting for data. Corrected my own I1 text,
which pointed at Common Crawl as the "nearest free source": that sends a
future reader into a 17 GB dead end. The §14 line now records what was
measured and why no number beats a fabricated one.

Also corrects the B3 note, whose follow-on ("so Common Crawl is the only free
source") was wrong for the same reason. The only free viable backlink source
is Bing's GetUrlLinks — first-party only, never a competitor, and blocked on
the client's Bing account. That raises W2's value; it does not unblock it.

Verified: full suite green, seo-data 144 pass / 0 fail.
2026-07-17 13:17:18 +02:00
Bastien Chanot 02c7a6fe6d Merge bugfix/seo-geo-integrity-phase1 into develop 2026-07-17 13:09:08 +02:00
Bastien Chanot 20d3082542 feat(seo-data,seo,geo): R2 — refuse to score what JS paints; no Playwright
Arbitrated (user): honest refusal on SPA, no headless browser.

STEP 2 has recorded `RENDERING: SSR/SSG/SPA/hybrid` since forever and NOTHING
ever acted on it. (The inventory claimed a "SPA severely limited" §0 flag
compensated — it does not exist. Seventh subagent claim this branch has had to
disprove.) So on a client-rendered site the FULL audit curls an empty shell,
every meta/H1/JSON-LD check reports "missing", and the agent emits a page of
false findings — plus a bundle that would "fix" tags which already exist.

rendercheck reads the verdict from what the server SENT. package.json cannot
tell a React SPA from a Next.js SSR app; the served bytes can. Stdlib only.

The refusal is the point:
- client-rendered → On-page is N/A, excluded from the weighted global, NOT
  scored zero. A zero says "your on-page is bad"; N/A says "we could not see
  it". Only one is true, and /client-handover gates on this number.
- No bundle item may come from a live on-page check on such a site.
- The report still says what IS auditable (robots, sitemap, headers, config,
  CrUX field data — real users, hydration included — GSC, legal, images)
  rather than returning an empty verdict.
- geo refuses Content Shape the same way, and states the sharper fact: AI
  crawlers are WORSE at JS than Googlebot. GPTBot/PerplexityBot/ClaudeBot
  fetch HTML and largely do not execute it, so a client-rendered site is not
  merely unauditable by us — it is near-invisible to the engines this audit
  exists to serve. §0 alert + SSR/SSG as the top user action.

Script/style text is not page text: a React shell with a fat inline
window.__INITIAL_STATE__ measures 7 chars. Without that skip a 200 KB bundle
reads as a rich page — the detector would fail exactly where it matters.

Verified on both extremes, not just the happy path: zenquality 7650 chars/1
h1/9 jsonld and lavageangels356 13973/1/1 → server-rendered, no warning; a
Vite/React shell fixture → client-rendered, 7/0/0, warned.

seo-data 136 -> 144 pass, 0 fail; full suite green; shellcheck + py_compile
clean.
2026-07-17 12:44:43 +02:00
Bastien Chanot fe41986be9 feat(seo-data): C3 — internal link graph; orphans + click depth, measured
seo-analyzer.md:613 asks "Every important page reachable within 3 clicks?"
and :616 asks "Orphan pages (no inbound internal links)?". Neither ever had a
command — same shape as the sameAs check before W3. This is that command.

My earlier reservation ("costs a lot of network") was wrong and the
measurement killed it: 24 pages in 2.7s, 86 in 3.8s. Cheap enough to always
run on FULL.

EXHAUSTIVE OR NOTHING is the design constraint, not a nicety. Orphans cannot
be sampled: proving a page has no inbound link means having read every other
page. So when the crawl is capped or any page fails, orphans are WITHHELD —
`orphans_withheld: true` and no list. A false orphan ("page X has no inbound
links" when it does) sends a client fixing what is not broken; that is the
worst finding this tool could emit. The cap does not degrade the result, it
invalidates it.

SPA refusal: on a client-rendered site the links are not in the HTML and
every page reads as orphaned. That is catastrophic, so an empty graph returns
degraded/no_links_in_html instead of a full false-positive list. No JS
rendering by design — that is the R1/R2 arbitration, not something to smuggle
in here.

Verified against BOTH live sites and against a planted failure, because two
clean results are not evidence a detector detects:
- native PHP: 24 pages, 335 links, depth 2, 0 orphans
- Astro: 86 pages, 2015 links, depth 2, 0 orphans
- fixture with a planted orphan + a 4-click chain: both found. Filters proven
  on real shapes seen live — /css/main.css?v=1778157313, #anchors, mailto:,
  tel:, external hosts, .png. /b/ in markup vs /b in sitemap unify to one node
  rather than a phantom orphan pair.

Fixed a flaw in my own mock while writing that test: a single page.html
fixture cannot express a GRAPH (every node gets identical links), so the mock
is now pages.json = {url: html}.

Verified: seo-data 122 -> 136 pass, 0 fail; full suite green; shellcheck +
py_compile clean.
2026-07-17 12:32:42 +02:00
Bastien Chanot dca977bb27 fix(seo-data,seo): backtest on a second, native site — two real bugs
Everything on this branch was grounded on ONE Astro repo. A native PHP site
(lavageangels356.fr) broke two things that looked fine there.

BUG 1 — sitemap counted images as pages. _locs matched
`el.tag.endswith("}loc")`, and <image:loc> from Google's image-sitemap
namespace ALSO ends with '}loc'. Astro's sitemap has no image extension, so
this was invisible. The native site's does: 24 <url> + 3 <image:loc> came back
as count=27. The COVERAGE denominator was 12.5% too high and img/logo.png was
about to be sampled and audited as a page.
Fixed with two locks: walk the DIRECT children of each <url>/<sitemap> instead
of root.iter() (which alone excludes <image:image><image:loc>), and test the
sitemaps.org namespace explicitly. Regression fixture carries the image
extension; the old endswith code returns 9 URLs against it, the new one 7 with
zero images.
Verified both sites: native 27 -> 24, zero images; Astro unchanged at 86.

BUG 2 — the C1c family heuristic was tuned to one URL layout. "First path
segment" works for NESTED city pages (/creation-site-internet/essonne-91/ →
25 pages, 1 family) and FAILS for FLAT ones (/lavage-auto-pomponne,
/lavage-auto-torcy → 8 pages, 8 singletons). Consequence: C1c's rule "sample
>=3 from the largest family" would have targeted /services (5) and missed the
8 city pages entirely — the exact doorway-page risk the 30/70 rule exists to
catch.
Family is now "shared parent path OR shared slug prefix (>=3 URLs sharing 2+
hyphen tokens)", with both real layouts as the worked examples, plus a
sanity-check: a sitemap yielding almost as many families as URLs has defeated
the heuristic, not proved the site has no templates. Fixed in seo-analyzer and
in the geo pointer that referenced it.

Backtest results on the native site for everything else: url-guard accepts the
domain; source-scope excludes only .git (no dist/build/out exists — the
exclusions are correctly no-ops, and cache/ holds only .htaccess+.gitignore so
it is rightly untouched); the sameAs check runs and finds zero (a real GEO gap
for that site, not a tool bug); links are present in the served HTML (PHP is
SSR), so C3 is feasible there.

Verified: seo-data 119 -> 122 pass, 0 fail; full suite green; py_compile clean.
2026-07-17 12:28:36 +02:00
Bastien Chanot 3a15643c2c feat(seo-data): C2 — cannibalisation from Google's own data, one param away
The inventory called this "no duplicate-content / cannibalisation detection".
Splitting that into its two halves shows one is free and the other is a trap.

CANNIBALISATION — free, and the data was already reachable. Search Analytics
has always accepted several dimensions at once ("no limit to the number of
dimensions that you can group by"); this engine only ever sent
`"dimensions": [dim]` and _norm_queries only ever read `keys[0]`. So
query+page — the pairing that exposes the conflict — was one parameter away
and nobody asked. Same shape of win as W1.

  fetch.sh cannibal → queries where 2+ of OUR pages compete, ranked by total
  impressions, strongest page first inside each. Same auth, same quota family,
  no new scope. `capped` reports a full row window rather than presenting a
  truncated list as exhaustive — same rule as COVERAGE and the sitemap caps.

Grouping happens in the engine, deterministically: asking an LLM to group
1000 rows by query is arithmetic it should never be handed.

Backward compatible: rows gained `keys` (the list the API actually returns);
`key` stays as keys[0], so the single-dim quick-wins consumer is untouched.
A test pins both.

30/70 DUPLICATION — deliberately NOT built, and this is the honest half.
Measuring it needs main-content extraction (strip nav/header/footer). Without
that, comparing two same-template pages returns ~95% similar for every site —
a confident false positive, which is exactly the failure class the rest of
this branch exists to remove. It stays an explicit LLM judgement over the >=3
same-family pages C1c now samples for it, labelled as judgement, never quoting
a similarity percentage nobody computed. A wrong number would be worse than
the current honest gap.

The two must not be merged in the report either: cannibalisation is a SERP
fact Google measured; 30/70 is a content question. The spec now says so.

Verified: fixture with 3 pages on one query, 2 on another, 1 on a third →
2 conflicts, correct ranking, single-page query excluded; live dispatch
degrades cleanly with no account; seo-data 110 -> 119 pass, 0 fail; full suite
green; shellcheck + py_compile clean.
2026-07-17 11:55:00 +02:00
Bastien Chanot 04ccc5ad9b fix(seo,geo): C1c — split COVERAGE source/live; the 30/70 rule needed the
opposite sample

I5 made COVERAGE mandatory and told the agent to "sample by risk, one per
template". Grounding that against a real Astro site shows the rule is half
wrong, and that one number was hiding two.

86 URLs collapse into 8 families; 75 of them (87%) come from 3 dynamic
[dept] templates. So the same 12 sampled pages are simultaneously 14% LIVE
coverage and ~100% SOURCE coverage. Reporting only 14% understates the audit;
reporting only 100% oversells it. Both lines now, in both agents — and they
bound different findings, so they must not be averaged:
  SOURCE bounds CODE (one template renders its whole family: a missing
  canonical in [dept]/index.astro breaks all 25 identically).
  LIVE bounds CONTENT (title wording, thin pages, duplication — written per
  page, so a template says nothing about its 25 instances).

The sharper half: "one per template" is CORRECT for code and WRONG for the
30/70 rule, which this same spec mandates at :954 (city pages: 30% shared,
70% unique). You cannot tell whether 25 dept pages are 70% unique by reading
one of them. The spec was mandating a check its own sampling made
structurally impossible. Sampling is now keyed to the finding class: 1 per
family for code, >=3 from the LARGEST family for duplication, spread for
per-page content.

Families come from the first path segment of the sitemap URLs (C1b) — a good
enough proxy for "same template" that needs no framework routing knowledge,
verified against the real distribution.

geo gets the same split, cut differently: JSON-LD lives in a shared layout so
SOURCE bounds Schema.org, while Definition Lead / TL;DR are written per page
so LIVE bounds Content Shape. Site-wide axes (crawler policy, llms.txt) stay
unbounded — single files, fully read.

Verified: full suite green. Caught and fixed one self-inflicted contradiction
before commit — geo's prose demanded both coverages while its output block
still had a single line.
2026-07-17 11:43:48 +02:00