feat(seo-data): C3 — internal link graph; orphans + click depth, measured

seo-analyzer.md:613 asks "Every important page reachable within 3 clicks?"
and :616 asks "Orphan pages (no inbound internal links)?". Neither ever had a
command — same shape as the sameAs check before W3. This is that command.

My earlier reservation ("costs a lot of network") was wrong and the
measurement killed it: 24 pages in 2.7s, 86 in 3.8s. Cheap enough to always
run on FULL.

EXHAUSTIVE OR NOTHING is the design constraint, not a nicety. Orphans cannot
be sampled: proving a page has no inbound link means having read every other
page. So when the crawl is capped or any page fails, orphans are WITHHELD —
`orphans_withheld: true` and no list. A false orphan ("page X has no inbound
links" when it does) sends a client fixing what is not broken; that is the
worst finding this tool could emit. The cap does not degrade the result, it
invalidates it.

SPA refusal: on a client-rendered site the links are not in the HTML and
every page reads as orphaned. That is catastrophic, so an empty graph returns
degraded/no_links_in_html instead of a full false-positive list. No JS
rendering by design — that is the R1/R2 arbitration, not something to smuggle
in here.

Verified against BOTH live sites and against a planted failure, because two
clean results are not evidence a detector detects:
- native PHP: 24 pages, 335 links, depth 2, 0 orphans
- Astro: 86 pages, 2015 links, depth 2, 0 orphans
- fixture with a planted orphan + a 4-click chain: both found. Filters proven
  on real shapes seen live — /css/main.css?v=1778157313, #anchors, mailto:,
  tel:, external hosts, .png. /b/ in markup vs /b in sitemap unify to one node
  rather than a phantom orphan pair.

Fixed a flaw in my own mock while writing that test: a single page.html
fixture cannot express a GRAPH (every node gets identical links), so the mock
is now pages.json = {url: html}.

Verified: seo-data 122 -> 136 pass, 0 fail; full suite green; shellcheck +
py_compile clean.
This commit is contained in:
Bastien Chanot
2026-07-17 12:32:42 +02:00
parent dca977bb27
commit fe41986be9
7 changed files with 271 additions and 1 deletions
+27
View File
@@ -135,6 +135,33 @@ has "billion-laughs refused" "$DTD" '"status": "degraded"'
has "dtd reason is distinct" "$DTD" 'unsafe_xml_dtd'
hasnt "dtd never parsed" "$DTD" '"count"'
echo "── linkgraph ──"
LG="$(SEO_DATA_MOCK_DIR="$SD/fixtures-linkgraph" python3 "$SD/linkgraph.py" \
--url https://ex.com/sitemap.xml)"
has "linkgraph ok" "$LG" '"status": "ok"'
has "linkgraph crawls all" "$LG" '"pages_crawled": 7'
# THE test: a planted page nobody links to must be found. Two live sites both
# returned zero orphans; without this, "always returns []" looks identical.
has "finds the planted orphan" "$LG" '"https://ex.com/orphan"'
has "orphan is also unreachable" "$LG" '"unreachable"'
has "depth chain measured" "$LG" '"max_depth": 4'
has "flags >3 clicks" "$LG" '"https://ex.com/deepest"'
# home links: /a and /b only. anchor, .css?v=, mailto:, tel:, external, .png
# are not page links — 9 total across the 7 pages.
has "filters non-page links" "$LG" '"total_internal_links": 9'
hasnt "no external host" "$LG" 'other.com'
hasnt "no asset link" "$LG" 'main.css'
hasnt "no image link" "$LG" 'logo.png'
# /b/ in the markup vs /b in the sitemap must be ONE node, not a phantom orphan
hasnt "trailing slash unified" "$LG" '"https://ex.com/b/"'
# An orphan from a partial crawl is a false orphan: withhold, do not truncate.
CAP="$(SEO_DATA_MOCK_DIR="$SD/fixtures-linkgraph" python3 "$SD/linkgraph.py" \
--url https://ex.com/sitemap.xml --max 3)"
has "cap is reported" "$CAP" '"capped": true'
has "capped withholds orphans" "$CAP" '"orphans_withheld": true'
hasnt "capped emits no orphans" "$CAP" '"orphans":'
echo "── fetch.sh ──"
FETCH="$SD/fetch.sh"
# SEO_DATA_ENV_FILE=/dev/null: tests must NEVER source the real ~/.claude/.env —