forked from bchanot/claude
Bastien Chanot
9da1dec9e6
Audited each statistic in agents/resources/ against primary sources after the VSI fiction (I2) showed WebSearch launders SEO-blog consensus. The failure mode is not invention — it is plausible recombination, which is what a model half-remembering a search result produces: - "Adding statistics increases AI visibility by up to 40% (Aggarwal et al.)" — paper real (KDD 2024), number real, SCOPE WRONG: 40% is the aggregate over the whole method set, domain-dependent. No per-technique figure exists. - "Pages not updated quarterly are 3x more likely to lose AI citations (LLMRefs)" — LLMrefs' actual 3x says brand mentions correlate ~3x more strongly with AI visibility than backlinks. DIFFERENT SUBJECT. No source supports a quarterly decay multiplier. - "QAPage cited 58% more often than Article" — uncited. Nearest real number: AccuraCast 2025, `Person` schema at 58.9% PREVALENCE among cited sources — wrong type, and its FAQPage figure (1.8%) points the opposite way to the claim it propped up. This one drove Tier 1 ranking. - "62% of searches involve voice" — uncited; 62% circulates as smart-speaker ADOPTION. Same family as the "50% by 2020" myth ComScore denied (origin: a 2014 Andrew Ng interview). Corrected my own framing too: I claimed three times these stats "drive axis weights". They do not — the weight tables carry no citations. They drive Tier/priority recommendations and, worse, geo-analyzer's "Cite sources" rule pushed them into CLIENT reports as research-backed. Fixes: recommendations kept on mechanism, fabricated numbers removed with the incident documented inline so they are not re-added. Unverified stats (48% AI Overviews, 2.5B queries/day, Gartner -25%) labelled [UNVERIFIED] rather than asserted or deleted — I did not check them. Structural, not just exhortation: resources/README.md now mandates `<claim> — <source, year, venue|vendor> — measured: <what the source ACTUALLY measured> — <link>`. `measured:` is the field that catches this — all four errors survive a source name; none survives stating the real measurement next to the claim. WebSearch demoted from verification to crawler/tool-name lookup only. Verified: make test 35 GREEN / 0 RED.
76 lines
3.5 KiB
Markdown
76 lines
3.5 KiB
Markdown
# SEO/GEO shared resources
|
|
|
|
Knowledge base shared by `seo-analyzer` and `geo-analyzer` agents.
|
|
Loaded on demand — keep each file focused and current.
|
|
|
|
| File | Owner agents | Topic |
|
|
|---|---|---|
|
|
| `ai-crawlers-2026.md` | seo + geo | User-agent strings, categories (training vs search), robots.txt strategy |
|
|
| `llms-txt-template.md` | geo | `/llms.txt` + `/llms-full.txt` structure, generation patterns |
|
|
| `geo-schemas.md` | geo | Schema.org types for AI extraction (QAPage, Speakable, Person, Article) + deprecated list |
|
|
| `entity-seo.md` | geo | Wikidata QID, sameAs network, Knowledge Graph wiring |
|
|
| `content-shape-for-ai.md` | geo | Definition Lead, TL;DR, Q→A, stats, citations — content patterns LLMs cite |
|
|
| `ai-visibility-tools.md` | geo | Monitoring tools (OtterlyAI, Peec, Trendos, ZipTie, HubSpot AEO, SE Ranking) |
|
|
| `automation-catalog.md` | seo + geo | For every user-action in SEO.md §11 — what tool can automate it |
|
|
|
|
## Update policy
|
|
|
|
These files capture state as of 2026-04. Crawler lists, Schema.org
|
|
deprecations, and tool landscape shift fast. Agents MUST cross-check
|
|
crawler lists and tool names via WebSearch on each run when FULL depth is
|
|
selected.
|
|
|
|
## Citation standard (mandatory for every statistic)
|
|
|
|
**WebSearch is NOT verification for a number.** It ranks SEO blogs, and SEO
|
|
blogs cross-cite each other into a consensus that looks like corroboration.
|
|
Two 2026-07-16 audits of this directory show how it fails:
|
|
|
|
- A "VSI (Visual Stability Index) — new 2026 Core Web Vital" lived in
|
|
`seo-analyzer.md`. Ten blogs asserted it; several claimed CrUX already
|
|
collected it. It is absent from the CrUX API metric list and from
|
|
web.dev. WebSearch returned the echo, not the truth.
|
|
- Every stat in this directory was real **and attached to the wrong
|
|
subject**: the GEO paper's 40% (all methods) pinned on one technique;
|
|
LLMrefs' 3x (brand mentions vs backlinks) pinned on freshness decay;
|
|
AccuraCast's 58.9% (Person schema prevalence) pinned on QAPage lift, with
|
|
its meaning inverted; a smart-speaker adoption figure sold as voice-search
|
|
share.
|
|
|
|
The failure mode is not invention — it is **plausible recombination**, which
|
|
is exactly what a model half-remembering a search result produces. So the
|
|
format has to make an unsourced number conspicuous:
|
|
|
|
```
|
|
<claim> — <source, year, venue|vendor> — measured: <what the source ACTUALLY
|
|
measured> — <link>
|
|
```
|
|
|
|
`measured:` is the field that catches it. All four errors above survive a
|
|
source name; none survives having to state the source's real measurement
|
|
next to the claim.
|
|
|
|
Rules:
|
|
1. **Primary source or no number.** Peer-reviewed paper, the vendor's own
|
|
published study, or an official API/doc. `developer.chrome.com/docs/crux`
|
|
is decisive for metrics: what CrUX cannot return, we cannot score.
|
|
2. **Name the tier.** Peer review ≠ vendor marketing. LLMrefs, AccuraCast,
|
|
Ahrefs publish useful data and sell products — say "vendor".
|
|
3. **Never widen scope.** An aggregate result is not a per-technique result.
|
|
4. **No number beats a wrong number.** A recommendation that only stands up
|
|
with a fabricated statistic was never standing up. Delete the stat, keep
|
|
the recommendation if it survives on mechanism.
|
|
5. **Unverified ⇒ labelled.** `[UNVERIFIED — <date>]` inline. Never quote an
|
|
unverified number to a client: `geo-analyzer.md` ("Cite sources") sends
|
|
these into client reports as research-backed.
|
|
|
|
## Loading pattern
|
|
|
|
Agents reference resources like this:
|
|
|
|
```
|
|
Load: ~/.claude/agents/resources/ai-crawlers-2026.md
|
|
```
|
|
|
|
Do not inline these contents into agent prompts — read them at step time.
|