fix(geo): I6 — every stat was real and attached to the wrong claim

Audited each statistic in agents/resources/ against primary sources after
the VSI fiction (I2) showed WebSearch launders SEO-blog consensus.

The failure mode is not invention — it is plausible recombination, which is
what a model half-remembering a search result produces:

- "Adding statistics increases AI visibility by up to 40% (Aggarwal et al.)"
  — paper real (KDD 2024), number real, SCOPE WRONG: 40% is the aggregate
  over the whole method set, domain-dependent. No per-technique figure
  exists.
- "Pages not updated quarterly are 3x more likely to lose AI citations
  (LLMRefs)" — LLMrefs' actual 3x says brand mentions correlate ~3x more
  strongly with AI visibility than backlinks. DIFFERENT SUBJECT. No source
  supports a quarterly decay multiplier.
- "QAPage cited 58% more often than Article" — uncited. Nearest real number:
  AccuraCast 2025, `Person` schema at 58.9% PREVALENCE among cited sources —
  wrong type, and its FAQPage figure (1.8%) points the opposite way to the
  claim it propped up. This one drove Tier 1 ranking.
- "62% of searches involve voice" — uncited; 62% circulates as smart-speaker
  ADOPTION. Same family as the "50% by 2020" myth ComScore denied (origin: a
  2014 Andrew Ng interview).

Corrected my own framing too: I claimed three times these stats "drive axis
weights". They do not — the weight tables carry no citations. They drive
Tier/priority recommendations and, worse, geo-analyzer's "Cite sources" rule
pushed them into CLIENT reports as research-backed.

Fixes: recommendations kept on mechanism, fabricated numbers removed with
the incident documented inline so they are not re-added. Unverified stats
(48% AI Overviews, 2.5B queries/day, Gartner -25%) labelled [UNVERIFIED]
rather than asserted or deleted — I did not check them.

Structural, not just exhortation: resources/README.md now mandates
`<claim> — <source, year, venue|vendor> — measured: <what the source ACTUALLY
measured> — <link>`. `measured:` is the field that catches this — all four
errors survive a source name; none survives stating the real measurement
next to the claim. WebSearch demoted from verification to crawler/tool-name
lookup only.

Verified: make test 35 GREEN / 0 RED.
This commit is contained in:
Bastien Chanot
2026-07-16 20:32:55 +02:00
parent e70e1d6c71
commit 9da1dec9e6
5 changed files with 123 additions and 19 deletions
+26 -5
View File
@@ -61,9 +61,18 @@ query. A one-sentence self-contained answer has the highest density.
### 4. Citations and statistics (strongest measured lever)
Adding peer-cited statistics with clear sources increases AI visibility
**by up to 40%** (Aggarwal et al., 2024 "GEO: Generative Engine
Optimization").
Aggarwal et al., 2024 ("GEO: Generative Engine Optimization", KDD 2024)
report that their optimisation methods **collectively** boost visibility
**by up to 40%** in generative-engine responses, and state the effect
**varies across domains**. Citations/statistics/quotations are among those
methods.
> **Attribute this correctly.** Until 2026-07-16 this section read "Adding
> peer-cited statistics with clear sources increases AI visibility by up to
> 40%" — pinning the paper's *aggregate* result on this *one* technique. The
> paper publishes no separate figure per technique. When quoting it to a
> client: "up to 40%, across the method set, domain-dependent" — never "+40%
> if you add stats".
Pattern: embed specific numbers with attribution.
@@ -100,8 +109,20 @@ Comparison tables are even stronger. Structure:
### 6. Freshness signals
Pages not updated at least quarterly are **3x more likely to lose AI
citations** (LLMRefs 2026 study).
Freshness is a real retrieval input: RAG systems fetch live and read
timestamps, so a page updated this quarter carries a stronger recency
signal than the same page last touched years ago. LLMrefs (a **vendor**,
not peer review) reports cited content running **~25.7% fresher** than
organic top-10 across ~17M citations. Substantive updates only — bumping a
date string is not freshness.
> **The "3x" that lived here was grafted from another claim.** Until
> 2026-07-16 this read "Pages not updated at least quarterly are 3x more
> likely to lose AI citations (LLMRefs 2026 study)". LLMrefs' actual "3x"
> says **brand mentions correlate ~3x more strongly with AI visibility than
> backlinks** — a different subject entirely. No source supports a quarterly
> decay multiplier. Recommend quarterly refresh on its merits; do not price
> it with a borrowed number.
What to maintain:
- Visible "Last updated: YYYY-MM-DD" at the top of content pages