forked from bchanot/claude
fix(geo): I6 — every stat was real and attached to the wrong claim
Audited each statistic in agents/resources/ against primary sources after the VSI fiction (I2) showed WebSearch launders SEO-blog consensus. The failure mode is not invention — it is plausible recombination, which is what a model half-remembering a search result produces: - "Adding statistics increases AI visibility by up to 40% (Aggarwal et al.)" — paper real (KDD 2024), number real, SCOPE WRONG: 40% is the aggregate over the whole method set, domain-dependent. No per-technique figure exists. - "Pages not updated quarterly are 3x more likely to lose AI citations (LLMRefs)" — LLMrefs' actual 3x says brand mentions correlate ~3x more strongly with AI visibility than backlinks. DIFFERENT SUBJECT. No source supports a quarterly decay multiplier. - "QAPage cited 58% more often than Article" — uncited. Nearest real number: AccuraCast 2025, `Person` schema at 58.9% PREVALENCE among cited sources — wrong type, and its FAQPage figure (1.8%) points the opposite way to the claim it propped up. This one drove Tier 1 ranking. - "62% of searches involve voice" — uncited; 62% circulates as smart-speaker ADOPTION. Same family as the "50% by 2020" myth ComScore denied (origin: a 2014 Andrew Ng interview). Corrected my own framing too: I claimed three times these stats "drive axis weights". They do not — the weight tables carry no citations. They drive Tier/priority recommendations and, worse, geo-analyzer's "Cite sources" rule pushed them into CLIENT reports as research-backed. Fixes: recommendations kept on mechanism, fabricated numbers removed with the incident documented inline so they are not re-added. Unverified stats (48% AI Overviews, 2.5B queries/day, Gartner -25%) labelled [UNVERIFIED] rather than asserted or deleted — I did not check them. Structural, not just exhortation: resources/README.md now mandates `<claim> — <source, year, venue|vendor> — measured: <what the source ACTUALLY measured> — <link>`. `measured:` is the field that catches this — all four errors survive a source name; none survives stating the real measurement next to the claim. WebSearch demoted from verification to crawler/tool-name lookup only. Verified: make test 35 GREEN / 0 RED.
This commit is contained in:
@@ -61,9 +61,18 @@ query. A one-sentence self-contained answer has the highest density.
|
||||
|
||||
### 4. Citations and statistics (strongest measured lever)
|
||||
|
||||
Adding peer-cited statistics with clear sources increases AI visibility
|
||||
**by up to 40%** (Aggarwal et al., 2024 "GEO: Generative Engine
|
||||
Optimization").
|
||||
Aggarwal et al., 2024 ("GEO: Generative Engine Optimization", KDD 2024)
|
||||
report that their optimisation methods **collectively** boost visibility
|
||||
**by up to 40%** in generative-engine responses, and state the effect
|
||||
**varies across domains**. Citations/statistics/quotations are among those
|
||||
methods.
|
||||
|
||||
> **Attribute this correctly.** Until 2026-07-16 this section read "Adding
|
||||
> peer-cited statistics with clear sources increases AI visibility by up to
|
||||
> 40%" — pinning the paper's *aggregate* result on this *one* technique. The
|
||||
> paper publishes no separate figure per technique. When quoting it to a
|
||||
> client: "up to 40%, across the method set, domain-dependent" — never "+40%
|
||||
> if you add stats".
|
||||
|
||||
Pattern: embed specific numbers with attribution.
|
||||
|
||||
@@ -100,8 +109,20 @@ Comparison tables are even stronger. Structure:
|
||||
|
||||
### 6. Freshness signals
|
||||
|
||||
Pages not updated at least quarterly are **3x more likely to lose AI
|
||||
citations** (LLMRefs 2026 study).
|
||||
Freshness is a real retrieval input: RAG systems fetch live and read
|
||||
timestamps, so a page updated this quarter carries a stronger recency
|
||||
signal than the same page last touched years ago. LLMrefs (a **vendor**,
|
||||
not peer review) reports cited content running **~25.7% fresher** than
|
||||
organic top-10 across ~17M citations. Substantive updates only — bumping a
|
||||
date string is not freshness.
|
||||
|
||||
> **The "3x" that lived here was grafted from another claim.** Until
|
||||
> 2026-07-16 this read "Pages not updated at least quarterly are 3x more
|
||||
> likely to lose AI citations (LLMRefs 2026 study)". LLMrefs' actual "3x"
|
||||
> says **brand mentions correlate ~3x more strongly with AI visibility than
|
||||
> backlinks** — a different subject entirely. No source supports a quarterly
|
||||
> decay multiplier. Recommend quarterly refresh on its merits; do not price
|
||||
> it with a borrowed number.
|
||||
|
||||
What to maintain:
|
||||
- Visible "Last updated: YYYY-MM-DD" at the top of content pages
|
||||
|
||||
Reference in New Issue
Block a user