forked from bchanot/claude
fix(geo): I6 — every stat was real and attached to the wrong claim
Audited each statistic in agents/resources/ against primary sources after the VSI fiction (I2) showed WebSearch launders SEO-blog consensus. The failure mode is not invention — it is plausible recombination, which is what a model half-remembering a search result produces: - "Adding statistics increases AI visibility by up to 40% (Aggarwal et al.)" — paper real (KDD 2024), number real, SCOPE WRONG: 40% is the aggregate over the whole method set, domain-dependent. No per-technique figure exists. - "Pages not updated quarterly are 3x more likely to lose AI citations (LLMRefs)" — LLMrefs' actual 3x says brand mentions correlate ~3x more strongly with AI visibility than backlinks. DIFFERENT SUBJECT. No source supports a quarterly decay multiplier. - "QAPage cited 58% more often than Article" — uncited. Nearest real number: AccuraCast 2025, `Person` schema at 58.9% PREVALENCE among cited sources — wrong type, and its FAQPage figure (1.8%) points the opposite way to the claim it propped up. This one drove Tier 1 ranking. - "62% of searches involve voice" — uncited; 62% circulates as smart-speaker ADOPTION. Same family as the "50% by 2020" myth ComScore denied (origin: a 2014 Andrew Ng interview). Corrected my own framing too: I claimed three times these stats "drive axis weights". They do not — the weight tables carry no citations. They drive Tier/priority recommendations and, worse, geo-analyzer's "Cite sources" rule pushed them into CLIENT reports as research-backed. Fixes: recommendations kept on mechanism, fabricated numbers removed with the incident documented inline so they are not re-added. Unverified stats (48% AI Overviews, 2.5B queries/day, Gartner -25%) labelled [UNVERIFIED] rather than asserted or deleted — I did not check them. Structural, not just exhortation: resources/README.md now mandates `<claim> — <source, year, venue|vendor> — measured: <what the source ACTUALLY measured> — <link>`. `measured:` is the field that catches this — all four errors survive a source name; none survives stating the real measurement next to the claim. WebSearch demoted from verification to crawler/tool-name lookup only. Verified: make test 35 GREEN / 0 RED.
This commit is contained in:
@@ -4,9 +4,17 @@ Tools that track whether your brand appears in AI-generated answers
|
||||
across ChatGPT, Perplexity, Gemini, Copilot, Claude, and Google AI
|
||||
Overviews.
|
||||
|
||||
Context: Google AI Overviews trigger on ~48% of searches; ChatGPT
|
||||
processes 2.5B queries/day; Gartner projects commercial organic
|
||||
search traffic will drop 25% by 2026. Monitoring is no longer optional.
|
||||
Context `[UNVERIFIED — 2026-07-16]`: Google AI Overviews trigger on ~48% of
|
||||
searches; ChatGPT processes 2.5B queries/day; Gartner projects commercial
|
||||
organic search traffic will drop 25% by 2026.
|
||||
|
||||
> Not checked against primary sources in the 2026-07-16 audit that corrected
|
||||
> the rest of this directory — flagged rather than asserted or deleted, per
|
||||
> the citation standard in `README.md` (rule 5). The Gartner projection at
|
||||
> least names its source; the other two float. Treat all three as
|
||||
> motivation, not evidence: **do NOT quote them to a client** until each
|
||||
> carries `source + measured: + link`. Their only job here is to explain why
|
||||
> this file exists, and that argument does not need numbers.
|
||||
|
||||
## Commercial tools
|
||||
|
||||
|
||||
Reference in New Issue
Block a user