Bastien Chanot
9da1dec9e6
Audited each statistic in agents/resources/ against primary sources after the VSI fiction (I2) showed WebSearch launders SEO-blog consensus. The failure mode is not invention — it is plausible recombination, which is what a model half-remembering a search result produces: - "Adding statistics increases AI visibility by up to 40% (Aggarwal et al.)" — paper real (KDD 2024), number real, SCOPE WRONG: 40% is the aggregate over the whole method set, domain-dependent. No per-technique figure exists. - "Pages not updated quarterly are 3x more likely to lose AI citations (LLMRefs)" — LLMrefs' actual 3x says brand mentions correlate ~3x more strongly with AI visibility than backlinks. DIFFERENT SUBJECT. No source supports a quarterly decay multiplier. - "QAPage cited 58% more often than Article" — uncited. Nearest real number: AccuraCast 2025, `Person` schema at 58.9% PREVALENCE among cited sources — wrong type, and its FAQPage figure (1.8%) points the opposite way to the claim it propped up. This one drove Tier 1 ranking. - "62% of searches involve voice" — uncited; 62% circulates as smart-speaker ADOPTION. Same family as the "50% by 2020" myth ComScore denied (origin: a 2014 Andrew Ng interview). Corrected my own framing too: I claimed three times these stats "drive axis weights". They do not — the weight tables carry no citations. They drive Tier/priority recommendations and, worse, geo-analyzer's "Cite sources" rule pushed them into CLIENT reports as research-backed. Fixes: recommendations kept on mechanism, fabricated numbers removed with the incident documented inline so they are not re-added. Unverified stats (48% AI Overviews, 2.5B queries/day, Gartner -25%) labelled [UNVERIFIED] rather than asserted or deleted — I did not check them. Structural, not just exhortation: resources/README.md now mandates `<claim> — <source, year, venue|vendor> — measured: <what the source ACTUALLY measured> — <link>`. `measured:` is the field that catches this — all four errors survive a source name; none survives stating the real measurement next to the claim. WebSearch demoted from verification to crawler/tool-name lookup only. Verified: make test 35 GREEN / 0 RED.
5.1 KiB
AI visibility monitoring tools — 2026
Tools that track whether your brand appears in AI-generated answers across ChatGPT, Perplexity, Gemini, Copilot, Claude, and Google AI Overviews.
Context [UNVERIFIED — 2026-07-16]: Google AI Overviews trigger on ~48% of
searches; ChatGPT processes 2.5B queries/day; Gartner projects commercial
organic search traffic will drop 25% by 2026.
Not checked against primary sources in the 2026-07-16 audit that corrected the rest of this directory — flagged rather than asserted or deleted, per the citation standard in
README.md(rule 5). The Gartner projection at least names its source; the other two float. Treat all three as motivation, not evidence: do NOT quote them to a client until each carriessource + measured: + link. Their only job here is to explain why this file exists, and that argument does not need numbers.
Commercial tools
| Tool | Platforms covered | Strong points | Weak points |
|---|---|---|---|
| OtterlyAI (otterly.ai) | ChatGPT, Perplexity, Gemini, AI Overviews, Copilot | Mature, 20k+ users, Gartner-recognised | Pricing mid-to-high |
| Peec AI (peec.ai) | ChatGPT, Perplexity, Gemini, AI Overviews | Good SaaS-brand focus, sentiment analysis | Narrower platform scope |
| Profound (tryprofound.com) | ChatGPT, Perplexity, Gemini, Copilot | Enterprise-grade, full-response capture | Enterprise pricing |
| ZipTie (ziptie.dev) | ChatGPT, Perplexity, AI Overviews | Competitive benchmarking, source attribution | Smaller team, newer |
| HubSpot AEO (hubspot.com/products/aeo) | ChatGPT, Gemini, Perplexity | Integrates with HubSpot ecosystem | Best if already HubSpot user |
| Trendos (trendos by Tesonet) | ChatGPT, Gemini, AI Search, Perplexity, DeepSeek | Added DeepSeek coverage, 2026 launch | Unproven longevity |
| SE Ranking AI Tracker (seranking.com) | ChatGPT, Perplexity, Gemini, AI Mode, AI Overviews | Bundled with classical SEO suite | Less specialised |
| LLMrefs (llmrefs.com) | ChatGPT, Perplexity, Gemini, Claude | GEO focus, research-backed | Newer, less tested |
Free / manual methods (zero budget)
For clients/projects with no monitoring budget, a manual process works at lower frequency. Recommended cadence: monthly for established brands, weekly during optimization sprints.
Query list construction
Build a list of 20-40 queries covering:
- Branded queries — "what is [brand]", "is [brand] good", "[brand] reviews"
- Generic category queries — "best [category] in [location]", "how to [problem]"
- Comparison queries — "[brand] vs [competitor]", "alternatives to [brand]"
- Problem queries — the actual questions the target persona asks
Manual check workflow
For each query, run across:
- ChatGPT (web version with search enabled, chatgpt.com)
- Perplexity (perplexity.ai)
- Google AI Overviews (google.com — appears for ~48% of searches)
- Claude (claude.ai with web search)
- Gemini (gemini.google.com)
- Copilot (copilot.microsoft.com)
- Brave Search AI (search.brave.com)
- DuckAssist (duckduckgo.com)
Record for each:
- Mentioned? (yes/no)
- Cited with link? (yes/no + which page)
- Position in answer? (1st mention / buried / listed)
- Sentiment? (positive / neutral / negative / misleading)
Spreadsheet template
| Date | Query | ChatGPT | Perplexity | Google AIO | Claude | Gemini | Copilot |
|---|---|---|---|---|---|---|---|
| 2026-04-21 | best plombier Évry | Mentioned, ranked 3, cited | Not mentioned | Top 3, no cite | — | — | — |
KPIs to track
From GEO research and industry consensus (GenOptima, HubSpot 2026):
| Metric | Definition | Benchmark |
|---|---|---|
| Mention Rate | % of AI answers that mention brand name | Varies; track trend, not absolute |
| Citation Rate | % of AI answers with a clickable link to domain | Target 20%+ for established brands |
| Position | When cited, is brand 1st mention vs buried? | First mention = best |
| Sentiment | Tone of brand mention (positive/neutral/negative) | Track for negative drift |
| Source Diversity | Which of your pages get cited? | Aim for 5+ distinct pages/domain |
| Competitor Share | % of category queries where competitor cited vs brand | Track gap |
Integration into SEO.md
In SEO.md §11 — Actions utilisateur requises:
Monitor AI visibility monthly
Automatisation possible avec: OtterlyAI, Peec AI, ZipTie, HubSpot AEO, SE Ranking AI Tracker. Budget: 50-500 EUR/mois selon le tool.
Alternative manuelle gratuite: template spreadsheet + 20 queries testées mensuellement sur ChatGPT, Perplexity, Google AI Overviews. Temps: ~1h/mois.
Methodology caveats
- AI engines are non-deterministic. Same query twice can return different answers. Always take 3 samples and track the median.
- Personalisation affects results. Test in logged-out / private mode for reproducibility.
- Geographic bias — ChatGPT's answers about local businesses vary by IP. Test from the target market's geography.
- Freshness lag — content updates take days to weeks to propagate into AI answers. Don't expect instant reflection of changes.