forked from bchanot/claude
Bastien Chanot
9da1dec9e6
Audited each statistic in agents/resources/ against primary sources after the VSI fiction (I2) showed WebSearch launders SEO-blog consensus. The failure mode is not invention — it is plausible recombination, which is what a model half-remembering a search result produces: - "Adding statistics increases AI visibility by up to 40% (Aggarwal et al.)" — paper real (KDD 2024), number real, SCOPE WRONG: 40% is the aggregate over the whole method set, domain-dependent. No per-technique figure exists. - "Pages not updated quarterly are 3x more likely to lose AI citations (LLMRefs)" — LLMrefs' actual 3x says brand mentions correlate ~3x more strongly with AI visibility than backlinks. DIFFERENT SUBJECT. No source supports a quarterly decay multiplier. - "QAPage cited 58% more often than Article" — uncited. Nearest real number: AccuraCast 2025, `Person` schema at 58.9% PREVALENCE among cited sources — wrong type, and its FAQPage figure (1.8%) points the opposite way to the claim it propped up. This one drove Tier 1 ranking. - "62% of searches involve voice" — uncited; 62% circulates as smart-speaker ADOPTION. Same family as the "50% by 2020" myth ComScore denied (origin: a 2014 Andrew Ng interview). Corrected my own framing too: I claimed three times these stats "drive axis weights". They do not — the weight tables carry no citations. They drive Tier/priority recommendations and, worse, geo-analyzer's "Cite sources" rule pushed them into CLIENT reports as research-backed. Fixes: recommendations kept on mechanism, fabricated numbers removed with the incident documented inline so they are not re-added. Unverified stats (48% AI Overviews, 2.5B queries/day, Gartner -25%) labelled [UNVERIFIED] rather than asserted or deleted — I did not check them. Structural, not just exhortation: resources/README.md now mandates `<claim> — <source, year, venue|vendor> — measured: <what the source ACTUALLY measured> — <link>`. `measured:` is the field that catches this — all four errors survive a source name; none survives stating the real measurement next to the claim. WebSearch demoted from verification to crawler/tool-name lookup only. Verified: make test 35 GREEN / 0 RED.
108 lines
5.1 KiB
Markdown
108 lines
5.1 KiB
Markdown
# AI visibility monitoring tools — 2026
|
|
|
|
Tools that track whether your brand appears in AI-generated answers
|
|
across ChatGPT, Perplexity, Gemini, Copilot, Claude, and Google AI
|
|
Overviews.
|
|
|
|
Context `[UNVERIFIED — 2026-07-16]`: Google AI Overviews trigger on ~48% of
|
|
searches; ChatGPT processes 2.5B queries/day; Gartner projects commercial
|
|
organic search traffic will drop 25% by 2026.
|
|
|
|
> Not checked against primary sources in the 2026-07-16 audit that corrected
|
|
> the rest of this directory — flagged rather than asserted or deleted, per
|
|
> the citation standard in `README.md` (rule 5). The Gartner projection at
|
|
> least names its source; the other two float. Treat all three as
|
|
> motivation, not evidence: **do NOT quote them to a client** until each
|
|
> carries `source + measured: + link`. Their only job here is to explain why
|
|
> this file exists, and that argument does not need numbers.
|
|
|
|
## Commercial tools
|
|
|
|
| Tool | Platforms covered | Strong points | Weak points |
|
|
|---|---|---|---|
|
|
| **OtterlyAI** (otterly.ai) | ChatGPT, Perplexity, Gemini, AI Overviews, Copilot | Mature, 20k+ users, Gartner-recognised | Pricing mid-to-high |
|
|
| **Peec AI** (peec.ai) | ChatGPT, Perplexity, Gemini, AI Overviews | Good SaaS-brand focus, sentiment analysis | Narrower platform scope |
|
|
| **Profound** (tryprofound.com) | ChatGPT, Perplexity, Gemini, Copilot | Enterprise-grade, full-response capture | Enterprise pricing |
|
|
| **ZipTie** (ziptie.dev) | ChatGPT, Perplexity, AI Overviews | Competitive benchmarking, source attribution | Smaller team, newer |
|
|
| **HubSpot AEO** (hubspot.com/products/aeo) | ChatGPT, Gemini, Perplexity | Integrates with HubSpot ecosystem | Best if already HubSpot user |
|
|
| **Trendos** (trendos by Tesonet) | ChatGPT, Gemini, AI Search, Perplexity, DeepSeek | Added DeepSeek coverage, 2026 launch | Unproven longevity |
|
|
| **SE Ranking AI Tracker** (seranking.com) | ChatGPT, Perplexity, Gemini, AI Mode, AI Overviews | Bundled with classical SEO suite | Less specialised |
|
|
| **LLMrefs** (llmrefs.com) | ChatGPT, Perplexity, Gemini, Claude | GEO focus, research-backed | Newer, less tested |
|
|
|
|
## Free / manual methods (zero budget)
|
|
|
|
For clients/projects with no monitoring budget, a manual process works
|
|
at lower frequency. Recommended cadence: monthly for established
|
|
brands, weekly during optimization sprints.
|
|
|
|
### Query list construction
|
|
|
|
Build a list of 20-40 queries covering:
|
|
|
|
1. **Branded queries** — "what is [brand]", "is [brand] good", "[brand] reviews"
|
|
2. **Generic category queries** — "best [category] in [location]", "how to [problem]"
|
|
3. **Comparison queries** — "[brand] vs [competitor]", "alternatives to [brand]"
|
|
4. **Problem queries** — the actual questions the target persona asks
|
|
|
|
### Manual check workflow
|
|
|
|
For each query, run across:
|
|
|
|
- **ChatGPT** (web version with search enabled, chatgpt.com)
|
|
- **Perplexity** (perplexity.ai)
|
|
- **Google AI Overviews** (google.com — appears for ~48% of searches)
|
|
- **Claude** (claude.ai with web search)
|
|
- **Gemini** (gemini.google.com)
|
|
- **Copilot** (copilot.microsoft.com)
|
|
- **Brave Search AI** (search.brave.com)
|
|
- **DuckAssist** (duckduckgo.com)
|
|
|
|
Record for each:
|
|
- Mentioned? (yes/no)
|
|
- Cited with link? (yes/no + which page)
|
|
- Position in answer? (1st mention / buried / listed)
|
|
- Sentiment? (positive / neutral / negative / misleading)
|
|
|
|
### Spreadsheet template
|
|
|
|
| Date | Query | ChatGPT | Perplexity | Google AIO | Claude | Gemini | Copilot |
|
|
|---|---|---|---|---|---|---|---|
|
|
| 2026-04-21 | best plombier Évry | Mentioned, ranked 3, cited | Not mentioned | Top 3, no cite | — | — | — |
|
|
|
|
## KPIs to track
|
|
|
|
From GEO research and industry consensus (GenOptima, HubSpot 2026):
|
|
|
|
| Metric | Definition | Benchmark |
|
|
|---|---|---|
|
|
| **Mention Rate** | % of AI answers that mention brand name | Varies; track trend, not absolute |
|
|
| **Citation Rate** | % of AI answers with a clickable link to domain | Target 20%+ for established brands |
|
|
| **Position** | When cited, is brand 1st mention vs buried? | First mention = best |
|
|
| **Sentiment** | Tone of brand mention (positive/neutral/negative) | Track for negative drift |
|
|
| **Source Diversity** | Which of your pages get cited? | Aim for 5+ distinct pages/domain |
|
|
| **Competitor Share** | % of category queries where competitor cited vs brand | Track gap |
|
|
|
|
## Integration into SEO.md
|
|
|
|
In `SEO.md §11 — Actions utilisateur requises`:
|
|
|
|
> ### Monitor AI visibility monthly
|
|
>
|
|
> **Automatisation possible avec:** OtterlyAI, Peec AI, ZipTie, HubSpot
|
|
> AEO, SE Ranking AI Tracker. Budget: 50-500 EUR/mois selon le tool.
|
|
>
|
|
> **Alternative manuelle gratuite:** template spreadsheet + 20 queries
|
|
> testées mensuellement sur ChatGPT, Perplexity, Google AI Overviews.
|
|
> Temps: ~1h/mois.
|
|
|
|
## Methodology caveats
|
|
|
|
- AI engines are **non-deterministic**. Same query twice can return
|
|
different answers. Always take 3 samples and track the median.
|
|
- **Personalisation** affects results. Test in logged-out / private
|
|
mode for reproducibility.
|
|
- **Geographic bias** — ChatGPT's answers about local businesses vary
|
|
by IP. Test from the target market's geography.
|
|
- **Freshness lag** — content updates take days to weeks to propagate
|
|
into AI answers. Don't expect instant reflection of changes.
|