forked from bchanot/claude
fix(seo,geo): C1a — find sees build output, grep does not; the two disagreed
Surfaced by dogfooding on a real Astro repo instead of reading the spec. Claude Code installs a shell function routing `grep` to ugrep with `--ignore-files`, so grep honours .gitignore and never descends into a gitignored dist/. `find` honours nothing. seo-analyzer uses both. Measured on zenquality (Astro, dist/ gitignored but built locally), seo-analyzer.md:497 returned 92 images, 45 of them under dist/ — every asset listed twice, source and generated copy, byte-identical. Two live consequences: - "top 20 by size" was ~10 real images dressed as 20. - Batch C (`cwebp -q 80 <img> -o <img>.webp`) could target dist/og-image.png; the .webp lands in dist/ and the `npm run build` the dispatcher runs to VERIFY the fix erases it. Fix lands, verification passes, nothing survives, report says applied. lib/source-scope.sh separates source from build output, framework-aware. `public/` is deliberately NOT excluded by default: it is Astro/Vite/Next SOURCE and holds favicon.ico, apple-touch-icon.png and robots.txt — the very files STEP 4 curls. It is build output only for Hugo/Gatsby, detected from config (legacy config.toml alone is ambiguous, so it needs archetypes/ too). Blanket-excluding it would blind the audit to its own resource checks. findargs emits one token per line and MUST be consumed via a quoted array. I shipped a flat-string version first and the dogfood caught it: the shell globs */dist/* against the CWD and passes the matches to find as search paths, which turned 90 hits into 135 and kept every dist/ file. Both the header and a functional test now pin that. Also: no bundle item may target build output, in either agent. That is the real safety net — even if some future find leaks a dist path, the fix cannot land there. Fix the source that generates the artifact; if the source cannot be found, that is a finding, not a reason to patch the artifact. SCOPE CORRECTION: the proposal claimed grep was auditing 86 generated files instead of 9 templates. That was FALSE — the ugrep shim already skips them. Killed my own premise before coding it; the real bug is narrower and lives in find only. Do NOT "fix" the grep lines to match: they are already correct, and adding these exclusions there would drop public/. Verified: 90 -> 45 images on the real repo, 0 dist survivors, public/ preserved (favicon.ico still visible); 34 new assertions PASS / 0 FAIL; full suite green; shellcheck clean. Test file addition used the documented one-shot config-edit sentinel.
This commit is contained in:
+10
-1
@@ -545,7 +545,8 @@ sample of a 300-page site says nothing about the other 294.
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Extract H1/H2/H3 from main pages to assess heading style
|
# Extract H1/H2/H3 from main pages to assess heading style
|
||||||
for f in index.html $(find . -maxdepth 3 -name "*.astro" -o -name "*.tsx" -o -name "*.md" -o -name "*.html" | head -10); do
|
mapfile -t FEXCL < <(bash ~/.claude/lib/source-scope.sh findargs) # C1a: skip build output
|
||||||
|
for f in index.html $(find . "${FEXCL[@]}" -maxdepth 3 \( -name "*.astro" -o -name "*.tsx" -o -name "*.md" -o -name "*.html" \) | head -10); do
|
||||||
echo "=== $f ==="
|
echo "=== $f ==="
|
||||||
grep -oE '<(h1|h2|h3)[^>]*>[^<]+</(h1|h2|h3)>|^#{1,3} .+' "$f" 2>/dev/null | head -20
|
grep -oE '<(h1|h2|h3)[^>]*>[^<]+</(h1|h2|h3)>|^#{1,3} .+' "$f" 2>/dev/null | head -20
|
||||||
done
|
done
|
||||||
@@ -970,6 +971,14 @@ PROCHAINE ETAPE : <highest-priority>
|
|||||||
NEVER `Write` on shared templates. `Write` is reserved for files
|
NEVER `Write` on shared templates. `Write` is reserved for files
|
||||||
you solely own: robots.txt, llms.txt, llms-full.txt. Full-template
|
you solely own: robots.txt, llms.txt, llms-full.txt. Full-template
|
||||||
refactor → escalate as user action in §11.
|
refactor → escalate as user action in §11.
|
||||||
|
- **NEVER emit a bundle item targeting build output (C1a).** No path under
|
||||||
|
`dist/ build/ .next/ .nuxt/ .output/ _site/ .astro/ .svelte-kit/ out/` —
|
||||||
|
run `bash ~/.claude/lib/source-scope.sh list` for the authoritative set.
|
||||||
|
Those files are regenerated: the `npm run build` the dispatcher runs to
|
||||||
|
VERIFY your fix is what erases it. The fix lands, verification passes,
|
||||||
|
nothing survives, and the report claims it was applied. Fix the SOURCE
|
||||||
|
template that generates the file. If you cannot find the source, that is
|
||||||
|
a finding — say so, do not patch the artifact.
|
||||||
- **Respect PERMISSIVE/RESTRICTIVE choice.** geo-analyzer defaults to
|
- **Respect PERMISSIVE/RESTRICTIVE choice.** geo-analyzer defaults to
|
||||||
PERMISSIVE (GEO's goal is AI visibility). Only switch if the client
|
PERMISSIVE (GEO's goal is AI visibility). Only switch if the client
|
||||||
explicitly flags premium/regulated content.
|
explicitly flags premium/regulated content.
|
||||||
|
|||||||
+36
-4
@@ -181,8 +181,9 @@ keep the two consistent.
|
|||||||
ls .htaccess nginx.conf netlify.toml vercel.json wrangler.toml 2>/dev/null
|
ls .htaccess nginx.conf netlify.toml vercel.json wrangler.toml 2>/dev/null
|
||||||
# SEO files
|
# SEO files
|
||||||
ls robots.txt sitemap.xml sitemap-index.xml sitemap-images.xml sitemap-videos.xml 2>/dev/null
|
ls robots.txt sitemap.xml sitemap-index.xml sitemap-images.xml sitemap-videos.xml 2>/dev/null
|
||||||
# Legal pages
|
# Legal pages — source only (C1a: find ignores .gitignore, grep does not)
|
||||||
find . -maxdepth 3 \( -iname "*mention*" -o -iname "*legal*" -o -iname "*confidentialite*" -o -iname "*privacy*" -o -iname "*cgv*" -o -iname "*cgu*" \) 2>/dev/null | head -10
|
mapfile -t FEXCL < <(bash ~/.claude/lib/source-scope.sh findargs)
|
||||||
|
find . "${FEXCL[@]}" -maxdepth 3 \( -iname "*mention*" -o -iname "*legal*" -o -iname "*confidentialite*" -o -iname "*privacy*" -o -iname "*cgv*" -o -iname "*cgu*" \) 2>/dev/null | head -10
|
||||||
# Analytics / trackers
|
# Analytics / trackers
|
||||||
grep -rl "gtag\|GTM-\|analytics\|matomo\|_paq\|plausible\|umami" --include="*.html" --include="*.js" --include="*.tsx" --include="*.astro" --include="*.php" . 2>/dev/null | head -10
|
grep -rl "gtag\|GTM-\|analytics\|matomo\|_paq\|plausible\|umami" --include="*.html" --include="*.js" --include="*.tsx" --include="*.astro" --include="*.php" . 2>/dev/null | head -10
|
||||||
# Cookie consent / CMP
|
# Cookie consent / CMP
|
||||||
@@ -493,10 +494,32 @@ grep -rE '<img[^>]*>' --include="*.html" --include="*.astro" --include="*.tsx" -
|
|||||||
# Images missing dimensions (CLS risk)
|
# Images missing dimensions (CLS risk)
|
||||||
grep -rE '<img[^>]*>' --include="*.html" --include="*.astro" --include="*.tsx" --include="*.jsx" --include="*.php" . 2>/dev/null | grep -vE 'width=|height=' | head -30
|
grep -rE '<img[^>]*>' --include="*.html" --include="*.astro" --include="*.tsx" --include="*.jsx" --include="*.php" . 2>/dev/null | grep -vE 'width=|height=' | head -30
|
||||||
|
|
||||||
# Check image asset sizes
|
# Check image asset sizes — source only, never build output (C1a)
|
||||||
find . -type f \( -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.png" -o -iname "*.gif" \) ! -path "./node_modules/*" ! -path "./.git/*" -printf "%s %p\n" 2>/dev/null | sort -rn | head -20
|
mapfile -t FEXCL < <(bash ~/.claude/lib/source-scope.sh findargs)
|
||||||
|
find . "${FEXCL[@]}" -type f \( -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.png" -o -iname "*.gif" \) -printf "%s %p\n" 2>/dev/null | sort -rn | head -20
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Why the guard, and why `find` specifically (C1a).** `grep` and `find`
|
||||||
|
disagree about this repo and you use both. Claude Code routes `grep` through
|
||||||
|
ugrep with `--ignore-files`, so it honours `.gitignore` and never descends
|
||||||
|
into a gitignored `dist/`. `find` honours nothing. Measured on a real Astro
|
||||||
|
repo: this command returned **92 images, 45 of them under `dist/`** — every
|
||||||
|
asset twice, source and generated copy, byte-identical. So "top 20 by size"
|
||||||
|
was ~10 real images dressed as 20, and a batch-C item
|
||||||
|
(`cwebp -q 80 <img> -o <img>.webp`) could target `dist/og-image.png`, whose
|
||||||
|
`.webp` the dispatcher's own `npm run build` then erases. The fix lands,
|
||||||
|
verification passes, nothing survives.
|
||||||
|
|
||||||
|
`FEXCL` MUST be consumed as a quoted array. `find . $FEXCL …` lets the shell
|
||||||
|
glob `*/dist/*` against the CWD and hand the matches to find as search paths
|
||||||
|
— that made the same run return 135 hits and kept every `dist/` file.
|
||||||
|
|
||||||
|
Do NOT add these exclusions to the `grep` lines: the shim already covers
|
||||||
|
them, `public/` is deliberately kept (it is Astro/Vite/Next SOURCE and holds
|
||||||
|
`favicon.ico`, `apple-touch-icon.png`, `robots.txt` — the very files STEP 4
|
||||||
|
curls), and it is build output only for Hugo/Gatsby, which the script
|
||||||
|
detects.
|
||||||
|
|
||||||
Flag images over 100 KB as compression candidates. WebP/AVIF preferred
|
Flag images over 100 KB as compression candidates. WebP/AVIF preferred
|
||||||
over JPEG/PNG.
|
over JPEG/PNG.
|
||||||
|
|
||||||
@@ -1195,6 +1218,15 @@ PROCHAINE ETAPE : <highest-priority>
|
|||||||
`Write` on shared templates. `Write` is reserved for files you
|
`Write` on shared templates. `Write` is reserved for files you
|
||||||
solely own: sitemap.xml, .htaccess, legal pages, new city/service
|
solely own: sitemap.xml, .htaccess, legal pages, new city/service
|
||||||
pages. Full-template refactor → escalate as user action in §11.
|
pages. Full-template refactor → escalate as user action in §11.
|
||||||
|
- **NEVER emit a bundle item targeting build output (C1a).** No path under
|
||||||
|
`dist/ build/ .next/ .nuxt/ .output/ _site/ .astro/ .svelte-kit/ out/` —
|
||||||
|
`bash ~/.claude/lib/source-scope.sh list` is the authoritative set. Those
|
||||||
|
files are regenerated: the `npm run build` the dispatcher runs to VERIFY
|
||||||
|
your fix is what erases it. The fix lands, verification passes, nothing
|
||||||
|
survives, and the report claims it was applied. This bites batch C hardest
|
||||||
|
(`cwebp -q 80 <img> -o <img>.webp` on a `dist/` asset writes a `.webp` the
|
||||||
|
next build deletes). Fix the SOURCE that generates the artifact; if you
|
||||||
|
cannot find it, that is a finding — say so, do not patch the artifact.
|
||||||
- **Landing page protection.** Zero visible change except meta tags,
|
- **Landing page protection.** Zero visible change except meta tags,
|
||||||
footer links, JSON-LD, image optimization.
|
footer links, JSON-LD, image optimization.
|
||||||
- **Preserve existing valid SEO.** Don't rewrite correct tags.
|
- **Preserve existing valid SEO.** Don't rewrite correct tags.
|
||||||
|
|||||||
@@ -0,0 +1,79 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Emit the directory exclusions that separate SOURCE from BUILD OUTPUT.
|
||||||
|
#
|
||||||
|
# EXCL="$(bash ~/.claude/lib/source-scope.sh grep)"
|
||||||
|
# grep -rl "gtag" $EXCL --include="*.html" . # note: $EXCL unquoted
|
||||||
|
#
|
||||||
|
# mapfile -t FEXCL < <(bash ~/.claude/lib/source-scope.sh findargs)
|
||||||
|
# find . "${FEXCL[@]}" -iname '*.jpg' -printf '%s %p\n' # quoted array!
|
||||||
|
#
|
||||||
|
# findargs emits ONE TOKEN PER LINE and MUST be consumed through a quoted
|
||||||
|
# array. A flat string does not work: `find . $FEXCL ...` lets the shell glob
|
||||||
|
# `*/dist/*` against the CWD before find ever sees it, and the matches are then
|
||||||
|
# passed as search PATHS. Measured on zenquality: that turned 90 hits into 135
|
||||||
|
# and kept every dist/ file. The array form passes each token literally.
|
||||||
|
#
|
||||||
|
# WHY: grep and find disagree about what is in the repo, and seo-analyzer uses
|
||||||
|
# both.
|
||||||
|
#
|
||||||
|
# grep → Claude Code installs a shell function routing grep to ugrep with
|
||||||
|
# `--ignore-files`, i.e. .gitignore-aware. A gitignored dist/ is
|
||||||
|
# invisible to it when recursing from `.`. Verified 2026-07-17.
|
||||||
|
# find → knows nothing about .gitignore. It sees everything.
|
||||||
|
#
|
||||||
|
# So on zenquality (Astro, dist/ gitignored, built locally) the spec's image
|
||||||
|
# audit at seo-analyzer.md:497 returns 92 images of which 45 live in dist/ —
|
||||||
|
# every asset listed twice, source and generated copy, identical bytes. Two
|
||||||
|
# real consequences:
|
||||||
|
# 1. "top 20 by size" is half generated duplicates: ~10 real images audited
|
||||||
|
# while 20 are claimed.
|
||||||
|
# 2. Batch C (`cwebp -q 80 <img> -o <img>.webp`) can target dist/og-image.png.
|
||||||
|
# The .webp lands in dist/ and the `npm run build` that /seo runs to VERIFY
|
||||||
|
# the fix erases it. The fix lands, verification passes, nothing survives.
|
||||||
|
#
|
||||||
|
# The grep side is already safe by accident — do NOT "fix" it to match find.
|
||||||
|
# `grep` mode below is defence in depth for the cases the shim misses: a repo
|
||||||
|
# that COMMITS its build output (no .gitignore entry to honour), or a directory
|
||||||
|
# that is not a git repo at all.
|
||||||
|
#
|
||||||
|
# `public/` is deliberately NOT in the always-list: it is SOURCE for
|
||||||
|
# Astro/Vite/Next and holds the very files this audit checks — favicon.ico,
|
||||||
|
# apple-touch-icon.png, robots.txt, OG images. It is build OUTPUT only for
|
||||||
|
# Hugo and Gatsby, detected below. Blanket-excluding it would blind the audit
|
||||||
|
# to its own resource checks.
|
||||||
|
#
|
||||||
|
# Exclusions are by NAME, not path, so a monorepo's frontend/dist is caught
|
||||||
|
# exactly like a root ./dist.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
_die() { echo "source-scope: $1" >&2; exit 2; }
|
||||||
|
|
||||||
|
# Build output + tool caches. Never source.
|
||||||
|
ALWAYS=(node_modules .git dist build .next .nuxt .output _site .astro
|
||||||
|
.svelte-kit .cache out coverage .vercel .netlify .turbo)
|
||||||
|
|
||||||
|
# public/ is output for exactly these two generators.
|
||||||
|
_public_is_output() {
|
||||||
|
find . -maxdepth 3 \( -name "gatsby-config.js" -o -name "gatsby-config.ts" \
|
||||||
|
-o -name "gatsby-config.mjs" -o -name "hugo.toml" -o -name "hugo.yaml" \
|
||||||
|
-o -name "hugo.json" \) 2>/dev/null | read -r _ && return 0
|
||||||
|
# Hugo's legacy config.toml is ambiguous on its own — pair it with archetypes/
|
||||||
|
[ -d ./archetypes ] && [ -f ./config.toml ] && return 0
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
_list() {
|
||||||
|
printf '%s\n' "${ALWAYS[@]}"
|
||||||
|
_public_is_output && printf 'public\n'
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
case "${1:-}" in
|
||||||
|
list) _list ;;
|
||||||
|
# Safe unquoted: --exclude-dir=NAME carries no glob character.
|
||||||
|
grep) _list | while read -r d; do printf -- '--exclude-dir=%s ' "$d"; done; echo ;;
|
||||||
|
# One token per line — consume with mapfile + a QUOTED array, never a flat
|
||||||
|
# string (see header: the shell would glob */dist/* against the CWD).
|
||||||
|
findargs) _list | while read -r d; do printf '!\n-path\n*/%s/*\n' "$d"; done ;;
|
||||||
|
*) _die "usage: source-scope.sh {list|grep|findargs}" ;;
|
||||||
|
esac
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# lib/tests/source-scope.test.sh
|
||||||
|
set -u
|
||||||
|
S="$(cd "$(dirname "$0")/../.." && pwd)/lib/source-scope.sh"
|
||||||
|
pass=0; fail=0
|
||||||
|
check() { if [ "$2" = "$3" ]; then pass=$((pass+1)); else fail=$((fail+1));
|
||||||
|
printf 'FAIL %s: got[%s] want[%s]\n' "$1" "$2" "$3"; fi; }
|
||||||
|
# does `list` (run inside dir $1) contain the name $2?
|
||||||
|
listed() { ( cd "$1" && bash "$S" list 2>/dev/null | grep -qxF "$2" ) \
|
||||||
|
&& echo yes || echo no; }
|
||||||
|
|
||||||
|
TMP="$(mktemp -d)"
|
||||||
|
|
||||||
|
# --- always-excluded build output + caches ---
|
||||||
|
mkdir -p "$TMP/plain"
|
||||||
|
for d in node_modules .git dist build .next .nuxt .output _site .astro \
|
||||||
|
.svelte-kit .cache out coverage .vercel .netlify .turbo; do
|
||||||
|
check "A-$d-listed" "$(listed "$TMP/plain" "$d")" yes
|
||||||
|
done
|
||||||
|
|
||||||
|
# --- public/ is SOURCE by default: Astro/Vite/Next keep favicon.ico,
|
||||||
|
# apple-touch-icon.png and robots.txt there, and the audit checks them ---
|
||||||
|
check B1-public-kept-by-default "$(listed "$TMP/plain" public)" no
|
||||||
|
|
||||||
|
# --- public/ is OUTPUT for Gatsby and Hugo only ---
|
||||||
|
mkdir -p "$TMP/gatsby"; : > "$TMP/gatsby/gatsby-config.js"
|
||||||
|
check B2-gatsby-js "$(listed "$TMP/gatsby" public)" yes
|
||||||
|
mkdir -p "$TMP/gatsby2"; : > "$TMP/gatsby2/gatsby-config.ts"
|
||||||
|
check B3-gatsby-ts "$(listed "$TMP/gatsby2" public)" yes
|
||||||
|
mkdir -p "$TMP/hugo"; : > "$TMP/hugo/hugo.toml"
|
||||||
|
check B4-hugo-toml "$(listed "$TMP/hugo" public)" yes
|
||||||
|
mkdir -p "$TMP/hugo2"; : > "$TMP/hugo2/hugo.yaml"
|
||||||
|
check B5-hugo-yaml "$(listed "$TMP/hugo2" public)" yes
|
||||||
|
# legacy config.toml alone is ambiguous (many tools use it) — needs archetypes/
|
||||||
|
mkdir -p "$TMP/amb"; : > "$TMP/amb/config.toml"
|
||||||
|
check B6-config-toml-alone-is-ambiguous "$(listed "$TMP/amb" public)" no
|
||||||
|
mkdir -p "$TMP/hugo3/archetypes"; : > "$TMP/hugo3/config.toml"
|
||||||
|
check B7-config-toml-plus-archetypes "$(listed "$TMP/hugo3" public)" yes
|
||||||
|
|
||||||
|
# --- grep mode: flags, and no glob character (safe unquoted) ---
|
||||||
|
G="$(cd "$TMP/plain" && bash "$S" grep)"
|
||||||
|
case "$G" in *--exclude-dir=dist*) check C1-grep-has-dist ok ok ;;
|
||||||
|
*) check C1-grep-has-dist "missing" ok ;; esac
|
||||||
|
case "$G" in *"*"*) check C2-grep-has-no-glob "has-glob" ok ;;
|
||||||
|
*) check C2-grep-has-no-glob ok ok ;; esac
|
||||||
|
|
||||||
|
# --- findargs: one token per line, 3 tokens per dir ---
|
||||||
|
N="$(cd "$TMP/plain" && bash "$S" findargs | wc -l)"
|
||||||
|
D="$(cd "$TMP/plain" && bash "$S" list | wc -l)"
|
||||||
|
check D1-findargs-3-tokens-per-dir "$N" "$((D * 3))"
|
||||||
|
check D2-findargs-first-token "$(cd "$TMP/plain" && bash "$S" findargs | head -1)" '!'
|
||||||
|
|
||||||
|
# --- FUNCTIONAL: the array form actually excludes build output ---
|
||||||
|
# A flat unquoted string does NOT work here: the shell globs */dist/* against
|
||||||
|
# the CWD and passes the matches to find as search paths. Measured on a real
|
||||||
|
# repo, that turned 90 hits into 135 and kept every dist/ file.
|
||||||
|
W="$TMP/work"; mkdir -p "$W/src" "$W/dist" "$W/public" "$W/node_modules"
|
||||||
|
: > "$W/src/a.png"; : > "$W/dist/a.png"; : > "$W/public/favicon.ico"
|
||||||
|
: > "$W/node_modules/dep.png"
|
||||||
|
cd "$W" || exit 1
|
||||||
|
mapfile -t FEXCL < <(bash "$S" findargs)
|
||||||
|
check E1-excludes-dist "$(find . "${FEXCL[@]}" -name 'a.png' | grep -c '/dist/')" 0
|
||||||
|
check E2-keeps-src "$(find . "${FEXCL[@]}" -name 'a.png' | grep -c '/src/')" 1
|
||||||
|
check E3-excludes-nodem "$(find . "${FEXCL[@]}" -name '*.png' | grep -c 'node_modules')" 0
|
||||||
|
# public/ survives: the audit's own resource checks live there
|
||||||
|
check E4-keeps-public "$(find . "${FEXCL[@]}" -name 'favicon.ico' | wc -l)" 1
|
||||||
|
cd / || exit 1
|
||||||
|
|
||||||
|
# --- usage ---
|
||||||
|
bash "$S" >/dev/null 2>&1; check X1-no-args "$?" 2
|
||||||
|
bash "$S" bogus >/dev/null 2>&1; check X2-bad-verb "$?" 2
|
||||||
|
# `find` was renamed to `findargs` when the flat-string form proved unsafe
|
||||||
|
bash "$S" find >/dev/null 2>&1; check X3-old-find-verb-gone "$?" 2
|
||||||
|
|
||||||
|
rm -rf "$TMP"
|
||||||
|
printf 'PASS=%s FAIL=%s\n' "$pass" "$fail"; [ "$fail" -eq 0 ]
|
||||||
Reference in New Issue
Block a user