fix(supabase): e2e:avatar runs on the sandbox only, DEV runner removed

scripts/avatar-e2e.sh read the DEV keys through `supabase status`, started
the worker on the DEV network and let the suite write and purge on DEV.
Port the V1-2 sandbox runner (`db-sandbox.sh avatar-e2e`: loopback gateway,
worker in a 600 env file) and the guarded suite (fixed URL, container and
key issuer, per-run accounts); refusals and the positive run proven under
strace in .claude/audits/2026-11-02-avatar-e2e-sandbox-only.md.
This commit is contained in:
bchanot committed 2026-11-02 10:45:03 +01:00
1 parent 6287e5bb9f
commit 3a4614bc86
17 files changed
+589 -117

No files matched your search

@@ -0,0 +1,198 @@
# PLAN — bugfix: `pnpm e2e:avatar` on develop can no longer reach DEV
- date: 2026-11-02 | flow: bugfix | branch: bugfix/avatar-e2e-sandbox-only (off develop 6287e5b)
- revision 3: three challengers (correctness, robustness, simplicity: CONCERNS each, no BLOCKER), then one
confirmation challenger (robustness, CONCERNS(5)); every MAJOR folded in as a named change
- worktree: /tmp/claude-1000/-home-bchanot-Documents-faunosteo/573c6c77-7294-4d96-a518-83afaafef8d1/scratchpad/bugfix-avatar
- reference (safe runner already built): feature/v1-2-anatomical-sheets at a15c35c, worktree
/tmp/claude-1000/-home-bchanot-Documents-faunosteo/e641af9a-743e-46d0-9804-99667836040e/scratchpad/v1-2
(commits cdd43ac, 2a84e37, 3c141c9, f924ad0)
## Bug
On develop, `pnpm e2e:avatar` = `bash scripts/avatar-e2e.sh`:
- reads API_URL, ANON_KEY, SERVICE_ROLE_KEY from `supabase status -o env` (the DEV stack), lines 12-23;
- starts the worker container on `supabase_network_faunosteo` against `http://supabase_kong_faunosteo:8000`
(DEV Kong) with a 5 s sweep and a 0 s quarantine grace;
- runs `packages/supabase/tests/e2e/avatar-flow.e2e.ts` with those keys: creates a confirmed account on DEV,
signs in as the seeded `bob@faunosteo.local` with the DEV password written in the suite
(`LocalDev-only-password-1`), uploads, and `expeditePurges()` moves EVERY due date of DEV's
`private.object_purge_queue` to now; `psql` goes to `supabase_db_faunosteo` by default (suite :29-31, :175, :215).
Root cause: neither the runner nor the suite knows an authorised target; URL, keys and container come
from the environment or from DEV defaults. Same family as the sheet-assets incidents of 2026-11-01
(fixed on develop by 6287e5b).
## Decisions already taken (constraints)
- Owner brief 2026-11-02 §1: minimal port of the V1-2 runner if its dependencies exist on develop, else an
explicit refusal; no DEV access (read or write), no implicit key recovery through `supabase status`, no
fallback to DEV URL / database / accounts / buckets; refusal before any connection when the configuration
is absent or invalid; refusals proven without any IP connection; independent security review; merge into
develop only (never V1-2).
- Owner brief §3: a JWT issuer check is not a signature check; never present it as full cryptographic
authentication nor as the only protection; strict destination restriction + sandbox isolation stay
mandatory; check that redirects, inherited variables and ambiguous configurations cannot leave the sandbox.
- Scope answer (gated 2026-11-02): avatar only; `cmd_sheet_assets_e2e` is NOT ported (`pnpm e2e:sheet-assets`
keeps refusing on develop until the V1-2 merge).
- LRN-110: absolute verified cwd for every command, no `cd X && cmd &`, quoted heredocs, env validated inside
the script; agents never run the old scripts (develop's `scripts/avatar-e2e.sh`, the main checkout's or
old-958228a's copies).
- Each ported block is copied from the V1-2 text (same bytes) where develop has no conflicting need; this
does NOT make the later V1-2 → develop merge conflict-free (see "Merge expectations").
Dependencies checked on develop: `scripts/db-sandbox.sh` (network, db/auth/rest/storage/mail containers,
keys signed with issuer `db-sandbox` :80-81, `load_secrets` :50, `wait_for` :63, `with_sandbox_env` :229);
the worker's default `run` logs `"event":"worker_start"` (`crates/worker/src/run.rs:121`, `main.rs:21-29`:
`run` takes no argument on develop); the image is `static-debian13:nonroot` with `USER nonroot`
(`crates/worker/Dockerfile:37-39`); `apps/app/e2e/support/sandbox.ts:111` creates its accounts with random
passwords; `packages/supabase/src/assets.ts` identical on both branches.
NOT ported (V1-2 only, with the reason): `SB_SEED_PW` / `seed_passwords` / `SANDBOX_SEED_PASSWORD`
(develop's `packages/supabase/tests/sandbox/workspaces.sandbox.ts:197-199` signs in to
`owner@faunosteo.local` with the seed password: rotating it would break `pnpm db:sandbox test`),
`throwaway_password`, `verify-e2e` / `cmd_verify_e2e` (needs `run --jobs sheet-verify`, V1-2 worker only),
`test` args pass-through, `cmd_sheet_assets_e2e` (scope answer).
## Fix plan (files in the bugfix worktree)
1. `scripts/db-sandbox.sh` — port from V1-2 a15c35c, copied by function name, bytes identical:
- usage header: insert after develop line 16 the V1-2 `avatar-e2e [args]` three lines only (V1-2 :20-22);
the usage fallback becomes `sed -n '2,26p'` (header grows by 3 lines);
- variables: after `MAIL=…` insert V1-2 :47-51 verbatim (`GATEWAY=`, `WORKER=faunosteo_dbsandbox_worker_verify`,
`AVATAR_WORKER=`, `GATEWAY_IMAGE=nginx:1.31-alpine`, `WORKER_IMAGE=faunosteo_worker:dev`); the PORT line
becomes V1-2 :52 (`PORT_GATEWAY=57800` appended). `WORKER=` is inert on develop; it lets `down` remove a
verify worker left by an interrupted V1-2 run (shared Docker names) and keeps the line identical;
- the `REPO="$(realpath "$PWD")"` check: V1-2 :58-62 (comment, `REPO=`, `case`, pattern, `esac`) replace develop :45-47 (a STATE inside the repository
reached through a symlink no longer passes; the port adds `$STATE/worker.XXXXXX` files holding the
sandbox service key);
- `cmd_down`: V1-2 :255 verbatim (`"$AVATAR_WORKER" "$WORKER" "$GATEWAY" "$REST" …`);
- after `with_sandbox_env`: V1-2 :270-332 (`GATEWAY_CONF`, `start_gateway`, `start_worker`,
`print_worker_log`, `worker_cleanup`, `worker_preflight`) and :342-359 (`cmd_avatar_e2e`, the comment
above it through the closing brace, the suite invocation :356-358 included); NOT :333-341
(`cmd_verify_e2e`) nor :361-376 (`cmd_sheet_assets_e2e`);
- dispatcher: `avatar-e2e) shift; cmd_avatar_e2e "$@" ;;` between `e2e-shell` and `sql`.
- `bash -n $W/scripts/db-sandbox.sh` and `bash $W/scripts/db-sandbox.sh` (usage, exit 1; absolute path, never a
bare `pnpm db:sandbox`) prove the file parses and the header range is right.
2. `scripts/avatar-e2e.sh` — deleted (`git rm`).
3. `package.json` — `"e2e:avatar": "bash scripts/db-sandbox.sh avatar-e2e"`.
4. `packages/supabase/tests/e2e/avatar-flow.e2e.ts` — replaced by the V1-2 file (same bytes): guard at
module load: `SUPABASE_URL === 'http://127.0.0.1:57800'`, `AVATAR_E2E_DB_CONTAINER === 'faunosteo_dbsandbox_db'`,
both keys' payload `iss === 'db-sandbox'`, else `throw` before any client is created; clients built on
the constant, never on the env URL; `psql` on the constant container; a random password per run; two
accounts created per run and deleted after; `expeditePurges(userId)` scoped to the account's objects;
`describe.skipIf(!configured)` gone: a wrong environment is an error, not a skip. The guard checks the
issuer, not the role and not the signature (brief §3): a role mix-up fails at the first admin call
against the sandbox; a forged `iss: db-sandbox` key passes the guard and fails at the sandbox's GoTrue.
5. `packages/supabase/vitest.e2e.config.ts`, `scripts/dev-env.sh` — the comment lines (V1-2 text).
6. Docs: only the lines this fix makes false (dead `scripts/avatar-e2e.sh` references, DEV-target
instructions), worded so they are true on develop (removal date 2026-11-02, no "V1-2.3", no `verify-e2e`):
- `CLAUDE.md`: the `db:sandbox` line (add `avatar-e2e` only), the `e2e:avatar` line, the two folder-structure
lines (tests/e2e, scripts list), the "Storage or worker change" workflow line;
- `docs/development/STORAGE.md` §6: the DEV `docker run` keeps its place with the V1-2 comment ("the worker
beside the local stack, for the manual photo flow (IDENTITY.md); the e2e suite refuses this stack"), the
`pnpm --filter @faunosteo/supabase test:e2e` line is DELETED (it would always throw), the `e2e:avatar`
line becomes the sandbox alias;
- `docs/development/SUPABASE.md` §3.2 block and §6 DEV-acceptance note; §5.1 table: a develop-only row for
`pnpm db:sandbox avatar-e2e` that describes the gateway on its own (no reference to a verify-e2e row);
- `docs/development/WORKSPACE.md`, `docs/development/TEST_STRATEGY.md`, `README.md:114`,
`docs/development/ENVIRONMENTS.md:38`, `docs/development/IDENTITY.md:130` (the manual photo flow points at
the STORAGE §6 `docker run` line, not at the e2e container: it sits on the sandbox network and is removed
at exit).
- NOT touched (still true after the fix): `docs/development/RUST.md:178`, `BROWSER_TESTS.md:103`,
`SUPABASE.md:38`, `CHANGELOG.md` (neither 6287e5b nor the V1-2 commits added a line).
## Merge expectations (V1-2 → develop, later)
Develop's `db-sandbox.sh` is the merge-base text (9135d73); the port changes base regions V1-2 changes
differently. Expected conflicts: the usage header, the function block (V1-2 has `cmd_verify_e2e` and
`cmd_sheet_assets_e2e` around `cmd_avatar_e2e`), the dispatcher, the usage `sed` range. Rule: take V1-2's
file whole; check with an empty `git diff <V1-2 tip> -- scripts/db-sandbox.sh` after the merge. The suite,
`package.json`, `dev-env.sh`, the vitest comment and the ported variable / `down` / `REPO` lines merge
cleanly. Doc lines reworded for develop conflict with V1-2's: keep V1-2's text (same rule as the
sheet-assets lines in TODO).
## Proof (log committed as `.claude/audits/2026-11-02-avatar-e2e-sandbox-only.md` on this branch)
Who runs what: the executor applies the code and the static checks (contract criteria 1-5, 8); the LIVE proofs
below (a, c-i, the positive run) and the audit log are run by the orchestrator in the main loop, never by a
sub-agent (incident 1 of 2026-11-01 was an executor's backgrounded `cd`).
`W=/tmp/claude-1000/-home-bchanot-Documents-faunosteo/573c6c77-7294-4d96-a518-83afaafef8d1/scratchpad/bugfix-avatar`.
Every command names `$W` explicitly: `bash "$W/scripts/db-sandbox.sh" …`, `corepack pnpm --dir "$W" …`,
`"$W/packages/supabase/node_modules/.bin/vitest" …`; NEVER a bare `pnpm e2e:avatar` / `pnpm db:sandbox` /
`pnpm worker:image` (the shell's cwd resets to the main checkout, which still holds the DEV runner).
Pre-flight before any live command: `test ! -e "$W/scripts/avatar-e2e.sh"` and `grep -F '"e2e:avatar": "bash
scripts/db-sandbox.sh avatar-e2e"' "$W/package.json"`; `docker ps -a --filter name=^faunosteo_worker_e2e$`
(a leftover of the old DEV runner; read-only; the owner removes it if present); the DEV containers' start
times recorded (`docker inspect -f '{{.State.StartedAt}}'`, read-only) and compared at the end.
Keys used in c, d, e, h, i are FORGED unsigned JWTs built in the shell (`base64url(header).base64url(payload).x`),
never read from `~/.cache/faunosteo-db-sandbox/secrets.sh` and never the DEV keys.
Trace: `strace -f -e trace=connect,sendto,sendmsg -o <file>` on `bash "$W/scripts/db-sandbox.sh" …` or on the
vitest binary directly, never through `pnpm` (its own process would be inside the trace). Criterion for a-e
and i: no `AF_INET` / `AF_INET6` in the trace, except where stated.
a. `bash "$W/scripts/db-sandbox.sh" avatar-e2e` with `FAUNOSTEO_SANDBOX_DIR` pointing at an empty temp dir
(no secrets file) → `db-sandbox: not running`, exit 1, no connect, no container started;
b. the suite (`vitest run --config vitest.e2e.config.ts` in `$W/packages/supabase`) with no environment →
throws at load, no connect (contract oracle 5, positive control on `curl 127.0.0.1:9`);
c. the suite with `SUPABASE_URL=http://127.0.0.1:54321`, `AVATAR_E2E_DB_CONTAINER=supabase_db_faunosteo` and
two forged JWTs saying `iss: supabase-demo` → throws at load, no connect;
d. the suite with the sandbox URL and forged `iss: db-sandbox` keys but `AVATAR_E2E_DB_CONTAINER=supabase_db_faunosteo`
→ throws, no connect;
e. the suite with the sandbox URL and container and a key whose payload is not JSON → throws, no connect;
i. the suite with the sandbox URL and container, a forged anon key `iss: db-sandbox` and a forged service
key `iss: supabase-demo`, then the swap → throws, no connect. This is the only case that isolates the
issuer comparison itself (c stops at the URL, d at the container, e in the JSON.parse catch).
h. (sandbox NOT running, nothing listens on 57800) the suite with the sandbox URL and container and two
forged JWTs saying `iss: db-sandbox` → the guard passes, `beforeAll` fails with ECONNREFUSED; strace shows
connects to 127.0.0.1:57800 only. The brief §3 point made visible: the issuer check routes, it does not
authenticate; whoever holds 127.0.0.1:57800 would receive the keys the environment supplies.
f. (after the setup below, sandbox up from this branch, image present) `bash "$W/scripts/db-sandbox.sh"
avatar-e2e` with 127.0.0.1:57800 held by a foreign listener (`python3 -I -m http.server` bound to
127.0.0.1:57800) → `already in use`, exit 1, `docker ps` shows no gateway and no avatar worker.
EXCEPTION, by construction: the probe `(exec 3<>"/dev/tcp/127.0.0.1/$PORT_GATEWAY")` is one
`connect(AF_INET, 127.0.0.1:57800)`; the criterion for f is "the only AF_INET connect is 127.0.0.1:57800;
none to :54321 / :54322 or the LAN address". Flagged to the owner. Known limitation (not exercised): a port
held by another run's `faunosteo_dbsandbox_gateway` is torn down by `worker_preflight` before the probe
(shared container names and default STATE dir across worktrees); fails closed, sandbox only.
Setup (destroys the current sandbox and the V1-2 build of the worker image; the V1-2 worktree belongs to an
earlier session at its checkpoint, no run in progress; listed at the approval gate): `corepack pnpm --dir "$W"
worker:image` (develop's crates), `bash "$W/scripts/db-sandbox.sh" down`, `bash "$W/scripts/db-sandbox.sh" up fresh`.
Positive run, ONCE, under strace, WITH hostile exports in the shell: `SUPABASE_URL=http://127.0.0.1:54321`,
`SUPABASE_ANON_KEY` and `SUPABASE_SERVICE_ROLE_KEY` forged (`iss: supabase-demo`),
`AVATAR_E2E_DB_CONTAINER=supabase_db_faunosteo`, `SUPABASE_NETWORK=supabase_network_faunosteo`,
`SUPABASE_INTERNAL_URL=http://supabase_kong_faunosteo:8000`, `WORKER_IMAGE=faunosteo_worker:local` (the old
runner's knobs; the script's constant wins) → `bash "$W/scripts/db-sandbox.sh" avatar-e2e` 5/5; trace
criterion: no connect to :54321 / :54322 or the LAN address; every other AF_INET connect named in the log
(expected: 127.0.0.1:57800 from the suite, the `/dev/tcp` probe). A watcher started beforehand in the
background with absolute paths (`until docker inspect faunosteo_dbsandbox_worker_avatar; do sleep 1; done`)
records, DURING the run and with filtered templates only: `docker port faunosteo_dbsandbox_gateway 8000/tcp`
(= 127.0.0.1:57800), the worker's networks (`{{range $k,$v := .NetworkSettings.Networks}}{{println $k}}{{end}}`
= faunosteo_dbsandbox_net) and `{{range .Config.Env}}{{println .}}{{end}} | grep '^SUPABASE_URL='`
(= http://faunosteo_dbsandbox_gateway:8000). NEVER a raw `docker inspect` or `.Config.Env` in the log: the
service key sits there. After the run: both containers gone, DEV start times unchanged.
What the isolation rests on (stated in the log with file:line, not "only one network": `NET` is a plain
bridge that can route to the host and the LAN where DEV's Kong publishes): the worker's URL is fixed to the
gateway container in the 600 env file, `crates/worker/src/http.rs:31` `max_redirects(0)` and `:33 proxy(None)`,
the gateway's upstreams are the sandbox containers by name, the sandbox keys are signed with a secret DEV
does not hold; `scripts/dev-env.sh:23` exports `FAUNOSTEO_DEV_API_URL` into every child of db-sandbox.sh,
the suite included, and nothing reads it.
Residuals (LOW, for the owner, no code change here, fixed in V1-2 first then ported): the suite's supabase-js
clients use Node's global fetch (`redirect: follow`; `HTTP(S)_PROXY` honoured when `NODE_USE_ENV_PROXY=1`):
no endpoint the suite calls redirects today, a proxy would receive sandbox keys only; `worker_env` is read by
the EXIT trap before `start_worker` assigns it (an inherited value would be `rm -f`'d on an early `die`);
every docker call follows `DOCKER_HOST` / the current context.
Static: `grep -rn "supabase status\|54321\|supabase_kong\|supabase_db_faunosteo\|supabase_network_faunosteo" "$W/scripts/db-sandbox.sh" "$W/packages/supabase/tests/e2e"` → nothing;
`git -C "$W" grep -n avatar-e2e.sh -- ':!.claude'` → nothing (the historical contract of 2026-10-13 under
`.claude/` keeps its citation).
Hand-off (in the log and to the owner): end with `bash "$W/scripts/db-sandbox.sh" down`; the V1-2 session
rebuilds `faunosteo_worker:dev` (`pnpm worker:image` there) and runs `pnpm db:sandbox up fresh` before its next
sandbox run (the develop secrets file has no SB_SEED_PW, V1-2's `load_secrets` refuses it: fails closed).
Copies that STILL reach DEV after the merge: the main checkout `/home/bchanot/Documents/faunosteo` (branch
feature/v1-1-clients-animals, merged, holds `scripts/avatar-e2e.sh` and the old `e2e:avatar` line: `pnpm
e2e:avatar` there targets DEV until that checkout moves to develop and the branch is deleted with
`gitflow.sh delete`), and the worktree old-958228a (same two scripts). Agents never run them.
## Gates
- `bash -n`, `pnpm typecheck`, `pnpm lint`, `pnpm --filter @faunosteo/supabase test:run` (PGlite) green in
the worktree.
- fresh verifier on the contract; fresh security-auditor on the diff (MODE: gate), both before commit.
- merge into develop with `gitflow.sh finish` only after the owner's go (brief §1 authorises it after the
verifications succeed); never V1-2.