Files
faunosteo/docs/development/SUPABASE.md
T
bchanot 57922e0902 fix(supabase): refuse bad sandbox arguments before starting anything
pnpm db:sandbox up with an unknown mode or a missing or malformed upgrade version started four containers and wrote the secrets before refusing. The mode and the version (fourteen digits) are now checked first.
2026-10-25 22:20:55 +01:00

27 KiB

Supabase: local stack, migrations, tests

Status: Step 3 deliverable, extended at Step 4 (2026-10-08). Covers the foundation (profiles, platform roles, audit log) and the anatomical sheet tables (ANATOMICAL_SHEET_MODEL.md). Conventions: BACKEND_ARCHITECTURE.md §3–§9.

1. Layout

supabase/
├─ config.toml            CLI config (public schema exposed only, PG 17, strong passwords)
├─ migrations/            forward-only SQL, <timestamp>_<name>.sql
│  ├─ 20261008100000_foundation.sql       schemas private/audit, updated_at, audit log
│  ├─ 20261008100100_profiles.sql         profiles + auth.users trigger + RLS
│  ├─ 20261008100200_platform_roles.sql   platform_role, profile_roles, helpers, RPCs
│  ├─ 20261008110000_anatomical_sheets.sql species, sheets, immutable versions, assets, lifecycle RPCs
│  ├─ 20261008110100_private_execute_hardening.sql EXECUTE revoked from PUBLIC on the Step 3 helpers
│  ├─ 20261009100000_sheet_asset_storage.sql bucket anatomical-sheet-assets + read policy through the sheet
│  ├─ 20261010100000_sheet_document_schema_v2.sql document schema 2: references, review_status, count_range checks
│  ├─ 20261011100000_profiles_professional.sql practice_name, professional_title, siret
│  ├─ 20261013100000_files_foundation.sql assets table, seven private buckets, avatar policies through assets, purge queue, browser + worker RPCs (STORAGE.md)
│  ├─ 20261025100000_workspaces.sql   V1-1.1 (M1): workspaces, workspace_members, personal workspace per account, access helpers, audit correlation cap (§3.3)
│  └─ 20261025110000_clients.sql      V1-1.2 (M2): clients, unaccent, client RPCs and search, write budget per workspace, N9 audit filter (§3.4)
├─ seed.sql               LOCAL dev accounts + first owner (never run in prod)
├─ seed/anatomical-sheets/equine/  official equine sheet as a DRAFT, generated by `pnpm import:equine`
│  ├─ legacy-sources/     the historical Horse.svg, Horse_head.svg, pathologies.json, vertebrae.ts (importer input)
│  ├─ legacy-import/      baseline import (document, id map, SVG copies, IMPORT-REPORT.md), no SQL
│  ├─ curation/           authored changes.json + references.json
│  └─ draft/              curated working revision (document, horse.svg, sheet-ref.json, report-migration-map.json, CHANGELOG, REVIEW-REPORT, review/*.svg; bundled into the app with SHEET_BUNDLE=draft) + equine_draft.sql: idempotent, converging seed (sheet, 1 asset row, working draft), applied by db reset and by PGlite
└─ tests/                 pgTAP, one file per concern, allowed AND denied paths
packages/supabase/
├─ src/client.ts          createBrowserClient() (anon key, PKCE)
├─ src/database.types.ts  `supabase gen types` output
├─ src/anatomical-sheets.ts publishSheetVersion(): the core's verdict (`sheet.check`) as gate before the RPC
├─ src/assets.ts          avatar flow: begin / quarantine upload / confirm / remove / own state / download, closed AssetFailure kinds
├─ tests/db/              PGlite harness: runs the pgTAP files without Docker
└─ tests/e2e/             real-stack end-to-end (avatar flow with the worker container), `pnpm e2e:avatar`
crates/core/src/domain/sheet/  canonical sheet document and its integrity rules (Rust core; `packages/domain` deleted at R5.7)

2. Prerequisites

  • Node 24 + pnpm via corepack (WORKSPACE.md).

  • Docker daemon reachable by your user for the real stack. If docker info says permission denied ... docker.sock, add yourself to the docker group and log in again:

    sudo usermod -aG docker "$USER"
    
  • No Docker: the PGlite suite still runs (see §5), the CLI commands do not.

The Supabase CLI is a root devDependency (pnpm supabase <cmd>), pinned like every other tool. Never install it globally.

3. Start, stop, reset

Every CLI call goes through scripts/supabase.sh (pnpm db:*, pnpm supabase <args>): it exports the DEV origin that config.toml reads with env() (api.external_url, auth.site_url, studio.api_url). A direct node_modules/.bin/supabase start would leave the literal env(...) text in GoTrue's mail links. The CLI publishes API 54321, PostgreSQL 54322, Studio 54323 and Mailpit 54324 on 0.0.0.0 (no bind option); the host firewall closes 54322-54324 on the LAN interface and the DEV gateway serves Mailpit behind basic auth on 54325 (ENVIRONMENTS.md §3).

pnpm db:start      # supabase start: pulls images, applies migrations + seed, prints URLs and keys
pnpm db:stop       # supabase stop (keeps the volume)
pnpm db:reset      # drop + replay every migration + seed from zero

Local endpoints (config.toml):

Service URL
API (PostgREST, Auth, Storage) http://127.0.0.1:54321
PostgreSQL postgresql://postgres:postgres@127.0.0.1:54322/postgres
Studio http://127.0.0.1:54323
Mail catcher (confirmation / OTP mails) http://127.0.0.1:54324

Addresses on the host itself. From another device of the network: app :5173, API :54321, Mailpit through the gateway :54325 (basic auth); Studio by SSH tunnel; PostgreSQL not at all (ENVIRONMENTS.md §3).

pnpm db:start prints the anon and service_role keys. Only the anon key ever reaches the browser.

Containers, volumes and network are named supabase_<service>_faunosteo (project_id of config.toml): the CLI imposes this pattern, so the faunosteo_ prefix convention of DEPLOY.md ("Docker resource naming") applies as a suffix here. Several projects can run side by side as long as their project_id and ports differ.

Two kinds of seed (config.toml [db.seed] sql_paths):

File Scope Content
seed.sql local only dev accounts + first owner
seed/…/*.sql every environment versioned reference data, idempotent (today: the official equine sheet as a draft); each file listed explicitly in sql_paths

The PGlite harness applies every .sql under seed/ after the migrations, seed.sql never (it needs the real auth schema); a test fails when a seed file is missing from sql_paths. On another environment, apply a versioned seed with psql -f from a privileged session, then upload its assets (§3.1). Never run supabase db push --include-seed against a shared database: the list also contains seed.sql and its known local password.

3.1 Sheet assets (Storage)

SVG files live in the private bucket anatomical-sheet-assets, keyed by content (<sha256>.svg). Reading goes through the sheet (policy anatomical_sheet_assets_read): a published version of a usable sheet, or any version for its editors. No API role uploads; the operator does, with the service key, from the generated seed folder. The tool is the Rust binary upload-sheet-assets (crates/core, feature upload, R5.5). It targets the local stack only: there is no default destination and a host that is not this machine (or the configured FAUNOSTEO_DEV_HOST, a private address) is refused before any request. Run it by hand; the assistant does not.

pnpm db:start                                  # the seed creates the asset rows (pending)
export SUPABASE_URL=http://127.0.0.1:54321     # http, explicit port; the DEV address needs FAUNOSTEO_DEV_HOST
read -rs SUPABASE_SERVICE_ROLE_KEY && export SUPABASE_SERVICE_ROLE_KEY   # typed, not in shell history
pnpm assets:upload --check supabase/seed/anatomical-sheets/equine/draft   # read-only plan
pnpm assets:upload supabase/seed/anatomical-sheets/equine/draft           # upload + confirm

What a run does, in order, and stops at the first refusal before the first write:

  1. The folder must lie under supabase/seed/anatomical-sheets. assets.json is read under a size cap (strict keys, plain .svg file names, lowercase sha256 and uuid, no repeated entry, 64 entries at most). Each drawing is read under a cap and must match its entry (size, sha256) and be a drawing the product accepts (the core's SVG reader, nothing removed).
  2. The destination is checked, then a handshake: the bucket exists, is private, and the key is accepted.
  3. Each entry is planned from what the destination holds: an object under the key is read back and hashed; different bytes, a missing asset row or a row recording another hash refuse the run (nothing is overwritten, no row is invented; apply the seed first).
  4. Missing objects are created without replacing anything (x-upsert: false), read back and verified, and only then is the row confirmed (confirm_sheet_asset_upload, service role only): the read policy ignores a row whose upload was never confirmed, so a user who registers a known hash on their own sheet opens nothing.

A rerun after any failure resumes: an object already verified is kept, a confirmed row is left alone. A connection failure or a 5xx answer is retried three times (four attempts); a refusal (401, 403, a state mismatch, an oversized answer) is final. Exit codes: 0 done, 1 refused (usage, manifest, destination, content, handshake, state, access), 2 an argument that is not UTF-8, 3 network. The DEV address counts only if it is an address of this machine. The key is never printed and goes through no proxy. pnpm e2e:sheet-assets runs the whole behaviour against the local stack (upload, rerun, anon and wrong key, refused destinations, unreachable port). The TypeScript tool (packages/supabase/scripts/upload-sheet-assets.ts) was deleted at R5.7 (RUST.md §11.23).

3.2 Files worker (Storage, Step 8)

User files land in private buckets and are processed by the Rust worker (crates/worker, docs/development/STORAGE.md). Against the local stack:

pnpm worker:image                         # faunosteo_worker:dev
pnpm e2e:avatar                           # starts the container on supabase_network_faunosteo, runs tests/e2e, stops it
docker compose --profile worker up worker # the same container from docker-compose.yml, reading .env.worker

.env.worker (not editable by the assistant): SUPABASE_URL=http://supabase_kong_faunosteo:8000, SUPABASE_SERVICE_ROLE_KEY=<the service key of pnpm supabase status>, WORKER_ID=dev. The service key stays in that file and in the shell; it never reaches the app .env, env.js or a commit.

Seed accounts (local only, password LocalDev-only-password-1):

Email Role
owner@faunosteo.local platform owner (bootstrapped by seed.sql)
alice@faunosteo.local plain user
bob@faunosteo.local plain user

3.3 Workspaces (V1-1.1, migration M1)

Contract: docs/development/V1-1_OWNERS_ANIMALS.md §4, §8.1, §9. Every account has exactly one personal workspace, created by the trigger profiles_create_personal_workspace in the sign-up transaction (GoTrue → handle_new_auth_user → profile → workspace + holder membership), through the single write path private.ensure_personal_workspace(uuid) (one statement, idempotent, race-safe). A deferred constraint trigger refuses at commit a personal workspace without its holder or with any other member. Reads: RLS through private.caller_workspace_ids() (member, active profile, confirmed email, auth.uid() only). No API role writes either table; anon and service_role hold no privilege on them. From M2 on, record tables use the same helpers; the app passes its workspace id explicitly.

M1 is written and tested on PGlite and on the isolated stack (§5.1), not applied to the DEV stack. Applying it there is an owner action, never run by Claude, never with db reset, db push, migration repair or any destructive command. Every step below was rehearsed on the isolated stack in upgrade mode (pnpm db:sandbox up upgrade 20261025100000), with the sandbox container name in place of supabase_db_faunosteo. Nothing below prints a secret: the database password stays in the container's own environment.

# 0. Target: the local CLI stack of this project, nothing remote.
pnpm supabase status                          # API on the DEV origin, DB on port 54322; no error
test ! -f supabase/.temp/project-ref && echo "no linked project"   # nothing remote is linked
docker ps --filter name=supabase_db_faunosteo --format '{{.Names}} {{.Status}}'

# 1. History: only the V1-1 migrations may be pending.
pnpm supabase migration list --local          # every row has local = remote, except the V1-1 ones (remote empty):
                                              #   20261025100000 (M1) on a checkout of 6bf6978-08abe42,
                                              #   20261025100000 and 20261025110000 (M1, M2) from 4adbfcb on
pnpm db:types:check                           # the diff must be exactly the objects of those migrations (M1: workspaces,
                                              #   workspace_members, workspace_kind, workspace_member_role; M2: clients,
                                              #   client_kind, create_client, update_client, set_client_archived,
                                              #   search_clients): anything else = drift, STOP

# 2. Existing accounts (the backfill creates one workspace per profile).
docker exec supabase_db_faunosteo sh -c 'PGPASSWORD="$POSTGRES_PASSWORD" psql -h 127.0.0.1 -U postgres -d postgres -t -A \
  -c "select (select count(*) from auth.users) || '"'"' users, '"'"' || (select count(*) from public.profiles) || '"'"' profiles'"'"'"'
#   users = profiles expected; note the profile count N

# 3. Backup outside Git (includes auth.users and the migration history), then check it.
mkdir -p ~/faunosteo-backups && chmod 700 ~/faunosteo-backups
docker exec supabase_db_faunosteo sh -c 'PGPASSWORD="$POSTGRES_PASSWORD" pg_dump -h 127.0.0.1 -U postgres -d postgres -Fc' \
  > ~/faunosteo-backups/dev-before-m1.dump && chmod 600 ~/faunosteo-backups/dev-before-m1.dump
docker exec -i supabase_db_faunosteo pg_restore --list < ~/faunosteo-backups/dev-before-m1.dump \
  | grep -E "TABLE DATA (auth users|public profiles|supabase_migrations schema_migrations)"   # three lines expected

# 4. Apply without reset (M1's backfill runs inside the migration). `migration up` applies every
#    pending migration of the checkout, in order: to apply M1 alone, run it from a checkout of 08abe42.
pnpm supabase migration up --local            # "Applying migration 20261025100000_workspaces.sql" (then ..._clients.sql)

After applying:

pnpm supabase migration list --local          # each applied migration now has its remote column
docker exec supabase_db_faunosteo sh -c 'PGPASSWORD="$POSTGRES_PASSWORD" psql -h 127.0.0.1 -U postgres -d postgres -t -A -c "
  select (select count(*) from public.profiles) || '"'"' profiles, '"'"'
      || (select count(*) from public.workspaces) || '"'"' workspaces, '"'"'
      || (select count(*) from public.workspace_members) || '"'"' members, '"'"'
      || (select count(*) from public.profiles p where (select count(*) from public.workspaces w
            where w.personal_user_id = p.id) <> 1) || '"'"' profiles without exactly one workspace'"'"'"'
#   expected: N profiles, N workspaces, N members, 0 profiles without exactly one workspace
pnpm db:test                                  # pgTAP 001-013 (grants, policies, reads); every file runs in
                                              #   begin ... rollback: rows unchanged (checked on the sandbox)
pnpm db:types:check                           # no diff
pnpm e2e:avatar                               # avatar flow and worker on the DEV stack (fictitious seed accounts)

Stop and report instead of continuing when: migration list shows another pending migration or a remote-only one; db:types:check before step 4 shows anything beyond the M1 objects; the dump check misses a line; migration up applies anything else than the expected V1-1 migrations; any count after it differs from the expectation. The backup restores into a scratch database (rehearsed: every row back; pg_restore --no-owner prints privilege errors on Supabase-managed objects, which do not affect the data). Storage file bytes (avatars) live in the storage container's volume, not in the dump; M1 does not touch them.

3.4 Clients (V1-1.2, migration M2)

Contract: V1-1_OWNERS_ANIMALS.md §5.1, §8.2, §9, §10. public.clients holds the clients of a workspace; the browser reads them through RLS (members of the workspace, active profile, confirmed email) and writes them only through create_client, update_client and set_client_archived (security definer); search_clients (invoker, POST) searches one workspace. Refusals are closed codes: 42501 not_found (also for a foreign id), P0001 invalid_input, archived, conflict, request_conflict, limit_reached, rate_limited. Writes are counted per workspace in private.workspace_write_budgets (UTC hour and day). M2 creates the extension unaccent in extensions (available on the CLI image and in PGlite 0.3.16, loaded by the harness). An account whose workspace holds a client can no longer be deleted (D9a); in DEV, delete its clients first from a privileged SQL session, or reset.

4. Writing a migration

  1. pnpm supabase migration new <name> creates supabase/migrations/<timestamp>_<name>.sql.
  2. A migration that adds a table adds, in the same file: RLS enable, explicit revoke all ... from public, anon, authenticated (and service_role for V1-1 record tables, contract N7) then the exact grants, separate SELECT / INSERT / UPDATE / DELETE policies, indexes, comments.
  3. Add or extend a pgTAP file under supabase/tests/: the allowed path and every denied path (other user, anon, direct write bypassing an RPC). Scope counts to the fixture ids: on the real stack the seed accounts exist, in PGlite they do not.
  4. pnpm --filter @faunosteo/supabase test:run (PGlite, seconds), then pnpm db:reset && pnpm db:test on the real stack.
  5. pnpm db:types and commit packages/supabase/src/database.types.ts.
  6. Never edit an applied migration. Corrections are new migrations.

Rules enforced by the schema-types test for every future public table: RLS enabled, no using (true) / with check (true) policy for anon or authenticated.

5. Tests

Two runners, one SQL source of truth (supabase/tests/*.sql, pgTAP):

Command Engine Needs Docker What runs
pnpm --filter @faunosteo/supabase test:run PGlite (PostgreSQL 17 in WASM) no shim of the Supabase base schema (auth + storage tables) + all migrations + seed/** + every pgTAP file; schema ↔ types check; client factory; publish gate
pnpm db:test supabase test db (pg_prove in the local stack) yes the same pgTAP files against the real Supabase image

pnpm test:run at the root runs the PGlite suite together with the app suite. The shim (packages/supabase/tests/db/supabase-shim.sql) recreates the API roles, auth.users, auth.uid() and the permissive default privileges of Supabase so that a forgotten revoke fails a test.

Two parity traps met so far: counts must be scoped to fixture ids (the real stack carries seed.sql rows), and order by on text must pin collate "C" (the real image sorts with a locale collation, PGlite with C).

5.1 Isolated stack (pnpm db:sandbox)

A throwaway Supabase stack for migration steps, so the DEV stack is never reset or migrated by a test (owner rule since V1-1.1). scripts/db-sandbox.sh starts the images the CLI stack uses (PostgreSQL 17.11.0.004, GoTrue v2.197.0, Storage v1.79.36, PostgREST v16.4, Mailpit v1.31.3, pinned in the script) on the network faunosteo_dbsandbox_net, containers faunosteo_dbsandbox_*, ports on 127.0.0.1 only and distinct from DEV (db 57432, auth 57999, rest 57300, storage 57500, mail 57025). Nothing is mounted and the images declare no volume, so down removes the containers and the network by their literal names and can never reach a supabase_*_faunosteo volume. The password, JWT secret and keys are generated at each up into a 600 file under ~/.cache/faunosteo-db-sandbox (FAUNOSTEO_SANDBOX_DIR, refused inside the repository), with 600 env files that hand them to the containers; deleted by down, never printed, never on a command line (the CLI and psql read PGPASSWORD from the environment). An incomplete secrets file stops the script before any CLI call, and a wrong up mode or version (fourteen digits) is refused before any container or secret exists. Works with rootless Docker.

Command What it does
pnpm db:sandbox up fresh every migration through supabase migration up --db-url, then the seeds of config.toml (as db reset)
pnpm db:sandbox up upgrade <version> the migrations before <version>, the seeds, two GoTrue accounts (confirmed, unconfirmed), then migration up: the path a DEV database takes
pnpm db:sandbox test-db supabase test db --db-url: the pgTAP files on the real image (rows unchanged afterwards)
pnpm db:sandbox types [--check] supabase gen types --db-url ... --schema public through Prettier, written or diffed with the committed file
pnpm db:sandbox test packages/supabase/tests/sandbox/*.sandbox.ts: real GoTrue and PostgREST sessions, concurrent SQL sessions
pnpm db:sandbox sql / down psql as postgres on stdin / remove the stack

Every CLI call passes --db-url with sslmode=disable (the sandbox has no TLS); none can fall back to the local DEV database. Differences from the CLI stack: no Kong (GoTrue and PostgREST are called directly), no Studio, no realtime or edge runtime, and the database init keeps only the role passwords the services use.

How a test impersonates a user (same technique in both runners):

set local role authenticated;
select set_config('request.jwt.claims', '{"sub":"<user uuid>","role":"authenticated"}', true);

6. Generated types

pnpm db:types          # regenerate packages/supabase/src/database.types.ts (local stack running)
pnpm db:types:check    # CI: fail on any diff

The CLI emits unformatted TypeScript; both scripts pipe it through Prettier (repo config) so the committed file is stable byte for byte. The schema-types test checks tables, columns, enum labels and RPC argument names against the migrated schema, so a stale file fails before CI.

7. First platform owner in a real environment

The RPCs refuse self-grants and need an existing owner or admin. The first owner is created once, from a privileged SQL session (service connection, psql on the host), never from the browser:

select private.assign_platform_role(null, '<auth.users.id of the owner>', 'owner');

The call is audited with a null actor. Afterwards every role change goes through grant_platform_role / revoke_platform_role.

8. Manual validation (Definition of Done: "an authenticated request hits a protected table safely")

URL=http://127.0.0.1:54321
ANON=<anon key printed by pnpm db:start>

token() {  # $1 = email
  curl -s -X POST "$URL/auth/v1/token?grant_type=password" \
    -H "apikey: $ANON" -H "Content-Type: application/json" \
    -d "{\"email\":\"$1\",\"password\":\"LocalDev-only-password-1\"}" | jq -r .access_token
}
ALICE=$(token alice@faunosteo.local); OWNER=$(token owner@faunosteo.local)
ALICE_ID=10000000-0000-0000-0000-00000000000a

# 1. alice reads exactly one profile (hers)
curl -s "$URL/rest/v1/profiles?select=id,display_name" -H "apikey: $ANON" -H "Authorization: Bearer $ALICE"

# 2. anon reads nothing (permission denied)
curl -s "$URL/rest/v1/profiles" -H "apikey: $ANON"

# 3. alice cannot deactivate herself (column privilege) nor self-grant admin
curl -s -X PATCH "$URL/rest/v1/profiles?id=eq.$ALICE_ID" -H "apikey: $ANON" \
  -H "Authorization: Bearer $ALICE" -H "Content-Type: application/json" -d '{"active":false}'
curl -s -X POST "$URL/rest/v1/rpc/grant_platform_role" -H "apikey: $ANON" \
  -H "Authorization: Bearer $ALICE" -H "Content-Type: application/json" \
  -d "{\"target_user_id\":\"$ALICE_ID\",\"role\":\"admin\"}"

# 4. the owner grants moderator to alice; the grant is audited
curl -s -X POST "$URL/rest/v1/rpc/grant_platform_role" -H "apikey: $ANON" \
  -H "Authorization: Bearer $OWNER" -H "Content-Type: application/json" \
  -d "{\"target_user_id\":\"$ALICE_ID\",\"role\":\"moderator\"}"
psql postgresql://postgres:postgres@127.0.0.1:54322/postgres \
  -c "select actor_user_id, action, target_user_id, metadata from audit.audit_logs order by occurred_at"

Expected: 1 → one row; 2 → 42501; 3 → both 42501; 4 → a uuid, then two audit rows (bootstrap with null actor, grant by the owner).

9. Wiring the app

Development: pnpm dev (scripts/dev.sh) sets VITE_SUPABASE_URL to the DEV origin (http://<FAUNOSTEO_DEV_HOST>:54321, the LAN address of the machine: ENVIRONMENTS.md §2) and the anon key from supabase status; the app is served on http://<host>:5173 for every device of the private network. pnpm dev:offline runs the local tool without backend. Docker deployments use .env:

APP_SUPABASE_URL=<Supabase origin reachable by the browser>
APP_SUPABASE_ANON_KEY=<anon key>               # public by design

Both empty keeps the app in its local-only mode: getSupabaseClient() returns null, the auth screens show an "unavailable" notice and /app redirects to /login. APP_AUTH_PROVIDERS / VITE_AUTH_PROVIDERS (google,apple) lists the OAuth buttons to show. Sign-in, sign-up, email confirmation, password recovery, the /app shell and Mon compte are documented in IDENTITY.md (routes, session handling, provider setup, MFA gap, manual validation).

10. Self-hosted parity

The local stack runs the same images as a self-hosted deployment (supabase/docker compose), so migrations, policies and tests transfer unchanged. The self-hosted runbook (VPS sizing, secrets, backups, TLS) is a separate deliverable once the hosting question (OPEN_QUESTIONS Q2) is settled; it reuses pnpm supabase db push --db-url <...> for migrations.