Files
faunosteo/docs/development/STEP8_FILES_ANALYSIS.md
T
bchanot 273914d825 fix(supabase): ceiling measured on bytes present in the bucket plus pending claims without bytes; confirm fails closed on an unsized object; cleartext host must start with a letter
Third audit (BLOCK 1): counting terminal rows while queued let begin spam
pin the ceiling with declared sizes. The ceiling now sums storage.objects
sizes of the bucket (whatever the row's state) plus the declared size of
pending_upload rows whose quarantine object has not landed: a claim dies
with its row, a present object counts whatever its row says, no full scan of
the assets history. confirm_asset_upload treats an object without a size as
upload_missing. pgTAP (plan 138): rejected rows without bytes weigh nothing,
pending claims fill, superseded claims vanish, present bytes fill, unsized
object refused. Worker: a single-label cleartext host must start with an
ASCII letter (0X… hex forms refused, test added); STORAGE.md names the
native-run search-domain residual.
2026-10-13 20:07:22 +02:00

40 KiB
Raw Blame History

Step 8 — Files foundation and Supabase Storage: technical analysis

Status: proposal for the owner's decision, 2026-10-12, revision 2 after three independent plan challenges (correctness, robustness, simplicity: 47 findings, folded in; the changes are listed in I.3). Nothing implemented. Applies the owner's brief of 2026-10-12 (Step 7 closed; hosting = self-hosted Supabase first, managed later; eight private buckets closed by default; quarantine + server validation; avatar as first consumer; service role never in the browser).

Part I states the facts the design rests on. Part II is the proposed design. Part III answers the ten security questions of the brief. Part IV lists the exact scope, the tests, the risks and the decisions the owner has to take.


Part I — Facts

I.1 What exists today

Area State
Storage One bucket, anatomical-sheet-assets (migration 20261009100000): private, SVG only, 20 MiB cap, content-addressed keys <sha256>.svg, SELECT policy through the sheet (private.can_read_sheet_asset_object), no API write policy, operator upload script with the service key, confirm_sheet_asset_upload RPC (service role). pgTAP 006 (28 checks) runs on the PGlite shim and on the real stack; pgTAP 001 pins the exact list of private functions authenticated may execute.
Storage API storage-api v1.79.36 locally (same image family as self-hosted). Verified on the client library (storage-js 2.117): an upload with the user JWT and upsert: false needs only an INSERT policy on storage.objects, evaluated on bucket_id and name; the bucket MIME check runs before RLS on the multipart part type (the browser's File.type, not sniffed); createSignedUrl evaluates the SELECT policy as the caller and answers 400 when denied; a signed upload URL is checked at signing time only and stays valid two hours. Image transformation needs imgproxy, absent here and a paid add-on on managed.
Profiles avatar_asset_id was deferred to this step (comment in 20261008100100_profiles.sql).
Shell UI UserBadge (monogram) in the sidebar and the drawer; the top bar holds only the language switch; "Mon compte" = stacked sections, PersonalInfoForm first. The validated mockup shows the avatar with "Modifier la photo" inside the Profil section and an avatar + name in the top bar. Production CSP: default-src 'self', img-src 'self' data: blob:.
SVG sanitiser packages/pdf/src/svg/{parse,sanitize,view}.ts: parseSvg refuses DOCTYPE / ENTITY, sanitizeSvgRoot = allowlist of elements and attributes, in-document href / url(#) only, backslash refused, caps 20 000 elements / depth 64 / 256 <use> / stdDeviation 100; cleanSvgRoot chains both. Hostile fixtures already covered by packages/pdf/tests/security.test.ts (script, on*, javascript:, external image, CSS escapes, ENTITY, 257 <use>, horse.svg unchanged). Not exported from the package index; serialisation drops the XML declaration (byte change).
Edge runtime Disabled in config.toml. BACKEND_ARCHITECTURE.md §6; C3: PostgreSQL-backed jobs before a queue; C12: the sanitiser runs between upload and publication.
Worker The plan creates apps/worker at Step 19 (notifications). Files processing needs server-side code before that.
Deploy No backend in production yet. The VPS .env is documented as non-sensitive. The VPS is shared by several SaaS (BDR-035).
sharp 0.35.5 No install script, Node ≥ 20.9, prebuilt linuxmusl-x64. rotate() with no angle auto-orients from EXIF; output strips all metadata by default and converts through the embedded ICC profile to sRGB; failOn defaults to warning (stricter than error); limitInputPixels is enforced before metadata() returns.
Tests pgTAP on PGlite + real stack; app vitest on happy-dom; Step 7 added a headless e2e script against the local stack. PostgREST max_rows = 1000 caps set-returning RPCs.

I.2 Corrections to the plan's Step 8 text

  • "RPC issue_signed_url(asset_id)" cannot exist in SQL: signing needs the Storage API's secret. The Storage API signs only after the SELECT policy passes, so the check and the signature already live in one place. For an owner-only avatar, Step 8 does not sign at all (II.6).
  • "Edge Function or SQL-only sanitizer": SQL cannot parse XML or decode images; a containerised worker is the fit (II.1).
  • "Upload a hostile SVG as a sheet asset and verify it is rewritten": no user can upload a sheet asset before Step 13, so this manual check moves there with the processor (D6).

I.3 Changes after the challenges (revision 2)

Decoder isolated from the service key and bounded by a hard deadline; write order PUT clean → commit → delete quarantine; uploaded rows superseded by a new upload instead of a blocking unique index; purge queue with a grace period longer than a backup run (the backup invariant was false otherwise); every quarantine leftover auto-purged (signed upload URLs can land bytes after the row moved on); client-side downscale as transport (current phones exceed a 24 MP cap); owner-only avatar read by download under RLS, no signed URL, no bearer token; schema narrowed to the one kind that has rules (no multi-kind dispatcher, no derived-bucket trigger, no unread columns); avatar pointer derived by a unique index instead of a profile column + trigger; SVG processor, migrate and reconcile commands, backup tooling and restore drill moved out of Step 8 (runbook text stays); policy census pins one bucket literal per policy.


Part II — Proposed design

II.1 Components

Browser (anon key + user JWT)
  │  pick file → decode → canvas downscale (transport) → JPEG blob
  │  begin_avatar_upload (RPC) ─► assets row pending_upload + quarantine key
  │  PUT object (Storage API, INSERT policy = that key, that row, that user)
  │  confirm_asset_upload (RPC) ─► status uploaded
  │  poll own assets row ─► clean | rejected
  │  download(object_key) (Storage API, SELECT policy) ─► blob URL ─► <img>
  ▼
Supabase (self-hosted compose, later managed)
  PostgREST ─ public.assets, RPCs, private helpers      Storage API ─ 8 private buckets
  ▲
  │  claim_avatar_jobs / complete_avatar_job / reject_avatar_job (service role)
  │  claim_purges / ack_purge (service role)
  │  GET quarantine, PUT clean, DELETE objects (service role)
apps/worker — container faunosteo_worker, outbound only, no port
  keyed I/O loop ──stdin/stdout──► unkeyed decode child (sharp), 20 s deadline, SIGKILL
  sweeper: purge queue (row-driven), quarantine leftovers (key-driven)

Why a worker: image validation needs a native decoder on the server side, behind the service key, with a hard resource bound. Postgres cannot read object bytes; a client-side re-encode is not validation (a polyglot passes); pg_net or a webhook would give the worker an inbound port. Edge Functions would be one more container on self-hosted too, hand-deployed, and on managed they decode in WASM under a CPU and memory budget not sized for photos (sharp is native and cannot run there). The worker is one Node 24 container, the same code against the local stack, the VPS and a managed project (URL + key only). It is the apps/worker of plan Step 19, arriving early with one job family; Step 19 extends it.

Dependency added: sharp 0.35.x (libvips). The only maintained Node library that decodes JPEG / PNG / WebP, resizes, re-encodes and strips metadata; pure-JS alternatives cover PNG only or are unmaintained. The worker imports no workspace package, so Node 24 runs its TypeScript directly (type stripping), no bundler, no tsx.

II.2 Buckets

All eight private. One migration, on conflict (id) do update, so it converges on the local stack, the VPS and managed alike.

Bucket Owning domain and step Step 8 state
avatars Identity (self) open: SELECT + INSERT policies through assets, 5 MiB, image/jpeg · png · webp
logos Organizations, Step 10 closed, no policy, no cap yet
professional-documents Verification, Step 9 closed
client-files Clients, Step 12 closed
animal-files Animals, Step 12 closed
report-files PDF artifacts, Step 15 closed
anatomical-sheet-assets Sheets (existing policy, kept) unchanged
imports Imports, Step 24 closed

"Closed" = the bucket row exists with public = false, no policy on storage.objects names it, so anon and authenticated can neither list, read, upload, update nor delete; only the service role can. Caps and MIME lists of closed buckets are set by their own step. A pgTAP census asserts that every storage.objects policy pins exactly one bucket literal (bucket_id = '…'), counts them per bucket (2 on avatars, 1 on anatomical-sheet-assets, 0 elsewhere) and pins the total, so a generic policy naming no bucket cannot open the others.

Open point (D2): the brief names the sheet bucket anatomical-sheets; the repository already has anatomical-sheet-assets with policy, operator script, tests and seed. Recommendation: keep the name, record the deviation.

II.3 public.assets and the RPCs

One row per file, created before the bytes exist, kept after they are gone (status deleted) until the purge queue has been served.

create type public.asset_status as enum
  ('pending_upload', 'uploaded', 'clean', 'rejected', 'deleted');

create table public.assets (
  id               uuid primary key default gen_random_uuid(),
  owner_id         uuid not null references public.profiles (id) on delete cascade,
  kind             text not null check (kind in ('profile_avatar')),   -- extended per step
  bucket_id        text not null check (bucket_id = 'avatars'),       -- extended with kind
  object_key       text not null unique,       -- final key, server-built
  quarantine_key   text,                       -- upload key, null once purged
  status           public.asset_status not null default 'pending_upload',
  declared_mime    text not null,
  declared_size    integer not null check (declared_size between 1 and 5242880),
  byte_size        integer,                    -- clean output
  sha256           text check (sha256 ~ '^[0-9a-f]{64}$'),
  rejection_reason text,                       -- closed codes, II.7
  attempts         smallint not null default 0,
  locked_at        timestamptz,                -- lease, II.8
  created_at       timestamptz not null default now(),
  updated_at       timestamptz not null default now()
);
-- "the" avatar of an account = its single clean row (no profile column, no trigger)
create unique index assets_one_clean_avatar
  on public.assets (owner_id) where kind = 'profile_avatar' and status = 'clean';
create index assets_claimable on public.assets (created_at) where status = 'uploaded';

Privileges: revoke all from API roles, grant select to authenticated, RLS assets_select_own (owner_id = auth.uid()). No INSERT / UPDATE / DELETE policy: every write is an RPC (security definer, search_path = '', authorisation before lookup, EXECUTE revoked from PUBLIC, granted per role, private.log_event on sensitive transitions). pgTAP 001's census of private functions executable by authenticated is updated for the two new predicates.

RPC Role Effect
begin_avatar_upload(declared_mime, declared_size) authenticated locks the caller's profiles row (for update, serialises a double click); rate: ≤ 10 rows of the caller created in the last hour, all statuses counted → rate_limited; global ceiling: bytes present in the bucket plus declared sizes of pending rows without bytes (as built, STORAGE.md §2.1) < private.storage_ceiling() → storage_full; caps (type in the list, size ≤ 5 MiB); supersedes any pending_upload / uploaded row of the caller (→ deleted, enqueued for purge); inserts the row with object_key = profiles/<uid>/<id>/avatar.webp, quarantine_key = quarantine/<uid>/<id>/upload; returns {asset_id, bucket_id, quarantine_key}
confirm_asset_upload(asset_id) authenticated own row, pending_upload, a storage.objects row exists at (bucket_id, quarantine_key) with metadata->>'size' ≤ cap → uploaded. Kept as an explicit client signal: the worker never has to guess from Storage API internals whether a PUT finished
remove_avatar() authenticated every profile_avatar row of the caller not already deleted / rejected (clean and in flight) → deleted; logs asset.avatar_removed
claim_avatar_jobs(max_jobs) service_role uploaded rows with locked_at is null or locked_at < now() - 10 min and attempts < 3, for update skip locked, oldest first → sets locked_at, attempts + 1, returns the rows; a row at its 3rd expired lease becomes rejected (processing_error) instead. VOLATILE (a STABLE function runs read-only through PostgREST)
complete_avatar_job(asset_id, byte_size, sha256) service_role guard status = 'uploaded' (a superseded row is not completed: the worker then deletes what it wrote) → marks the caller's previous clean avatar deleted (enqueued), sets the row clean; one transaction; logs asset.avatar_set
reject_avatar_job(asset_id, reason) service_role guard uploaded → rejected with a closed code; logs asset.rejected
mark_quarantine_purged(asset_id) service_role nulls quarantine_key after the worker deleted the quarantine object of a clean row
claim_purges(max_items) / ack_purge(purge_id) service_role the purge queue, II.9

Why assets does not absorb anatomical_sheet_assets: that table is content-addressed and sheet-scoped (BDR-032/034) with its own confirmed upload path; the kinds that join assets later (logos, documents, client and animal files, report files, imports) extend the kind CHECK and add a predicate branch at their step. The profile pointer is derived by the unique index rather than the pre-announced profiles.avatar_asset_id column: no cross-table write, no trigger, three fewer tests; a column can be added additively when a public profile needs a join (D5).

II.4 Object keys

  • Final: profiles/<user_id>/<asset_id>/avatar.webp. The name is fixed per kind; the user's filename never reaches the server.
  • Quarantine: quarantine/<user_id>/<asset_id>/upload in the same bucket. No API role can read it: the SELECT predicate only matches the object_key of a clean row, the INSERT predicate only the quarantine_key of the caller's pending_upload row.
  • Keys are built by the RPC from uuids; the policies compare whole strings, no prefix match, no storage.foldername(), so the PGlite shim and the real stack evaluate the same expression. The asset id in the key lets the sweeper classify a leftover without a row.
  • Sheet assets keep <sha256>.svg (shared drawings, immutable versions).

II.5 Upload and processing (avatar)

  1. Browser: File.size ≤ 20 MiB (sanity), createImageBitmap decodes it (else unsupported_type), draw on a canvas with the longest side capped at 1024 px, toBlob('image/jpeg', 0.9). This is transport only (phones produce 24 MP / 8 MiB photos; a 1024 px JPEG weighs ~150 KB); the server re-decodes and re-encodes regardless. The preview shows that blob in the circle with object-fit: cover. Nothing uploaded yet.
  2. "Enregistrer la photo" → begin_avatar_upload('image/jpeg', blob.size).
  3. storage.from('avatars').upload(quarantine_key, blob, { upsert: false, contentType: 'image/jpeg' }) with the user JWT. The INSERT policy accepts only (bucket_id = 'avatars', name = quarantine_key) of a pending_upload row owned by auth.uid(); upsert would need UPDATE and SELECT rights the plan withholds on purpose.
  4. confirm_asset_upload(asset_id) → uploaded.
  5. Worker loop, fixed 2 s interval: claim_avatar_jobs(1) → GET the quarantine object (service client, 30 s timeout on every network call).
  6. Decode child: the I/O process spawns decode.ts with an empty environment (no key, PATH only), feeds the bytes on stdin, reads the result on stdout, kills it with SIGKILL after 20 s. The child: type from magic bytes (JPEG FF D8 FF, PNG 8-byte signature, WebP RIFF….WEBP), sharp(bytes, { limitInputPixels: 16_777_216 }) with default failOn, sharp.cache(false), sharp.concurrency(1), metadata() (format matches the magic; 64 ≤ sides ≤ 4096; the pixel-limit error maps to dimensions), .rotate(), .resize(512, 512, { fit: 'cover', position: 'centre' }), .webp({ quality: 82 }), no withMetadata(): EXIF, XMP, IPTC and the ICC profile are not copied (colours converted to sRGB through the profile first). Output: one 512 × 512 WebP, sha256, byte size. Data after a PNG IEND is ignored by the decoder and absent from the output.
  7. I/O process: PUT the clean bytes at object_key (upsert: true, contentType: 'image/webp'; idempotent on retry) → complete_avatar_job (commit) → DELETE the quarantine object → mark_quarantine_purged. A refusal → reject_avatar_job(reason); the quarantine object is enqueued for purge by the status trigger. A complete refused by the guard (row superseded) → the worker deletes the clean object it wrote.
  8. Browser polls its own row (1 s, up to 30 s): clean → download the photo; rejected → message from the closed code; still uploaded after 30 s → "Traitement en cours, revenez dans un instant"; on the next visit the account page loads any in-flight row and shows the same state. The save button is disabled while a row of the caller is in flight; a new pick supersedes it.

Crash matrix (the retry is idempotent at every point):

Worker dies Row Objects Recovery
before PUT clean uploaded, lease set quarantine lease expires (10 min) → re-claim
after PUT clean, before complete same quarantine + clean re-claim, PUT again (upsert), complete
after complete, before DELETE quarantine clean, quarantine_key set both sweeper: quarantine leftovers of clean rows, then mark_quarantine_purged
3rd lease expired rejected (processing_error) enqueued purge queue

II.6 Read path (owner-only, no signed URL)

storage.from('avatars').download(object_key) with the user JWT: the Storage API evaluates the SELECT policy, the bytes come back, the app keeps a URL.createObjectURL(blob) in the auth store (revoked on change or sign out) and uses it as <img src> in UserBadge, the account section and, if D3 is accepted, the top bar. One download of ~30 KB per session; the production CSP already allows blob: images, connect-src must name the Supabase origin (already required by Step 7's API calls).

Why not a signed URL: in Step 8 nobody but the owner reads the avatar, and a signed URL is a bearer token that keeps serving after remove_avatar until it expires. Signed URLs arrive with the first kind shared with other users (documents, Step 9): issued by the Storage API after the SELECT policy, TTL per kind, download flag for documents, query strings dropped from the proxy access logs.

II.7 Rejection codes (closed set, phrased by i18n)

Row codes: unsupported_type, decode_failed, dimensions, processing_error. Client-side failure kinds (never rows): too_large, unsupported_type, rate_limited, storage_full, network, unknown. The raw library or supabase-js error never reaches a row or the screen; the worker logs an allowlist of fields (asset id, code, duration) with control characters escaped (LRN-044).

II.8 Worker runtime and operations

  • apps/worker (@faunosteo/worker): TypeScript run by Node 24, deps @supabase/supabase-js and sharp only. No DB driver: every database access is an RPC through PostgREST with the service key (one credential, one transport; a managed project needs no open database port).
  • Two processes in one container: the keyed I/O loop and the unkeyed decode child per job (II.5 step 6). A decoder exploit lands in a process with no key, no file access beyond stdin/stdout and a 20 s life; it can reach Kong with anonymous rights only, which is what any browser has.
  • Container faunosteo_worker (compose service worker): non-root, read_only: true + tmpfs /tmp, mem_limit: 512m, cpus: 1, pids_limit: 64, no published port, attached to the Supabase compose network only (internal, Kong reachable, nothing else; a host firewall rule would not filter container egress), restart: unless-stopped. A watchdog exits the process when one loop iteration exceeds 60 s so the restart policy fires (Docker never restarts an unhealthy container on its own). Env file .env.worker (SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY, WORKER_ID), chmod 600, separate from the app .env, excluded from any backup copy of env files, readable through docker inspect by the docker group (documented).
  • Fairness: one in-flight avatar per account (begin supersedes), oldest first → no account can queue more than one job.
  • Sweeper every 10 min, two sources: (a) the purge queue (row-driven, II.9); (b) key-driven: list quarantine/ of the bucket, parse the asset id, delete any object older than 3 h whose row is not pending_upload / uploaded (covers bytes landed through a signed upload URL or a slow PUT after the row moved on; the 3 h grace exceeds the 2 h validity of such a URL). No operator step, no --apply.
  • sharp updates: a Renovate / dependabot rule on the worker package, not a manual chore.
  • Local dev: pnpm worker:dev against the local stack (service key typed into the shell as SUPABASE.md §8 already does).

II.9 Purge queue, orphans, replacement, deletion

private.object_purge_queue(id, bucket_id, object_name, not_before, enqueued_at) fed by triggers on assets: a transition to deleted or rejected enqueues object_key and quarantine_key; a row DELETE (account cascade) enqueues them too. not_before = now + 48 h for the final key of a deleted row (longer than any backup run, II.10), now + 1 h for quarantine keys and rejected uploads. The sweeper calls claim_purges(max_items), deletes each object (a missing object is a success), ack_purge(id). The only automated deletions are those two sources; a clean row's object_key is never deleted (test).

Event Row Object Cleanup
New avatar clean new clean, previous deleted (same transaction) previous object kept 48 h queue
remove_avatar() clean and in-flight rows deleted kept 48 h queue
New pick while in flight superseded row deleted its bytes, if any queue; a complete for it is refused
Upload never confirmed pending_upload superseded or expired by the next begin / after 24 h maybe present queue
Bytes landed after the row moved on none or not pending present key-driven sweep after 3 h
Account deleted (Step 26) rows cascade enqueued by the DELETE trigger queue

Access is cut by the row: a deleted or rejected row matches no read predicate, and no URL outlives the policy check in Step 8 (download, not signing). Object lifetime is eventual, bounded by the grace periods.

Reconciliation (operator, documented in STORAGE.md, not a tool): one SQL anti-join between storage.objects and assets (same database) lists objects without a row and rows without an object per bucket; sheet assets are checked against anatomical_sheet_assets. Reading only; deletions stay row-driven.

II.10 Backups and restore on the VPS (self-hosted)

  • Storage backend file on a named volume faunosteo_supabase_storage (kept across renames, BDR-035), on a filesystem separate from the PostgreSQL volume with a quota, and a disk-usage alert: the VPS hosts other products, a full disk would take PostgreSQL down with it. The S3 backend (Scaleway Object Storage) remains the alternative when the volume outgrows the disk: copy the files, change the backend env, rows unchanged.
  • Global ceiling: private.storage_ceiling() (constant, 10 GiB to start) checked by begin_avatar_upload on top of the per-account rate.
  • Backup order: database dump first (whole instance, storage schema included), files second. With the 48 h purge grace, every clean row of the dump still has its object in the snapshot, provided the snapshot runs within 48 h of the dump (the invariant the restore checks: every clean row has an object).
  • Restore: stop the worker, start the database, restore the dump, restore the volume, start the rest, run the reconciliation query, start the worker. Files newer than the dump have no row and are invisible to the Storage API (it lists from storage.objects); the directory listing of the volume against storage.objects finds them.
  • Tooling (restic to Scaleway Object Storage, encrypted, dump in the same run, write-only key with retention pruned from another host, the Supabase secrets JWT_SECRET / POSTGRES_PASSWORD in the encrypted set, .env.worker never in a plaintext copy) and the restore drill belong to the self-hosted deployment runbook (plan Step 27 "backup/restore rehearsal"). Step 8 writes the ordering, the invariant and the backend choice in STORAGE.md; the owner runs every backup or restore command.

II.11 Migration to managed Supabase (runbook, no tool in Step 8)

Order: freeze writes (stop the worker, maintenance notice); migrate the schema (supabase db push, buckets and policies are SQL; the storage schema itself is never dumped into managed); migrate the data rows (auth.users through the Supabase auth migration path, then public: profiles, roles, sheets, assets); copy the objects driven by the rows (assets and anatomical_sheet_assets: for every clean / confirmed row, download, verify sha256, upload at the same key, verify; paginate by keyset, PostgREST caps a set at 1000 rows); run the reconciliation query on the destination; switch SUPABASE_URL and the keys of the app and the worker; resume. The service key of a self-hosted stack is a long-lived JWT: revoking it means rotating JWT_SECRET (every key and session); documented, and a reason to prefer managed secret keys once there.

II.12 UI (follows the validated mockup, no redesign)

  • Mon compte → Informations personnelles: avatar circle (photo, else the monogram from UserBadge) with "Modifier la photo" under it, as drawn. Picking a file shows the local preview in the circle with "Enregistrer la photo" / "Annuler"; after saving, the circle shows "Traitement en cours…" until the row is clean; "Supprimer la photo" appears when a photo exists (one confirmation). Errors use account.avatar.error.<code>.
  • UserBadge (sidebar + drawer): photo when present, monogram otherwise.
  • Top bar (D3): the mockup shows avatar + name with a chevron on desktop and the avatar alone on mobile; Step 7 shipped the bar without a user element. Proposal: add the avatar (and the name on desktop) as a link to Mon compte, no dropdown (its content is undefined). Decision for the owner.
  • Dashboard unchanged (the mockup shows no avatar in the content area).
  • Accepted inputs at the picker: JPEG, PNG, WebP; never SVG for avatars. What the browser cannot decode is refused there with unsupported_type; what it decodes is sent as JPEG and judged again by the worker.

Part III — The ten security questions

  1. Access control: the assets row is the authority. The two policies on storage.objects are bucket-scoped (bucket_id = 'avatars' and private.can_read_avatar_object(uid, name), same for upload) and look the key up in assets (status, owner, kind). A bucket without policy is unreachable by API roles; the census pins one bucket literal per policy and the counts.
  2. No cross-user access: the read predicate requires a clean row with owner_id = auth.uid() whose object_key equals the requested name; the path is never trusted, only the row. Listing is denied the same way. pgTAP: Bob reads nothing of Alice, cannot upload at her key, cannot confirm her row; real-stack e2e: Bob's download of Alice's key → 400.
  3. Keys: server-built from uuids by the RPC, canonical file name per kind, no user filename anywhere, whole-string comparison in the policies.
  4. Signed URLs: not used in Step 8. An owner-only avatar is read by download under RLS and shown through a blob URL, so no bearer token exists and a removed avatar stops being readable at the row flip. When a shared kind arrives (documents, Step 9): Storage API signing after the SELECT policy, TTL per kind (60 s and download for documents), query strings dropped from access logs, the row cuts signing and the purge cuts serving.
  5. Quarantine: upload lands under quarantine/ with the row pending_upload → uploaded; no API role can read it; only the worker reads it, writes the clean bytes under the final key, commits the row, deletes the quarantine object. Every transition is an RPC; the browser cannot set clean. Bytes landed late (signed upload URL, slow PUT) are swept by key after 3 h without an operator.
  6. Dangerous SVG: avatars never accept SVG (the browser decodes raster only, the worker checks magic bytes, the bucket lists three raster types). For sheet drawings the allowlist sanitiser exists and is tested on hostile fixtures (packages/pdf); it becomes the worker's SVG processor at Step 13 with its producer, after a pre-parse byte and tag-count cap (the DOM is built before the element cap counts) and a bounded removed list (D6).
  7. Images without metadata: the worker never stores the uploaded bytes. The browser already re-encoded through a canvas (no EXIF survives), and the worker decodes again (type from magic bytes, 16.7 MP pixel cap, sides ≤ 4096), applies the EXIF orientation if any, resizes and re-encodes to WebP without withMetadata(): no EXIF, XMP, IPTC or ICC chunk in the output. Tests: an EXIF-laden JPEG fed directly to the worker yields an output without EXIF / XMP / ICCP chunks, upright.
  8. Orphans: replacement, removal, supersede and rejection flip the row in one transaction and enqueue the keys; the sweeper serves the queue after the grace period and sweeps quarantine leftovers by key; a clean row's object is never deleted; the reconciliation query reports the rest.
  9. Backups and restore: file backend on a named, quota-limited volume; dump first, files second, 48 h purge grace so every clean row of a dump has its object; restore in the documented order followed by the reconciliation query; tooling and drill in the deployment runbook.
  10. Managed migration: same migrations; rows then objects driven by the rows with hash verification; keys and URLs switched by env; write freeze during the copy.

Service role: only .env.worker and operator commands hold it; the decode child never sees it; env.js is generated from APP_* variables only.


Part IV — Scope, tests, risks, decisions

IV.1 Exact scope of the implementation

In:

  • Migration 20261012100000_files_foundation.sql: asset_status, assets (CHECKs, indexes), seven bucket rows (the existing one untouched), private.can_read_avatar_object / can_upload_avatar_object, storage.objects SELECT + INSERT policies for avatars, private.object_purge_queue + triggers, private.storage_ceiling(), the nine RPCs of II.3, explicit EXECUTE revokes. pgTAP 010_files_foundation.sql (allowed and denied cases, policy census), pgTAP 001 census update, shim extension if a column is missing, database.types.ts regenerated.
  • apps/worker: I/O loop, decode child, sweeper (queue + key-driven), watchdog, Dockerfile (node:24-alpine, non-root), compose service, .env.worker.example, vitest on fixtures (with wall-time and RSS assertions on the pathological ones), an integration test against the local stack run by the CI supabase job.
  • packages/supabase/src/assets.ts: beginAvatarUpload, uploadToQuarantine, confirmAssetUpload, removeAvatar, fetchOwnAvatar (clean or in flight), downloadAvatar; closed AssetFailure kinds.
  • apps/app: avatar block in PersonalInfoForm (picker, canvas transport, preview, save, processing, delete), UserBadge photo, top bar avatar if D3 = yes, auth store blob URL, i18n keys FR/EN, component and store tests.
  • Docs: docs/development/STORAGE.md (model, keys, statuses, worker operations, purge and grace rules, reconciliation query, backup ordering and invariant, managed migration runbook), DEPLOY.md (worker service, env file, network, CSP connect-src), SUPABASE.md (worker in dev), CLAUDE.md, CHANGELOG; plan note (worker early, SVG processor at Step 13).
  • CI: worker typecheck / lint / test / build, docker build of the worker image, worker integration test in the supabase job.

Out (named so nothing is implied): SVG processor and user SVG upload path (Steps 13 / 17), logos, documents, client / animal / report / import flows (their steps), signed URLs, image transformations, public profile photo, profiles.avatar_asset_id, the production deployment of the self-hosted stack, backup tooling and restore drill, migration and reconcile tools, account erasure (Step 26), Realtime, a user menu dropdown.

IV.2 Test scenarios

pgTAP (PGlite + real stack), every case with its denial twin:

  • buckets: eight rows, all private; avatars cap and types; census: one bucket literal per policy, 2 / 1 / 0 per bucket, total pinned; authenticated cannot insert nor select in a closed bucket.
  • begin_avatar_upload: keys of the right shape; wrong type, oversize → denied; 11th begin within the hour fails even after supersedes; ceiling reached → storage_full; a begin while a row is pending_upload or uploaded supersedes it (deleted, queue row written); anon denied.
  • upload policy: Alice inserts at her quarantine key; Bob at her key → 42501; Alice at a final key, at the key of a clean row, at a superseded key, at an unknown key → denied; no UPDATE / DELETE policy exists.
  • confirm: own pending row with the object present → uploaded; without the object → error; Bob confirming Alice's row → denied; twice → error.
  • read policy: Alice reads her clean object only; nothing while pending_upload / uploaded / rejected / deleted; the quarantine object never; Bob nothing; anon nothing.
  • worker RPCs: authenticated cannot execute any; claim leases oldest first and skips a live lease; an expired lease is re-claimed; the third expiry rejects with processing_error; complete marks the previous clean avatar deleted and enqueues it atomically; complete on a superseded row is refused; reject stores a closed code; the queue holds the right keys with the right not_before; a clean row's object_key never enters the queue.
  • remove_avatar: clean and in-flight rows → deleted; a later complete of the in-flight row is refused (the photo does not reappear).
  • cascade: deleting a profile enqueues its keys (trigger).
  • audit: asset.avatar_set, asset.avatar_removed, asset.rejected rows.

Worker (vitest, fixtures under apps/worker/tests/fixtures):

  • decode child: JPEG / PNG / WebP accepted; PNG bytes under a .jpg name accepted as PNG (the name is never seen); GIF, BMP, SVG, PDF, HTML → unsupported_type; truncated → decode_failed; 32 px → dimensions; header-declared 20 000 × 20 000 and a 5000 × 5000 → dimensions through the pixel limit; EXIF-laden rotated JPEG → upright output without EXIF / XMP / ICCP chunks; PNG with a trailer after IEND → accepted, trailer absent from the output; output always 512 × 512 WebP under a size bound; pathological-but-within-cap fixtures (progressive JPEG with many scans, large ICC) finish under the deadline with RSS asserted; a child that sleeps is killed at 20 s → processing_error.
  • I/O loop: claim → decode → PUT → complete → DELETE sequence; complete refused → the clean object is deleted; Storage error → row left for the lease; the child never receives the environment; sweeper serves the queue and the key-driven list only, never a clean object_key.

Integration (local stack, CI supabase job): Alice's full flow with the service key for the worker side → row clean, object at the final key, no quarantine object; a second upload replaces, the first object disappears once its not_before is reached (clock-shifted in the test); remove_avatar then sweep → bucket empty; a signed upload URL obtained during pending_upload and used after supersede → object swept by key.

Real-stack e2e (scratchpad script, like Step 7): Alice download 200, Bob download of Alice's key 400, anon 400, Alice's quarantine key 400, a direct GET of the object path without JWT 400.

App (vitest + happy-dom): picker refuses a file the browser cannot decode with the phrased error; preview appears before any request; save calls begin → upload → confirm in order with a JPEG blob; processing state then photo on clean; rejection message per code; delete flow; in-flight row shown on mount; save disabled while in flight; UserBadge shows the photo when the store has a blob URL, the monogram otherwise; i18n parity.

Manual validation (owner): upload, replace, delete from Mon compte on desktop and mobile with a phone photo; photo visible in the sidebar, the drawer and the top bar; open the object path in a private window (400); upload a renamed PNG (accepted) and an HTML file (refused with a clear message); sign in as another seeded user and check nothing of the first user shows.

IV.3 Risks

  • Decoder vulnerabilities (libvips / libwebp history): confined to a keyless child with a 20 s life; the keyed process never decodes; sharp updates automated. Residual: a child exploit reaches Kong with anonymous rights, which any browser has.
  • Worker down or stuck: uploads stay uploaded; the UI says so; the watchdog exits on a hung loop and the restart policy restarts the container; a stuck decode is killed by the deadline; an account can always supersede. Monitoring and alerting are Step 27.
  • Disk: separate quota-limited filesystem, global ceiling, per-account rate, 48 h grace. Residual: a burst of accounts within the ceiling fills the quota, not the host.
  • CSP in production: connect-src must include the Supabase origin (already needed by Step 7); documented in DEPLOY.md.
  • PGlite shim vs real storage schema: policies use only bucket_id and name; pgTAP never deletes storage objects (LRN-041).
  • Storage API behaviour relied on: INSERT policy on direct upload, SELECT policy on download, signed upload URL semantics; the integration test on the CI stack pins them on the pinned image.
  • Service key handling: a second env file, chmod 600, excluded from env backups; rotation = JWT_SECRET rotation on self-hosted (documented).
  • Colour: ICC profiles are converted to sRGB then dropped; only out-of-gamut colours clip. Accepted for an avatar.
  • Plan deviations: worker at Step 8 instead of 19; SVG processor at Step 13; avatar pointer derived instead of a profile column. Recorded.

IV.4 Decisions for the owner

Id Question Recommendation
D1 Server-side processing runtime Containerised worker faunosteo_worker, keyless decode child; no Edge Functions (no native decoder, WASM budget on managed, one more container on self-hosted anyway)
D2 Sheet bucket name Keep anatomical-sheet-assets, record the deviation from the announced anatomical-sheets
D3 Top bar avatar Add avatar (+ name on desktop) as a link to Mon compte, no dropdown; or sidebar / drawer / account only
D4 Avatar pipeline Browser: decode, canvas ≤ 1024 px, JPEG transport. Server: magic check, ≤ 16.7 MP, one 512 × 512 WebP, centre crop, no metadata. Inputs JPEG / PNG / WebP ≤ 5 MiB at upload
D5 Avatar pointer Derived from the single clean row (unique index), no profiles.avatar_asset_id yet
D6 SVG processor Deferred to Step 13 with its producer; the sanitiser and its hostile fixtures already exist in packages/pdf
D7 Read path download under RLS + blob URL (no signed URL, no bearer) for the owner-only avatar
D8 Processing feedback Browser polling of the own row (1 s, 30 s cap), worker at a fixed 2 s; Realtime stays off
D9 Self-hosted Storage backend File backend on a named, quota-limited volume; dump-first backups with 48 h purge grace; tooling at the deployment step