Files
faunosteo/docs/development/STORAGE.md
T

25 KiB
Raw Blame History

Files foundation: Supabase Storage, the assets table and the Rust worker

Status: Step 8 / Rust R1, 2026-10-13, branch feature/step-08-files-foundation. Design origin: STEP8_FILES_ANALYSIS.md (D1 to D9 validated by the owner, D1 amended to the Rust worker of RUST_ARCHITECTURE_REVIEW.md Part IV). This file is the implementation contract: table, keys, RPCs, policies, worker behaviour and client flow are what the code and the tests pin.

First consumer: the profile photo (avatar). Every other bucket exists, closed.


1. Model

1.1 Statuses

pending_upload ─(confirm_asset_upload)─► uploaded ─(complete_avatar_job)─► clean
      │                                      │
      │ begin supersedes / 24 h expiry        ├─(reject_avatar_job, 3rd expired lease)─► rejected
      ▼                                      ▼
   deleted ◄──────(remove_avatar, replacement by a newer clean row, begin supersedes)

deleted and rejected are terminal: the row stays (history, purge bookkeeping) and matches no read or upload predicate.

1.2 public.assets

Column Type Rule
id uuid pk
owner_id uuid → profiles.id on delete cascade
kind text 'profile_avatar' only in Step 8 (CHECK)
bucket_id text 'avatars' for that kind (CHECK)
object_key text unique final key, server-built: profiles/<owner_id>/<id>/avatar.webp
quarantine_key text quarantine/<owner_id>/<id>/upload; null once the worker confirmed the quarantine object is gone
status public.asset_status enum above
declared_mime text image/jpeg, image/png, image/webp (CHECK)
declared_size integer 1 … 5 242 880 (CHECK), the transport size announced by the browser
byte_size, sha256 integer, text the clean output, set by complete_avatar_job
rejection_reason text closed set unsupported_type, decode_failed, dimensions, processing_error; non-null iff rejected (CHECK)
attempts smallint leases taken; the 3rd expired lease rejects with processing_error
locked_at, lock_token timestamptz, uuid the lease: set by claim_avatar_jobs, required by complete / reject, 10 min validity
created_at, updated_at timestamptz trigger set_updated_at

Indexes: unique partial assets_one_clean_avatar (owner_id) where kind = 'profile_avatar' and status = 'clean' ("the" avatar of an account is its single clean row: no profile column, D5); assets_claimable (created_at) where status = 'uploaded'; assets_owner_idx (owner_id, created_at desc).

Privileges: revoke all from public, anon, authenticated; grant select (id, owner_id, kind, bucket_id, object_key, status, declared_mime, declared_size, byte_size, sha256, rejection_reason, created_at, updated_at) to authenticated (the lease columns and quarantine_key stay server-side; a client must list its columns, select=* is refused). RLS assets_select_own (owner_id = auth.uid()). No INSERT / UPDATE / DELETE policy: every write is an RPC.

1.3 Buckets (all private)

Bucket Step 8 state
avatars open: SELECT + INSERT policies through assets, 5 MiB, image/jpeg, image/png, image/webp
logos, professional-documents, client-files, animal-files, report-files, imports closed: row exists, public = false, no policy names it, no cap or MIME list yet (their step sets them)
anatomical-sheet-assets untouched (migration 20261009100000, read through the sheet)

The migration upserts the rows (on conflict (id) do update set public = false) so it converges on the local stack, the VPS and a managed project.

1.4 Keys

  • Built by begin_avatar_upload from uuids; the user's filename never reaches the server; one canonical name per kind (avatar.webp).
  • Policies compare whole strings (name = quarantine_key, name = object_key): no prefix match, no storage.foldername(), so PGlite and the real stack evaluate the same expression.
  • The worker validates every key it receives from the database before putting it in a URL: exactly four segments, profiles|quarantine, two uuids, avatar.webp|upload.

2. RPCs

Every function is security definer, set search_path = '', authorisation before lookup, EXECUTE revoked from public / anon / authenticated and granted to the one role that calls it. Business refusals raise SQLSTATE P0001 with the closed code as the message (PostgREST answers 400 with {"code":"P0001","message":"<code>"}); authorisation refusals raise 42501 (403).

2.1 Browser (authenticated)

RPC Effect Refusals (message)
begin_avatar_upload(declared_mime text, declared_size integer) returns jsonb locks the caller's profiles row (for update); checks the type and the size; rate: at most 10 rows created by the caller in the last hour, every status counted; per-account budget: the caller's bytes present under profiles/<uid>/ and quarantine/<uid>/ plus one cap below private.account_byte_budget() (64 MiB) → rate_limited; global ceiling: the bytes present in the bucket (sum of sized storage.objects, whatever each row's state) plus one 5 MiB cap per pending_upload row whose bytes have not landed, below private.storage_ceiling() (10 GiB): a claim dies with its row, so begin spam pins nothing; residuals: about 100 to 160 colluding confirmed accounts at their budget fill the ceiling (bytes still streaming for a superseded row weigh 0 until they land, bounded by rate × cap), about 2 000 accounts each keeping one bytes-less pending claim alive do the same, and a burst of accounts fills the filesystem quota, the hard bound; supersedes the caller's pending_upload / uploaded rows (→ deleted); inserts the row; returns {"asset_id", "bucket_id", "quarantine_key"} unsupported_type, too_large, rate_limited, storage_full
confirm_asset_upload(asset_id uuid) returns void active account; own pending_upload row with a storage.objects row at (bucket_id, quarantine_key) whose metadata->>'size' is set and within the cap → uploaded not_found (unknown or not own), invalid_state (not pending), upload_missing (no object yet, or an object without a size: fail closed), too_large (object over the cap or over the declared size: the row stays pending and expires after 24 h)
remove_avatar() returns integer every profile_avatar row of the caller in pending_upload / uploaded / clean → deleted; returns the count; logs asset.avatar_removed none (0 rows is fine)

auth.uid() is null → 42501 on all three.

2.2 Worker (service_role)

RPC Effect
claim_avatar_jobs(max_jobs integer) returns table (asset_id, owner_id, bucket_id, object_key, quarantine_key, lock_token, attempts) first rejects (processing_error) every uploaded row whose lease expired with attempts >= 3; then claims uploaded rows with locked_at is null or locked_at < now() - 10 min and attempts < 3, oldest first, for update skip locked, at most max_jobs (capped at 10): locked_at = now(), attempts + 1, fresh lock_token. VOLATILE (a STABLE function would run read-only through PostgREST)
complete_avatar_job(asset_id uuid, lock_token uuid, byte_size integer, sha256 text) returns void guard: status = 'uploaded' and the token matches, else invalid_state. In one transaction: the owner's previous clean avatar → deleted (trigger enqueues its object 48 h later), the row → clean with byte_size / sha256, lease cleared; logs asset.avatar_set (actor null = worker, target = owner)
reject_avatar_job(asset_id uuid, lock_token uuid, reason text) returns void same guard; reason in the closed set else invalid_reason; → rejected; logs asset.rejected with the reason
mark_quarantine_purged(asset_id uuid) returns void clean row → quarantine_key = null (after the worker deleted the quarantine object); other statuses: no-op
claim_purges(max_items integer) returns table (purge_id, bucket_id, object_name) queue rows with not_before <= now() and no live lease (locked_at null or older than 10 min), for update skip locked, at most max_items (capped at 100); sets locked_at
ack_purge(purge_id uuid) returns void deletes the queue row (a missing object counted as success by the worker)
claim_quarantine_leftovers(older_than_seconds integer, max_items integer) returns table (bucket_id, object_name, asset_id) storage.objects under quarantine/ in avatars, created before now() - older_than, whose row is not pending_upload / uploaded (anti-join on quarantine_key); asset_id is the matching row when one exists (the worker calls mark_quarantine_purged for a clean one)
expire_pending_uploads(max_age_seconds integer) returns integer pending_upload rows older than the age → deleted (trigger enqueues their keys); returns the count

The three stale-task guards together: a superseded or removed row is deleted (never uploaded, so no complete); a complete from an expired lease carries a stale lock_token and is refused even while the row is still uploaded; the purge of an old key never touches a newer row's object (keys embed the asset id).

2.3 storage.objects policies (bucket avatars)

Policy Command Expression
avatars_read SELECT, authenticated bucket_id = 'avatars' and private.can_read_avatar_object((select auth.uid()), name) = a clean row of the caller with object_key = name
avatars_upload INSERT, authenticated bucket_id = 'avatars' and private.can_upload_avatar_object((select auth.uid()), name) = a pending_upload row of the caller with quarantine_key = name

No UPDATE / DELETE policy exists for any API role. The pgTAP census asserts that every policy on storage.objects pins exactly one bucket literal and counts them: 2 on avatars, 1 on anatomical-sheet-assets, 0 elsewhere, total 3.

2.4 Purge queue (private.object_purge_queue)

(id, bucket_id, object_name, not_before, enqueued_at, locked_at, attempts), unique on (bucket_id, object_name) (a second enqueue keeps the earliest not_before). Fed by triggers on assets:

Transition object_key quarantine_key
→ deleted now() + 48 h now() + 1 h
→ rejected now() + 1 h (a clean object may exist after a PUT before a 3rd expired lease) now() + 1 h
row DELETE (account cascade) now() + 48 h now() + 1 h

A clean row's object_key is never enqueued (test). The 48 h grace is the backup invariant of §5.


3. The worker (crates/worker, container faunosteo_worker)

3.1 Process model

One static musl binary, two roles in one container:

  • faunosteo-worker run: the keyed parent. Holds SUPABASE_SERVICE_ROLE_KEY, talks to PostgREST and the Storage API over HTTP, never decodes a pixel.
  • faunosteo-worker decode: the child spawned per job by the parent with an empty environment and piped stdin/stdout (R0 sandbox unchanged: Threads: 1, no core dump, RLIMIT_AS 512 MiB, RLIMIT_CPU 10 s / 12 s, seccomp allowlist, 5 MiB input, 512 KiB output, 20 s deadline). It sees no key, no file, no network.

The parent checks the child's output at container level only (webp_check: RIFF structure, chunk allowlist, 512 × 512 canvas, no metadata chunk), hashes it and uploads it.

3.2 Configuration (environment of the parent only)

Variable Required Default Meaning
SUPABASE_URL yes origin of the stack (http://supabase_kong_faunosteo:8000 from the Supabase compose network, http://127.0.0.1:54321 natively). Cleartext http:// is accepted only for localhost, a dotted-quad loopback address, or a single-label host starting with a letter (a Docker service name; inside the musl container Docker's ndots:0 makes musl skip the search list; glibc, natively, may still try search domains on a mistyped name, so prefer loopback there); any other host must be https://, the service key travels in the headers
SUPABASE_SERVICE_ROLE_KEY yes the service key; never printed; the JWT exp claim, when the key is a JWT, is logged at start and warned about 30 days ahead
WORKER_ID no worker-<pid> tag of every log line
WORKER_POLL_SECONDS no 2 idle interval between claims
WORKER_SWEEP_SECONDS no 600 interval of the sweeper
WORKER_QUARANTINE_GRACE_SECONDS no 10800 age before a quarantine leftover is deleted (above the 2 h validity of a signed upload URL)
WORKER_PENDING_MAX_AGE_SECONDS no 86400 age before a never-confirmed pending_upload row is expired
WORKER_ONCE no unset 1 = one claim pass and one sweep, then exit (tests)

faunosteo-worker run --check prints the resolved configuration without any secret value. Exit 0 when the worker could start (libwebp ≥ 1.6.0, both required variables set, URL parseable) or when no worker variable is set at all (an unconfigured image is a valid artifact check, scripts/check-all.sh); a partial or invalid configuration exits 2 naming the variable, a too-old libwebp exits 18.

3.3 HTTP endpoints used (service key: apikey and Authorization: Bearer)

Call Request
RPC POST <url>/rest/v1/rpc/<name>, JSON body, Content-Type: application/json, Prefer: return=representation
download quarantine GET <url>/storage/v1/object/<bucket>/<key>, body read with a 5 MiB + 1 cap; a body over the cap (impossible through the bucket cap and confirm_asset_upload, kept as a guard) is a content refusal: the row is rejected decode_failed and the worker logs download_overflow
upload clean POST <url>/storage/v1/object/<bucket>/<key>, Content-Type: image/webp, x-upsert: true (idempotent on retry)
delete object DELETE <url>/storage/v1/object/<bucket>/<key> (a missing object counts as done: HTTP 404, or the HTTP 400 with statusCode 404 this Storage API version answers)

Every call: 30 s global timeout, 5 s connect timeout, no redirect follow, no proxy (the environment's HTTP_PROXY is ignored on purpose), http_status_as_error(false) so each status is classified by the caller. Log lines carry the closed error kind and the HTTP status, never the library's error text. Keys are validated (§1.4) before any URL is built.

3.4 One job

claim_avatar_jobs(1)                        stage claim     deadline 35 s
GET quarantine                              stage download  35 s   404 → reject decode_failed (bytes never landed; the row said uploaded)
spawn decode child, feed bytes              stage decode    25 s
  Produced(webp)  → POST clean (upsert) → complete_avatar_job(asset, token, size, sha256)
                                            stage upload 35 s, stage commit 35 s
                    → DELETE quarantine → mark_quarantine_purged     (best effort: the sweeper finishes it)
  Rejected{code}   → reject_avatar_job(asset, token, code)           code ∈ unsupported_type | decode_failed | dimensions | resource_limit→decode_failed
  Signalled / Timeout / OutputOverflow / Failed
                   → reject_avatar_job(asset, token, 'processing_error')  (logged with signal number / detail)
  Broken{code}     → log level=error event=child_broken, leave the lease (the row is retried, 3rd lease rejects), stop the loop with exit code 20
complete refused (invalid_state: superseded, removed, stale lease)
                   → log, delete nothing: the purge queue owns every object of a terminal row

Every network failure in a stage leaves the row to its lease (re-claimed after 10 min, rejected processing_error on the 3rd expiry). A stage that exceeds its deadline makes the watchdog thread exit the process (code 21); Docker's restart policy brings it back.

3.5 Sweeper (every WORKER_SWEEP_SECONDS, and once at start)

  1. expire_pending_uploads(WORKER_PENDING_MAX_AGE_SECONDS).
  2. claim_purges(100) → DELETE each object (404 = done) → ack_purge.
  3. claim_quarantine_leftovers(WORKER_QUARANTINE_GRACE_SECONDS, 100) → DELETE each object → mark_quarantine_purged(asset_id) when present.

Those are the only deletions the worker ever makes: a key it did not get from the queue or the leftover query is never deleted.

3.6 Sheet verification mode (V1-2.3)

faunosteo-worker run --jobs sheet-verify, service worker-verify under the Compose profile worker (same image, environment file and hardening as worker, but mem_limit 1 280 MiB so the child's 1 GiB limit binds first), runs the server verdict of submitted sheet rounds (V1-2 contract §6.4, §6.5) and nothing else: no avatar job, no sweep. The default run is unchanged.

  • Parent: claim_sheet_verifications(1) (M6, service role only), then get_sheet_verification_input (the frozen document and the base as text, the drawing rows), then each drawing from anatomical-sheet-assets at its key <sha256>.svg. A row not uploaded, a key that is not its own sha256, an object missing (404) or over the policy (at most 4 drawings, 2 MiB each, 6 MiB in all: the archive caps) is passed to the core without bytes, so the verdict names it (drawing_missing, caps): a business failure, never a worker failure. The parent never parses the document or a drawing.
  • Child faunosteo-worker verify: same sandbox as decode (empty environment, one thread, no core dump, the seccomp allowlist) with its own limits: framed stdin input up to 12 MiB, RLIMIT_AS 1 GiB, RLIMIT_CPU 60 s, 90 s deadline, verdict output up to 3 MiB (under the SQL cap of 4 MiB on jsonb::text). It runs the core's compute_verdict and writes the verdict JSON; a failed verdict is still a normal exit. The seccomp derivation of 2026-11-01 traced brk in the verify child at the caps (musl mallocng growing its heap): added to the allowlist, the only change.
  • Completion: complete_sheet_verification answers passed, failed, stale (nothing written, logged) or malformed (reported verification.worker.verdict_refused). Child timeout → verification.worker.timeout (transient); SIGXCPU, SIGABRT or SIGSYS (the child's own CPU, memory and seccomp budgets), an output overflow or an input over its cap → verification.worker.resource_limit (definitive: the frozen input gives the same end every time); any other signal, SIGKILL first (the OOM killer or a stop, from outside) and a transport error → verification.worker.unavailable (transient); a broken child → unavailable, then the worker stops (as the avatar loop). A dead lease (10 minutes) is claimed again as verification.worker.lease_expired. Retry by cause: resource_limit and verdict_refused close the round verification_failed / unavailable at the first occurrence; the other codes retry up to the fifth attempt, which closes it the same way.
  • Retry delay (owner decision 2026-11-01): a failure reported by fail_sheet_verification makes the round claimable again after 2 s, 5 s, 10 s, 20 s (after attempts 1 to 4), kept in SQL (not_before, hidden from the API roles), so the worker never sleeps for it and the claim of another round is not blocked; the effective delay is that delay plus up to one poll interval (WORKER_POLL_SECONDS, 2 s). A verdict (failed or not) is never retried; a dead lease is claimed again at once. Only the transient codes wait: the delay applies to timeout, unavailable and lease_expired (SQL decides, private.sheet_worker_final_codes).
  • Measured (2026-11-01): the at-caps document (64 views, 1 024 annotations, 8 192 nodes) verified in 185 ms by the worker on the sandbox, verdict 24 689 bytes; the equine draft in about 60 ms.
  • Sandbox only: pnpm db:sandbox verify-e2e starts a gateway and this worker on the sandbox network (SUPABASE.md §5.1). Never started on DEV (V1-2 contract §16.11 item 3).

3.7 Logs

One JSON object per line on stdout, fields from an allowlist only: ts, level, event, worker_id, asset_id, review_id, stage, code, signal, http_status, duration_ms, count, detail (worker-authored text only). The child's stderr is child-controlled: it is logged under child_stderr JSON-escaped and truncated to 200 bytes, never raw. No key, no URL with a token, no user content.


4. Client flow (browser, anon key + user JWT)

  1. Pick: <input type="file" accept="image/jpeg,image/png,image/webp">. The first bytes are sniffed (FF D8 FF, PNG signature, RIFF….WEBP); anything else, SVG included, is refused with unsupported_type before any decode. File.size over 20 MiB → too_large.
  2. Transport: createImageBitmap → canvas with the longest side at 1024 px → toBlob('image/jpeg', 0.9); over 5 MiB after that → too_large. The preview shows that blob. Nothing uploaded yet.
  3. "Enregistrer la photo": begin_avatar_upload('image/jpeg', blob.size) → storage.from('avatars').upload(quarantine_key, blob, { upsert: false, contentType: 'image/jpeg' }) → confirm_asset_upload(asset_id).
  4. Poll the own row every 1 s, 30 s at most: clean → download object_key (storage.from('avatars').download), keep a blob URL in the auth store; rejected → message per code; still uploaded after 30 s → "processing, come back later" (the next mount polls again). Only a row the section knows at mount resumes: after a full page reload the section mounts before the auth store has read the rows, so it shows the clean photo with « Modifier » and « Supprimer » enabled, and a new save supersedes the waiting row (deleted, keys enqueued). An in-app return to Mon compte resumes the polling. Without a running worker (a DEV stack where docker compose --profile worker up worker was not started, §3.2 of SUPABASE.md) every new photo stays uploaded and ends on that message; the clean photo is kept (AVATAR-01, 2026-10-27, isolated reproduction).
  5. "Supprimer la photo": confirmation dialog → remove_avatar() → monogram.

Closed failure kinds the UI phrases (AssetFailure.kind): too_large, unsupported_type, rate_limited, storage_full, upload_missing, invalid_state, not_found, network, unknown; rejection codes of a row: unsupported_type, decode_failed, dimensions, processing_error.

No signed URL, no bearer token: the download runs under the SELECT policy and stops at the row flip (D7). The production CSP already allows blob: images; connect-src must name the Supabase origin (DEPLOY.md).


5. Operations

5.1 Reconciliation (read-only, operator)

-- objects without a row (any status) and rows whose object is missing
select o.bucket_id, o.name from storage.objects o
  where o.bucket_id = 'avatars'
    and not exists (select 1 from public.assets a where a.object_key = o.name or a.quarantine_key = o.name);
select a.id, a.object_key from public.assets a
  where a.status = 'clean'
    and not exists (select 1 from storage.objects o where o.bucket_id = a.bucket_id and o.name = a.object_key);

Deletions stay row-driven (queue and leftover sweep); the query only reports.

5.2 Backups (self-hosted, file backend)

Order: database dump first (whole instance, storage schema included), files second, within 48 h of each other. With the 48 h purge grace every clean row of the dump still has its object in the file snapshot (the invariant the restore checks with the second query above). Restore: stop the worker, restore the database, restore the volume, run the reconciliation, start the worker. Tooling (restic, retention, drill) is the deployment runbook's (plan Step 27).

5.3 Managed migration

Same migrations (supabase db push); rows (auth.users, public), then the objects driven by the rows (clean rows and confirmed sheet assets: download, verify sha256, upload at the same key, verify), write freeze while copying (worker stopped), reconciliation on the destination, switch SUPABASE_URL and the keys of the app and the worker.


6. Development

pnpm db:start                                  # local stack (migrations + seed)
pnpm worker:image                              # faunosteo_worker:dev
docker run --rm --read-only --cap-drop ALL --security-opt no-new-privileges:true \
  --network supabase_network_faunosteo \
  -e SUPABASE_URL=http://supabase_kong_faunosteo:8000 \
  -e SUPABASE_SERVICE_ROLE_KEY="$(pnpm -s supabase status -o env | sed -n 's/^SERVICE_ROLE_KEY="\(.*\)"/\1/p')" \
  faunosteo_worker:dev run                     # the worker beside the local stack, for the manual photo flow (IDENTITY.md); the e2e suite refuses this stack
pnpm e2e:avatar                                # = pnpm db:sandbox avatar-e2e: the suite and the worker (5 s sweep, no quarantine grace) on the throwaway stack only (SUPABASE.md §5.1); the DEV script was removed on 2026-11-01

docker compose --profile worker up worker runs the same container from docker-compose.yml with .env.worker (SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY, WORKER_ID; chmod 600, never in the app .env, never in a backup of env files).