docs(docs): Step 8 files foundation analysis — storage model, worker, quarantine, policies, backups, scope

This commit is contained in:
bchanot committed 2026-10-12 01:11:28 +02:00
1 parent 505b83087b
commit 54df4d851a
1 file changed
+578
+578
View File
@@ -0,0 +1,578 @@
# Step 8 — Files foundation and Supabase Storage: technical analysis
Status: proposal for the owner's decision, 2026-10-12. Nothing implemented.
Applies the owner's brief of 2026-10-12 (Step 7 closed; hosting = self-hosted
Supabase first, managed later; eight private buckets; quarantine + server
validation; avatar as first consumer; service role never in the browser).
Part I states the facts the design rests on. Part II is the proposed design.
Part III answers the ten security questions of the brief. Part IV lists the
exact scope, the tests, the risks and the decisions the owner has to take.
---
## Part I — Facts
### I.1 What exists today
| Area | State |
|---|---|
| Storage | One bucket, `anatomical-sheet-assets` (migration `20261009100000`): private, SVG only, 20 MiB cap, content-addressed keys `<sha256>.svg`, SELECT policy through the sheet (`private.can_read_sheet_asset_object`), no API write policy, operator upload script with the service key, `confirm_sheet_asset_upload` RPC (service role). pgTAP 006 (28 checks) runs on the PGlite shim and on the real stack. |
| Storage API | `storage-api v1.79.36` in the local stack (same image family as a self-hosted deployment). Signed URLs, signed upload URLs and RLS on `storage.objects` are available; image transformation needs `imgproxy`, absent here and a paid add-on on managed. |
| Profiles | `avatar_asset_id` was deliberately deferred to this step (comment in `20261008100100_profiles.sql`). |
| Shell UI | `UserBadge` (monogram) in the sidebar and the drawer; the top bar holds only the language switch; "Mon compte" = stacked sections, `PersonalInfoForm` first. The validated mockup shows the avatar with "Modifier la photo" inside the Profil section and an avatar + name in the top bar. |
| SVG sanitiser | `packages/pdf/src/svg/sanitize.ts`: allowlist of elements and attributes, `href` / `url()` kept in-document, backslash refused, caps 20 000 elements / depth 64 / 256 `<use>` / `stdDeviation` 100, DOM Level 2 on `@xmldom/xmldom`, `SvgRejectedError`. Browser-free. |
| Edge runtime | Disabled in `config.toml`. `BACKEND_ARCHITECTURE.md` §6: self-hosted Edge Functions are hand-deployed; first steps lean on SQL. C3: PostgreSQL-backed jobs before a queue. C12: the sanitiser runs between upload and publication, never "on upload". |
| Worker | The plan creates `apps/worker` at Step 19 (notifications). Files processing needs server-side code before that. |
| Deploy | No backend in production yet. The VPS `.env` is documented as non-sensitive. CSP: `default-src 'self'`, `img-src 'self' data: blob:` (a Supabase origin would be blocked for both API calls and images). |
| Tests | pgTAP on PGlite + real stack; app vitest on happy-dom; Step 7 added a headless e2e script against the local stack (scratchpad, Mailpit). |
### I.2 Corrections to the plan's Step 8 text
- "RPC `issue_signed_url(asset_id)`" cannot exist in SQL: signing needs the
Storage API's secret. The Storage API signs a URL only after evaluating the
SELECT policy of `storage.objects` as the caller, so the access check and
the signature live in one place already. The design uses it directly.
- "Edge Function or SQL-only sanitizer": SQL cannot parse XML or decode
images safely; Edge Functions cannot load a native image library and are
hand-deployed on self-hosted. A containerised worker is the fit (II.1).
---
## Part II — Proposed design
### II.1 Components
```text
Browser (anon key + user JWT)
│ begin_avatar_upload (RPC) ─► assets row pending_upload + quarantine key
│ PUT object (Storage API, INSERT policy = that key, that row, that user)
│ confirm_asset_upload (RPC) ─► status uploaded
│ poll own assets row ─► clean | rejected
│ createSignedUrl(object_key, 3600) (Storage API, SELECT policy) ─► <img>
▼
Supabase (self-hosted compose, later managed)
PostgREST ─ public.assets, RPCs, private helpers Storage API ─ 8 private buckets
▲
│ claim_asset_jobs / complete_asset_job / reject_asset_job (RPC, service role)
│ GET quarantine object, PUT clean object, DELETE quarantine (service role)
apps/worker (container faunosteo_worker, outbound only, no port)
processors: image (sharp) · svg (@faunosteo/pdf allowlist)
sweeper: purge objects of deleted/rejected rows, abandoned uploads, stale jobs
operator commands: reconcile, migrate
```
Why a worker and not Edge Functions: Edge Functions on the self-hosted stack
run in a separate `edge-runtime` container, are deployed by hand, cannot
load `sharp` (native), and the config keeps them off. Why not SQL: no image
decoding, no XML parsing. The worker is one Node 24 container with the
service key in its own env file, no inbound port, the same code against the
local stack, the VPS stack and a managed project (URL + key change only).
It is the `apps/worker` the plan creates at Step 19; it arrives earlier with
one job family, Step 19 adds its own.
Dependency added: `sharp` (0.35.x, libvips, prebuilt `linuxmusl-x64`
package, no install script, Node ≥ 20.9). It decodes JPEG / PNG / WebP,
resizes, re-encodes, and drops metadata by default. The only widely
maintained option that does all four in Node; pure-JS stacks cover PNG only
or are unmaintained.
### II.2 Buckets
All eight private (`public = false`). Created by one migration with
`on conflict (id) do update` so the migration converges on the local stack,
the VPS and managed alike. `file_size_limit` and `allowed_mime_types` on the
bucket are a first filter on what the Storage API accepts, never a proof of
content.
| Bucket | Kind(s) | Owning domain and step | Step 8 state |
|---|---|---|---|
| `avatars` | profile_avatar | Identity (self) | **open**: SELECT + INSERT policies through `assets` |
| `logos` | organization_logo | Organizations, Step 10 | closed, no policy |
| `professional-documents` | professional_document | Verification, Step 9 | closed |
| `client-files` | client_file | Clients, Step 12 | closed |
| `animal-files` | animal_file | Animals, Step 12 | closed |
| `report-files` | report_file | PDF artifacts, Step 15 | closed |
| `anatomical-sheet-assets` | anatomical_sheet_asset | Sheets (existing policy kept) | unchanged |
| `imports` | import_file | Imports, Step 24 | closed |
"Closed" = the bucket row exists, RLS on `storage.objects` has no policy
naming it, so `anon` and `authenticated` can neither list, read, upload,
update nor delete. Only the service role touches it. A pgTAP census pins
the exact policy count per bucket, so a later step cannot open a bucket by
accident without changing the test.
Open point (D2): the brief names the sheet bucket `anatomical-sheets`; the
repository already has `anatomical-sheet-assets` with its policy, its
operator script, its tests and its seed. Recommendation: keep the existing
name and record the deviation; a rename costs a migration, a re-upload and
touches Step 13 for no behaviour.
Proposed caps (closed buckets get the final value at their own step):
| Bucket | Size | Declared types |
|---|---|---|
| avatars | 5 MiB | image/jpeg, image/png, image/webp |
| logos | 2 MiB | image/jpeg, image/png, image/webp |
| professional-documents | 20 MiB | application/pdf, image/jpeg, image/png |
| client-files, animal-files | 20 MiB | pdf + the three image types |
| report-files | 50 MiB | application/pdf |
| anatomical-sheet-assets | 20 MiB (existing) | image/svg+xml |
| imports | 50 MiB | application/pdf, application/json, text/csv |
### II.3 `public.assets` — the authority on every user file
One row per file, created before the bytes exist, kept after they are gone
(status `deleted`) until the sweeper has purged the object.
```sql
create type public.asset_kind as enum (
'profile_avatar', 'organization_logo', 'professional_document',
'client_file', 'animal_file', 'report_file', 'anatomical_sheet_asset', 'import_file');
create type public.asset_status as enum (
'pending_upload', 'uploaded', 'processing', 'clean', 'rejected', 'deleted');
create table public.assets (
id uuid primary key default gen_random_uuid(),
owner_id uuid not null references public.profiles (id) on delete cascade,
kind public.asset_kind not null,
bucket_id text not null, -- derived from kind, checked by trigger
resource_id uuid, -- profile id for avatars
object_key text not null unique, -- final key, server-built
quarantine_key text, -- upload key, null once purged
status public.asset_status not null default 'pending_upload',
declared_mime text not null,
declared_size integer not null check (declared_size between 1 and 52428800),
original_filename text check (char_length(original_filename) <= 120), -- display only
detected_mime text,
byte_size integer,
sha256 text check (sha256 ~ '^[0-9a-f]{64}$'),
width integer,
height integer,
rejection_reason text, -- closed codes, see II.6
attempts smallint not null default 0,
locked_at timestamptz,
created_at timestamptz not null default now(),
updated_at timestamptz not null default now(),
processed_at timestamptz,
deleted_at timestamptz,
purged_at timestamptz -- object removed by the sweeper
);
-- one avatar in flight per account
create unique index assets_one_avatar_in_flight
on public.assets (owner_id) where kind = 'profile_avatar' and status in ('pending_upload','uploaded','processing');
alter table public.profiles add column avatar_asset_id uuid references public.assets (id);
```
Privileges: `revoke all` from API roles, `grant select` to `authenticated`,
RLS `assets_select_own` (`owner_id = auth.uid()`). No INSERT / UPDATE /
DELETE policy: every write is an RPC. `profiles.avatar_asset_id` is not in
the column grant: only the server sets it, and a trigger checks that it
points to a `clean` `profile_avatar` of the same profile.
RPCs (security definer, `set search_path = ''`, authorisation before any
lookup, EXECUTE revoked from PUBLIC, granted per role, audited through
`private.log_event` where the state is sensitive):
| RPC | Role | Effect |
|---|---|---|
| `begin_avatar_upload(declared_mime, declared_size, original_filename)` | authenticated | caps (type in list, size ≤ 5 MiB), rate (≤ 10 rows created in the last hour), replaces a stale `pending_upload` of the same account; inserts the row with `object_key = profiles/<uid>/<id>/avatar.webp` and `quarantine_key = quarantine/<uid>/<id>/upload`; returns `{asset_id, bucket_id, quarantine_key}` |
| `confirm_asset_upload(asset_id)` | authenticated | own row, `pending_upload` → checks the object exists at `quarantine_key` with `metadata->>'size'` ≤ cap → `uploaded` |
| `remove_avatar()` | authenticated | clears `profiles.avatar_asset_id`, marks the asset `deleted`; logs `asset.avatar_removed` |
| `claim_asset_jobs(limit)` | service_role | `uploaded` → `processing` with `for update skip locked`, `locked_at`, `attempts + 1`; returns the rows |
| `complete_asset_job(asset_id, result)` | service_role | `processing` → `clean` with detected fields; for an avatar: sets `profiles.avatar_asset_id`, marks the previous avatar asset `deleted`, in the same transaction; logs `asset.avatar_set` |
| `reject_asset_job(asset_id, reason)` | service_role | `processing` → `rejected`; logs `asset.rejected` |
| `reclaim_stale_jobs()` | service_role | `processing` older than 10 min → `uploaded` while `attempts < 3`, else `rejected (processing_error)` |
| `list_purgeable_assets()` / `mark_asset_purged(asset_id)` | service_role | rows whose object must go (`deleted`, `rejected`, abandoned `pending_upload` > 24 h, `clean` with a quarantine key still present) |
Why `assets` does not absorb `anatomical_sheet_assets`: that table is
content-addressed and sheet-scoped (BDR-032/034) with its own confirmed
upload path; merging would rewrite a validated model for no gain. The SVG
processor of the worker will serve it at Step 13 through a
`sanitization_status` added then.
### II.4 Object keys
- Final: `<resource-plural>/<resource-id>/<asset-id>/<canonical-name>`. Avatar:
`profiles/<user_id>/<asset_id>/avatar.webp`. The name is fixed per kind;
the user's filename never enters a key (it is stored, length-capped and
control-character-free, for display only).
- Quarantine: `quarantine/<user_id>/<asset_id>/upload` in the same bucket.
No API role can read it: the SELECT predicate only matches the
`object_key` of `clean` rows, the INSERT predicate only the
`quarantine_key` of the caller's `pending_upload` row.
- Keys are built by the RPC from uuids; the policies compare whole strings
(no prefix match, no `storage.foldername()` so the PGlite shim and the real
stack evaluate the same expression).
- Sheet assets keep `<sha256>.svg` (shared drawings, immutable versions).
Avatars are per-account and mutable: a hash key would need reference
counting for nothing.
### II.5 Upload, processing, display (avatar)
1. Browser pre-checks (feedback only): size ≤ 5 MiB, magic bytes of
JPEG / PNG / WebP, the file decodes (`createImageBitmap`). Preview from
`URL.createObjectURL`, shown in the circle with `object-fit: cover`,
nothing uploaded yet.
2. "Enregistrer" → `begin_avatar_upload` → row + keys.
3. `storage.from('avatars').upload(quarantine_key, file, { upsert: false })`
with the user JWT. The INSERT policy on `storage.objects` accepts only
`(bucket_id = 'avatars', name = quarantine_key)` for a `pending_upload`
row owned by `auth.uid()`.
4. `confirm_asset_upload(asset_id)` → `uploaded`.
5. Worker loop (every 2 s, backoff to 15 s when idle): `claim_asset_jobs(1)`
→ download the quarantine object with the service client → processor.
6. Image processor: detect the type from bytes (JPEG `FF D8 FF`, PNG
8-byte signature, WebP `RIFF….WEBP`); any other → `not_an_image`.
`sharp(bytes, { limitInputPixels: 24e6, failOn: 'error', animated: false })`
→ header check (`metadata()`: format matches the magic, 64 ≤ sides ≤ 8192)
→ `.rotate()` (applies EXIF orientation, then EXIF is dropped)
→ `.resize(512, 512, { fit: 'cover', position: 'centre' })`
→ `.webp({ quality: 82 })`, no `withMetadata()` → EXIF, XMP, IPTC and ICC
are gone. Output: one 512 × 512 WebP, sha256, byte size.
7. Worker uploads the clean bytes at `object_key` (service client), deletes
the quarantine object, calls `complete_asset_job` (clean + profile
pointer + previous avatar marked `deleted`, one transaction). On a
processor refusal: `reject_asset_job(reason)` and the quarantine object
is deleted.
8. Browser polls its own `assets` row (1 s, up to 30 s): `clean` → refresh
the profile and the signed URL; `rejected` → message from the closed
reason code; still `processing` after 30 s → "toujours en cours" with a
retry on next visit.
9. Display: `createSignedUrl(object_key, 3600)` with the user JWT. The
Storage API evaluates the SELECT policy, then signs. The URL is cached in
the auth store with its expiry and renewed five minutes before. Used as
`<img src>` in `UserBadge` (sidebar, drawer), the account section and,
if the owner approves D3, the top bar.
Why no client-side resize before upload: the server re-encodes anyway and
is the boundary; a second pipeline would duplicate the crop rule and could
drift. The 5 MiB cap keeps the upload short.
### II.6 Rejection codes (closed set, phrased by i18n)
`not_an_image`, `unsupported_type`, `too_large`, `decode_failed`,
`dimensions` (too small or too large), `svg_rejected`, `processing_error`.
The raw library message never reaches the row nor the screen; it is logged
by the worker with the asset id only.
### II.7 SVG processor (anatomical sheets)
Same worker, kind `anatomical_sheet_asset`: parse with `@xmldom/xmldom`
(parse errors fatal), `sanitizeSvgRoot` from `@faunosteo/pdf` (allowlist +
caps), serialise, 20 MiB cap in and out, refusal → `svg_rejected`. The
removed element and attribute names are kept in the result (never values).
In Step 8 there is no user producer: uploads into the sheet bucket stay
operator-only until the editor step (17) and the database sheets step (13)
open a signed upload path with `sanitization_status` on
`anatomical_sheet_assets`. The processor is tested on hostile fixtures and
through the worker with the service key, so the boundary exists before the
producer does (D6).
### II.8 Worker runtime and operations
- `apps/worker` (`@faunosteo/worker`): TypeScript, Node 24, deps
`@supabase/supabase-js`, `sharp`, `@faunosteo/pdf` (sanitiser),
`@xmldom/xmldom`. No DB driver: every database access is an RPC through
PostgREST with the service key, so one credential and one transport, and a
managed project needs no open database port.
- Container `faunosteo_worker` (compose service `worker`): non-root,
`read_only: true` + `tmpfs /tmp`, `mem_limit: 512m`, `pids_limit`, no
published port, `restart: unless-stopped`, healthcheck = heartbeat file
refreshed each loop. Env file `.env.worker` (`SUPABASE_URL`,
`SUPABASE_SERVICE_ROLE_KEY`, `WORKER_ID`), `chmod 600`, separate from the
app `.env` which DEPLOY.md keeps non-sensitive. Egress limited to the
Supabase host (documented firewall rule).
- One job at a time, `sharp.concurrency(1)`; time is bounded by the input
caps (≤ 5 MiB, ≤ 24 MP), not by a kill switch; a crashed run is reclaimed
by `reclaim_stale_jobs` (max 3 attempts).
- Sweeper every 10 min: purge objects of `deleted` / `rejected` rows and of
abandoned uploads, remove quarantine leftovers of `clean` rows, mark
`purged_at`. Objects with no row (orphans from a crash between PUT and
RPC) are only reported; `pnpm worker reconcile --apply` deletes them after
an operator reads the dry run.
- Local dev: `pnpm worker:dev` against the local stack (service key from
`supabase status`, typed into the shell as SUPABASE.md §8 already does).
- Logging: structured lines, asset id and reason codes only; never bytes,
names or user email.
### II.9 Orphans on replacement and deletion
| Event | Row state | Object state | Who cleans |
|---|---|---|---|
| New avatar clean | new `clean`, previous `deleted` (same transaction) | previous object still present | sweeper (purge, idempotent) |
| `remove_avatar()` | `deleted`, `profiles.avatar_asset_id = null` | present until purge | sweeper |
| Upload never confirmed | `pending_upload` | maybe present | sweeper after 24 h |
| Worker dies after PUT clean, before `complete` | `processing` → reclaimed | clean object present | retry: `upsert: true` on the same deterministic key, then `complete` |
| Worker dies after `complete`, before quarantine DELETE | `clean`, `quarantine_key` set | quarantine present | sweeper (quarantine leftovers of clean rows) |
| Account deleted (Step 26) | rows cascade | objects orphaned | reconcile; Step 26 will mark then purge before dropping rows |
Access is always cut by the row: a `deleted` or `rejected` row matches no
read predicate, whatever the object's fate. The object lifetime is an
eventual-consistency concern handled by the sweeper and verified by
`reconcile` (lists objects vs rows per bucket, dry run by default).
### II.10 Backups and restore on the VPS (self-hosted)
- Storage backend `file` on a named volume `faunosteo_supabase_storage`
(kept across renames, BDR-035). One VPS, one disk, no extra service. The
S3 backend (Scaleway Object Storage) remains available when the volume
outgrows the disk or a second host appears: copy the files, change the
backend env, rows unchanged.
- Backup order: database first (`pg_dump` of the whole instance, the
`storage` schema included), files second (restic snapshot of the volume
to Scaleway Object Storage, encrypted, 7 daily / 4 weekly / 6 monthly).
Dumping the rows first guarantees every row in the dump has its file in
the snapshot; files newer than the dump are extra and show up in
`reconcile` as orphans.
- Restore: stop the worker, restore the database, restore the volume, start
the stack, run `pnpm worker reconcile` (dry run), decide on the report.
- Runbook delivered by this step (`docs/development/STORAGE.md`, pointer in
DEPLOY.md); the owner runs it. Claude never runs a backup, restore or
transfer tool. A restore drill on the local stack is part of the manual
validation.
### II.11 Migration to managed Supabase
- Schema: the same migrations (`supabase db push`); buckets and policies
are SQL and apply on managed. The `storage` schema itself is never dumped
into managed: `storage.objects` rows are re-created by the uploads.
- Objects: operator command `pnpm worker migrate --from <url> --to <url>`
(two service keys from env), driven by `assets` and
`anatomical_sheet_assets`: for every `clean` / confirmed row, download,
verify sha256, upload at the same key, verify; report. Then `reconcile`
on the destination. Everything is verifiable because every object has a
row with its hash.
- Worker and app: change `SUPABASE_URL` / keys. No code change.
- Not used on purpose because managed meters or lacks them: image
transformations (imgproxy), Edge Functions, Realtime.
### II.12 UI (follows the validated mockup, no redesign)
- Mon compte → Informations personnelles: avatar circle (photo, else the
monogram) with "Modifier la photo" under it, as drawn. Picking a file
shows the local preview in the circle with "Enregistrer la photo" /
"Annuler"; after saving, the circle shows a "Traitement en cours…" state
until the row is `clean`; "Supprimer la photo" appears when a photo
exists. Errors use `account.avatar.error.<code>`.
- `UserBadge` (sidebar + drawer): photo when present, monogram otherwise.
- Top bar (D3): the mockup shows avatar + name with a chevron on desktop and
the avatar alone on mobile; Step 7 shipped the bar without a user element.
Proposal: add the avatar (and the name on desktop) as a link to Mon
compte, no dropdown yet (the menu's content is undefined). Decision for
the owner.
- Dashboard unchanged (the mockup shows no avatar in the content area).
- Accepted inputs: JPEG, PNG, WebP; never SVG for avatars.
---
## Part III — The ten security questions
1. **Access control**: the `assets` row is the authority. `storage.objects`
policies call `private.can_read_object(uid, bucket, name)` /
`private.can_upload_object(uid, bucket, name)` which look the key up in
`assets` (status, owner, kind) and dispatch per kind to the owning
domain's helper. Step 8 implements the `profile_avatar` branch (owner
only); every other kind returns false. Buckets without policy are
unreachable by API roles.
2. **No cross-user access**: the read predicate requires a `clean` row
whose `owner_id = auth.uid()` (avatar branch) and whose `object_key`
equals the requested name; keys contain the owner's uuid but the policy
never trusts the path, only the row. Listing is denied the same way
(SELECT policy). pgTAP: Bob reads nothing of Alice, cannot upload at her
key, cannot confirm her row, cannot sign her URL (real-stack e2e).
3. **Keys**: server-built from uuids by the RPC
(`profiles/<uid>/<asset>/avatar.webp`, `quarantine/<uid>/<asset>/upload`),
canonical file name per kind, user filename never in a key, policies
compare whole strings.
4. **Signed URLs**: issued by the Storage API after the SELECT policy, with
the user's JWT; TTL per kind in one table (`profile_avatar: 3600 s`),
cached client-side until five minutes before expiry; no `download`
flag, no transform, no public URL. A signed URL is a bearer token until
expiry: accepted for a 512 px avatar, documented; documents (Step 9) get
60 s and `download`.
5. **Quarantine**: upload lands under `quarantine/` with the row in
`pending_upload` → `uploaded`; no API role can read it; only the worker
(service key) reads it, writes the clean bytes under the final key,
deletes the quarantine object. Status machine and every transition are
RPCs; the browser cannot set `clean`.
6. **Dangerous SVG**: the existing allowlist sanitiser (elements,
attributes, in-document `href` / `url(#)` only, no backslash, no
`<script>` / `<foreignObject>` / `<image>` / event attributes by
construction, caps on elements, depth, `<use>`, filters, 20 MiB) runs in
the worker; refusal = `svg_rejected`; hostile fixtures (script,
foreignObject, `on*`, `javascript:`, external `use`, CSS `url()`, entity
expansion, deep nesting, oversized) are unit tests. Avatars never accept
SVG.
7. **Images without metadata**: the worker never stores the uploaded
bytes. It decodes (type from magic bytes, header limits, 24 MP pixel cap),
applies the EXIF orientation, resizes and re-encodes to WebP without
`withMetadata()`: EXIF, XMP, IPTC and ICC are not copied. A test asserts
the output of an EXIF-laden JPEG carries no `EXIF` / `XMP ` chunk and
that a rotated portrait comes out upright.
8. **Orphans**: replacement and deletion flip the row first (one
transaction), the sweeper removes the object later and marks
`purged_at`; abandoned uploads expire; `reconcile` lists objects vs rows
and only deletes with `--apply` after a dry run (II.9).
9. **Backups and restore**: file backend on a named volume, database dump
first then restic snapshot of the volume to Scaleway Object Storage,
restore in the same order followed by `reconcile`; runbook in
`STORAGE.md`, drill on the local stack (II.10).
10. **Managed migration**: same migrations, objects copied by an operator
command driven by the rows with hash verification, URLs and keys
switched by env (II.11).
Service role: only the worker's env file and the operator commands hold
it; `env.js` is generated from `APP_*` variables only (unchanged).
---
## Part IV — Scope, tests, risks, decisions
### IV.1 Exact scope of the implementation
In:
- Migration `20261012100000_files_foundation.sql`: enums, `assets`, trigger
(`bucket_id` from kind, avatar pointer check), seven bucket rows (the
existing one untouched), `private.can_read_object` /
`private.can_upload_object` (avatar branch, closed default),
`storage.objects` SELECT + INSERT policies for `avatars`, the eight RPCs
of II.3, `profiles.avatar_asset_id`, explicit EXECUTE revokes. pgTAP
`010_files_foundation.sql` (allowed and denied cases, policy census per
bucket), shim extension if a column is missing, `database.types.ts`
regenerated.
- `apps/worker`: loop, claim / complete / reject, image processor, SVG
processor, sweeper, `reconcile` and `migrate` commands, Dockerfile
(`node:24-alpine`, non-root), compose service, `.env.worker.example`,
vitest on fixtures, an integration test against the local stack run by
the CI `supabase` job.
- `packages/supabase/src/assets.ts`: `beginAvatarUpload`, `uploadToQuarantine`,
`confirmAssetUpload`, `removeAvatar`, `fetchOwnAsset`, `signedUrlFor`
with the TTL table; closed `AssetFailure` kinds.
- `apps/app`: avatar block in `PersonalInfoForm` (picker, preview, save,
processing, delete), `UserBadge` photo, top bar avatar if D3 = yes, auth
store avatar URL cache, i18n keys FR/EN, component and store tests.
- Docs: `docs/development/STORAGE.md` (model, keys, statuses, worker
operations, backup / restore runbook, managed migration), DEPLOY.md
(worker service, env file, CSP additions for the Supabase origin),
SUPABASE.md (worker in dev), CLAUDE.md, CHANGELOG; plan note on the
early worker.
- CI: worker typecheck / lint / test / build, `docker build` of the worker
image, worker integration test in the `supabase` job.
Out (named so nothing is implied): user SVG upload path and
`sanitization_status` on sheet assets (Steps 13 / 17), logos, documents,
client / animal / report / import flows (their steps), image
transformations, public profile photo, the production deployment of the
self-hosted stack itself (runbook only), account erasure (Step 26),
Realtime, a user menu dropdown in the top bar.
### IV.2 Test scenarios
pgTAP (PGlite + real stack), every case with its denial twin:
- buckets: eight rows, all private, caps and types as the table; policy
census = 2 on `avatars`, 1 on `anatomical-sheet-assets`, 0 elsewhere;
`authenticated` cannot insert nor select in a closed bucket.
- `begin_avatar_upload`: happy path returns keys of the right shape; wrong
type, oversize, rate exceeded, anon → denied; a second begin replaces the
stale pending row (unique index holds).
- upload policy: Alice inserts at her quarantine key; Bob at her key →
42501; Alice at a final key, at a key of a `clean` row, at an unknown key
→ denied; no UPDATE / DELETE policy exists.
- confirm: own pending row with the object present → `uploaded`; without
the object → error; Bob confirming Alice's row → denied; confirming twice
→ error.
- read policy: Alice reads her `clean` object only; nothing while
`pending_upload` / `uploaded` / `processing` / `rejected` / `deleted`;
the quarantine object is never readable; Bob reads nothing; anon nothing.
- worker RPCs: `authenticated` cannot execute any of them; claim locks and
increments attempts; complete sets the profile pointer and marks the
previous avatar `deleted` atomically; reject stores a closed code;
reclaim after 10 min, rejection after three attempts; purge list contents.
- profile pointer trigger: pointing at a `rejected` row, a foreign row, a
non-avatar kind → refused; `authenticated` cannot update the column.
- audit: `asset.avatar_set`, `asset.avatar_removed`, `asset.rejected` rows.
Worker (vitest, fixtures under `apps/worker/tests/fixtures`):
- image: JPEG / PNG / WebP accepted; PNG bytes with `.jpg` name accepted as
PNG; GIF, BMP, SVG, PDF, HTML, polyglot (PNG + trailing script), zip bomb
PNG (declares 20 000 × 20 000) → rejected with the right code; truncated
file → `decode_failed`; 32 px image → `dimensions`; EXIF-laden JPEG → no
EXIF / XMP / ICC in the output, orientation applied; output is 512 × 512
WebP, deterministic size bounds.
- svg: each hostile fixture rejected or rewritten as the sanitiser rules
say; caps enforced; clean draft `horse.svg` passes unchanged.
- loop: claim → process → complete call sequence; processor throw →
reject; Storage error → row left for reclaim; sweeper purges only listed
rows and reports orphans without deleting.
Integration (local stack, CI `supabase` job): Alice's full flow with the
service key for the worker side → row `clean`, object at the final key, no
quarantine object, profile pointer set; a second upload replaces and the
first object disappears after the sweep; `remove_avatar` then sweep →
bucket empty.
Real-stack e2e (scratchpad script, like Step 7): Alice signs her URL
(200), Bob signing Alice's key (400 / 403), anon (400), Alice signing her
quarantine key (denied), signed URL fetch works then expires.
App (vitest + happy-dom): picker rejects a 6 MiB file and a `.jpg` that is
a PNG by magic bytes with the phrased error; preview appears before any
request; save calls begin → upload → confirm in order; processing state
then photo on `clean`; rejection message per code; delete flow; `UserBadge`
shows the photo when the store has a URL, the monogram otherwise; i18n
parity.
Manual validation (owner): upload, replace, delete from Mon compte on
desktop and mobile; photo visible in the sidebar, the drawer and the top
bar; copy the signed URL, open it in a private window (works until expiry,
then 400); try the object URL without token (400); upload a renamed PNG and
an HTML file (refused with a clear message); restore drill on the local
stack.
### IV.3 Risks
- **Decoder vulnerabilities** (libvips / libwebp history): the worker is
the only process that decodes untrusted images, runs non-root, read-only,
memory-capped, with no inbound port and egress limited to the Supabase
host; `sharp` updates become a routine chore. Residual risk accepted and
documented.
- **Worker down**: uploads stay `uploaded`; the UI says "toujours en
cours"; nothing is lost; the healthcheck and `restart: unless-stopped`
cover the common case; a monitoring hook is Step 27.
- **Signed URL as bearer**: a leaked avatar URL exposes a 512 px picture
for at most one hour. Documents will use short TTLs.
- **CSP in production**: `connect-src` and `img-src` must include the
Supabase origin, or neither the API nor the avatars load. Documented in
DEPLOY.md; the owner applies it at deployment.
- **PGlite shim vs real `storage` schema**: policies use only `bucket_id`
and `name`, no storage helper function, so both engines evaluate the same
expression; the real stack's `protect_delete` trigger forbids direct
deletes in tests (LRN-041), writes are asserted by census.
- **Storage API version drift** between the CLI image, the self-hosted
compose and managed: signing and RLS semantics are stable since 2023; the
integration test on the CI stack catches a regression.
- **Service key handling on the VPS**: a second env file with `chmod 600`;
the app `.env` stays key-free; `env.js` never carries it (test).
- **Orientation and colour**: EXIF orientation is applied before the
rotation data is dropped; wide-gamut ICC profiles are dropped, a slight
colour shift on rare photos is accepted for an avatar.
- **Plan deviation**: the worker arrives at Step 8 instead of Step 19;
Step 19 extends it rather than creating it. Recorded in the plan.
### IV.4 Decisions for the owner
| Id | Question | Recommendation |
|---|---|---|
| D1 | Server-side processing runtime | Containerised worker `faunosteo_worker` (II.1), no Edge Functions |
| D2 | Sheet bucket name | Keep `anatomical-sheet-assets`, record the deviation from the announced `anatomical-sheets` |
| D3 | Top bar avatar | Add avatar (+ name on desktop) as a link to Mon compte, no dropdown; or sidebar / drawer / account only |
| D4 | Avatar output | One 512 × 512 WebP, centre square crop, no metadata; inputs JPEG / PNG / WebP ≤ 5 MiB |
| D5 | Self-hosted Storage backend and backups | File backend on a named volume + restic to Scaleway Object Storage, database dumped first |
| D6 | SVG processor in Step 8 | Yes, in the worker with hostile fixtures, producer at Steps 13 / 17 |
| D7 | Signed URL TTL for avatars | 3600 s, cached client-side |
| D8 | Polling vs Realtime for the processing state | Polling the own row (1 s, 30 s cap); Realtime stays off |