docs(docs): Step 8 files foundation analysis — storage model, worker, quarantine, policies, backups, scope
This commit is contained in:
1 parent
505b83087b
commit
54df4d851a
1 file changed
+578
@@ -0,0 +1,578 @@
|
||||
# Step 8 — Files foundation and Supabase Storage: technical analysis
|
||||
|
||||
Status: proposal for the owner's decision, 2026-10-12. Nothing implemented.
|
||||
Applies the owner's brief of 2026-10-12 (Step 7 closed; hosting = self-hosted
|
||||
Supabase first, managed later; eight private buckets; quarantine + server
|
||||
validation; avatar as first consumer; service role never in the browser).
|
||||
|
||||
Part I states the facts the design rests on. Part II is the proposed design.
|
||||
Part III answers the ten security questions of the brief. Part IV lists the
|
||||
exact scope, the tests, the risks and the decisions the owner has to take.
|
||||
|
||||
---
|
||||
|
||||
## Part I — Facts
|
||||
|
||||
### I.1 What exists today
|
||||
|
||||
| Area | State |
|
||||
|---|---|
|
||||
| Storage | One bucket, `anatomical-sheet-assets` (migration `20261009100000`): private, SVG only, 20 MiB cap, content-addressed keys `<sha256>.svg`, SELECT policy through the sheet (`private.can_read_sheet_asset_object`), no API write policy, operator upload script with the service key, `confirm_sheet_asset_upload` RPC (service role). pgTAP 006 (28 checks) runs on the PGlite shim and on the real stack. |
|
||||
| Storage API | `storage-api v1.79.36` in the local stack (same image family as a self-hosted deployment). Signed URLs, signed upload URLs and RLS on `storage.objects` are available; image transformation needs `imgproxy`, absent here and a paid add-on on managed. |
|
||||
| Profiles | `avatar_asset_id` was deliberately deferred to this step (comment in `20261008100100_profiles.sql`). |
|
||||
| Shell UI | `UserBadge` (monogram) in the sidebar and the drawer; the top bar holds only the language switch; "Mon compte" = stacked sections, `PersonalInfoForm` first. The validated mockup shows the avatar with "Modifier la photo" inside the Profil section and an avatar + name in the top bar. |
|
||||
| SVG sanitiser | `packages/pdf/src/svg/sanitize.ts`: allowlist of elements and attributes, `href` / `url()` kept in-document, backslash refused, caps 20 000 elements / depth 64 / 256 `<use>` / `stdDeviation` 100, DOM Level 2 on `@xmldom/xmldom`, `SvgRejectedError`. Browser-free. |
|
||||
| Edge runtime | Disabled in `config.toml`. `BACKEND_ARCHITECTURE.md` §6: self-hosted Edge Functions are hand-deployed; first steps lean on SQL. C3: PostgreSQL-backed jobs before a queue. C12: the sanitiser runs between upload and publication, never "on upload". |
|
||||
| Worker | The plan creates `apps/worker` at Step 19 (notifications). Files processing needs server-side code before that. |
|
||||
| Deploy | No backend in production yet. The VPS `.env` is documented as non-sensitive. CSP: `default-src 'self'`, `img-src 'self' data: blob:` (a Supabase origin would be blocked for both API calls and images). |
|
||||
| Tests | pgTAP on PGlite + real stack; app vitest on happy-dom; Step 7 added a headless e2e script against the local stack (scratchpad, Mailpit). |
|
||||
|
||||
### I.2 Corrections to the plan's Step 8 text
|
||||
|
||||
- "RPC `issue_signed_url(asset_id)`" cannot exist in SQL: signing needs the
|
||||
Storage API's secret. The Storage API signs a URL only after evaluating the
|
||||
SELECT policy of `storage.objects` as the caller, so the access check and
|
||||
the signature live in one place already. The design uses it directly.
|
||||
- "Edge Function or SQL-only sanitizer": SQL cannot parse XML or decode
|
||||
images safely; Edge Functions cannot load a native image library and are
|
||||
hand-deployed on self-hosted. A containerised worker is the fit (II.1).
|
||||
|
||||
---
|
||||
|
||||
## Part II — Proposed design
|
||||
|
||||
### II.1 Components
|
||||
|
||||
```text
|
||||
Browser (anon key + user JWT)
|
||||
│ begin_avatar_upload (RPC) ─► assets row pending_upload + quarantine key
|
||||
│ PUT object (Storage API, INSERT policy = that key, that row, that user)
|
||||
│ confirm_asset_upload (RPC) ─► status uploaded
|
||||
│ poll own assets row ─► clean | rejected
|
||||
│ createSignedUrl(object_key, 3600) (Storage API, SELECT policy) ─► <img>
|
||||
▼
|
||||
Supabase (self-hosted compose, later managed)
|
||||
PostgREST ─ public.assets, RPCs, private helpers Storage API ─ 8 private buckets
|
||||
▲
|
||||
│ claim_asset_jobs / complete_asset_job / reject_asset_job (RPC, service role)
|
||||
│ GET quarantine object, PUT clean object, DELETE quarantine (service role)
|
||||
apps/worker (container faunosteo_worker, outbound only, no port)
|
||||
processors: image (sharp) · svg (@faunosteo/pdf allowlist)
|
||||
sweeper: purge objects of deleted/rejected rows, abandoned uploads, stale jobs
|
||||
operator commands: reconcile, migrate
|
||||
```
|
||||
|
||||
Why a worker and not Edge Functions: Edge Functions on the self-hosted stack
|
||||
run in a separate `edge-runtime` container, are deployed by hand, cannot
|
||||
load `sharp` (native), and the config keeps them off. Why not SQL: no image
|
||||
decoding, no XML parsing. The worker is one Node 24 container with the
|
||||
service key in its own env file, no inbound port, the same code against the
|
||||
local stack, the VPS stack and a managed project (URL + key change only).
|
||||
It is the `apps/worker` the plan creates at Step 19; it arrives earlier with
|
||||
one job family, Step 19 adds its own.
|
||||
|
||||
Dependency added: `sharp` (0.35.x, libvips, prebuilt `linuxmusl-x64`
|
||||
package, no install script, Node ≥ 20.9). It decodes JPEG / PNG / WebP,
|
||||
resizes, re-encodes, and drops metadata by default. The only widely
|
||||
maintained option that does all four in Node; pure-JS stacks cover PNG only
|
||||
or are unmaintained.
|
||||
|
||||
### II.2 Buckets
|
||||
|
||||
All eight private (`public = false`). Created by one migration with
|
||||
`on conflict (id) do update` so the migration converges on the local stack,
|
||||
the VPS and managed alike. `file_size_limit` and `allowed_mime_types` on the
|
||||
bucket are a first filter on what the Storage API accepts, never a proof of
|
||||
content.
|
||||
|
||||
| Bucket | Kind(s) | Owning domain and step | Step 8 state |
|
||||
|---|---|---|---|
|
||||
| `avatars` | profile_avatar | Identity (self) | **open**: SELECT + INSERT policies through `assets` |
|
||||
| `logos` | organization_logo | Organizations, Step 10 | closed, no policy |
|
||||
| `professional-documents` | professional_document | Verification, Step 9 | closed |
|
||||
| `client-files` | client_file | Clients, Step 12 | closed |
|
||||
| `animal-files` | animal_file | Animals, Step 12 | closed |
|
||||
| `report-files` | report_file | PDF artifacts, Step 15 | closed |
|
||||
| `anatomical-sheet-assets` | anatomical_sheet_asset | Sheets (existing policy kept) | unchanged |
|
||||
| `imports` | import_file | Imports, Step 24 | closed |
|
||||
|
||||
"Closed" = the bucket row exists, RLS on `storage.objects` has no policy
|
||||
naming it, so `anon` and `authenticated` can neither list, read, upload,
|
||||
update nor delete. Only the service role touches it. A pgTAP census pins
|
||||
the exact policy count per bucket, so a later step cannot open a bucket by
|
||||
accident without changing the test.
|
||||
|
||||
Open point (D2): the brief names the sheet bucket `anatomical-sheets`; the
|
||||
repository already has `anatomical-sheet-assets` with its policy, its
|
||||
operator script, its tests and its seed. Recommendation: keep the existing
|
||||
name and record the deviation; a rename costs a migration, a re-upload and
|
||||
touches Step 13 for no behaviour.
|
||||
|
||||
Proposed caps (closed buckets get the final value at their own step):
|
||||
|
||||
| Bucket | Size | Declared types |
|
||||
|---|---|---|
|
||||
| avatars | 5 MiB | image/jpeg, image/png, image/webp |
|
||||
| logos | 2 MiB | image/jpeg, image/png, image/webp |
|
||||
| professional-documents | 20 MiB | application/pdf, image/jpeg, image/png |
|
||||
| client-files, animal-files | 20 MiB | pdf + the three image types |
|
||||
| report-files | 50 MiB | application/pdf |
|
||||
| anatomical-sheet-assets | 20 MiB (existing) | image/svg+xml |
|
||||
| imports | 50 MiB | application/pdf, application/json, text/csv |
|
||||
|
||||
### II.3 `public.assets` — the authority on every user file
|
||||
|
||||
One row per file, created before the bytes exist, kept after they are gone
|
||||
(status `deleted`) until the sweeper has purged the object.
|
||||
|
||||
```sql
|
||||
create type public.asset_kind as enum (
|
||||
'profile_avatar', 'organization_logo', 'professional_document',
|
||||
'client_file', 'animal_file', 'report_file', 'anatomical_sheet_asset', 'import_file');
|
||||
create type public.asset_status as enum (
|
||||
'pending_upload', 'uploaded', 'processing', 'clean', 'rejected', 'deleted');
|
||||
|
||||
create table public.assets (
|
||||
id uuid primary key default gen_random_uuid(),
|
||||
owner_id uuid not null references public.profiles (id) on delete cascade,
|
||||
kind public.asset_kind not null,
|
||||
bucket_id text not null, -- derived from kind, checked by trigger
|
||||
resource_id uuid, -- profile id for avatars
|
||||
object_key text not null unique, -- final key, server-built
|
||||
quarantine_key text, -- upload key, null once purged
|
||||
status public.asset_status not null default 'pending_upload',
|
||||
declared_mime text not null,
|
||||
declared_size integer not null check (declared_size between 1 and 52428800),
|
||||
original_filename text check (char_length(original_filename) <= 120), -- display only
|
||||
detected_mime text,
|
||||
byte_size integer,
|
||||
sha256 text check (sha256 ~ '^[0-9a-f]{64}$'),
|
||||
width integer,
|
||||
height integer,
|
||||
rejection_reason text, -- closed codes, see II.6
|
||||
attempts smallint not null default 0,
|
||||
locked_at timestamptz,
|
||||
created_at timestamptz not null default now(),
|
||||
updated_at timestamptz not null default now(),
|
||||
processed_at timestamptz,
|
||||
deleted_at timestamptz,
|
||||
purged_at timestamptz -- object removed by the sweeper
|
||||
);
|
||||
-- one avatar in flight per account
|
||||
create unique index assets_one_avatar_in_flight
|
||||
on public.assets (owner_id) where kind = 'profile_avatar' and status in ('pending_upload','uploaded','processing');
|
||||
|
||||
alter table public.profiles add column avatar_asset_id uuid references public.assets (id);
|
||||
```
|
||||
|
||||
Privileges: `revoke all` from API roles, `grant select` to `authenticated`,
|
||||
RLS `assets_select_own` (`owner_id = auth.uid()`). No INSERT / UPDATE /
|
||||
DELETE policy: every write is an RPC. `profiles.avatar_asset_id` is not in
|
||||
the column grant: only the server sets it, and a trigger checks that it
|
||||
points to a `clean` `profile_avatar` of the same profile.
|
||||
|
||||
RPCs (security definer, `set search_path = ''`, authorisation before any
|
||||
lookup, EXECUTE revoked from PUBLIC, granted per role, audited through
|
||||
`private.log_event` where the state is sensitive):
|
||||
|
||||
| RPC | Role | Effect |
|
||||
|---|---|---|
|
||||
| `begin_avatar_upload(declared_mime, declared_size, original_filename)` | authenticated | caps (type in list, size ≤ 5 MiB), rate (≤ 10 rows created in the last hour), replaces a stale `pending_upload` of the same account; inserts the row with `object_key = profiles/<uid>/<id>/avatar.webp` and `quarantine_key = quarantine/<uid>/<id>/upload`; returns `{asset_id, bucket_id, quarantine_key}` |
|
||||
| `confirm_asset_upload(asset_id)` | authenticated | own row, `pending_upload` → checks the object exists at `quarantine_key` with `metadata->>'size'` ≤ cap → `uploaded` |
|
||||
| `remove_avatar()` | authenticated | clears `profiles.avatar_asset_id`, marks the asset `deleted`; logs `asset.avatar_removed` |
|
||||
| `claim_asset_jobs(limit)` | service_role | `uploaded` → `processing` with `for update skip locked`, `locked_at`, `attempts + 1`; returns the rows |
|
||||
| `complete_asset_job(asset_id, result)` | service_role | `processing` → `clean` with detected fields; for an avatar: sets `profiles.avatar_asset_id`, marks the previous avatar asset `deleted`, in the same transaction; logs `asset.avatar_set` |
|
||||
| `reject_asset_job(asset_id, reason)` | service_role | `processing` → `rejected`; logs `asset.rejected` |
|
||||
| `reclaim_stale_jobs()` | service_role | `processing` older than 10 min → `uploaded` while `attempts < 3`, else `rejected (processing_error)` |
|
||||
| `list_purgeable_assets()` / `mark_asset_purged(asset_id)` | service_role | rows whose object must go (`deleted`, `rejected`, abandoned `pending_upload` > 24 h, `clean` with a quarantine key still present) |
|
||||
|
||||
Why `assets` does not absorb `anatomical_sheet_assets`: that table is
|
||||
content-addressed and sheet-scoped (BDR-032/034) with its own confirmed
|
||||
upload path; merging would rewrite a validated model for no gain. The SVG
|
||||
processor of the worker will serve it at Step 13 through a
|
||||
`sanitization_status` added then.
|
||||
|
||||
### II.4 Object keys
|
||||
|
||||
- Final: `<resource-plural>/<resource-id>/<asset-id>/<canonical-name>`. Avatar:
|
||||
`profiles/<user_id>/<asset_id>/avatar.webp`. The name is fixed per kind;
|
||||
the user's filename never enters a key (it is stored, length-capped and
|
||||
control-character-free, for display only).
|
||||
- Quarantine: `quarantine/<user_id>/<asset_id>/upload` in the same bucket.
|
||||
No API role can read it: the SELECT predicate only matches the
|
||||
`object_key` of `clean` rows, the INSERT predicate only the
|
||||
`quarantine_key` of the caller's `pending_upload` row.
|
||||
- Keys are built by the RPC from uuids; the policies compare whole strings
|
||||
(no prefix match, no `storage.foldername()` so the PGlite shim and the real
|
||||
stack evaluate the same expression).
|
||||
- Sheet assets keep `<sha256>.svg` (shared drawings, immutable versions).
|
||||
Avatars are per-account and mutable: a hash key would need reference
|
||||
counting for nothing.
|
||||
|
||||
### II.5 Upload, processing, display (avatar)
|
||||
|
||||
1. Browser pre-checks (feedback only): size ≤ 5 MiB, magic bytes of
|
||||
JPEG / PNG / WebP, the file decodes (`createImageBitmap`). Preview from
|
||||
`URL.createObjectURL`, shown in the circle with `object-fit: cover`,
|
||||
nothing uploaded yet.
|
||||
2. "Enregistrer" → `begin_avatar_upload` → row + keys.
|
||||
3. `storage.from('avatars').upload(quarantine_key, file, { upsert: false })`
|
||||
with the user JWT. The INSERT policy on `storage.objects` accepts only
|
||||
`(bucket_id = 'avatars', name = quarantine_key)` for a `pending_upload`
|
||||
row owned by `auth.uid()`.
|
||||
4. `confirm_asset_upload(asset_id)` → `uploaded`.
|
||||
5. Worker loop (every 2 s, backoff to 15 s when idle): `claim_asset_jobs(1)`
|
||||
→ download the quarantine object with the service client → processor.
|
||||
6. Image processor: detect the type from bytes (JPEG `FF D8 FF`, PNG
|
||||
8-byte signature, WebP `RIFF….WEBP`); any other → `not_an_image`.
|
||||
`sharp(bytes, { limitInputPixels: 24e6, failOn: 'error', animated: false })`
|
||||
→ header check (`metadata()`: format matches the magic, 64 ≤ sides ≤ 8192)
|
||||
→ `.rotate()` (applies EXIF orientation, then EXIF is dropped)
|
||||
→ `.resize(512, 512, { fit: 'cover', position: 'centre' })`
|
||||
→ `.webp({ quality: 82 })`, no `withMetadata()` → EXIF, XMP, IPTC and ICC
|
||||
are gone. Output: one 512 × 512 WebP, sha256, byte size.
|
||||
7. Worker uploads the clean bytes at `object_key` (service client), deletes
|
||||
the quarantine object, calls `complete_asset_job` (clean + profile
|
||||
pointer + previous avatar marked `deleted`, one transaction). On a
|
||||
processor refusal: `reject_asset_job(reason)` and the quarantine object
|
||||
is deleted.
|
||||
8. Browser polls its own `assets` row (1 s, up to 30 s): `clean` → refresh
|
||||
the profile and the signed URL; `rejected` → message from the closed
|
||||
reason code; still `processing` after 30 s → "toujours en cours" with a
|
||||
retry on next visit.
|
||||
9. Display: `createSignedUrl(object_key, 3600)` with the user JWT. The
|
||||
Storage API evaluates the SELECT policy, then signs. The URL is cached in
|
||||
the auth store with its expiry and renewed five minutes before. Used as
|
||||
`<img src>` in `UserBadge` (sidebar, drawer), the account section and,
|
||||
if the owner approves D3, the top bar.
|
||||
|
||||
Why no client-side resize before upload: the server re-encodes anyway and
|
||||
is the boundary; a second pipeline would duplicate the crop rule and could
|
||||
drift. The 5 MiB cap keeps the upload short.
|
||||
|
||||
### II.6 Rejection codes (closed set, phrased by i18n)
|
||||
|
||||
`not_an_image`, `unsupported_type`, `too_large`, `decode_failed`,
|
||||
`dimensions` (too small or too large), `svg_rejected`, `processing_error`.
|
||||
The raw library message never reaches the row nor the screen; it is logged
|
||||
by the worker with the asset id only.
|
||||
|
||||
### II.7 SVG processor (anatomical sheets)
|
||||
|
||||
Same worker, kind `anatomical_sheet_asset`: parse with `@xmldom/xmldom`
|
||||
(parse errors fatal), `sanitizeSvgRoot` from `@faunosteo/pdf` (allowlist +
|
||||
caps), serialise, 20 MiB cap in and out, refusal → `svg_rejected`. The
|
||||
removed element and attribute names are kept in the result (never values).
|
||||
In Step 8 there is no user producer: uploads into the sheet bucket stay
|
||||
operator-only until the editor step (17) and the database sheets step (13)
|
||||
open a signed upload path with `sanitization_status` on
|
||||
`anatomical_sheet_assets`. The processor is tested on hostile fixtures and
|
||||
through the worker with the service key, so the boundary exists before the
|
||||
producer does (D6).
|
||||
|
||||
### II.8 Worker runtime and operations
|
||||
|
||||
- `apps/worker` (`@faunosteo/worker`): TypeScript, Node 24, deps
|
||||
`@supabase/supabase-js`, `sharp`, `@faunosteo/pdf` (sanitiser),
|
||||
`@xmldom/xmldom`. No DB driver: every database access is an RPC through
|
||||
PostgREST with the service key, so one credential and one transport, and a
|
||||
managed project needs no open database port.
|
||||
- Container `faunosteo_worker` (compose service `worker`): non-root,
|
||||
`read_only: true` + `tmpfs /tmp`, `mem_limit: 512m`, `pids_limit`, no
|
||||
published port, `restart: unless-stopped`, healthcheck = heartbeat file
|
||||
refreshed each loop. Env file `.env.worker` (`SUPABASE_URL`,
|
||||
`SUPABASE_SERVICE_ROLE_KEY`, `WORKER_ID`), `chmod 600`, separate from the
|
||||
app `.env` which DEPLOY.md keeps non-sensitive. Egress limited to the
|
||||
Supabase host (documented firewall rule).
|
||||
- One job at a time, `sharp.concurrency(1)`; time is bounded by the input
|
||||
caps (≤ 5 MiB, ≤ 24 MP), not by a kill switch; a crashed run is reclaimed
|
||||
by `reclaim_stale_jobs` (max 3 attempts).
|
||||
- Sweeper every 10 min: purge objects of `deleted` / `rejected` rows and of
|
||||
abandoned uploads, remove quarantine leftovers of `clean` rows, mark
|
||||
`purged_at`. Objects with no row (orphans from a crash between PUT and
|
||||
RPC) are only reported; `pnpm worker reconcile --apply` deletes them after
|
||||
an operator reads the dry run.
|
||||
- Local dev: `pnpm worker:dev` against the local stack (service key from
|
||||
`supabase status`, typed into the shell as SUPABASE.md §8 already does).
|
||||
- Logging: structured lines, asset id and reason codes only; never bytes,
|
||||
names or user email.
|
||||
|
||||
### II.9 Orphans on replacement and deletion
|
||||
|
||||
| Event | Row state | Object state | Who cleans |
|
||||
|---|---|---|---|
|
||||
| New avatar clean | new `clean`, previous `deleted` (same transaction) | previous object still present | sweeper (purge, idempotent) |
|
||||
| `remove_avatar()` | `deleted`, `profiles.avatar_asset_id = null` | present until purge | sweeper |
|
||||
| Upload never confirmed | `pending_upload` | maybe present | sweeper after 24 h |
|
||||
| Worker dies after PUT clean, before `complete` | `processing` → reclaimed | clean object present | retry: `upsert: true` on the same deterministic key, then `complete` |
|
||||
| Worker dies after `complete`, before quarantine DELETE | `clean`, `quarantine_key` set | quarantine present | sweeper (quarantine leftovers of clean rows) |
|
||||
| Account deleted (Step 26) | rows cascade | objects orphaned | reconcile; Step 26 will mark then purge before dropping rows |
|
||||
|
||||
Access is always cut by the row: a `deleted` or `rejected` row matches no
|
||||
read predicate, whatever the object's fate. The object lifetime is an
|
||||
eventual-consistency concern handled by the sweeper and verified by
|
||||
`reconcile` (lists objects vs rows per bucket, dry run by default).
|
||||
|
||||
### II.10 Backups and restore on the VPS (self-hosted)
|
||||
|
||||
- Storage backend `file` on a named volume `faunosteo_supabase_storage`
|
||||
(kept across renames, BDR-035). One VPS, one disk, no extra service. The
|
||||
S3 backend (Scaleway Object Storage) remains available when the volume
|
||||
outgrows the disk or a second host appears: copy the files, change the
|
||||
backend env, rows unchanged.
|
||||
- Backup order: database first (`pg_dump` of the whole instance, the
|
||||
`storage` schema included), files second (restic snapshot of the volume
|
||||
to Scaleway Object Storage, encrypted, 7 daily / 4 weekly / 6 monthly).
|
||||
Dumping the rows first guarantees every row in the dump has its file in
|
||||
the snapshot; files newer than the dump are extra and show up in
|
||||
`reconcile` as orphans.
|
||||
- Restore: stop the worker, restore the database, restore the volume, start
|
||||
the stack, run `pnpm worker reconcile` (dry run), decide on the report.
|
||||
- Runbook delivered by this step (`docs/development/STORAGE.md`, pointer in
|
||||
DEPLOY.md); the owner runs it. Claude never runs a backup, restore or
|
||||
transfer tool. A restore drill on the local stack is part of the manual
|
||||
validation.
|
||||
|
||||
### II.11 Migration to managed Supabase
|
||||
|
||||
- Schema: the same migrations (`supabase db push`); buckets and policies
|
||||
are SQL and apply on managed. The `storage` schema itself is never dumped
|
||||
into managed: `storage.objects` rows are re-created by the uploads.
|
||||
- Objects: operator command `pnpm worker migrate --from <url> --to <url>`
|
||||
(two service keys from env), driven by `assets` and
|
||||
`anatomical_sheet_assets`: for every `clean` / confirmed row, download,
|
||||
verify sha256, upload at the same key, verify; report. Then `reconcile`
|
||||
on the destination. Everything is verifiable because every object has a
|
||||
row with its hash.
|
||||
- Worker and app: change `SUPABASE_URL` / keys. No code change.
|
||||
- Not used on purpose because managed meters or lacks them: image
|
||||
transformations (imgproxy), Edge Functions, Realtime.
|
||||
|
||||
### II.12 UI (follows the validated mockup, no redesign)
|
||||
|
||||
- Mon compte → Informations personnelles: avatar circle (photo, else the
|
||||
monogram) with "Modifier la photo" under it, as drawn. Picking a file
|
||||
shows the local preview in the circle with "Enregistrer la photo" /
|
||||
"Annuler"; after saving, the circle shows a "Traitement en cours…" state
|
||||
until the row is `clean`; "Supprimer la photo" appears when a photo
|
||||
exists. Errors use `account.avatar.error.<code>`.
|
||||
- `UserBadge` (sidebar + drawer): photo when present, monogram otherwise.
|
||||
- Top bar (D3): the mockup shows avatar + name with a chevron on desktop and
|
||||
the avatar alone on mobile; Step 7 shipped the bar without a user element.
|
||||
Proposal: add the avatar (and the name on desktop) as a link to Mon
|
||||
compte, no dropdown yet (the menu's content is undefined). Decision for
|
||||
the owner.
|
||||
- Dashboard unchanged (the mockup shows no avatar in the content area).
|
||||
- Accepted inputs: JPEG, PNG, WebP; never SVG for avatars.
|
||||
|
||||
---
|
||||
|
||||
## Part III — The ten security questions
|
||||
|
||||
1. **Access control**: the `assets` row is the authority. `storage.objects`
|
||||
policies call `private.can_read_object(uid, bucket, name)` /
|
||||
`private.can_upload_object(uid, bucket, name)` which look the key up in
|
||||
`assets` (status, owner, kind) and dispatch per kind to the owning
|
||||
domain's helper. Step 8 implements the `profile_avatar` branch (owner
|
||||
only); every other kind returns false. Buckets without policy are
|
||||
unreachable by API roles.
|
||||
2. **No cross-user access**: the read predicate requires a `clean` row
|
||||
whose `owner_id = auth.uid()` (avatar branch) and whose `object_key`
|
||||
equals the requested name; keys contain the owner's uuid but the policy
|
||||
never trusts the path, only the row. Listing is denied the same way
|
||||
(SELECT policy). pgTAP: Bob reads nothing of Alice, cannot upload at her
|
||||
key, cannot confirm her row, cannot sign her URL (real-stack e2e).
|
||||
3. **Keys**: server-built from uuids by the RPC
|
||||
(`profiles/<uid>/<asset>/avatar.webp`, `quarantine/<uid>/<asset>/upload`),
|
||||
canonical file name per kind, user filename never in a key, policies
|
||||
compare whole strings.
|
||||
4. **Signed URLs**: issued by the Storage API after the SELECT policy, with
|
||||
the user's JWT; TTL per kind in one table (`profile_avatar: 3600 s`),
|
||||
cached client-side until five minutes before expiry; no `download`
|
||||
flag, no transform, no public URL. A signed URL is a bearer token until
|
||||
expiry: accepted for a 512 px avatar, documented; documents (Step 9) get
|
||||
60 s and `download`.
|
||||
5. **Quarantine**: upload lands under `quarantine/` with the row in
|
||||
`pending_upload` → `uploaded`; no API role can read it; only the worker
|
||||
(service key) reads it, writes the clean bytes under the final key,
|
||||
deletes the quarantine object. Status machine and every transition are
|
||||
RPCs; the browser cannot set `clean`.
|
||||
6. **Dangerous SVG**: the existing allowlist sanitiser (elements,
|
||||
attributes, in-document `href` / `url(#)` only, no backslash, no
|
||||
`<script>` / `<foreignObject>` / `<image>` / event attributes by
|
||||
construction, caps on elements, depth, `<use>`, filters, 20 MiB) runs in
|
||||
the worker; refusal = `svg_rejected`; hostile fixtures (script,
|
||||
foreignObject, `on*`, `javascript:`, external `use`, CSS `url()`, entity
|
||||
expansion, deep nesting, oversized) are unit tests. Avatars never accept
|
||||
SVG.
|
||||
7. **Images without metadata**: the worker never stores the uploaded
|
||||
bytes. It decodes (type from magic bytes, header limits, 24 MP pixel cap),
|
||||
applies the EXIF orientation, resizes and re-encodes to WebP without
|
||||
`withMetadata()`: EXIF, XMP, IPTC and ICC are not copied. A test asserts
|
||||
the output of an EXIF-laden JPEG carries no `EXIF` / `XMP ` chunk and
|
||||
that a rotated portrait comes out upright.
|
||||
8. **Orphans**: replacement and deletion flip the row first (one
|
||||
transaction), the sweeper removes the object later and marks
|
||||
`purged_at`; abandoned uploads expire; `reconcile` lists objects vs rows
|
||||
and only deletes with `--apply` after a dry run (II.9).
|
||||
9. **Backups and restore**: file backend on a named volume, database dump
|
||||
first then restic snapshot of the volume to Scaleway Object Storage,
|
||||
restore in the same order followed by `reconcile`; runbook in
|
||||
`STORAGE.md`, drill on the local stack (II.10).
|
||||
10. **Managed migration**: same migrations, objects copied by an operator
|
||||
command driven by the rows with hash verification, URLs and keys
|
||||
switched by env (II.11).
|
||||
|
||||
Service role: only the worker's env file and the operator commands hold
|
||||
it; `env.js` is generated from `APP_*` variables only (unchanged).
|
||||
|
||||
---
|
||||
|
||||
## Part IV — Scope, tests, risks, decisions
|
||||
|
||||
### IV.1 Exact scope of the implementation
|
||||
|
||||
In:
|
||||
|
||||
- Migration `20261012100000_files_foundation.sql`: enums, `assets`, trigger
|
||||
(`bucket_id` from kind, avatar pointer check), seven bucket rows (the
|
||||
existing one untouched), `private.can_read_object` /
|
||||
`private.can_upload_object` (avatar branch, closed default),
|
||||
`storage.objects` SELECT + INSERT policies for `avatars`, the eight RPCs
|
||||
of II.3, `profiles.avatar_asset_id`, explicit EXECUTE revokes. pgTAP
|
||||
`010_files_foundation.sql` (allowed and denied cases, policy census per
|
||||
bucket), shim extension if a column is missing, `database.types.ts`
|
||||
regenerated.
|
||||
- `apps/worker`: loop, claim / complete / reject, image processor, SVG
|
||||
processor, sweeper, `reconcile` and `migrate` commands, Dockerfile
|
||||
(`node:24-alpine`, non-root), compose service, `.env.worker.example`,
|
||||
vitest on fixtures, an integration test against the local stack run by
|
||||
the CI `supabase` job.
|
||||
- `packages/supabase/src/assets.ts`: `beginAvatarUpload`, `uploadToQuarantine`,
|
||||
`confirmAssetUpload`, `removeAvatar`, `fetchOwnAsset`, `signedUrlFor`
|
||||
with the TTL table; closed `AssetFailure` kinds.
|
||||
- `apps/app`: avatar block in `PersonalInfoForm` (picker, preview, save,
|
||||
processing, delete), `UserBadge` photo, top bar avatar if D3 = yes, auth
|
||||
store avatar URL cache, i18n keys FR/EN, component and store tests.
|
||||
- Docs: `docs/development/STORAGE.md` (model, keys, statuses, worker
|
||||
operations, backup / restore runbook, managed migration), DEPLOY.md
|
||||
(worker service, env file, CSP additions for the Supabase origin),
|
||||
SUPABASE.md (worker in dev), CLAUDE.md, CHANGELOG; plan note on the
|
||||
early worker.
|
||||
- CI: worker typecheck / lint / test / build, `docker build` of the worker
|
||||
image, worker integration test in the `supabase` job.
|
||||
|
||||
Out (named so nothing is implied): user SVG upload path and
|
||||
`sanitization_status` on sheet assets (Steps 13 / 17), logos, documents,
|
||||
client / animal / report / import flows (their steps), image
|
||||
transformations, public profile photo, the production deployment of the
|
||||
self-hosted stack itself (runbook only), account erasure (Step 26),
|
||||
Realtime, a user menu dropdown in the top bar.
|
||||
|
||||
### IV.2 Test scenarios
|
||||
|
||||
pgTAP (PGlite + real stack), every case with its denial twin:
|
||||
|
||||
- buckets: eight rows, all private, caps and types as the table; policy
|
||||
census = 2 on `avatars`, 1 on `anatomical-sheet-assets`, 0 elsewhere;
|
||||
`authenticated` cannot insert nor select in a closed bucket.
|
||||
- `begin_avatar_upload`: happy path returns keys of the right shape; wrong
|
||||
type, oversize, rate exceeded, anon → denied; a second begin replaces the
|
||||
stale pending row (unique index holds).
|
||||
- upload policy: Alice inserts at her quarantine key; Bob at her key →
|
||||
42501; Alice at a final key, at a key of a `clean` row, at an unknown key
|
||||
→ denied; no UPDATE / DELETE policy exists.
|
||||
- confirm: own pending row with the object present → `uploaded`; without
|
||||
the object → error; Bob confirming Alice's row → denied; confirming twice
|
||||
→ error.
|
||||
- read policy: Alice reads her `clean` object only; nothing while
|
||||
`pending_upload` / `uploaded` / `processing` / `rejected` / `deleted`;
|
||||
the quarantine object is never readable; Bob reads nothing; anon nothing.
|
||||
- worker RPCs: `authenticated` cannot execute any of them; claim locks and
|
||||
increments attempts; complete sets the profile pointer and marks the
|
||||
previous avatar `deleted` atomically; reject stores a closed code;
|
||||
reclaim after 10 min, rejection after three attempts; purge list contents.
|
||||
- profile pointer trigger: pointing at a `rejected` row, a foreign row, a
|
||||
non-avatar kind → refused; `authenticated` cannot update the column.
|
||||
- audit: `asset.avatar_set`, `asset.avatar_removed`, `asset.rejected` rows.
|
||||
|
||||
Worker (vitest, fixtures under `apps/worker/tests/fixtures`):
|
||||
|
||||
- image: JPEG / PNG / WebP accepted; PNG bytes with `.jpg` name accepted as
|
||||
PNG; GIF, BMP, SVG, PDF, HTML, polyglot (PNG + trailing script), zip bomb
|
||||
PNG (declares 20 000 × 20 000) → rejected with the right code; truncated
|
||||
file → `decode_failed`; 32 px image → `dimensions`; EXIF-laden JPEG → no
|
||||
EXIF / XMP / ICC in the output, orientation applied; output is 512 × 512
|
||||
WebP, deterministic size bounds.
|
||||
- svg: each hostile fixture rejected or rewritten as the sanitiser rules
|
||||
say; caps enforced; clean draft `horse.svg` passes unchanged.
|
||||
- loop: claim → process → complete call sequence; processor throw →
|
||||
reject; Storage error → row left for reclaim; sweeper purges only listed
|
||||
rows and reports orphans without deleting.
|
||||
|
||||
Integration (local stack, CI `supabase` job): Alice's full flow with the
|
||||
service key for the worker side → row `clean`, object at the final key, no
|
||||
quarantine object, profile pointer set; a second upload replaces and the
|
||||
first object disappears after the sweep; `remove_avatar` then sweep →
|
||||
bucket empty.
|
||||
|
||||
Real-stack e2e (scratchpad script, like Step 7): Alice signs her URL
|
||||
(200), Bob signing Alice's key (400 / 403), anon (400), Alice signing her
|
||||
quarantine key (denied), signed URL fetch works then expires.
|
||||
|
||||
App (vitest + happy-dom): picker rejects a 6 MiB file and a `.jpg` that is
|
||||
a PNG by magic bytes with the phrased error; preview appears before any
|
||||
request; save calls begin → upload → confirm in order; processing state
|
||||
then photo on `clean`; rejection message per code; delete flow; `UserBadge`
|
||||
shows the photo when the store has a URL, the monogram otherwise; i18n
|
||||
parity.
|
||||
|
||||
Manual validation (owner): upload, replace, delete from Mon compte on
|
||||
desktop and mobile; photo visible in the sidebar, the drawer and the top
|
||||
bar; copy the signed URL, open it in a private window (works until expiry,
|
||||
then 400); try the object URL without token (400); upload a renamed PNG and
|
||||
an HTML file (refused with a clear message); restore drill on the local
|
||||
stack.
|
||||
|
||||
### IV.3 Risks
|
||||
|
||||
- **Decoder vulnerabilities** (libvips / libwebp history): the worker is
|
||||
the only process that decodes untrusted images, runs non-root, read-only,
|
||||
memory-capped, with no inbound port and egress limited to the Supabase
|
||||
host; `sharp` updates become a routine chore. Residual risk accepted and
|
||||
documented.
|
||||
- **Worker down**: uploads stay `uploaded`; the UI says "toujours en
|
||||
cours"; nothing is lost; the healthcheck and `restart: unless-stopped`
|
||||
cover the common case; a monitoring hook is Step 27.
|
||||
- **Signed URL as bearer**: a leaked avatar URL exposes a 512 px picture
|
||||
for at most one hour. Documents will use short TTLs.
|
||||
- **CSP in production**: `connect-src` and `img-src` must include the
|
||||
Supabase origin, or neither the API nor the avatars load. Documented in
|
||||
DEPLOY.md; the owner applies it at deployment.
|
||||
- **PGlite shim vs real `storage` schema**: policies use only `bucket_id`
|
||||
and `name`, no storage helper function, so both engines evaluate the same
|
||||
expression; the real stack's `protect_delete` trigger forbids direct
|
||||
deletes in tests (LRN-041), writes are asserted by census.
|
||||
- **Storage API version drift** between the CLI image, the self-hosted
|
||||
compose and managed: signing and RLS semantics are stable since 2023; the
|
||||
integration test on the CI stack catches a regression.
|
||||
- **Service key handling on the VPS**: a second env file with `chmod 600`;
|
||||
the app `.env` stays key-free; `env.js` never carries it (test).
|
||||
- **Orientation and colour**: EXIF orientation is applied before the
|
||||
rotation data is dropped; wide-gamut ICC profiles are dropped, a slight
|
||||
colour shift on rare photos is accepted for an avatar.
|
||||
- **Plan deviation**: the worker arrives at Step 8 instead of Step 19;
|
||||
Step 19 extends it rather than creating it. Recorded in the plan.
|
||||
|
||||
### IV.4 Decisions for the owner
|
||||
|
||||
| Id | Question | Recommendation |
|
||||
|---|---|---|
|
||||
| D1 | Server-side processing runtime | Containerised worker `faunosteo_worker` (II.1), no Edge Functions |
|
||||
| D2 | Sheet bucket name | Keep `anatomical-sheet-assets`, record the deviation from the announced `anatomical-sheets` |
|
||||
| D3 | Top bar avatar | Add avatar (+ name on desktop) as a link to Mon compte, no dropdown; or sidebar / drawer / account only |
|
||||
| D4 | Avatar output | One 512 × 512 WebP, centre square crop, no metadata; inputs JPEG / PNG / WebP ≤ 5 MiB |
|
||||
| D5 | Self-hosted Storage backend and backups | File backend on a named volume + restic to Scaleway Object Storage, database dumped first |
|
||||
| D6 | SVG processor in Step 8 | Yes, in the worker with hostile fixtures, producer at Steps 13 / 17 |
|
||||
| D7 | Signed URL TTL for avatars | 3600 s, cached client-side |
|
||||
| D8 | Polling vs Realtime for the processing state | Polling the own row (1 s, 30 s cap); Realtime stays off |
|
||||
Reference in new issue
Block a user