docs(docs): V1-2.3 verification worker, M6, sandbox e2e and rehearsals
This commit is contained in:
1 parent
5afb34ed0f
commit
7043d39ae0
7 files changed
+106
-10
No files matched your search
@@ -4454,3 +4454,44 @@ Ignored cargo tests: the three are one test, `pdf_serialize::l_report_with_real_
|
||||
once per cargo run whose features build `pdf_serialize` (workspace, `cli`, `ts,pdf-read,pdf-write`). A timing print for
|
||||
§10.7, 40 s in debug; run on request in release (2026-11-01: passed, 120 views, 46.5 MB, 5.2 s). The L report's export
|
||||
itself is covered unignored (`pdf_budget`); it masks no required scenario.
|
||||
|
||||
## 13. V1-2.3: the server verdict in the worker (2026-11-01)
|
||||
|
||||
Owner brief « Validation définitive V1-2.2 » (V1-2.2 VERIFIED at `c73415a`): V1-2.3 along the V1-2 contract §16.4.
|
||||
Contract `.claude/tasks/contracts/2026-11-01-v1-2-3-worker-verification.md`, plan
|
||||
`.claude/tasks/plans/2026-11-01-v1-2-3-worker.md`. Sandbox only; DEV untouched.
|
||||
|
||||
| Part | What | Where |
|
||||
|---|---|---|
|
||||
| M6 | `claim_sheet_verifications`, `get_sheet_verification_input`, `complete_sheet_verification`, `fail_sheet_verification`, service role only (claims checked too); the verdict checked for its binding (stale) and its structure (malformed), nothing written otherwise | `supabase/migrations/20261101100000_sheet_verification_jobs.sql`, pgTAP `021` (51) |
|
||||
| Link | the worker links `faunosteo-core` (default features: no PDF, no curation); `version` prints the core version | `crates/worker/Cargo.toml` |
|
||||
| Child | `verify`: framed stdin (`[u32 LE header length][header JSON][document][base][drawings]`), `compute_verdict`, verdict JSON out; same sandbox as `decode`, its own limits (12 MiB in, 4 MiB out, 1 GiB, 60 s CPU, 90 s); a frame the child cannot read is a broken parent (`frame_invalid`), never content | `verify_frame.rs`, `verify.rs`, `limits.rs` |
|
||||
| Spawner | per command: `Spawner::for_decode` (unchanged values), `Spawner::for_verify`; a verify output counts only when it parses as a JSON object | `spawn.rs` |
|
||||
| Mode | `run --jobs sheet-verify`: no sweep; drawings fetched under the archive caps, anything else passed without bytes for the core to name; §6.5 rows mapped to closed worker codes; watchdog stage `VERIFY` 95 s | `run.rs`, `verify_job.rs`, `backend.rs`, `http.rs` |
|
||||
| Compose | service `worker-verify`, profile `worker`, same hardening as `worker` | `docker-compose.yml` |
|
||||
|
||||
Seccomp: the verify child at the caps was killed by SIGSYS in the Alpine stage; the derivation
|
||||
(`derive-seccomp.sh`, extended to trace `verify --no-seccomp` on the clean and at-caps frames) traced `brk`
|
||||
(musl mallocng growing the heap for the core's many small allocations) and nothing else new. `brk` was added to the
|
||||
allowlist shared by both children; it only moves the program break, bounded by `RLIMIT_AS`. A filter per command
|
||||
would keep `decode` without it, for more code: not done, listed for the owner.
|
||||
|
||||
Proofs (2026-11-01):
|
||||
- `pnpm worker:test`: unit 28, `job_loop` 11 (unchanged), `verify_loop` 15, sandbox 23 under the full filter
|
||||
(clean sheet passed, at-caps sheet within the limits, empty environment, over-cap input, malformed frame,
|
||||
violation killed), derivation « allowlist covers the trace ». `pnpm worker:image` ok. `pnpm rust:check` green.
|
||||
- `pnpm db:sandbox verify-e2e` (real worker container, gateway on the sandbox network, no host port): 7 tests,
|
||||
cases a to h of the contract: clean equine draft opened (about 60 ms in the worker); `svg_not_canonical` and
|
||||
`drawing_missing` with their exits; `no_geometry` and `out_of_frame`; no decision or publication without a current
|
||||
passed verdict, then approval and publication after it; stale and malformed completions leave the round
|
||||
byte-identical; avatar and purge rows untouched by the verification mode; the working copy verified in place on
|
||||
the shared drawing object; the at-caps document (1 059 150 bytes of text, 129 groups) verified in 185 ms by the
|
||||
worker, 842 ms from submission (budget 120 s), on a 10 × 10 drawing (on the horse frame its markers fail
|
||||
`marker_too_small`, the rule working).
|
||||
- Upgrade rehearsals on the sandbox: from the state before M6, and from the state before M4 (the DEV state, M4 to
|
||||
M6 in one `migration up`): pgTAP 1 859 PASS, `verify-e2e` 7/7, `types --check` equal.
|
||||
- Not measured: the largest input the policy allows (a 2 MiB document, its 2 MiB base and four drawings, 6 MiB).
|
||||
|
||||
Choices for the owner: worker codes, attempt limit five, lease 10 minutes, verdict cap 4 MiB, service name
|
||||
`worker-verify`, the shared `brk`, and that an object larger than the policy is passed without bytes (named
|
||||
`drawing_missing` by the core, not `size_mismatch`).
|
||||
@@ -234,10 +234,50 @@ Docker's restart policy brings it back.
|
||||
Those are the only deletions the worker ever makes: a key it did not get
|
||||
from the queue or the leftover query is never deleted.
|
||||
|
||||
### 3.6 Logs
|
||||
### 3.6 Sheet verification mode (V1-2.3)
|
||||
|
||||
`faunosteo-worker run --jobs sheet-verify`, service `worker-verify` under the
|
||||
Compose profile `worker` (same image, environment file and hardening as
|
||||
`worker`), runs the server verdict of submitted sheet rounds (V1-2 contract
|
||||
§6.4, §6.5) and nothing else: no avatar job, no sweep. The default `run` is
|
||||
unchanged.
|
||||
|
||||
- Parent: `claim_sheet_verifications(1)` (M6, service role only), then
|
||||
`get_sheet_verification_input` (the frozen document and the base as text,
|
||||
the drawing rows), then each drawing from `anatomical-sheet-assets` at its
|
||||
key `<sha256>.svg`. A row not uploaded, a key that is not its own sha256,
|
||||
an object missing (404) or over the policy (at most 4 drawings, 2 MiB each,
|
||||
6 MiB in all: the archive caps) is passed to the core without bytes, so the
|
||||
verdict names it (`drawing_missing`, caps): a business failure, never a
|
||||
worker failure. The parent never parses the document or a drawing.
|
||||
- Child `faunosteo-worker verify`: same sandbox as `decode` (empty
|
||||
environment, one thread, no core dump, the seccomp allowlist) with its own
|
||||
limits: framed stdin input up to 12 MiB, `RLIMIT_AS` 1 GiB, `RLIMIT_CPU`
|
||||
60 s, 90 s deadline, verdict output up to 4 MiB (the SQL cap). It runs the
|
||||
core's `compute_verdict` and writes the verdict JSON; a failed verdict is
|
||||
still a normal exit. The seccomp derivation of 2026-11-01 traced `brk` in
|
||||
the verify child at the caps (musl mallocng growing its heap): added to the
|
||||
allowlist, the only change.
|
||||
- Completion: `complete_sheet_verification` answers `passed`, `failed`,
|
||||
`stale` (nothing written, logged) or `malformed` (reported
|
||||
`verification.worker.verdict_refused`). Child timeout →
|
||||
`verification.worker.timeout`; a signal, an output overflow or an input over
|
||||
its cap → `verification.worker.resource_limit`; a transport error →
|
||||
`verification.worker.unavailable`; a broken child → `unavailable`, then the
|
||||
worker stops (as the avatar loop). A dead lease (10 minutes) is claimed
|
||||
again as `verification.worker.lease_expired`; the fifth attempt closes the
|
||||
round `verification_failed` / `unavailable`.
|
||||
- Measured (2026-11-01): the at-caps document (64 views, 1 024 annotations,
|
||||
8 192 nodes) verified in 185 ms by the worker on the sandbox, verdict
|
||||
24 689 bytes; the equine draft in about 60 ms.
|
||||
- Sandbox only: `pnpm db:sandbox verify-e2e` starts a gateway and this
|
||||
worker on the sandbox network (SUPABASE.md §5.1). Never started on DEV
|
||||
(V1-2 contract §16.11 item 3).
|
||||
|
||||
### 3.7 Logs
|
||||
|
||||
One JSON object per line on stdout, fields from an allowlist only: `ts`,
|
||||
`level`, `event`, `worker_id`, `asset_id`, `stage`, `code`, `signal`,
|
||||
`level`, `event`, `worker_id`, `asset_id`, `review_id`, `stage`, `code`, `signal`,
|
||||
`http_status`, `duration_ms`, `count`, `detail` (worker-authored text only).
|
||||
The child's stderr is child-controlled: it is logged under `child_stderr`
|
||||
JSON-escaped and truncated to 200 bytes, never raw. No key, no URL with a
|
||||
|
||||
@@ -800,6 +800,7 @@ Docker.
|
||||
| `pnpm db:sandbox types [--check]` | `supabase gen types --db-url ... --schema public` through Prettier, written or diffed with the committed file |
|
||||
| `pnpm db:sandbox test` | `packages/supabase/tests/sandbox/*.sandbox.ts`: real GoTrue, PostgREST and Storage sessions, concurrent SQL sessions (two-session lock order, V1-2.1), timing at the caps, the PGlite shim against the image; re-runnable on one stack |
|
||||
| `pnpm db:sandbox e2e-shell` (= `pnpm e2e:shell`) | the authenticated `/app` shell in Chromium (`apps/app/e2e/shell`, `docs/development/BROWSER_TESTS.md`): a loopback dev server whose Supabase calls are replayed on the sandbox, fictitious accounts per test |
|
||||
| `pnpm db:sandbox verify-e2e` | V1-2.3: a gateway (`nginx:1.31-alpine`, sandbox network only, no host port) and the real verification worker `faunosteo_worker:dev run --jobs sheet-verify` against the sandbox, then `packages/supabase/tests/sandbox/sheet-verification.sandbox.ts`; both containers removed at the end, the service key passed through a 600 file deleted at start |
|
||||
| `pnpm db:sandbox sql` / `down` | psql as postgres on stdin / remove the stack |
|
||||
|
||||
Every CLI call passes `--db-url` with `sslmode=disable` (the sandbox has no
|
||||
|
||||
@@ -926,6 +926,16 @@ own (§16.11), because a started DEV worker also processes the avatar upload
|
||||
waiting since the V1-1 acceptance (AVATAR-01) and the due rows of the purge
|
||||
queue.
|
||||
|
||||
Implemented in V1-2.3 (2026-11-01, migration `20261101100000_sheet_verification_jobs.sql`,
|
||||
`crates/worker` mode `sheet-verify`, STORAGE.md §3.6): lease 10 minutes, attempt limit
|
||||
five, verdict text at most 4 MiB; the completion also refuses an unknown verdict format
|
||||
or code, a duplicated group key and a drawing whose result is not `ok`, and accepts a
|
||||
failed verdict without groups or drawings; worker codes `verification.worker.unavailable`,
|
||||
`.timeout`, `.resource_limit`, `.lease_expired` (a dead lease), `.verdict_refused` (a
|
||||
completion answered `malformed`), a closed list in SQL (`private.sheet_worker_codes`).
|
||||
These values and codes are the implementer's, for the owner's confirmation. The
|
||||
service role executes these four RPCs besides the two upload ones (pgTAP 018).
|
||||
|
||||
Rejected: a verdict computed in browsers (contrary to the owner's rule), a
|
||||
synchronous HTTP verification service (a public endpoint holding the service
|
||||
key, a sandbox to rebuild; not in the MVP), an operator command-line
|
||||
@@ -945,7 +955,7 @@ did not pass; the closed codes keep the categories apart.
|
||||
| Timeout or resource limit | child over its wall-clock or memory limit | stays `verifying`, code recorded | yes, up to the limit | reduce the document or the drawing if it repeats |
|
||||
| Interruption | worker crash, lease expired | claimable again; counts as an attempt | yes, up to the limit | automatic |
|
||||
| Stale result | completion with an old token, after a withdrawal, or naming another revision | nothing written, logged | no | none needed |
|
||||
| Definitive failure | attempt limit reached (five, proposal, tuned in V1-2.3) | `verification_failed`, outcome `unavailable`, last code | no | submit again later |
|
||||
| Definitive failure | attempt limit reached (five; V1-2.3 implements five, for the owner's confirmation) | `verification_failed`, outcome `unavailable`, last code | no | submit again later |
|
||||
| Core change | release that changes rules or reader | rounds withdrawn by the release migration | no | submit again |
|
||||
|
||||
- Withdrawal is always available: the submitter or any editor withdraws a
|
||||
@@ -1695,7 +1705,7 @@ of the document's JSON text in one form, the same for every layer:
|
||||
|---|---|---|---|
|
||||
| M4 `sheet_access_hardening` | V1-2.1 | `anatomical_catalog_grants` and its RPCs; `has_mfa`, `grant_active`, `caller_has`; `can_read_official_catalog` and the reader row predicate; `can_edit_sheet` redefined (F14); AR-1 B on the platform role RPCs and the private write functions (F19, §5.7), F17, F18; F1, F2, F3 (fork closed), F5, F6, F8, F9, F12 (every catalogue table), F13, F15; asset-row immutability trigger (F16); historical-sheet protection trigger (§14.2); reader column grants narrowed; personal creation and visibility changes refused | no row changes |
|
||||
| M5 `sheet_review_workflow` | V1-2.1 | version columns `revision`, `authors`, `source_version_id`, `published_review_id`, `document_fingerprint`; sheet column `working_copy_source_version_id`; revision, author and freeze triggers, the historical-sheet write rule; extended immutability guard (§6.8); rounds, comments, acknowledgments; `create_official_sheet`, `create_sheet_draft` adapted, `save_sheet_draft`, submit, withdraw, comment, acknowledge, decide, publish; `create_sheet_working_copy`; species RPCs; catalogue and version read RPCs; upload: `register_sheet_asset` extended, `confirm_sheet_asset_upload` with the storage check (F7), INSERT policy, bucket limit 2 MiB; mirror removal and status check (§12.2); superseded signatures dropped; the content-size CHECK replaced by the text measure (`octet_length(content::text)`, §12.2) | new columns take their defaults on the equine v1 row (`revision` 0, empty author set); its document, status, number and assets unchanged; the new size CHECK is validated against the existing rows when M5 is applied (the equine document measures 184 105 bytes as `content::text`, measured on the sandbox); bucket limit lowered; no round, no grant |
|
||||
| M6 `sheet_verification_jobs` | V1-2.3 | worker RPCs (claim, input, complete with the verdict structure check, fail), service role only | none |
|
||||
| M6 `sheet_verification_jobs` | V1-2.3 | worker RPCs (claim, input, complete with the verdict structure check, fail), service role only; written as `20261101100000_sheet_verification_jobs.sql`, pgTAP `021`, rehearsed on the sandbox from the state before M4 (M4 to M6 in one `migration up`) and from the state before M6 | none |
|
||||
|
||||
Each migration ships its pgTAP file (allowed and denied cases), regenerated
|
||||
`database.types.ts`, a `db:sandbox up upgrade` run from the DEV state (with a
|
||||
@@ -2154,6 +2164,7 @@ V1-2.7 on.
|
||||
verification time at the caps within budget.
|
||||
- Rollback: branch revert; the worker image exists on the sandbox only.
|
||||
- Checkpoint: owner validates the server workflow (V1-2.1 to V1-2.3).
|
||||
- State 2026-11-01: built and tested on the sandbox (RUST.md §13); DEV untouched.
|
||||
|
||||
### 16.5 V1-2.4 Wireframes (parallel to V1-2.1 to V1-2.3)
|
||||
|
||||
|
||||
Reference in new issue
Block a user