feat(supabase): retry delay of 2, 5, 10, 20 s for technical verification failures
M6 amended in place (applied only to the throwaway sandbox): a hidden not_before on the round, set by fail_sheet_verification and honoured by the claim; RPC signatures unchanged. pgTAP 021 (68), sandbox cases i (delay with the real worker), j (inputs at the caps, 2 MiB base), k (just beyond).
This commit is contained in:
1 parent
e967ce6736
commit
5adad3709f
8 files changed
+398
-29
No files matched your search
@@ -871,12 +871,13 @@ fingerprint or a document hash from a page.
|
||||
|
||||
Job. The round row is the job, with the avatar lease pattern of
|
||||
`20261013100000_files_foundation.sql` (lease token, `locked_at`, fixed lease,
|
||||
attempts below a limit), without backoff. Submission sets the round
|
||||
`verifying`. Worker RPCs, service role only (M6):
|
||||
attempts below a limit), plus a progressive delay for technical retries only
|
||||
(owner decision 2026-11-01). Submission sets the round `verifying`. Worker RPCs, service role only (M6):
|
||||
|
||||
- `claim_sheet_verifications(limit)`: `for update skip locked` over rounds
|
||||
`verifying` with no live lease; sets the token and `locked_at`, increments
|
||||
`attempts`; returns round, version, sheet, revision and token.
|
||||
`verifying` with no live lease and no pending delay (`not_before` null or
|
||||
reached); sets the token and `locked_at`, increments `attempts`, clears
|
||||
`not_before`; returns round, version, sheet, revision and token.
|
||||
- `get_sheet_verification_input(review_id, lease_token)`: the document at the
|
||||
round's revision (frozen) and the base document, both as text; the drawings
|
||||
the document references among the rows registered for the sheet (asset id,
|
||||
@@ -889,8 +890,17 @@ attempts below a limit), without backoff. Submission sets the round
|
||||
else answers `stale` or `malformed` and writes nothing. Moves the round to
|
||||
`open` or to `verification_failed` (§6.5) and audits.
|
||||
- `fail_sheet_verification(review_id, lease_token, code)`: a technical
|
||||
failure; the round stays claimable until the attempt limit, then becomes
|
||||
`verification_failed` with outcome `unavailable`.
|
||||
failure; the round stays `verifying` and becomes claimable after a delay
|
||||
that grows with the attempt (2 s after attempt 1, 5 s after 2, 10 s after 3,
|
||||
20 s after 4; `private.sheet_verification_retry_delay`), kept in the round's
|
||||
`not_before` (a column the API roles cannot read, set here, cleared by the
|
||||
claim); the fifth failure becomes `verification_failed` with outcome
|
||||
`unavailable` and clears it. Nothing is written for a stale call. The delay
|
||||
is persisted, so no worker sleeps and any worker honours it; the effective
|
||||
wait is the delay plus up to one poll interval of the worker (2 s by
|
||||
default). Only these technical failures wait: a validation or integrity
|
||||
refusal is a verdict and is never retried, and a dead lease is claimed again
|
||||
at once (the 10-minute lease is its wait).
|
||||
|
||||
Processing. Verification runs as its own mode of the worker image (`run
|
||||
--jobs sheet-verify`), in its own container under the `worker` compose
|
||||
@@ -951,9 +961,9 @@ did not pass; the closed codes keep the categories apart.
|
||||
|---|---|---|---|---|
|
||||
| Business validation | integrity error, caps exceeded, annotation without geometry, annotation outside its view's frame | `verification_failed`, outcome `failed`, codes `verification.validation.*` and the issues | no | fix the draft, submit again |
|
||||
| Integrity | strict parse refused, drawing missing, size or sha256 mismatch, SVG refused, needing a removal or outside the stored form (§9.2), `object_conflict` | `verification_failed`, outcome `failed`, codes `verification.integrity.*` | no | fix or upload again; `object_conflict` follows §9.3 |
|
||||
| Transient worker error | database or Storage unreachable, worker restart | stays `verifying`, code recorded | yes, up to the limit | automatic |
|
||||
| Timeout or resource limit | child over its wall-clock or memory limit | stays `verifying`, code recorded | yes, up to the limit | reduce the document or the drawing if it repeats |
|
||||
| Interruption | worker crash, lease expired | claimable again; counts as an attempt | yes, up to the limit | automatic |
|
||||
| Transient worker error | database or Storage unreachable, worker restart | stays `verifying`, code recorded | yes, after 2 s, 5 s, 10 s, 20 s, up to the limit | automatic |
|
||||
| Timeout or resource limit | child over its wall-clock or memory limit | stays `verifying`, code recorded | yes, same delays, up to the limit | reduce the document or the drawing if it repeats |
|
||||
| Interruption | worker crash, lease expired | claimable again at once (the lease was the wait); counts as an attempt | yes, up to the limit | automatic |
|
||||
| Stale result | completion with an old token, after a withdrawal, or naming another revision | nothing written, logged | no | none needed |
|
||||
| Definitive failure | attempt limit reached (five; V1-2.3 implements five, for the owner's confirmation) | `verification_failed`, outcome `unavailable`, last code | no | submit again later |
|
||||
| Core change | release that changes rules or reader | rounds withdrawn by the release migration | no | submit again |
|
||||
@@ -1705,7 +1715,7 @@ of the document's JSON text in one form, the same for every layer:
|
||||
|---|---|---|---|
|
||||
| M4 `sheet_access_hardening` | V1-2.1 | `anatomical_catalog_grants` and its RPCs; `has_mfa`, `grant_active`, `caller_has`; `can_read_official_catalog` and the reader row predicate; `can_edit_sheet` redefined (F14); AR-1 B on the platform role RPCs and the private write functions (F19, §5.7), F17, F18; F1, F2, F3 (fork closed), F5, F6, F8, F9, F12 (every catalogue table), F13, F15; asset-row immutability trigger (F16); historical-sheet protection trigger (§14.2); reader column grants narrowed; personal creation and visibility changes refused | no row changes |
|
||||
| M5 `sheet_review_workflow` | V1-2.1 | version columns `revision`, `authors`, `source_version_id`, `published_review_id`, `document_fingerprint`; sheet column `working_copy_source_version_id`; revision, author and freeze triggers, the historical-sheet write rule; extended immutability guard (§6.8); rounds, comments, acknowledgments; `create_official_sheet`, `create_sheet_draft` adapted, `save_sheet_draft`, submit, withdraw, comment, acknowledge, decide, publish; `create_sheet_working_copy`; species RPCs; catalogue and version read RPCs; upload: `register_sheet_asset` extended, `confirm_sheet_asset_upload` with the storage check (F7), INSERT policy, bucket limit 2 MiB; mirror removal and status check (§12.2); superseded signatures dropped; the content-size CHECK replaced by the text measure (`octet_length(content::text)`, §12.2) | new columns take their defaults on the equine v1 row (`revision` 0, empty author set); its document, status, number and assets unchanged; the new size CHECK is validated against the existing rows when M5 is applied (the equine document measures 184 105 bytes as `content::text`, measured on the sandbox); bucket limit lowered; no round, no grant |
|
||||
| M6 `sheet_verification_jobs` | V1-2.3 | worker RPCs (claim, input, complete with the verdict structure check, fail), service role only; written as `20261101100000_sheet_verification_jobs.sql`, pgTAP `021`, rehearsed on the sandbox from the state before M4 (M4 to M6 in one `migration up`) and from the state before M6 | none |
|
||||
| M6 `sheet_verification_jobs` | V1-2.3 | worker RPCs (claim, input, complete with the verdict structure check, fail), service role only, with the retry delay (hidden `not_before` column on the round, `private.sheet_verification_retry_delay`: owner decision 2026-11-01, replaces « without backoff »); written as `20261101100000_sheet_verification_jobs.sql`, pgTAP `021`, rehearsed on the sandbox from the state before M4 (M4 to M6 in one `migration up`) and from the state before M6 | none |
|
||||
|
||||
Each migration ships its pgTAP file (allowed and denied cases), regenerated
|
||||
`database.types.ts`, a `db:sandbox up upgrade` run from the DEV state (with a
|
||||
@@ -2021,8 +2031,9 @@ The final approval needs all 30 groups acknowledged, no `to_confirm` and no
|
||||
- Worker: the verification mode, verdicts accepted and each failure, the
|
||||
hostile SVG corpus in the sandboxed child, per-command limits and watchdog,
|
||||
the purge guard by bucket, sandbox tests and seccomp derivation, an e2e on
|
||||
the sandbox stack with the verification worker running; `pnpm e2e:avatar`
|
||||
unchanged in outcome; `pnpm e2e:sheet-assets` targets DEV only, its rerun
|
||||
the sandbox stack with the verification worker running; the avatar suite
|
||||
unchanged in outcome on the sandbox (`pnpm db:sandbox avatar-e2e`, owner
|
||||
decision 2026-11-01: `pnpm e2e:avatar` targets DEV and is not run); `pnpm e2e:sheet-assets` targets DEV only, its rerun
|
||||
is an owner step (§16.11).
|
||||
- Release replay: a script loads every published version and drawing of a
|
||||
stack through the new core and fails on any regression.
|
||||
|
||||
Reference in new issue
Block a user