docs(docs): R5.5 upload tool runbook, caps, serde tolerance characterisation, contract

This commit is contained in:
bchanot committed 2026-10-23 21:16:59 +02:00
1 parent 2c445063a4
commit a481af52c7
5 files changed
+221 -2

No files matched your search

@@ -0,0 +1,71 @@
# CONTRACT: Rust R5.5, the sheet-asset upload tool (`upload-sheet-assets`) and the R5.4 closing adjustments
- date: 2026-10-23 | branch `feature/rust-r5-pipeline-core` | plan `.claude/tasks/plans/2026-10-22-rust-r5-pipeline-core.md` §B.5 row R5.5
- brief: owner 2026-10-23 « Validation R5.4 et autorisation conditionnelle R5.5 » §1 to §10 and §12
- predecessor: `.claude/tasks/contracts/2026-10-23-rust-r5-review.md` (R5.4)
## Scope
Port `packages/supabase/scripts/upload-sheet-assets.ts` into the core crate (`crates/core`, feature `upload`): rules in
`src/curation/upload.rs`, binary in `src/bin/upload_sheet_assets/{main,storage}.rs`; scripts `scripts/upload-sheet-assets.sh`,
`scripts/sheet-assets-e2e.sh`; pnpm `assets:upload` (Rust), `e2e:sheet-assets`, TypeScript kept as `assets:upload:legacy`
until the R5.7 census. Plus the R5.4 closing adjustments (owner §2, §3, §6, §7). Out of scope: the fingerprint convention
(R5.6), any seed or `documentSha256` change, a gated reader change, deleting `packages/anatomy` or `packages/domain`, any
deploy, any production or managed-Supabase access.
## Clauses
C1. **Core free of I/O.** `curation::upload` takes values and returns values or `UploadError`; no `std::fs`, `std::env`,
network or clock. `ureq` is an optional dependency enabled only by feature `upload` (never by the wasm facade or the
worker); only the binary reads files, the environment and the network.
C2. **Local destination only, verified first.** There is no default destination. `SUPABASE_URL` must be `http`, carry an
explicit port, no credentials, path, query or fragment, and a host that is `localhost`, a loopback address, or the private
IPv4 address equal to `FAUNOSTEO_DEV_HOST`; anything else is refused (kind `Destination`) before any request. The key is
never sent to a refused destination.
C3. **Inputs verified before the first request.** The folder lies under `supabase/seed/anatomical-sheets` (canonical path,
symlinks resolved). `assets.json` is size-capped before parsing, strict on keys, plain `.svg` file names, lowercase sha256
and uuid, size range, no repeated file, hash or asset id, 1 to 64 entries. Each drawing is read under a cap and equals its
entry (size, sha256) and is accepted by the core's SVG reader with nothing removed. A failure stops the run with no request.
C4. **Handshake.** Before any object read or write: the bucket `anatomical-sheet-assets` exists, is private, and the key is
accepted (401/403 are `Access`).
C5. **No overwrite, delete or visibility change.** Objects are created with `x-upsert: false`; an object under the key is
read back and hashed: other bytes refuse the run (`State`), never replaced. A missing asset row, or a row recording another
hash, refuses the run. The tool issues no DELETE, no PUT, no bucket or policy change. The plan of every entry is computed
(read-only) before the first write; a run that cannot complete writes nothing.
C6. **Confirm after verify; safe resume.** The row is confirmed only after the object is read back and hashes to its key. After
any failure a rerun resumes: a verified object is kept, a confirmed row is left alone. Transport errors and 5xx are retried
three times, refusals are final.
C7. **Secrets.** The service key comes from the environment only, is held in a type whose `Debug` is redacted, travels in two
request headers, never in a URL, a message or a log; the agent uses no proxy and no redirect; server error text is not echoed.
C8. **Closed errors and exit codes.** Failures fall in `Manifest | Destination | Content | Handshake | State | Access |
Network`; exit 0 done, 1 refused, 2 usage (argument not UTF-8), 3 network. No failure silently drops an entry.
C9. **Identity and bytes preserved.** The object key is `<sha256>.svg` with the manifest hash, the row is addressed by the
manifest `svg_asset_id`, the bytes uploaded are the file's bytes (no re-cleaning, no re-serialisation).
C10. **Tests.** `tests/curation_upload.rs` (manifest, bytes, destination, plan), `tests/upload_tool.rs` (the binary against a
loopback stub: check writes nothing, run then rerun, different bytes, missing and foreign row, 401/403, resume after a failed
confirm, unreachable port, refused destinations, folder outside the seed, usage), `tests/curation_caps.rs`;
`scripts/sheet-assets-e2e.sh` for the real local stack (run by the owner; the assistant's own run was refused by the permission
layer and is reported as such). `scripts/rust-check.sh` runs clippy and the tests with `--features upload`.
C11. **Runbook.** `docs/development/SUPABASE.md` §3.1, `docs/development/RUST.md` §11.16, CHANGELOG, CLAUDE.md commands.
C12. **Frozen references untouched.** No change under `supabase/seed/**`, `apps/app/tests/fixtures/**`,
`packages/core-wasm/tests/n-1/**`, `crates/core/src/domain/**`, `crates/core/src/pdf/**`, `crates/core/tests/oracles/**`;
`documentSha256` stays `27c7383c…2660`; `pnpm r5:import-equine --check` still reports 0 differences on 18 files.
C13. **R5.4 closing adjustments.** (a) `changes.json` and `references.json` are refused above 4 MiB before any parse allocation
with an explicit message, in the core (`MAX_CURATION_FILE_BYTES`) and in the binary read; other budgets unchanged; (b) the FLOOR
clarification of the R5.4 contract holds with its negative proof (`scripts/floor-clarification-check.sh`); (c) no gated reader
and no serde tolerance is changed (characterisation only, RUST.md §11.16).
C14. **Gates and limits.** `pnpm rust:check` and `pnpm check:all` green; security auditor PASS on the diff; this contract
CONFORME by a fresh verifier. No merge, no R5.6, no release, no deploy, nothing outside the local stack.
+5
View File
@@ -6,6 +6,11 @@ Versions follow [Semantic Versioning](https://semver.org/).
## [Unreleased]
### Changed (2026-10-23, Rust R5.5: the sheet-asset upload tool in Rust)
- `pnpm assets:upload [--check] <seed folder>` runs the Rust binary `upload-sheet-assets` (`crates/core`, feature `upload`, `ureq`) in place of the TypeScript script (kept as `assets:upload:legacy` until R5.7). Local stack only: no default destination, `http` with an explicit port on loopback or the configured DEV address, a private-bucket handshake before any transfer; every file checked against `assets.json` and the SVG reader before the first request; an existing object is hashed and never overwritten, a missing or foreign asset row stops the run, a row is confirmed only after its object is read back; bounded retries, resumable; the service key is never printed. Runbook `docs/development/SUPABASE.md` §3.1, `pnpm e2e:sheet-assets` for the local stack, `docs/development/RUST.md` §11.16.
- `changes.json` and `references.json` are refused above 4 MiB with an explicit message (core and binary). The serde tolerance of the R2 contracts is characterised (test `characterisation_serde_tolerance`), not changed.
### Changed (2026-10-23, Rust R5.4: the review files in Rust)
- The review drawings (`review/annotations-*.svg`, `review/app-*.svg`), `CHANGELOG.md` and `REVIEW-REPORT.md` of the draft are produced by the Rust pipeline: the two reports and three drawings byte for byte with the committed files; `review/app-main.svg` carries the head trigger's English name ("Head (left lateral view)"), the name the editor canvas shows since R3, where the committed file repeats the French one. `pnpm r5:import-equine --check` now compares the whole draft (12 files) and fails on any file missing or different without a declared rule; `--out` writes the 18 files. The seed, `horse.svg`, `pnpm import:equine` and `documentSha256` are unchanged (`docs/development/RUST.md` §11.14).
+4 -2
View File
@@ -120,6 +120,8 @@ pnpm build:preview # SHEET_BUNDLE=draft build (manual validation, preview de
pnpm preview:draft # same, then serves it
pnpm import:equine # regenerate legacy-import/ and draft/ (document, migration map, review drawings): the TypeScript writer of the seed until the R5 switch
pnpm r5:import-equine --check | --out <dir> # the Rust pipeline (R5.2 importer, R5.3 draft, R5.4 review files): compare the 18 generated files with the committed folders (bytes; app drawings under the declared R3 rule), or write <dir>/{legacy-import,draft} into an empty folder; never deletes (scripts/import-equine.sh, first run compiles)
pnpm assets:upload [--check] <seed folder> # R5.5: uploads the sheet drawings of a generated seed folder to the LOCAL stack's private bucket (Rust `upload-sheet-assets`, feature `upload`; SUPABASE_URL local with a port, SUPABASE_SERVICE_ROLE_KEY typed, never printed; no overwrite, confirm after verify; run by hand, scripts/upload-sheet-assets.sh); the TypeScript one is `assets:upload:legacy` until R5.7
pnpm e2e:sheet-assets # scripts/sheet-assets-e2e.sh: that tool against the local stack (upload, rerun, anon and wrong key, refused destinations); run by hand
pnpm report:compare <pdf|dir>... # archive check: attachment vs its v3 reading, unresolved items flagged (docs/PDF-FORMAT.md); the core's `report-compare` binary since R4.7 (scripts/report-compare.sh, first run compiles)
# Rust (docs/development/RUST.md) — native toolchain (user-level rustup, channel of rust-toolchain.toml) first
@@ -280,10 +282,10 @@ faunosteo/
├─ packages/README.md ← shared packages index
├─ packages/core-wasm/ ← @faunosteo/core-wasm: CoreRuntime (lazy, poison, recovery) + CoreExportSession (R4.6) + Node source, pkg/ shipped and pkg-proofs/ (built, git-ignored, BUILD_STAMP), ts-rs types (committed), frozen N-1 reader (tests/n-1), tests
├─ crates/ ← Rust workspace (Cargo.toml, rust-toolchain.toml, deny.toml, clippy.toml, .cargo/config.toml at the root)
│ ├─ core/ ← faunosteo-core: domain/{js,sheet,report,archive,editor,session} (contracts and rules), anatomy/{svg/{prescan,tree,sanitize,refs,number,serialize},layer,compose} (R3: the one SVG reader, the layer, canvas + highlight composition), pdf/{read,json_js,import,write,compare} (R4: reader, writer and session, the archive check tool behind feature `cli`), ops.rs + facade.rs (`call`, `call_with_sheet`), json (JS number printing), curation/{text,ids,prettier_json,markdown,report_guard,legacy/*,draft/*} (R5, feature `curation`: Prettier layout, the legacy importer, the changeset applier, migration map, stats, seed SQL, CHANGELOG, REVIEW-REPORT, review drawings), bin/report_compare.rs (`pnpm report:compare`) + bin/import_equine.rs (`pnpm r5:import-equine`, the two files allowed std::fs); tests/{oracles,conformance,fixtures,pdf/LEDGER.tsv} frozen from the TypeScript (oracles/r3, oracles/r4, fixtures/svg/hostile); no I/O, no clock (lint-enforced)
│ ├─ core/ ← faunosteo-core: domain/{js,sheet,report,archive,editor,session} (contracts and rules), anatomy/{svg/{prescan,tree,sanitize,refs,number,serialize},layer,compose} (R3: the one SVG reader, the layer, canvas + highlight composition), pdf/{read,json_js,import,write,compare} (R4: reader, writer and session, the archive check tool behind feature `cli`), ops.rs + facade.rs (`call`, `call_with_sheet`), json (JS number printing), curation/{text,ids,prettier_json,markdown,report_guard,legacy/*,draft/*} (R5, feature `curation`: Prettier layout, the legacy importer, the changeset applier, migration map, stats, seed SQL, CHANGELOG, REVIEW-REPORT, review drawings), curation/upload.rs (R5.5, feature `upload`: the upload rules, pure) + bin/report_compare.rs (`pnpm report:compare`) + bin/import_equine.rs (`pnpm r5:import-equine`) + bin/upload_sheet_assets/ (`pnpm assets:upload`, ureq; the three tools allowed std::fs); tests/{oracles,conformance,fixtures,pdf/LEDGER.tsv} frozen from the TypeScript (oracles/r3, oracles/r4, fixtures/svg/hostile); no I/O, no clock (lint-enforced)
│ ├─ wasm/ ← faunosteo-wasm: wasm-bindgen facade (core_version, call, SheetHandle; trap exports in the proofs build only)
│ └─ worker/ ← faunosteo-worker: plain main; `decode` sandboxed child (rlimits, seccomp allowlist, image → libwebp), parent Spawner + RIFF check; R1 `run` loop: config, log (JSON lines, allowlist), keys, backend trait + http (ureq, PostgREST rpc + Storage objects), job, sweep, watchdog; Dockerfile, scripts/derive-seccomp.sh
├─ scripts/{rust-env,rust-check,wasm-build,check-all,avatar-e2e,report-compare,import-equine}.sh, wasm-stamp.mjs ← native Rust gate, wasm build (pkg + pkg-proofs + stamp), the pre-merge gate (`pnpm check:all`), the real-stack avatar e2e, the archive check tool (`pnpm report:compare`), the Rust pipeline (`pnpm r5:import-equine`)
├─ scripts/{rust-env,rust-check,wasm-build,check-all,avatar-e2e,report-compare,import-equine,upload-sheet-assets,sheet-assets-e2e,floor-clarification-check}.sh, wasm-stamp.mjs ← native Rust gate, wasm build (pkg + pkg-proofs + stamp), the pre-merge gate (`pnpm check:all`), the real-stack avatar e2e, the archive check tool (`pnpm report:compare`), the Rust pipeline (`pnpm r5:import-equine`)
├─ scripts/{dev-env,supabase,dev}.sh ← the DEV origin (LAN address) in one place; CLI wrapper; `pnpm dev` on that origin
├─ docker-compose.yml ← `app` service; `worker` service under the `worker` profile (read-only, cap_drop ALL, external Supabase network, `.env.worker`)
├─ docker/rust-builder.Dockerfile, rust-run.sh, csp-proof/ ← reproducible Rust builder (alternative) + the CSP proofs (R0 module, R2 app image: app-image.sh)
@@ -0,0 +1,58 @@
//! Characterisation, not a requirement (owner 2026-10-23, R5.5 brief §7): the serde derives of
//! the R2 contracts read a struct from a positional array and a unit enum from a one-key map,
//! where the frozen Zod reader refused both. These cases pin today's Rust verdict so that any
//! change to the gated readers is visible; they are to be turned around, not extended, if the
//! owner decides a strict deserialisation (`docs/development/RUST.md` §11.16).
#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]
mod common;
use faunosteo_core::domain::report::Report;
use serde::Deserialize;
use serde_json::{Value, json};
fn demo() -> Value {
serde_json::from_str(&common::repo_file(
"apps/app/tests/fixtures/report-v3.demo.json",
))
.unwrap()
}
fn read(value: &Value) -> Result<Report, serde_json::Error> {
Report::deserialize(value)
}
#[test]
fn the_canonical_report_is_read() {
assert!(read(&demo()).is_ok());
}
#[test]
fn a_struct_written_as_a_positional_array_is_accepted_where_zod_refused() {
let mut sheet = demo();
let parts = [
sheet["sheet"]["sheetId"].clone(),
Value::Null,
sheet["sheet"]["documentSha256"].clone(),
];
sheet["sheet"] = json!(parts);
let mut practitioner = demo();
let p = practitioner["practitioner"].clone();
practitioner["practitioner"] = json!([p["name"], p["cabinet"], p["contact"], p["siret"]]);
assert!(read(&sheet).is_ok());
assert!(read(&practitioner).is_ok());
}
#[test]
fn a_unit_enum_written_as_a_one_key_map_is_accepted_where_zod_refused() {
let mut report = demo();
report["observations"][0]["severity"] = json!({ "mild": null });
assert!(read(&report).is_ok());
}
#[test]
fn an_unknown_field_is_accepted_as_the_non_strict_zod_objects_did() {
let mut report = demo();
report["extra"] = json!(1);
assert!(read(&report).is_ok());
}
+83
View File
@@ -3266,3 +3266,86 @@ service key from the environment, bucket `anatomical-sheet-assets`, key `<sha256
`upsert: false`, bytes verified against `assets.json`, RPC `confirm_sheet_asset_upload`) run
against the local stack only, with its runbook.
### 11.16 R5.5: the sheet-asset upload tool, the R5.4 closing adjustments (2026-10-23)
Contract `.claude/tasks/contracts/2026-10-23-rust-r5-upload.md`. Owner brief « Validation R5.4 et
autorisation conditionnelle R5.5 ».
**Tool.** `upload-sheet-assets` (`crates/core`, feature `upload` = `curation` + `ureq`, the version
the worker already uses; `pnpm assets:upload [--check] <seed folder>`). The rules are the core's
(`curation::upload`: manifest, destination, bytes through the one SVG reader, plan from the observed
state, closed error kinds); the binary owns the file system, the environment and the network, split
in `main.rs` (flow, files) and `storage.rs` (HTTP). It replaces `packages/supabase/scripts/
upload-sheet-assets.ts`, kept as `assets:upload:legacy` until the R5.7 census. Behaviour and runbook:
SUPABASE.md §3.1. Differences from the TypeScript tool, all refusals added: no default destination
(a local address with an explicit port is required); an existing object is hashed, not trusted by its
name; a missing or foreign asset row stops the run instead of failing after the upload; the whole run
is planned before the first write; the file must pass the SVG reader with nothing removed; sizes and
entry counts are capped; nothing is overwritten, deleted or made public, ever (`x-upsert: false`, no
DELETE, no bucket call but the read).
**Destination rule.** `http`, explicit port, no credentials, path, query or fragment; host `localhost`,
a loopback address, or the private IPv4 address equal to `FAUNOSTEO_DEV_HOST`. A name other than
`localhost` is never accepted (it cannot be proven local without resolving it). The handshake then
checks the private bucket and the key before any object is read or written. A local port held by
something else would receive the key: accepted on a single-user machine, the same stance as the
worker's `SUPABASE_URL`.
**Tests.** `curation_upload` (7: manifest family, bytes, 15 destination refusals + DEV host rule,
plan), `upload_tool` (10, the binary against a loopback stub with a made-up key: `--check` writes
nothing, run then rerun, different bytes never overwritten, missing and foreign row stop before any
write, 401/403, resume after a failed confirmation, unreachable port, refused destinations, folder
outside the seed, usage), `curation_caps` (4). **Not run by the assistant**: the session against the
real local stack (`pnpm e2e:sheet-assets`: upload, rerun, anon key, wrong key, refused destinations,
unreachable port); the assistant's own run of the tool with the service key was refused by the
permission layer (CLAUDE.md: no transfer tool run by Claude) and was not retried by another route.
The owner runs it.
**Read caps (owner §3).** `changes.json` and `references.json` are refused above 4 MiB, by name, in
the core (`MAX_CURATION_FILE_BYTES`, checked before the parse) and in the binary (bounded read, the
bytes read are counted as well as the size seen). Measured: 221.5 KB and 17.3 KB committed (about 19 and 240
times under the cap); `--check` peaks at 18.5 MB resident; a synthetic changeset of 765 changes
(3.98 MB, ids renumbered) parses at +39 MB over a 7 MB baseline, about 10 times the text. The 65 times
of the R5.4 round-9 security gate (§11.15) is a worst-case shape of many tiny values; at the 4 MiB cap
that is about 260 MB where the 16 MiB read cap allowed about 1 GB. Other budgets (document depth and
values, records, drawings, the 16 MiB read cap for every other source) unchanged. 4 MiB leaves a
wide margin; a future multi-species changeset is a business-contract question and is not decided
here.
**Owner decisions applied.** Quadratic scans (§5): kept, debt recorded in §11.15, no rewrite without
a measured problem. URLs in titles, questions and justifications (§4): kept verbatim in the data and
in the Markdown reports, no active HTML produced; an interface shows them as untrusted text. Divergent
refusals (§6): invalid field segments, ids outside the stable-id grammar, inputs over the budgets, each
a controlled `PipelineError::Source` or `Guard`, none dropping authored data; the grammar is the
sheet contract's `^[a-z0-9][a-z0-9_-]{0,63}$`, species-neutral (`curation_caps` checks ids of other
species). Markdown stays an allowlist: an unreproduced form is a loud refusal, never a different file.
FLOOR (§2): the clarification is in the R5.4 contract, with `scripts/floor-clarification-check.sh`.
**Serde tolerance of the R2 contracts (owner §7), characterisation only; nothing changed.**
Structures: every serde-derived contract of `domain/{sheet,report,archive}`, read by the gated
readers (`report/read.rs`: `Report::deserialize` for v3, `ReportV2::deserialize` for v2;
`archive/read.rs`; the PDF import path). Two tolerances: a struct is read from a positional array,
and a unit enum variant from a one-key map. Probed on Report v3 (test
`characterisation_serde_tolerance`, which pins today's verdict):
| Input (minimal change of `report-v3.demo.json`) | Rust now | Historical Zod contract |
|---|---|---|
| `"sheet": [sheetId, null, documentSha256]` | accepted | refused |
| `"practitioner": [name, cabinet, contact, siret]` | accepted | refused |
| `observations[0].severity` = `{"mild": null}` | accepted | refused |
| a zone written as an array | refused only when the order is wrong, accepted when it matches the field order | refused |
| `"extra": 1` (unknown field) | accepted | accepted (non-strict objects) |
Consequence of accepting: a hand-built or foreign attachment imports; the report is re-serialised in
the canonical object form, so nothing stored is non-canonical, but a positional array read in the
wrong order assigns values to the wrong fields without an error when the types agree (an integrity
risk for crafted files, not a memory or code-execution one). Consequence of refusing: only files the
application never produced are turned away. Risk for R2, R3 and R4 archives: none either way, every
archive the product wrote is in object form (the frozen envelopes round-trip, `report_contract`, and the
N-1 suites), so a strict reading rejects no golden and no fixture. Proposal: the guard already used for
the authored files (`draft/shape.rs`: compare the input with its typed reading serialised back) at the
three reader entries, per-file go of the owner, decided before R5.6 since the membership test touches
the same readers there; until then the gated readers are unchanged.
Open points carried: debt list of §11.12 and §11.13 stands (migration map cascade replay, `clippy.toml`
I/O list); the upload tool is not run against the real stack by the assistant.