From a481af52c7ecff740dd6e90ff8f39cb8353988a2 Mon Sep 17 00:00:00 2001 From: bchanot Date: Fri, 23 Oct 2026 21:16:59 +0200 Subject: [PATCH] docs(docs): R5.5 upload tool runbook, caps, serde tolerance characterisation, contract --- .../contracts/2026-10-23-rust-r5-upload.md | 71 ++++++++++++++++ CHANGELOG.md | 5 ++ CLAUDE.md | 6 +- .../tests/characterisation_serde_tolerance.rs | 58 +++++++++++++ docs/development/RUST.md | 83 +++++++++++++++++++ 5 files changed, 221 insertions(+), 2 deletions(-) create mode 100644 .claude/tasks/contracts/2026-10-23-rust-r5-upload.md create mode 100644 crates/core/tests/characterisation_serde_tolerance.rs diff --git a/.claude/tasks/contracts/2026-10-23-rust-r5-upload.md b/.claude/tasks/contracts/2026-10-23-rust-r5-upload.md new file mode 100644 index 00000000..09b35bec --- /dev/null +++ b/.claude/tasks/contracts/2026-10-23-rust-r5-upload.md @@ -0,0 +1,71 @@ +# CONTRACT: Rust R5.5, the sheet-asset upload tool (`upload-sheet-assets`) and the R5.4 closing adjustments + +- date: 2026-10-23 | branch `feature/rust-r5-pipeline-core` | plan `.claude/tasks/plans/2026-10-22-rust-r5-pipeline-core.md` §B.5 row R5.5 +- brief: owner 2026-10-23 « Validation R5.4 et autorisation conditionnelle R5.5 » §1 to §10 and §12 +- predecessor: `.claude/tasks/contracts/2026-10-23-rust-r5-review.md` (R5.4) + +## Scope + +Port `packages/supabase/scripts/upload-sheet-assets.ts` into the core crate (`crates/core`, feature `upload`): rules in +`src/curation/upload.rs`, binary in `src/bin/upload_sheet_assets/{main,storage}.rs`; scripts `scripts/upload-sheet-assets.sh`, +`scripts/sheet-assets-e2e.sh`; pnpm `assets:upload` (Rust), `e2e:sheet-assets`, TypeScript kept as `assets:upload:legacy` +until the R5.7 census. Plus the R5.4 closing adjustments (owner §2, §3, §6, §7). Out of scope: the fingerprint convention +(R5.6), any seed or `documentSha256` change, a gated reader change, deleting `packages/anatomy` or `packages/domain`, any +deploy, any production or managed-Supabase access. + +## Clauses + +C1. **Core free of I/O.** `curation::upload` takes values and returns values or `UploadError`; no `std::fs`, `std::env`, +network or clock. `ureq` is an optional dependency enabled only by feature `upload` (never by the wasm facade or the +worker); only the binary reads files, the environment and the network. + +C2. **Local destination only, verified first.** There is no default destination. `SUPABASE_URL` must be `http`, carry an +explicit port, no credentials, path, query or fragment, and a host that is `localhost`, a loopback address, or the private +IPv4 address equal to `FAUNOSTEO_DEV_HOST`; anything else is refused (kind `Destination`) before any request. The key is +never sent to a refused destination. + +C3. **Inputs verified before the first request.** The folder lies under `supabase/seed/anatomical-sheets` (canonical path, +symlinks resolved). `assets.json` is size-capped before parsing, strict on keys, plain `.svg` file names, lowercase sha256 +and uuid, size range, no repeated file, hash or asset id, 1 to 64 entries. Each drawing is read under a cap and equals its +entry (size, sha256) and is accepted by the core's SVG reader with nothing removed. A failure stops the run with no request. + +C4. **Handshake.** Before any object read or write: the bucket `anatomical-sheet-assets` exists, is private, and the key is +accepted (401/403 are `Access`). + +C5. **No overwrite, delete or visibility change.** Objects are created with `x-upsert: false`; an object under the key is +read back and hashed: other bytes refuse the run (`State`), never replaced. A missing asset row, or a row recording another +hash, refuses the run. The tool issues no DELETE, no PUT, no bucket or policy change. The plan of every entry is computed +(read-only) before the first write; a run that cannot complete writes nothing. + +C6. **Confirm after verify; safe resume.** The row is confirmed only after the object is read back and hashes to its key. After +any failure a rerun resumes: a verified object is kept, a confirmed row is left alone. Transport errors and 5xx are retried +three times, refusals are final. + +C7. **Secrets.** The service key comes from the environment only, is held in a type whose `Debug` is redacted, travels in two +request headers, never in a URL, a message or a log; the agent uses no proxy and no redirect; server error text is not echoed. + +C8. **Closed errors and exit codes.** Failures fall in `Manifest | Destination | Content | Handshake | State | Access | +Network`; exit 0 done, 1 refused, 2 usage (argument not UTF-8), 3 network. No failure silently drops an entry. + +C9. **Identity and bytes preserved.** The object key is `.svg` with the manifest hash, the row is addressed by the +manifest `svg_asset_id`, the bytes uploaded are the file's bytes (no re-cleaning, no re-serialisation). + +C10. **Tests.** `tests/curation_upload.rs` (manifest, bytes, destination, plan), `tests/upload_tool.rs` (the binary against a +loopback stub: check writes nothing, run then rerun, different bytes, missing and foreign row, 401/403, resume after a failed +confirm, unreachable port, refused destinations, folder outside the seed, usage), `tests/curation_caps.rs`; +`scripts/sheet-assets-e2e.sh` for the real local stack (run by the owner; the assistant's own run was refused by the permission +layer and is reported as such). `scripts/rust-check.sh` runs clippy and the tests with `--features upload`. + +C11. **Runbook.** `docs/development/SUPABASE.md` §3.1, `docs/development/RUST.md` §11.16, CHANGELOG, CLAUDE.md commands. + +C12. **Frozen references untouched.** No change under `supabase/seed/**`, `apps/app/tests/fixtures/**`, +`packages/core-wasm/tests/n-1/**`, `crates/core/src/domain/**`, `crates/core/src/pdf/**`, `crates/core/tests/oracles/**`; +`documentSha256` stays `27c7383c…2660`; `pnpm r5:import-equine --check` still reports 0 differences on 18 files. + +C13. **R5.4 closing adjustments.** (a) `changes.json` and `references.json` are refused above 4 MiB before any parse allocation +with an explicit message, in the core (`MAX_CURATION_FILE_BYTES`) and in the binary read; other budgets unchanged; (b) the FLOOR +clarification of the R5.4 contract holds with its negative proof (`scripts/floor-clarification-check.sh`); (c) no gated reader +and no serde tolerance is changed (characterisation only, RUST.md §11.16). + +C14. **Gates and limits.** `pnpm rust:check` and `pnpm check:all` green; security auditor PASS on the diff; this contract +CONFORME by a fresh verifier. No merge, no R5.6, no release, no deploy, nothing outside the local stack. diff --git a/CHANGELOG.md b/CHANGELOG.md index 84c7bc7c..c55234a9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,11 @@ Versions follow [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Changed (2026-10-23, Rust R5.5: the sheet-asset upload tool in Rust) + +- `pnpm assets:upload [--check] ` runs the Rust binary `upload-sheet-assets` (`crates/core`, feature `upload`, `ureq`) in place of the TypeScript script (kept as `assets:upload:legacy` until R5.7). Local stack only: no default destination, `http` with an explicit port on loopback or the configured DEV address, a private-bucket handshake before any transfer; every file checked against `assets.json` and the SVG reader before the first request; an existing object is hashed and never overwritten, a missing or foreign asset row stops the run, a row is confirmed only after its object is read back; bounded retries, resumable; the service key is never printed. Runbook `docs/development/SUPABASE.md` §3.1, `pnpm e2e:sheet-assets` for the local stack, `docs/development/RUST.md` §11.16. +- `changes.json` and `references.json` are refused above 4 MiB with an explicit message (core and binary). The serde tolerance of the R2 contracts is characterised (test `characterisation_serde_tolerance`), not changed. + ### Changed (2026-10-23, Rust R5.4: the review files in Rust) - The review drawings (`review/annotations-*.svg`, `review/app-*.svg`), `CHANGELOG.md` and `REVIEW-REPORT.md` of the draft are produced by the Rust pipeline: the two reports and three drawings byte for byte with the committed files; `review/app-main.svg` carries the head trigger's English name ("Head (left lateral view)"), the name the editor canvas shows since R3, where the committed file repeats the French one. `pnpm r5:import-equine --check` now compares the whole draft (12 files) and fails on any file missing or different without a declared rule; `--out` writes the 18 files. The seed, `horse.svg`, `pnpm import:equine` and `documentSha256` are unchanged (`docs/development/RUST.md` §11.14). diff --git a/CLAUDE.md b/CLAUDE.md index 4bbfb320..e6642502 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -120,6 +120,8 @@ pnpm build:preview # SHEET_BUNDLE=draft build (manual validation, preview de pnpm preview:draft # same, then serves it pnpm import:equine # regenerate legacy-import/ and draft/ (document, migration map, review drawings): the TypeScript writer of the seed until the R5 switch pnpm r5:import-equine --check | --out # the Rust pipeline (R5.2 importer, R5.3 draft, R5.4 review files): compare the 18 generated files with the committed folders (bytes; app drawings under the declared R3 rule), or write /{legacy-import,draft} into an empty folder; never deletes (scripts/import-equine.sh, first run compiles) +pnpm assets:upload [--check] # R5.5: uploads the sheet drawings of a generated seed folder to the LOCAL stack's private bucket (Rust `upload-sheet-assets`, feature `upload`; SUPABASE_URL local with a port, SUPABASE_SERVICE_ROLE_KEY typed, never printed; no overwrite, confirm after verify; run by hand, scripts/upload-sheet-assets.sh); the TypeScript one is `assets:upload:legacy` until R5.7 +pnpm e2e:sheet-assets # scripts/sheet-assets-e2e.sh: that tool against the local stack (upload, rerun, anon and wrong key, refused destinations); run by hand pnpm report:compare ... # archive check: attachment vs its v3 reading, unresolved items flagged (docs/PDF-FORMAT.md); the core's `report-compare` binary since R4.7 (scripts/report-compare.sh, first run compiles) # Rust (docs/development/RUST.md) — native toolchain (user-level rustup, channel of rust-toolchain.toml) first @@ -280,10 +282,10 @@ faunosteo/ ├─ packages/README.md ← shared packages index ├─ packages/core-wasm/ ← @faunosteo/core-wasm: CoreRuntime (lazy, poison, recovery) + CoreExportSession (R4.6) + Node source, pkg/ shipped and pkg-proofs/ (built, git-ignored, BUILD_STAMP), ts-rs types (committed), frozen N-1 reader (tests/n-1), tests ├─ crates/ ← Rust workspace (Cargo.toml, rust-toolchain.toml, deny.toml, clippy.toml, .cargo/config.toml at the root) -│ ├─ core/ ← faunosteo-core: domain/{js,sheet,report,archive,editor,session} (contracts and rules), anatomy/{svg/{prescan,tree,sanitize,refs,number,serialize},layer,compose} (R3: the one SVG reader, the layer, canvas + highlight composition), pdf/{read,json_js,import,write,compare} (R4: reader, writer and session, the archive check tool behind feature `cli`), ops.rs + facade.rs (`call`, `call_with_sheet`), json (JS number printing), curation/{text,ids,prettier_json,markdown,report_guard,legacy/*,draft/*} (R5, feature `curation`: Prettier layout, the legacy importer, the changeset applier, migration map, stats, seed SQL, CHANGELOG, REVIEW-REPORT, review drawings), bin/report_compare.rs (`pnpm report:compare`) + bin/import_equine.rs (`pnpm r5:import-equine`, the two files allowed std::fs); tests/{oracles,conformance,fixtures,pdf/LEDGER.tsv} frozen from the TypeScript (oracles/r3, oracles/r4, fixtures/svg/hostile); no I/O, no clock (lint-enforced) +│ ├─ core/ ← faunosteo-core: domain/{js,sheet,report,archive,editor,session} (contracts and rules), anatomy/{svg/{prescan,tree,sanitize,refs,number,serialize},layer,compose} (R3: the one SVG reader, the layer, canvas + highlight composition), pdf/{read,json_js,import,write,compare} (R4: reader, writer and session, the archive check tool behind feature `cli`), ops.rs + facade.rs (`call`, `call_with_sheet`), json (JS number printing), curation/{text,ids,prettier_json,markdown,report_guard,legacy/*,draft/*} (R5, feature `curation`: Prettier layout, the legacy importer, the changeset applier, migration map, stats, seed SQL, CHANGELOG, REVIEW-REPORT, review drawings), curation/upload.rs (R5.5, feature `upload`: the upload rules, pure) + bin/report_compare.rs (`pnpm report:compare`) + bin/import_equine.rs (`pnpm r5:import-equine`) + bin/upload_sheet_assets/ (`pnpm assets:upload`, ureq; the three tools allowed std::fs); tests/{oracles,conformance,fixtures,pdf/LEDGER.tsv} frozen from the TypeScript (oracles/r3, oracles/r4, fixtures/svg/hostile); no I/O, no clock (lint-enforced) │ ├─ wasm/ ← faunosteo-wasm: wasm-bindgen facade (core_version, call, SheetHandle; trap exports in the proofs build only) │ └─ worker/ ← faunosteo-worker: plain main; `decode` sandboxed child (rlimits, seccomp allowlist, image → libwebp), parent Spawner + RIFF check; R1 `run` loop: config, log (JSON lines, allowlist), keys, backend trait + http (ureq, PostgREST rpc + Storage objects), job, sweep, watchdog; Dockerfile, scripts/derive-seccomp.sh -├─ scripts/{rust-env,rust-check,wasm-build,check-all,avatar-e2e,report-compare,import-equine}.sh, wasm-stamp.mjs ← native Rust gate, wasm build (pkg + pkg-proofs + stamp), the pre-merge gate (`pnpm check:all`), the real-stack avatar e2e, the archive check tool (`pnpm report:compare`), the Rust pipeline (`pnpm r5:import-equine`) +├─ scripts/{rust-env,rust-check,wasm-build,check-all,avatar-e2e,report-compare,import-equine,upload-sheet-assets,sheet-assets-e2e,floor-clarification-check}.sh, wasm-stamp.mjs ← native Rust gate, wasm build (pkg + pkg-proofs + stamp), the pre-merge gate (`pnpm check:all`), the real-stack avatar e2e, the archive check tool (`pnpm report:compare`), the Rust pipeline (`pnpm r5:import-equine`) ├─ scripts/{dev-env,supabase,dev}.sh ← the DEV origin (LAN address) in one place; CLI wrapper; `pnpm dev` on that origin ├─ docker-compose.yml ← `app` service; `worker` service under the `worker` profile (read-only, cap_drop ALL, external Supabase network, `.env.worker`) ├─ docker/rust-builder.Dockerfile, rust-run.sh, csp-proof/ ← reproducible Rust builder (alternative) + the CSP proofs (R0 module, R2 app image: app-image.sh) diff --git a/crates/core/tests/characterisation_serde_tolerance.rs b/crates/core/tests/characterisation_serde_tolerance.rs new file mode 100644 index 00000000..5586830c --- /dev/null +++ b/crates/core/tests/characterisation_serde_tolerance.rs @@ -0,0 +1,58 @@ +//! Characterisation, not a requirement (owner 2026-10-23, R5.5 brief §7): the serde derives of +//! the R2 contracts read a struct from a positional array and a unit enum from a one-key map, +//! where the frozen Zod reader refused both. These cases pin today's Rust verdict so that any +//! change to the gated readers is visible; they are to be turned around, not extended, if the +//! owner decides a strict deserialisation (`docs/development/RUST.md` §11.16). +#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)] + +mod common; + +use faunosteo_core::domain::report::Report; +use serde::Deserialize; +use serde_json::{Value, json}; + +fn demo() -> Value { + serde_json::from_str(&common::repo_file( + "apps/app/tests/fixtures/report-v3.demo.json", + )) + .unwrap() +} + +fn read(value: &Value) -> Result { + Report::deserialize(value) +} + +#[test] +fn the_canonical_report_is_read() { + assert!(read(&demo()).is_ok()); +} + +#[test] +fn a_struct_written_as_a_positional_array_is_accepted_where_zod_refused() { + let mut sheet = demo(); + let parts = [ + sheet["sheet"]["sheetId"].clone(), + Value::Null, + sheet["sheet"]["documentSha256"].clone(), + ]; + sheet["sheet"] = json!(parts); + let mut practitioner = demo(); + let p = practitioner["practitioner"].clone(); + practitioner["practitioner"] = json!([p["name"], p["cabinet"], p["contact"], p["siret"]]); + assert!(read(&sheet).is_ok()); + assert!(read(&practitioner).is_ok()); +} + +#[test] +fn a_unit_enum_written_as_a_one_key_map_is_accepted_where_zod_refused() { + let mut report = demo(); + report["observations"][0]["severity"] = json!({ "mild": null }); + assert!(read(&report).is_ok()); +} + +#[test] +fn an_unknown_field_is_accepted_as_the_non_strict_zod_objects_did() { + let mut report = demo(); + report["extra"] = json!(1); + assert!(read(&report).is_ok()); +} diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 5dc8d5ef..dfcdbdd3 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -3266,3 +3266,86 @@ service key from the environment, bucket `anatomical-sheet-assets`, key ``). The rules are the core's +(`curation::upload`: manifest, destination, bytes through the one SVG reader, plan from the observed +state, closed error kinds); the binary owns the file system, the environment and the network, split +in `main.rs` (flow, files) and `storage.rs` (HTTP). It replaces `packages/supabase/scripts/ +upload-sheet-assets.ts`, kept as `assets:upload:legacy` until the R5.7 census. Behaviour and runbook: +SUPABASE.md §3.1. Differences from the TypeScript tool, all refusals added: no default destination +(a local address with an explicit port is required); an existing object is hashed, not trusted by its +name; a missing or foreign asset row stops the run instead of failing after the upload; the whole run +is planned before the first write; the file must pass the SVG reader with nothing removed; sizes and +entry counts are capped; nothing is overwritten, deleted or made public, ever (`x-upsert: false`, no +DELETE, no bucket call but the read). + +**Destination rule.** `http`, explicit port, no credentials, path, query or fragment; host `localhost`, +a loopback address, or the private IPv4 address equal to `FAUNOSTEO_DEV_HOST`. A name other than +`localhost` is never accepted (it cannot be proven local without resolving it). The handshake then +checks the private bucket and the key before any object is read or written. A local port held by +something else would receive the key: accepted on a single-user machine, the same stance as the +worker's `SUPABASE_URL`. + +**Tests.** `curation_upload` (7: manifest family, bytes, 15 destination refusals + DEV host rule, +plan), `upload_tool` (10, the binary against a loopback stub with a made-up key: `--check` writes +nothing, run then rerun, different bytes never overwritten, missing and foreign row stop before any +write, 401/403, resume after a failed confirmation, unreachable port, refused destinations, folder +outside the seed, usage), `curation_caps` (4). **Not run by the assistant**: the session against the +real local stack (`pnpm e2e:sheet-assets`: upload, rerun, anon key, wrong key, refused destinations, +unreachable port); the assistant's own run of the tool with the service key was refused by the +permission layer (CLAUDE.md: no transfer tool run by Claude) and was not retried by another route. +The owner runs it. + +**Read caps (owner §3).** `changes.json` and `references.json` are refused above 4 MiB, by name, in +the core (`MAX_CURATION_FILE_BYTES`, checked before the parse) and in the binary (bounded read, the +bytes read are counted as well as the size seen). Measured: 221.5 KB and 17.3 KB committed (about 19 and 240 +times under the cap); `--check` peaks at 18.5 MB resident; a synthetic changeset of 765 changes +(3.98 MB, ids renumbered) parses at +39 MB over a 7 MB baseline, about 10 times the text. The 65 times +of the R5.4 round-9 security gate (§11.15) is a worst-case shape of many tiny values; at the 4 MiB cap +that is about 260 MB where the 16 MiB read cap allowed about 1 GB. Other budgets (document depth and +values, records, drawings, the 16 MiB read cap for every other source) unchanged. 4 MiB leaves a +wide margin; a future multi-species changeset is a business-contract question and is not decided +here. + +**Owner decisions applied.** Quadratic scans (§5): kept, debt recorded in §11.15, no rewrite without +a measured problem. URLs in titles, questions and justifications (§4): kept verbatim in the data and +in the Markdown reports, no active HTML produced; an interface shows them as untrusted text. Divergent +refusals (§6): invalid field segments, ids outside the stable-id grammar, inputs over the budgets, each +a controlled `PipelineError::Source` or `Guard`, none dropping authored data; the grammar is the +sheet contract's `^[a-z0-9][a-z0-9_-]{0,63}$`, species-neutral (`curation_caps` checks ids of other +species). Markdown stays an allowlist: an unreproduced form is a loud refusal, never a different file. +FLOOR (§2): the clarification is in the R5.4 contract, with `scripts/floor-clarification-check.sh`. + +**Serde tolerance of the R2 contracts (owner §7), characterisation only; nothing changed.** +Structures: every serde-derived contract of `domain/{sheet,report,archive}`, read by the gated +readers (`report/read.rs`: `Report::deserialize` for v3, `ReportV2::deserialize` for v2; +`archive/read.rs`; the PDF import path). Two tolerances: a struct is read from a positional array, +and a unit enum variant from a one-key map. Probed on Report v3 (test +`characterisation_serde_tolerance`, which pins today's verdict): + +| Input (minimal change of `report-v3.demo.json`) | Rust now | Historical Zod contract | +|---|---|---| +| `"sheet": [sheetId, null, documentSha256]` | accepted | refused | +| `"practitioner": [name, cabinet, contact, siret]` | accepted | refused | +| `observations[0].severity` = `{"mild": null}` | accepted | refused | +| a zone written as an array | refused only when the order is wrong, accepted when it matches the field order | refused | +| `"extra": 1` (unknown field) | accepted | accepted (non-strict objects) | + +Consequence of accepting: a hand-built or foreign attachment imports; the report is re-serialised in +the canonical object form, so nothing stored is non-canonical, but a positional array read in the +wrong order assigns values to the wrong fields without an error when the types agree (an integrity +risk for crafted files, not a memory or code-execution one). Consequence of refusing: only files the +application never produced are turned away. Risk for R2, R3 and R4 archives: none either way, every +archive the product wrote is in object form (the frozen envelopes round-trip, `report_contract`, and the +N-1 suites), so a strict reading rejects no golden and no fixture. Proposal: the guard already used for +the authored files (`draft/shape.rs`: compare the input with its typed reading serialised back) at the +three reader entries, per-file go of the owner, decided before R5.6 since the membership test touches +the same readers there; until then the gated readers are unchanged. + +Open points carried: debt list of §11.12 and §11.13 stands (migration map cascade replay, `clippy.toml` +I/O list); the upload tool is not run against the real stack by the assistant.