Files
faunosteo/docs/development/RUST.md
T

298 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Rust in the Faunosteo monorepo
Status: plan R0 (foundation), 2026-10-12, on `feature/rust-r0-foundation`. Rule
and roadmap: `RUST_ARCHITECTURE_REVIEW.md` (rule "Rust by default", Part III
boundary rules, Part VI phases R0–R5, Annex B R0 scope, Annex C owner
amendments). R0 ships the toolchain, the workspace skeleton, the worker's
sandboxed child, the browser module loader and seven experimental proofs. No
application behaviour, interface, archive or database changes; the frontend
does not load the module before R2; the worker's job loop arrives at R1.
## 1. Toolchain: reproducible in Docker, never on the VPS
Every cargo command runs inside the reference builder image, so CI and a
developer machine use the same compiler and the production VPS never needs
Rust (images are built in CI and pulled by digest, DEPLOY.md "Image deploy by
digest").
| Piece | Where | Pinned by |
|---|---|---|
| Rust 1.99.0, edition 2024, rustfmt, clippy, `wasm32-unknown-unknown` | `docker/rust-builder.Dockerfile` (`FROM rust:1.99.0-bookworm`) | `rust-toolchain.toml` (same channel; a difference would make rustup download inside the container) |
| `wasm-bindgen-cli` 0.2.129 | same image | `wasm-bindgen = "=0.2.129"` in `Cargo.toml`; `docker/wasm-build.sh` refuses to run when `Cargo.lock` and the CLI differ |
| `cargo-deny` 0.20.2 | same image | `deny.toml` |
| worker builder | `crates/worker/Dockerfile` (`FROM rust:1.99.0-alpine3.24`, `build-base` for libwebp) | same channel |
Commands (root `package.json`):
```bash
pnpm rust:builder # builds faunosteo_rust_builder:1.99.0 once (about 6 minutes cold)
pnpm rust:check # docker/rust-check.sh: fmt --check, clippy -D warnings (all targets),
# cargo test (core, wasm), ts-rs export, cargo deny licenses/bans/sources
pnpm wasm:build # crates/wasm → packages/core-wasm/pkg (git-ignored), prints raw and gzip sizes
pnpm worker:test # crates/worker/Dockerfile stage test: 13 sandbox tests + seccomp derivation
pnpm worker:image # runtime image faunosteo_worker:dev (distroless/static:nonroot)
bash docker/rust-run.sh cargo <anything> # any cargo command in the builder
```
`docker/rust-run.sh` mounts the repository at `/w` and keeps Cargo's registry
and the target directory in the named volumes `faunosteo_cargo_registry` and
`faunosteo_rust_target` (incremental builds take seconds). A local `rustup`
is optional for the inner loop (same `rust-toolchain.toml`); the container is
the arbiter.
## 2. Workspace
```text
Cargo.toml workspace (resolver 3), pinned [workspace.dependencies], lints, profiles
rust-toolchain.toml channel 1.99.0, components rustfmt + clippy
rustfmt.toml edition 2024, max_width 100
clippy.toml disallowed-methods: std::time::*::now, std::fs::*, std::env::var* (the core)
deny.toml licences allow-list, bans (duplicates warn, wildcards deny, path deps allowed), sources
.cargo/config.toml wasm shadow stack 1 MiB (explicit), TS_RS_EXPORT_DIR = packages/core-wasm/types
crates/core faunosteo-core: lib, no I/O (lint), serde + serde_json only
crates/wasm faunosteo-wasm: cdylib facade over core (wasm-bindgen)
crates/worker faunosteo-worker: bin + lib (image, libwebp-sys, seccompiler, rlimit)
packages/core-wasm loader, generated types (committed), vitest proofs
```
Lints: `unwrap_used`, `expect_used` and `disallowed_methods` are errors in the
libraries (tests allow them explicitly at the top of the file); `unsafe_code`
is forbidden everywhere except the worker (libwebp FFI, each block with a
`SAFETY` comment). Dependencies are optimised even in debug builds
(`profile.dev.package."*"`) because the sandbox tests decode 4096² images
under a 10 s CPU limit.
### 2.1 Boundary rules in code (review III.3)
- `faunosteo_core::facade::call(op, json) -> json`: one operation name, one
JSON text in, the outcome envelope out (`{"ok":true,"value":…}` or
`{"ok":false,"kind":…,"detail":…}`), never a panic on content. R0 ops:
`core_version`, `pretty_json`, `sheet_ref`, `report_zone`.
- `faunosteo_core::json`: `js_number(f64)` prints floats as ECMAScript
`Number::toString`; `JsPrettyFormatter` is `serde_json`'s pretty formatter
with that rule; `to_js_pretty` and `pretty_js_round_trip` produce
`JSON.stringify(v, null, 2)` byte for byte (proof §5.2).
- `faunosteo_core::domain::report`: `SheetRef`, `ReportZone`, `Vertebra`,
`LateralSide`, `Unresolved`, field order of the Zod schemas, optional
fields `skip_serializing_if = "Option::is_none"`, nullable fields plain
`Option`, unknown keys ignored where Zod strips them. `cargo test -p
faunosteo-core --features ts` writes the TypeScript types to
`packages/core-wasm/types` (committed; CI fails on drift).
- `packages/core-wasm/src/index.ts`: `loadCore()` instantiates a **fresh**
module each time (cache-busting import of the glue), `callCore()` parses
the envelope, `isCoreBuilt()` lets the proofs skip when `pkg/` is absent.
## 3. The worker (R0 skeleton, R1 job loop)
`faunosteo-worker` is one static musl binary with subcommands dispatched by
a plain `main` (no async runtime, no thread), so `decode` installs its
sandbox in a single-threaded process:
- `decode`: refuses a non-empty environment; checks `Threads: 1` in
`/proc/self/status`; disables core dumps (`PR_SET_DUMPABLE 0`,
`RLIMIT_CORE 0`: a dump would hold the uploaded photo); `RLIMIT_AS`
512 MiB, `RLIMIT_CPU` 10 s soft / 12 s hard (soft = hard made the kernel
SIGKILL first, finding §5.4); seccomp allowlist installed with TSYNC,
`KillProcess` on anything else, with argument filters: `tkill`/`tgkill`
may only send SIGABRT to the child's own pid (never a signal to the keyed
parent or a sibling), `mmap` may not map `PROT_EXEC`; then reads
≤ 5 MiB on stdin, sniffs the format (JPEG, PNG, WebP), decodes with
`image::Limits` (sides ≤ 4096, alloc 256 MiB), applies the EXIF
orientation, `resize_to_fill` 512 × 512, encodes lossy WebP (q 82) through
`libwebp-sys` (C, this process only), writes stdout. Exit codes in
`exit_codes.rs` (10–19, never 1, 101 or a signal number); every decode-time
error maps to a content code (`unsupported_type`, `decode_failed`,
`dimensions`, `resource_limit`), so no upload can produce a broken-environment
code.
- the parent side (`spawn::Spawner`, used by the tests now and by R1):
`env_clear`, stdin/stdout/stderr piped, output read up to 512 KiB (the
child is killed past it), 20 s deadline, classification of every exit
(`Produced`, `Rejected{code}` for the content codes
(`exit_codes::is_content_rejection`, an explicit list), `Broken{code}` when
the child could not run properly (usage, non-empty environment, sandbox
failure, encode or output failure, internal: R1 alerts instead of blaming
the upload; no content can produce it), `Signalled{signal}`,
`OutputOverflow`, `Timeout`, `Failed`). The
parent **never decodes**: `webp_check::inspect` parses the RIFF container
only (chunk allowlist `VP8X`, `VP8 `, `VP8L`, `ALPH`; the encoder's structure
enforced: optional `VP8X` first, optional `ALPH` only in an extended file,
exactly one frame; canvas size from the headers, the frame size must agree
with the canvas; any `EXIF`, `XMP `, `ICCP` or unknown chunk refuses the
output; owner amendment of D-R4). The
child's stderr (≤ 4 KiB) is child-controlled text: R1 never logs it raw
(LRN-044).
- `run --check`: prints the resolved configuration without secrets (which
variables are set, the linked libwebp version, refused below 1.6.0). A
plain `run` does nothing before R1 and says so.
- `version`: binary, workspace and libwebp versions.
- test hooks (`--features test-hooks`, never in a release build): fixtures
generated in memory (no binary in git), `--no-seccomp` (rlimits only,
prints the `SECCOMP-MARKER` line for the strace derivation), `--violate`
(`getcwd` after the filter), `--flood`, `--alloc-gib n`, `--spin`,
`print-allowlist`.
Image: `crates/worker/Dockerfile`, stages `build` (release, static),
`test` (sandbox tests and `scripts/derive-seccomp.sh` on the **release
profile** with the `test-hooks` feature, so the binary under test differs
from the shipped one by the feature-gated hooks only), `runtime`
(`gcr.io/distroless/static-debian12:nonroot`, `USER nonroot`, `ENTRYPOINT
/faunosteo-worker`, `CMD run --check`). Measured 2026-10-12: image 8.7 MB (one static
binary), `run --check` exits 0 with `--read-only --cap-drop ALL
--security-opt no-new-privileges:true`.
## 4. CI
- Job `rust` (before `check`): builds the builder image, runs
`docker/rust-check.sh`, checks the committed ts-rs types for drift, builds
the browser module (size printed).
- Job `check`: builds the module the same way, then the existing Node
pipeline; the `core-wasm` proofs run because `pkg/` exists.
- Job `docker`: application image, worker `test` stage (sandbox tests and
seccomp derivation on the musl binary that ships), worker runtime image
started once with `--read-only --cap-drop ALL`.
- `rust-advisories.yml`: nightly and manual `cargo deny check advisories`,
out of the blocking path on purpose.
- `release-images.yml`: manual only; the dispatch input is a typed choice
re-validated through an environment variable; the RustSec and npm
(`pnpm audit --prod`) advisories gates run first; a named ephemeral
`buildx` builder (`--builder`, no silent fallback) with its own cache
builds the images; the registry token reaches the login step only, into
a per-job `DOCKER_CONFIG` under `RUNNER_TEMP` created before any step and
removed by a literal path; digests read from the build metadata;
`actions/checkout` pinned to a commit. The `if: github.ref` line is not a
boundary (push access and the token's scope are; a dedicated release
runner label is the Step 27 operation). It pushes `faunosteo_app:<sha>`
and `faunosteo_worker:<sha>` and prints their digests. Deploys nothing
(DEPLOY.md "Image deploy by digest").
Not measured here: the forge runner's cache behaviour and cold build times
(the runner was not available from this host). The builder image build took
about 6 minutes cold on the development host; both image builds reuse
BuildKit cache mounts (`faunosteo_cargo_registry_musl`,
`faunosteo_target_musl`).
## 5. R0 proofs (2026-10-12, development host, Docker 29.8 rootless, kernel 7.0)
Each proof is a permanent test; the numbers are what the run produced.
### 5.1 Panic strategy on wasm32 (`packages/core-wasm/tests/panic.test.ts`)
- A Rust panic reaches JavaScript as `WebAssembly.RuntimeError` (`unreachable`):
the module is panic = abort, `catch_unwind` cannot be used there.
- Exhausting the 1 MiB shadow stack (2 000 000 frames of 128 bytes) traps
(`RuntimeError`) instead of corrupting memory.
- After a stack-exhaustion trap, the next call on the same instance fails in
the allocator (`memory access out of bounds`, deterministic). After a
plain panic the same instance *still answers*, with a stack pointer that
was never unwound: the dangerous case. Rule for R2: no instance is reused
after any trap; a fresh instance is a new module record (the wasm-bindgen
glue caches its instance in a module variable, and the ESM cache returns
the same module for the same URL: `loadCore()` imports with a cache-busting
query; in the browser the glue must be a static asset imported the same
way, or the core runs in a Worker recreated on trap).
### 5.2 Serialisation contract (`tests/serialisation.test.ts`, `crates/core/tests`)
- Byte equality with `JSON.stringify(JSON.parse(text), null, 2)` on
`report-v3.demo.json`, `report-v2.full.json`, the 296 KB equine
`document.json` and the 84 KB `report-migration-map.json`, plus a number
corpus (`1523`, `1715.4285714285713`, `0.85`, `1e+21`, `1e-7`,
`0.000001`, `-0` → `0`, `123456789012345680000`, `5e-324`,
`1.7976931348623157e+308`).
- Finding: `serde_json`'s default float parser is a documented fast path
that can be one ULP off; the first run printed `1715.428571428571` and
`123456789012345670000`. The `float_roundtrip` feature (enabled in
`Cargo.toml`) fixes it; the module grew from 187 KB to 225 KB raw.
- Optional fields absent, nullable fields `null`, unknown keys ignored,
`unresolved: false` refused; the current `ReportZoneSchema` and
`SheetRefSchema` (Zod 3, `@faunosteo/domain`) parse what the core writes.
### 5.3 musl static binary with libwebp (`crates/worker/Dockerfile`)
Builds on `rust:1.99.0-alpine3.24` with `build-base`; `libwebp-sys` 0.14.4
compiles its vendored libwebp with `cc`; `WebPGetEncoderVersion()` reports
**1.6.0** (logged by `run --check`, test
`the_linked_libwebp_is_recent_enough`). The runtime image carries the one
static binary.
### 5.4 seccomp allowlist, derived and proven (`crates/worker/tests/sandbox.rs`)
- Derivation (`scripts/derive-seccomp.sh`, strace over png, jpeg, webp, a
4096 × 4096 JPEG and a 3000 × 3000 PNG, after the marker): exactly
`read write mmap munmap mremap exit_group getrandom`. musl's mallocng used
neither `brk` nor `mprotect`; no `futex` in a single-threaded decode. The
allowlist is that set plus the abort path (`rt_sigaction`,
`rt_sigprocmask`, `getpid`, `gettid`, `tkill`, `tgkill`, `exit`).
- Proven on the release-profile musl binary with the test hooks, inside
Docker's default seccomp profile: 16 tests green (`png_jpeg_and_webp_decode_through_the_full_sandbox`,
`inputs_at_the_caps_decode_within_the_limits` (4096²), dimensions 32 px
and 20 000² header, GIF and SVG refused, truncated PNG, non-empty
environment refused, `--violate` → SIGSYS 31, `--flood` → output cut,
`--alloc-gib 2` → SIGABRT 6 under `RLIMIT_AS`, `--spin` → parent deadline,
`--spin` → SIGXCPU 24 at the CPU soft limit, libwebp ≥ 1.6.0; negative
proofs of the argument filters: `--signal-parent` (`tkill` to the parent)
and `--mmap-exec` → SIGSYS; an `iCCP` bomb of 300 MB inflated → a content
rejection, never a broken-environment outcome).
- Findings: with `RLIMIT_CPU` soft = hard the kernel sends SIGKILL (hard
limit checked first), indistinguishable from the parent's deadline kill;
the soft limit is now two seconds below the hard one. The 4096² decode
exceeded 10 s of CPU in an unoptimised debug build: dependencies are
optimised in every profile.
### 5.5 CSP with `'wasm-unsafe-eval'` (`docker/csp-proof/run.sh`)
Served `packages/core-wasm/pkg` through `nginx:1.31-alpine` with
`docker/nginx.conf` plus the exact `Content-Security-Policy` of
`DEPLOY.md`, loaded by headless Chrome for Testing 156:
| Header | Result |
|---|---|
| production (`script-src 'self'`) | `FAIL WebAssembly.instantiateStreaming(): Compiling or instantiating WebAssembly module violates the following Content Security policy directive because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self'"` |
| `script-src 'self' 'wasm-unsafe-eval'` | `OK 0.1.0` (module instantiated, `core_version()` answered) |
The keyword allows WebAssembly compilation only: an inline `<script>`
variant stayed blocked under both headers. nginx serves `.wasm` as
`application/wasm` out of the box.
### 5.6 Served size and instantiate time (`tests/size.test.ts`)
`faunosteo_wasm_bg.wasm`: 224 713 bytes raw, 85 396 bytes gzip-9 (with
`float_roundtrip`; 186 805 / 72 917 without), instantiate 2.7 ms in Node 24.
The D-R13 budget (600 KB compressed) is asserted by the test from R0 on.
### 5.7 Runner caches and build times
Not measurable from this host (no forge runner here). Recorded instead:
builder image cold build about 6 minutes; worker test stage cold about 2
minutes, warm under 30 s with the BuildKit cache mounts; the wasm build
under 10 s warm. The first run on the forge records its own numbers here.
## 6. Reproduction
```bash
pnpm rust:builder && pnpm rust:check # §2, §5.2 (cargo side)
pnpm wasm:build && pnpm --filter @faunosteo/core-wasm test:run # §5.1, §5.2, §5.6
pnpm worker:test # §5.3, §5.4 (tests + derivation in the build log)
pnpm worker:image && docker run --rm --read-only --cap-drop ALL faunosteo_worker:dev # §3
CHROME=<chromium binary> bash docker/csp-proof/run.sh # §5.5 (playwright-core on NODE_PATH)
```
## 7. What stays hypothetical after R0
- Follow-ups from the R0 security audit, owned by R1 unless stated: base
images pinned by digest in the three Dockerfiles (today by tag, as the
existing `Dockerfile`); a verification decode of the child's output in a
second sandboxed child (the RIFF check validates the container, not the
bitstream); the shared BuildKit cache mounts of the CI builds (the
release workflow already uses its own builder); `actions/*@v4` tags in
`ci.yml` (pre-existing practice) to pin by commit at Step 27.
- The browser-side fresh-instance mechanism (static glue URL with a
cache-busting query under Vite, or a Worker per instance) is designed, not
built: R2 builds and tests it.
- HTTPS from the worker (CA bundle of distroless, `webpki-roots`): no HTTP
client in the R0 binary; R1 adds `reqwest` and the CI HTTPS call.
- The forge runner's cache and timings (§5.7).
- The allowlist is x86_64; an aarch64 host would need its own derivation
(`target_arch` selects the seccomp architecture, the syscall numbers come
from `libc`).