Files
faunosteo/docs/development/RUST.md
T

24 KiB
Raw Blame History

Rust in the Faunosteo monorepo

Status: plan R0 (foundation), 2026-10-12, on feature/rust-r0-foundation. Rule and roadmap: RUST_ARCHITECTURE_REVIEW.md (rule "Rust by default", Part III boundary rules, Part VI phases R0–R5, Annex B R0 scope, Annex C owner amendments). R0 ships the toolchain, the workspace skeleton, the worker's sandboxed child, the browser module loader and seven experimental proofs. No application behaviour, interface, archive or database changes; the frontend does not load the module before R2; the worker's job loop arrives at R1.

1. Toolchain: native on the development machine, reproducible in Docker, never on the VPS

Three environments, three roles (owner decision of 2026-10-13, review Annex C addendum). The development machine is a self-hosted Ubuntu server (native, not WSL, not the VPS) with Docker rootless.

Environment Role Rust
Ubuntu development machine, native daily development (cargo check / test / build / clippy / fmt), the browser module build, every check before a merge (pnpm check:all) user-level rustup in ~/.cargo and ~/.rustup, channel pinned by rust-toolchain.toml
Docker on that machine the Supabase stack, the worker's sandbox tests and seccomp derivation, integration tests, the application and worker images, and the reproducible run of the same Rust scripts through the builder image inside images only
Production VPS runs images; never compiles none

1.1 Native toolchain (the primary path)

Everything lives in the user's home: no global package, no sudo. Checked on 2026-10-13 (Ubuntu 26.04.1, x86_64): build-essential (gcc 15, the C compiler that builds the worker's vendored libwebp), libc6-dev, pkg-config, libssl-dev and strace were already installed; no system package is missing.

# once: rustup, the user-level installer (https://rustup.rs); ~/.bashrc and ~/.profile source ~/.cargo/env
rustup show                                                   # from the repository: installs 1.99.0 + rustfmt + clippy
rustup target add wasm32-unknown-unknown                      # browser build of crates/wasm
cargo install wasm-bindgen-cli --version 0.2.129 --locked     # = the wasm-bindgen crate in Cargo.lock
cargo install cargo-deny --version 0.20.2 --locked

rustup show run inside the repository installs the channel of rust-toolchain.toml (rustup auto-install), so cargo here is always 1.99.0 whatever the default toolchain of the machine. Non-interactive shells (editor tasks, git hooks, agent shells) skip ~/.bashrc and start without ~/.cargo/bin on PATH: the scripts source scripts/rust-env.sh, which prepends it when cargo is not found.

Piece Native Builder image Pinned by
Rust 1.99.0, edition 2024, rustfmt, clippy rustup show (auto-install) FROM rust:1.99.0-bookworm + rustup component add rust-toolchain.toml
wasm32-unknown-unknown rustup target add RUN rustup target add same channel
wasm-bindgen-cli 0.2.129 cargo install --locked same wasm-bindgen = "=0.2.129" in Cargo.toml; scripts/wasm-build.sh refuses to run when Cargo.lock and the CLI on PATH differ
cargo-deny 0.20.2 cargo install --locked same deny.toml
worker builder (musl, libwebp) not native: sandbox tests run in Docker only crates/worker/Dockerfile (FROM rust:1.99.0-alpine3.24, build-base) same channel

Commands (root package.json; every Rust script is one file run natively or inside the builder image):

pnpm rust:check          # scripts/rust-check.sh, native: fmt --check, clippy -D warnings (all targets),
                         # cargo test (core, wasm, worker unit tests), ts-rs export, cargo deny licenses/bans/sources
pnpm wasm:build          # scripts/wasm-build.sh, native: crates/wasm → packages/core-wasm/pkg (git-ignored), sizes printed
pnpm check:all           # scripts/check-all.sh: the pre-merge gate (§4): rust:check, types drift, wasm:build,
                         # pnpm check, application image, worker:test, worker:image started read-only
pnpm rust:check:docker   # the same rust-check.sh inside the builder image (reproducible run)
pnpm wasm:build:docker   # the same wasm-build.sh inside the builder image
pnpm rust:builder        # builds faunosteo_rust_builder:1.99.0 once (about 6 minutes cold)
pnpm worker:test         # Docker only: crates/worker/Dockerfile stage test, 16 sandbox tests + seccomp derivation
pnpm worker:image        # Docker only: runtime image faunosteo_worker:dev (distroless/static:nonroot)
bash docker/rust-run.sh cargo <anything>   # any cargo command in the builder
cargo check | test | build | clippy | fmt  # the inner loop, from the repository root

The native target/ (git-ignored, .dockerignored) and the builder's named volumes faunosteo_cargo_registry / faunosteo_rust_target are two separate caches; neither invalidates the other.

1.2 What stays in Docker, and why

  • The worker's sandbox tests (crates/worker/tests/sandbox.rs) are gated by the test-hooks feature: a native cargo test compiles that target empty and runs the worker's unit tests only. The seccomp allowlist is derived by strace on the musl static binary that ships, inside the Alpine test stage, and glibc would need other syscalls (brk, mprotect, futex): running the sandbox natively would either kill a legitimate decode or prove nothing about the shipped binary. pnpm worker:test is the only path, and the worker keeps running in its container.
  • The Supabase stack, the application image, the worker image and the integration tests on them: pnpm db:start, docker build -t faunosteo ., pnpm worker:image.
  • The builder image stays as the reproducible alternative (same compiler, same tools, a clean environment) and as the reference a future runner uses; it is not needed for a compile or a test on the development machine.

Exposure change, stated for the record: a native build runs the build scripts and proc-macros of the dependencies under the user account (the builder container ran them with the repository mounted read-write, so the difference is the rest of the home directory). Mitigations in place: --locked everywhere, deny.toml [sources] limited to crates.io with unknown registries and git sources denied, cargo deny check licenses bans sources in every check.

2. Workspace

Cargo.toml              workspace (resolver 3), pinned [workspace.dependencies], lints, profiles
rust-toolchain.toml     channel 1.99.0, components rustfmt + clippy
rustfmt.toml            edition 2024, max_width 100
clippy.toml             disallowed-methods: std::time::*::now, std::fs::*, std::env::var* (the core)
deny.toml               licences allow-list, bans (duplicates warn, wildcards deny, path deps allowed), sources
.cargo/config.toml      wasm shadow stack 1 MiB (explicit), TS_RS_EXPORT_DIR = packages/core-wasm/types
crates/core             faunosteo-core: lib, no I/O (lint), serde + serde_json only
crates/wasm             faunosteo-wasm: cdylib facade over core (wasm-bindgen)
crates/worker           faunosteo-worker: bin + lib (image, libwebp-sys, seccompiler, rlimit)
packages/core-wasm      loader, generated types (committed), vitest proofs

Lints: unwrap_used, expect_used and disallowed_methods are errors in the libraries (tests allow them explicitly at the top of the file); unsafe_code is forbidden everywhere except the worker (libwebp FFI, each block with a SAFETY comment). Dependencies are optimised even in debug builds (profile.dev.package."*") because the sandbox tests decode 4096² images under a 10 s CPU limit.

2.1 Boundary rules in code (review III.3)

  • faunosteo_core::facade::call(op, json) -> json: one operation name, one JSON text in, the outcome envelope out ({"ok":true,"value":…} or {"ok":false,"kind":…,"detail":…}), never a panic on content. R0 ops: core_version, pretty_json, sheet_ref, report_zone.
  • faunosteo_core::json: js_number(f64) prints floats as ECMAScript Number::toString; JsPrettyFormatter is serde_json's pretty formatter with that rule; to_js_pretty and pretty_js_round_trip produce JSON.stringify(v, null, 2) byte for byte (proof §5.2).
  • faunosteo_core::domain::report: SheetRef, ReportZone, Vertebra, LateralSide, Unresolved, field order of the Zod schemas, optional fields skip_serializing_if = "Option::is_none", nullable fields plain Option, unknown keys ignored where Zod strips them. cargo test -p faunosteo-core --features ts writes the TypeScript types to packages/core-wasm/types (committed; CI fails on drift).
  • packages/core-wasm/src/index.ts: loadCore() instantiates a fresh module each time (cache-busting import of the glue), callCore() parses the envelope, isCoreBuilt() lets the proofs skip when pkg/ is absent.

3. The worker (R0 skeleton, R1 job loop)

faunosteo-worker is one static musl binary with subcommands dispatched by a plain main (no async runtime, no thread), so decode installs its sandbox in a single-threaded process:

  • decode: refuses a non-empty environment; checks Threads: 1 in /proc/self/status; disables core dumps (PR_SET_DUMPABLE 0, RLIMIT_CORE 0: a dump would hold the uploaded photo); RLIMIT_AS 512 MiB, RLIMIT_CPU 10 s soft / 12 s hard (soft = hard made the kernel SIGKILL first, finding §5.4); seccomp allowlist installed with TSYNC, KillProcess on anything else, with argument filters: tkill/tgkill may only send SIGABRT to the child's own pid (never a signal to the keyed parent or a sibling), mmap may not map PROT_EXEC; then reads ≤ 5 MiB on stdin, sniffs the format (JPEG, PNG, WebP), decodes with image::Limits (sides ≤ 4096, alloc 256 MiB), applies the EXIF orientation, resize_to_fill 512 × 512, encodes lossy WebP (q 82) through libwebp-sys (C, this process only), writes stdout. Exit codes in exit_codes.rs (10–19, never 1, 101 or a signal number); every decode-time error maps to a content code (unsupported_type, decode_failed, dimensions, resource_limit), so no upload can produce a broken-environment code.
  • the parent side (spawn::Spawner, used by the tests now and by R1): env_clear, stdin/stdout/stderr piped, output read up to 512 KiB (the child is killed past it), 20 s deadline, classification of every exit (Produced, Rejected{code} for the content codes (exit_codes::is_content_rejection, an explicit list), Broken{code} when the child could not run properly (usage, non-empty environment, sandbox failure, encode or output failure, internal: R1 alerts instead of blaming the upload; no content can produce it), Signalled{signal}, OutputOverflow, Timeout, Failed). The parent never decodes: webp_check::inspect parses the RIFF container only (chunk allowlist VP8X, VP8 , VP8L, ALPH; the encoder's structure enforced: optional VP8X first, optional ALPH only in an extended file, exactly one frame; canvas size from the headers, the frame size must agree with the canvas; any EXIF, XMP , ICCP or unknown chunk refuses the output; owner amendment of D-R4). The child's stderr (≤ 4 KiB) is child-controlled text: R1 never logs it raw (LRN-044).
  • run --check: prints the resolved configuration without secrets (which variables are set, the linked libwebp version, refused below 1.6.0). A plain run does nothing before R1 and says so.
  • version: binary, workspace and libwebp versions.
  • test hooks (--features test-hooks, never in a release build): fixtures generated in memory (no binary in git), --no-seccomp (rlimits only, prints the SECCOMP-MARKER line for the strace derivation), --violate (getcwd after the filter), --flood, --alloc-gib n, --spin, print-allowlist.

Image: crates/worker/Dockerfile, stages build (release, static), test (sandbox tests and scripts/derive-seccomp.sh on the release profile with the test-hooks feature, so the binary under test differs from the shipped one by the feature-gated hooks only), runtime (gcr.io/distroless/static-debian12:nonroot, USER nonroot, ENTRYPOINT /faunosteo-worker, CMD run --check). Measured 2026-10-12: image 8.7 MB (one static binary), run --check exits 0 with --read-only --cap-drop ALL --security-opt no-new-privileges:true.

4. Checks before a merge, and the workflow files (prepared, no CI active)

No CI/CD runs on the forge today (2026-10-13): no runner, no secret, no registry. The YAML files under .gitea/workflows/ are kept as prepared material for that separate decision; a green workflow is never claimed until one has actually run on the forge. Until then the gate is local:

pnpm check:all   # scripts/check-all.sh, stops at the first failure

Stages, in order: scripts/rust-check.sh (native), the ts-rs types drift (git status --porcelain -- packages/core-wasm/types must be empty: a modified or a new type file both fail), cargo deny check advisories (warn-only: out of the blocking path by design, review VIII.3, but printed here because no nightly job runs yet; needs the network), scripts/wasm-build.sh (native), pnpm check (typecheck, tests including the core-wasm proofs on the module just built and the pgTAP suite on PGlite, lint, build), the application image, pnpm worker:test (sandbox tests and seccomp derivation on the musl binary), pnpm worker:image started once with --read-only --cap-drop ALL. The local Supabase stack is not required; pnpm db:test is the real-stack complement after a change under supabase/.

Workflow files, each mapping onto those stages so a future runner calls the same scripts:

  • ci.yml (push on main/develop, pull requests): job rust (builder image, scripts/rust-check.sh, types drift, scripts/wasm-build.sh), job check (module built the same way, then the Node pipeline), job docker (application image, worker test stage, worker runtime started read-only), job supabase (real stack, pgTAP, generated types diff).
  • rust-advisories.yml: nightly and manual cargo deny check advisories, out of the blocking path on purpose.
  • release-images.yml: manual only; the dispatch input is a typed choice re-validated through an environment variable; the RustSec and npm (pnpm audit --prod) advisories gates run first in a job without secrets; a named ephemeral buildx builder (--builder, no silent fallback) with its own cache builds the images; the registry token reaches the login step only, into a per-job DOCKER_CONFIG under RUNNER_TEMP created before any step and removed by a literal path; digests read from the build metadata; actions/checkout pinned to a commit. The if: github.ref line is not a boundary (push access and the token's scope are; a dedicated release runner label is the Step 27 operation). It would push faunosteo_app:<sha> and faunosteo_worker:<sha> and print their digests. Deploys nothing (DEPLOY.md "Image deploy by digest").

Target of the future CI/CD step (planned, not yet built; recorded so the scripts stay compatible): tests on every push and pull request, Rust and TypeScript checks, PostgreSQL/RLS tests, Docker builds, security checks, branch verification before a merge, image publication only in an explicitly authorised workflow.

Not measured: the forge runner's cache behaviour, cold build times and the workflow syntax (job-level env, GITHUB_ENV path) have never run. The builder image build took about 6 minutes cold on the development host; both image builds reuse BuildKit cache mounts (faunosteo_cargo_registry_musl, faunosteo_target_musl).

5. R0 proofs (2026-10-12, development host, Docker 29.8 rootless, kernel 7.0)

Each proof is a permanent test; the numbers are what the run produced.

5.1 Panic strategy on wasm32 (packages/core-wasm/tests/panic.test.ts)

  • A Rust panic reaches JavaScript as WebAssembly.RuntimeError (unreachable): the module is panic = abort, catch_unwind cannot be used there.
  • Exhausting the 1 MiB shadow stack (2 000 000 frames of 128 bytes) traps (RuntimeError) instead of corrupting memory.
  • After a stack-exhaustion trap, the next call on the same instance fails in the allocator (memory access out of bounds, deterministic). After a plain panic the same instance still answers, with a stack pointer that was never unwound: the dangerous case. Rule for R2: no instance is reused after any trap; a fresh instance is a new module record (the wasm-bindgen glue caches its instance in a module variable, and the ESM cache returns the same module for the same URL: loadCore() imports with a cache-busting query; in the browser the glue must be a static asset imported the same way, or the core runs in a Worker recreated on trap).

5.2 Serialisation contract (tests/serialisation.test.ts, crates/core/tests)

  • Byte equality with JSON.stringify(JSON.parse(text), null, 2) on report-v3.demo.json, report-v2.full.json, the 296 KB equine document.json and the 84 KB report-migration-map.json, plus a number corpus (1523, 1715.4285714285713, 0.85, 1e+21, 1e-7, 0.000001, -0 → 0, 123456789012345680000, 5e-324, 1.7976931348623157e+308).
  • Finding: serde_json's default float parser is a documented fast path that can be one ULP off; the first run printed 1715.428571428571 and 123456789012345670000. The float_roundtrip feature (enabled in Cargo.toml) fixes it; the module grew from 187 KB to 225 KB raw.
  • Optional fields absent, nullable fields null, unknown keys ignored, unresolved: false refused; the current ReportZoneSchema and SheetRefSchema (Zod 3, @faunosteo/domain) parse what the core writes.

5.3 musl static binary with libwebp (crates/worker/Dockerfile)

Builds on rust:1.99.0-alpine3.24 with build-base; libwebp-sys 0.14.4 compiles its vendored libwebp with cc; WebPGetEncoderVersion() reports 1.6.0 (logged by run --check, test the_linked_libwebp_is_recent_enough). The runtime image carries the one static binary.

5.4 seccomp allowlist, derived and proven (crates/worker/tests/sandbox.rs)

  • Derivation (scripts/derive-seccomp.sh, strace over png, jpeg, webp, a 4096 × 4096 JPEG and a 3000 × 3000 PNG, after the marker): exactly read write mmap munmap mremap exit_group getrandom. musl's mallocng used neither brk nor mprotect; no futex in a single-threaded decode. The allowlist is that set plus the abort path (rt_sigaction, rt_sigprocmask, getpid, gettid, tkill, tgkill, exit).
  • Proven on the release-profile musl binary with the test hooks, inside Docker's default seccomp profile: 16 tests green (png_jpeg_and_webp_decode_through_the_full_sandbox, inputs_at_the_caps_decode_within_the_limits (4096²), dimensions 32 px and 20 000² header, GIF and SVG refused, truncated PNG, non-empty environment refused, --violate → SIGSYS 31, --flood → output cut, --alloc-gib 2 → SIGABRT 6 under RLIMIT_AS, --spin → parent deadline, --spin → SIGXCPU 24 at the CPU soft limit, libwebp ≥ 1.6.0; negative proofs of the argument filters: --signal-parent (tkill to the parent) and --mmap-exec → SIGSYS; an iCCP bomb of 300 MB inflated → a content rejection, never a broken-environment outcome).
  • Findings: with RLIMIT_CPU soft = hard the kernel sends SIGKILL (hard limit checked first), indistinguishable from the parent's deadline kill; the soft limit is now two seconds below the hard one. The 4096² decode exceeded 10 s of CPU in an unoptimised debug build: dependencies are optimised in every profile.

5.5 CSP with 'wasm-unsafe-eval' (docker/csp-proof/run.sh)

Served packages/core-wasm/pkg through nginx:1.31-alpine with docker/nginx.conf plus the exact Content-Security-Policy of DEPLOY.md, loaded by headless Chrome for Testing 156:

Header Result
production (script-src 'self') FAIL WebAssembly.instantiateStreaming(): Compiling or instantiating WebAssembly module violates the following Content Security policy directive because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self'"
script-src 'self' 'wasm-unsafe-eval' OK 0.1.0 (module instantiated, core_version() answered)

The keyword allows WebAssembly compilation only: an inline <script> variant stayed blocked under both headers. nginx serves .wasm as application/wasm out of the box.

5.6 Served size and instantiate time (tests/size.test.ts)

faunosteo_wasm_bg.wasm: 224 713 bytes raw, 85 396 bytes gzip-9 (with float_roundtrip; 186 805 / 72 917 without), instantiate 2.7 ms in Node 24. The D-R13 budget (600 KB compressed) is asserted by the test from R0 on.

5.7 Runner caches and build times

Not measurable from this host (no forge runner here). Recorded instead: builder image cold build about 6 minutes; worker test stage cold about 2 minutes, warm under 30 s with the BuildKit cache mounts; the wasm build under 10 s warm. The first run on the forge records its own numbers here.

5.8 Native toolchain run (2026-10-13, Ubuntu 26.04.1, rustup 1.29.1, 16 cores)

Same proofs, native toolchain (rustc 1.99.0 (b940084d7 2026-09-28), cargo 1.99.0, clippy 0.1.99, rustfmt 1.10.0-stable, wasm-bindgen 0.2.129, cargo-deny 0.20.2), compared with the builder image runs above:

Measure Native Builder image (R0)
cold cargo build --workspace --all-targets (libwebp compiled by gcc 15) 31.5 s not timed at R0
cold wasm release build 4.3 s under 10 s warm
scripts/rust-check.sh warm (fmt, clippy, tests, ts export, deny) 32 s 2.2 s warm volumes (same script, same results)
cargo tests core 10 + 2, worker unit 9, core with ts 14 + 2, all green identical counts
committed ts-rs types unchanged after the export unchanged
faunosteo_wasm_bg.wasm raw 224 777 B 224 713 B
gzip -9 84 924 B (gzip), 85 390 B (the vitest measure) 84 927 B / 85 396 B
sha256 2d559a6b…bb3a 451d49c1…a752
core-wasm proofs 14 passed, instantiate 3.9 ms 14 passed, 2.7 ms

The 64-byte difference and the distinct hashes come from the source paths embedded in the panic locations (/home/<user>/Documents/faunosteo/… natively, /w/… in the builder): the module is not bit-reproducible across build roots without --remap-path-prefix, which R0 does not set. The builder image is the reference when two builds must match byte for byte. cargo deny prints the same warnings in both environments (license-not-encountered for six allowed licences, duplicate miniz_oxide and syn): policy warn, not failures.

6. Reproduction

pnpm rust:check                                            # §2, §5.2 (cargo side), native
pnpm wasm:build && pnpm --filter @faunosteo/core-wasm test:run   # §5.1, §5.2, §5.6, native module
pnpm rust:builder && pnpm rust:check:docker && pnpm wasm:build:docker   # the same, reproducible in the builder
pnpm worker:test                                           # §5.3, §5.4 (tests + derivation in the build log)
pnpm worker:image && docker run --rm --read-only --cap-drop ALL faunosteo_worker:dev   # §3
CHROME=<chromium binary> bash docker/csp-proof/run.sh      # §5.5 (playwright-core on NODE_PATH)
pnpm check:all                                             # everything above but the CSP proof, in one command

7. What stays hypothetical after R0

  • Follow-ups from the R0 security audit, owned by R1 unless stated: base images pinned by digest in the three Dockerfiles (today by tag, as the existing Dockerfile); a verification decode of the child's output in a second sandboxed child (the RIFF check validates the container, not the bitstream); the shared BuildKit cache mounts of the CI builds (the release workflow already uses its own builder); actions/*@v4 tags in ci.yml (pre-existing practice) to pin by commit at Step 27.
  • The browser-side fresh-instance mechanism (static glue URL with a cache-busting query under Vite, or a Worker per instance) is designed, not built: R2 builds and tests it.
  • HTTPS from the worker (CA bundle of distroless, webpki-roots): no HTTP client in the R0 binary; R1 adds reqwest and the CI HTTPS call.
  • The forge runner's cache and timings (§5.7), and the workflow files themselves: no runner exists, none of the three workflows has ever run (§4).
  • The allowlist is x86_64; an aarch64 host would need its own derivation (target_arch selects the seccomp architecture, the syscall numbers come from libc).