From 6e2bbec4b088a14a2afc4ee449d48b4c4d6ce71c Mon Sep 17 00:00:00 2001 From: bchanot Date: Tue, 13 Oct 2026 01:19:15 +0200 Subject: [PATCH] docs(docs): Ubuntu native development, Docker for services and isolation, separate VPS; pnpm check:all; CI status MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RUST.md §1 (three environments, native install recipe, what stays in Docker and why), §4 (pre-merge gate, workflow files prepared and never run), §5.8 (native run numbers against the builder image), §6; CLAUDE.md stack row, commands, map, workflow rule; README quick start and verification; DEPLOY.md CI status; WORKSPACE.md; TEST_STRATEGY.md; CHANGELOG; review Annex C.1 addendum of 2026-10-13. --- CHANGELOG.md | 4 + CLAUDE.md | 26 ++- DEPLOY.md | 11 +- README.md | 11 + docs/development/RUST.md | 207 ++++++++++++++----- docs/development/RUST_ARCHITECTURE_REVIEW.md | 32 +++ docs/development/TEST_STRATEGY.md | 6 +- docs/development/WORKSPACE.md | 4 +- 8 files changed, 233 insertions(+), 68 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ef5d35e8..55bb8921 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,10 @@ Versions follow [Semantic Versioning](https://semver.org/). - **Rust by default** (rule adopted 2026-10-12, `docs/development/RUST_ARCHITECTURE_REVIEW.md`): Cargo workspace with three crates, `crates/core` (contracts and the JavaScript-compatible JSON formatter, no I/O by lint), `crates/wasm` (browser facade built into `packages/core-wasm/pkg`) and `crates/worker` (the future file-processing worker: sandboxed `decode` child with rlimits and a seccomp allowlist derived by `strace`, image decoding in pure Rust, lossy WebP through libwebp confined to the child, parent-side container check; `run --check` only, no job loop before R1). Reference toolchain in Docker (`docker/rust-builder.Dockerfile`, `pnpm rust:check`, `pnpm wasm:build`), worker image on `distroless/static`, CI job `rust`, nightly advisories, manual digest-tagged image workflow (no deploy). R0 proofs recorded in `docs/development/RUST.md`: `float_roundtrip` required for JSON parity, traps leave a wasm instance unusable (fresh instance per recovery), CSP needs `'wasm-unsafe-eval'`. No application behaviour, interface, archive or database change; the frontend does not load the module. +### Changed (Rust foundation, R0 correction 2026-10-13) + +- **Native Rust toolchain as the primary path** on the Ubuntu development machine (user-level rustup, channel of `rust-toolchain.toml`, `wasm32-unknown-unknown`, `wasm-bindgen-cli` 0.2.129, `cargo-deny` 0.20.2): `pnpm rust:check` and `pnpm wasm:build` run `scripts/rust-check.sh` and `scripts/wasm-build.sh` natively (moved from `docker/`, PATH guard `scripts/rust-env.sh`); the builder image stays the reproducible alternative (`pnpm rust:check:docker`, `pnpm wasm:build:docker`). The native check now covers the worker's unit tests; the sandbox tests and the seccomp derivation stay Docker-only (`pnpm worker:test`). New `pnpm check:all` (`scripts/check-all.sh`): every check a branch must pass before a merge, in one command. The Gitea workflow files are kept but marked prepared and never executed (no runner, secret or registry on the forge). Worker `unsafe` blocks: SAFETY comments rewritten as verifiable invariants (no approval marker), WebP stride computed with checked arithmetic. `docs/development/RUST.md` §1 and §4 describe the three environments (Ubuntu native development, Docker for services and isolation, separate production VPS). + ### Added (SaaS migration, plan Step 7) - **Identity (Step 7)** — sign-in and sign-up by email/password (confirmation mandatory, policy 12 chars + lower/upper/digit), password recovery, OAuth buttons (Google, Apple) driven by `APP_AUTH_PROVIDERS`, landing route for confirmation / OAuth links; the guarded `/app` shell (desktop sidebar, light top bar, mobile drawer) with a dashboard (greeting, profile completeness, local tool, account) and **Mon compte** (personal, professional, preferences, security). Profile columns `practice_name`, `professional_title`, `siret` (migration `20261011100000`, pgTAP 009), language synced with `profiles.locale`, platform roles shown read-only. GoTrue TOTP enabled server-side; MFA enforcement for platform roles documented as a gap (`docs/development/IDENTITY.md`). The local tool (`/`, `/new`, `/editor`) stays unauthenticated; the Home header links to the space only when Supabase is configured. diff --git a/CLAUDE.md b/CLAUDE.md index dbb5f0b0..78cc362e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -52,8 +52,8 @@ MVP species: equine only. Open-source MIT. | DB client | @supabase/supabase-js (packages/supabase) | 2.x | | SQL tests | pgTAP under supabase/tests, run by `supabase test db` and by PGlite (Dockerless) | PGlite 0.3 | | Deploy | Docker multi-stage → nginx:alpine | — | -| Rust (core, wasm facade, worker) | Rust pinned by `rust-toolchain.toml`, edition 2024; every cargo command runs in `docker/rust-builder.Dockerfile` (`pnpm rust:check`, `pnpm wasm:build`), never on the VPS | 1.99.0 | -| WASM glue | wasm-bindgen crate = CLI version (locked in `docker/wasm-build.sh`), `ts-rs` types committed under `packages/core-wasm/types` | 0.2.129 / 12 | +| Rust (core, wasm facade, worker) | Rust pinned by `rust-toolchain.toml`, edition 2024; native user-level rustup on the Ubuntu development machine (`pnpm rust:check`, `pnpm wasm:build`, plain `cargo`), the builder image `docker/rust-builder.Dockerfile` as the reproducible alternative (`:docker` scripts), never on the VPS | 1.99.0 | +| WASM glue | wasm-bindgen crate = CLI version (locked in `scripts/wasm-build.sh`), `ts-rs` types committed under `packages/core-wasm/types` | 0.2.129 / 12 | | Worker image | `crates/worker/Dockerfile`: `rust:1.99.0-alpine3.24` (musl, libwebp via `cc`) → `distroless/static:nonroot`; sandbox tests + seccomp derivation in the `test` stage | — | --- @@ -103,12 +103,15 @@ pnpm preview:draft # same, then serves it pnpm import:equine # regenerate legacy-import/ and draft/ (document, migration map, review drawings) pnpm report:compare ... # archive check: attachment vs its v3 reading, unresolved items flagged (docs/PDF-FORMAT.md) -# Rust (docs/development/RUST.md) — all through the builder image, no toolchain on the host -pnpm rust:builder # build docker/rust-builder.Dockerfile once (faunosteo_rust_builder:1.99.0) -pnpm rust:check # fmt --check, clippy -D warnings, cargo test (core, wasm), ts-rs export, cargo deny -pnpm wasm:build # crates/wasm → packages/core-wasm/pkg (git-ignored); then the core-wasm vitest proofs run -pnpm worker:test # crates/worker/Dockerfile stage test: sandbox tests + seccomp derivation on the musl binary -pnpm worker:image # runtime image faunosteo_worker:dev (distroless/static, `run --check`) +# Rust (docs/development/RUST.md) — native toolchain (user-level rustup, channel of rust-toolchain.toml) first +pnpm rust:check # scripts/rust-check.sh: fmt --check, clippy -D warnings, cargo test (core, wasm, worker unit), ts-rs export, cargo deny +pnpm wasm:build # scripts/wasm-build.sh: crates/wasm → packages/core-wasm/pkg (git-ignored); then the core-wasm vitest proofs run +pnpm check:all # scripts/check-all.sh: the whole pre-merge gate (rust:check, types drift, wasm:build, check, app image, worker:test, worker:image) +pnpm rust:builder # build docker/rust-builder.Dockerfile once (faunosteo_rust_builder:1.99.0), the reproducible alternative +pnpm rust:check:docker # the same rust-check.sh inside the builder image +pnpm wasm:build:docker # the same wasm-build.sh inside the builder image +pnpm worker:test # Docker only: crates/worker/Dockerfile stage test, sandbox tests + seccomp derivation on the musl binary +pnpm worker:image # Docker only: runtime image faunosteo_worker:dev (distroless/static, `run --check`) ``` ## Docker commands @@ -259,8 +262,9 @@ faunosteo/ │ ├─ core/ ← faunosteo-core: contracts (report types), js_number + JS pretty formatter, facade `call(op, json)`; no I/O, no clock (lint-enforced) │ ├─ wasm/ ← faunosteo-wasm: wasm-bindgen facade (core_version, call, R0 trap proofs) │ └─ worker/ ← faunosteo-worker: plain main; `decode` sandboxed child (rlimits, seccomp allowlist, image → libwebp), parent Spawner + RIFF check, `run --check`; Dockerfile, scripts/derive-seccomp.sh -├─ docker/rust-builder.Dockerfile, rust-run.sh, rust-check.sh, wasm-build.sh, csp-proof/ ← reference Rust toolchain + the CSP proof -├─ .gitea/workflows/ci.yml ← CI: rust job (builder image, fmt/clippy/test/deny, types drift, wasm), install, typecheck, test, lint, build, docker (app + worker test stage + runtime) +├─ scripts/{rust-env,rust-check,wasm-build,check-all}.sh ← native Rust gate, wasm build, the pre-merge gate (`pnpm check:all`) +├─ docker/rust-builder.Dockerfile, rust-run.sh, csp-proof/ ← reproducible Rust builder (alternative) + the CSP proof +├─ .gitea/workflows/ci.yml ← PREPARED, never executed (no runner): rust job (builder image, scripts/rust-check.sh, types drift, wasm), install, typecheck, test, lint, build, docker (app + worker test stage + runtime) ├─ .gitea/workflows/{rust-advisories,release-images}.yml ← nightly RustSec advisories; manual digest-tagged image publication (no deploy) ├─ .claude/{tasks/TODO.md,memory/,audits/} ├─ Dockerfile, docker-compose.yml, docker/ @@ -509,7 +513,7 @@ the owner's validation before any implementation; release 0.2.0 unaffected). - Import never parses untrusted PDF object graphs: the attachment is located by byte scan and inflated under the 16 MiB cap (BLK-009). Opening the import to external users is gated by `docs/development/PDF_IMPORT_HARDENING.md` §5 (BDR-040). - After PDF roundtrip tests fail: check attachment bytes, then the layered reader (report first, envelope metadata, snapshot/branding/labels), then DecompressionStream inflation. Goldens under `apps/app/tests/fixtures/goldens/step6` are never regenerated without a user decision. - Node 24 / pnpm 10.13.1: `corepack enable` or `corepack pnpm`; never a global install. -- Rust: never on the host or the VPS; `pnpm rust:check` and `pnpm wasm:build` run inside the builder image. Before every Rust commit: `pnpm rust:check` green, `pnpm worker:test` green when `crates/worker` changed, the ts-rs types under `packages/core-wasm/types` committed, `Cargo.lock` committed. The core takes ids, time and bytes as inputs (clippy `disallowed_methods`); the worker child is the only process that decodes untrusted bytes. +- Rust: native on the Ubuntu development machine (user-level rustup, never a global package, never on the VPS); `pnpm rust:check` and `pnpm wasm:build` run natively, the `:docker` variants reproduce them in the builder image. Before every Rust commit: `pnpm rust:check` green, `pnpm worker:test` green when `crates/worker` changed (Docker only: the sandbox tests and the seccomp derivation never run natively), the ts-rs types under `packages/core-wasm/types` committed, `Cargo.lock` committed. Before a merge: `pnpm check:all`. No CI runs on the forge: a workflow file is never cited as a passed check. The core takes ids, time and bytes as inputs (clippy `disallowed_methods`); the worker child is the only process that decodes untrusted bytes. - SQL change: `pnpm --filter @faunosteo/supabase test:run` (PGlite) before commit; `pnpm db:reset && pnpm db:test` on the real stack when Docker is available; regenerate types (`pnpm db:types`). diff --git a/DEPLOY.md b/DEPLOY.md index 8015d106..42da1d8d 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -25,8 +25,11 @@ Rust (plan R0, `docs/development/RUST.md`): the repository also builds a worker image (`crates/worker/Dockerfile`, static musl binary on `distroless/static`, container `faunosteo_worker`). It is not deployed yet (its job loop arrives at R1); the live deployment above is unchanged. The -production VPS never needs a Rust toolchain: images are built in CI and -pulled by digest ("Image deploy by digest", prepared below). +production VPS never needs a Rust toolchain: Rust is compiled on the Ubuntu +development machine (native toolchain) or inside Docker builder images, and +production images will be pulled by digest ("Image deploy by digest", +prepared below) once a runner and a registry exist on the forge. No CI/CD is +operational today (2026-10-13). ### Reference sheet in the image (SHEET_BUNDLE) @@ -582,7 +585,9 @@ deployment keeps building on the VPS as documented above. 1. Build and publish: run the manual workflow `release-images` (`.gitea/workflows/release-images.yml`, `workflow_dispatch`, never on - push) on the forge. It pushes `faunosteo_app:` and + push) on the forge. Prerequisite not met yet: no runner, registry or + secret is configured on the forge (2026-10-13); the workflow has never + run, which is the CI/CD step to decide separately. It pushes `faunosteo_app:` and `faunosteo_worker:` to the container registry named by the `REGISTRY` secret (the forge's own Gitea registry is the recommended choice, Scaleway Container Registry the alternative) and prints their diff --git a/README.md b/README.md index 7b2caccc..d4403bd8 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,11 @@ cp .env.example .env # edit APP_* vars if desired pnpm dev # http://localhost:5173 ``` +Rust (core, browser module, worker): a user-level `rustup` installs the +pinned toolchain from `rust-toolchain.toml` on first use; see +`docs/development/RUST.md` §1 for the three `cargo install` / `rustup +target add` lines and the Docker alternative. + Docker Compose (matches production topology): ```bash @@ -81,8 +86,14 @@ pnpm lint # ESLint 9 flat config, 0 warnings pnpm test:run # vitest run (domain, anatomy, supabase/PGlite, app) pnpm build # tsc + vite build → apps/app/dist/ pnpm report:compare ... # archive PDF vs its v3 reading, unresolved items flagged +pnpm rust:check # native: fmt, clippy -D warnings, cargo test, ts-rs export, cargo deny +pnpm wasm:build # native: browser module → packages/core-wasm/pkg (the core-wasm proofs run on it) +pnpm check:all # everything a branch must pass before a merge, in one command (needs Docker for the image stages) ``` +No CI/CD runs on the forge yet: the workflow files under `.gitea/workflows/` +are prepared for a future runner and call these same scripts. + --- ## Build & deploy diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 19c4bb03..c73e2dd1 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -8,37 +8,88 @@ sandboxed child, the browser module loader and seven experimental proofs. No application behaviour, interface, archive or database changes; the frontend does not load the module before R2; the worker's job loop arrives at R1. -## 1. Toolchain: reproducible in Docker, never on the VPS +## 1. Toolchain: native on the development machine, reproducible in Docker, never on the VPS -Every cargo command runs inside the reference builder image, so CI and a -developer machine use the same compiler and the production VPS never needs -Rust (images are built in CI and pulled by digest, DEPLOY.md "Image deploy by -digest"). +Three environments, three roles (owner decision of 2026-10-13, review Annex C +addendum). The development machine is a self-hosted Ubuntu server (native, +not WSL, not the VPS) with Docker rootless. -| Piece | Where | Pinned by | +| Environment | Role | Rust | |---|---|---| -| Rust 1.99.0, edition 2024, rustfmt, clippy, `wasm32-unknown-unknown` | `docker/rust-builder.Dockerfile` (`FROM rust:1.99.0-bookworm`) | `rust-toolchain.toml` (same channel; a difference would make rustup download inside the container) | -| `wasm-bindgen-cli` 0.2.129 | same image | `wasm-bindgen = "=0.2.129"` in `Cargo.toml`; `docker/wasm-build.sh` refuses to run when `Cargo.lock` and the CLI differ | -| `cargo-deny` 0.20.2 | same image | `deny.toml` | -| worker builder | `crates/worker/Dockerfile` (`FROM rust:1.99.0-alpine3.24`, `build-base` for libwebp) | same channel | +| Ubuntu development machine, native | daily development (`cargo check / test / build / clippy / fmt`), the browser module build, every check before a merge (`pnpm check:all`) | user-level `rustup` in `~/.cargo` and `~/.rustup`, channel pinned by `rust-toolchain.toml` | +| Docker on that machine | the Supabase stack, the worker's sandbox tests and seccomp derivation, integration tests, the application and worker images, and the reproducible run of the same Rust scripts through the builder image | inside images only | +| Production VPS | runs images; never compiles | none | -Commands (root `package.json`): +### 1.1 Native toolchain (the primary path) + +Everything lives in the user's home: no global package, no `sudo`. Checked on +2026-10-13 (Ubuntu 26.04.1, x86_64): `build-essential` (gcc 15, the C +compiler that builds the worker's vendored libwebp), `libc6-dev`, +`pkg-config`, `libssl-dev` and `strace` were already installed; no system +package is missing. ```bash -pnpm rust:builder # builds faunosteo_rust_builder:1.99.0 once (about 6 minutes cold) -pnpm rust:check # docker/rust-check.sh: fmt --check, clippy -D warnings (all targets), - # cargo test (core, wasm), ts-rs export, cargo deny licenses/bans/sources -pnpm wasm:build # crates/wasm → packages/core-wasm/pkg (git-ignored), prints raw and gzip sizes -pnpm worker:test # crates/worker/Dockerfile stage test: 13 sandbox tests + seccomp derivation -pnpm worker:image # runtime image faunosteo_worker:dev (distroless/static:nonroot) -bash docker/rust-run.sh cargo # any cargo command in the builder +# once: rustup, the user-level installer (https://rustup.rs); ~/.bashrc and ~/.profile source ~/.cargo/env +rustup show # from the repository: installs 1.99.0 + rustfmt + clippy +rustup target add wasm32-unknown-unknown # browser build of crates/wasm +cargo install wasm-bindgen-cli --version 0.2.129 --locked # = the wasm-bindgen crate in Cargo.lock +cargo install cargo-deny --version 0.20.2 --locked ``` -`docker/rust-run.sh` mounts the repository at `/w` and keeps Cargo's registry -and the target directory in the named volumes `faunosteo_cargo_registry` and -`faunosteo_rust_target` (incremental builds take seconds). A local `rustup` -is optional for the inner loop (same `rust-toolchain.toml`); the container is -the arbiter. +`rustup show` run inside the repository installs the channel of +`rust-toolchain.toml` (rustup auto-install), so `cargo` here is always +1.99.0 whatever the default toolchain of the machine. Non-interactive +shells (editor tasks, git hooks, agent shells) skip `~/.bashrc` and start +without `~/.cargo/bin` on PATH: the scripts source `scripts/rust-env.sh`, +which prepends it when `cargo` is not found. + +| Piece | Native | Builder image | Pinned by | +|---|---|---|---| +| Rust 1.99.0, edition 2024, rustfmt, clippy | `rustup show` (auto-install) | `FROM rust:1.99.0-bookworm` + `rustup component add` | `rust-toolchain.toml` | +| `wasm32-unknown-unknown` | `rustup target add` | `RUN rustup target add` | same channel | +| `wasm-bindgen-cli` 0.2.129 | `cargo install --locked` | same | `wasm-bindgen = "=0.2.129"` in `Cargo.toml`; `scripts/wasm-build.sh` refuses to run when `Cargo.lock` and the CLI on PATH differ | +| `cargo-deny` 0.20.2 | `cargo install --locked` | same | `deny.toml` | +| worker builder (musl, libwebp) | not native: sandbox tests run in Docker only | `crates/worker/Dockerfile` (`FROM rust:1.99.0-alpine3.24`, `build-base`) | same channel | + +Commands (root `package.json`; every Rust script is one file run natively +or inside the builder image): + +```bash +pnpm rust:check # scripts/rust-check.sh, native: fmt --check, clippy -D warnings (all targets), + # cargo test (core, wasm, worker unit tests), ts-rs export, cargo deny licenses/bans/sources +pnpm wasm:build # scripts/wasm-build.sh, native: crates/wasm → packages/core-wasm/pkg (git-ignored), sizes printed +pnpm check:all # scripts/check-all.sh: the pre-merge gate (§4): rust:check, types drift, wasm:build, + # pnpm check, application image, worker:test, worker:image started read-only +pnpm rust:check:docker # the same rust-check.sh inside the builder image (reproducible run) +pnpm wasm:build:docker # the same wasm-build.sh inside the builder image +pnpm rust:builder # builds faunosteo_rust_builder:1.99.0 once (about 6 minutes cold) +pnpm worker:test # Docker only: crates/worker/Dockerfile stage test, 16 sandbox tests + seccomp derivation +pnpm worker:image # Docker only: runtime image faunosteo_worker:dev (distroless/static:nonroot) +bash docker/rust-run.sh cargo # any cargo command in the builder +cargo check | test | build | clippy | fmt # the inner loop, from the repository root +``` + +The native `target/` (git-ignored, `.dockerignore`d) and the builder's named +volumes `faunosteo_cargo_registry` / `faunosteo_rust_target` are two +separate caches; neither invalidates the other. + +### 1.2 What stays in Docker, and why + +- The worker's sandbox tests (`crates/worker/tests/sandbox.rs`) are gated by + the `test-hooks` feature: a native `cargo test` compiles that target empty + and runs the worker's unit tests only. The seccomp allowlist is derived by + `strace` on the **musl static binary that ships**, inside the Alpine + `test` stage, and glibc would need other syscalls (`brk`, `mprotect`, + `futex`): running the sandbox natively would either kill a legitimate + decode or prove nothing about the shipped binary. `pnpm worker:test` is + the only path, and the worker keeps running in its container. +- The Supabase stack, the application image, the worker image and the + integration tests on them: `pnpm db:start`, `docker build -t faunosteo .`, + `pnpm worker:image`. +- The builder image stays as the reproducible alternative (same compiler, + same tools, a clean environment) and as the reference a future runner + uses; it is not needed for a compile or a test on the development + machine. ## 2. Workspace @@ -140,35 +191,60 @@ from the shipped one by the feature-gated hooks only), `runtime` binary), `run --check` exits 0 with `--read-only --cap-drop ALL --security-opt no-new-privileges:true`. -## 4. CI +## 4. Checks before a merge, and the workflow files (prepared, no CI active) -- Job `rust` (before `check`): builds the builder image, runs - `docker/rust-check.sh`, checks the committed ts-rs types for drift, builds - the browser module (size printed). -- Job `check`: builds the module the same way, then the existing Node - pipeline; the `core-wasm` proofs run because `pkg/` exists. -- Job `docker`: application image, worker `test` stage (sandbox tests and - seccomp derivation on the musl binary that ships), worker runtime image - started once with `--read-only --cap-drop ALL`. +No CI/CD runs on the forge today (2026-10-13): no runner, no secret, no +registry. The YAML files under `.gitea/workflows/` are kept as prepared +material for that separate decision; a green workflow is never claimed +until one has actually run on the forge. Until then the gate is local: + +```bash +pnpm check:all # scripts/check-all.sh, stops at the first failure +``` + +Stages, in order: `scripts/rust-check.sh` (native), the ts-rs types drift +(`git diff --exit-code -- packages/core-wasm/types`), `scripts/wasm-build.sh` +(native), `pnpm check` (typecheck, tests including the `core-wasm` proofs on +the module just built and the pgTAP suite on PGlite, lint, build), the +application image, `pnpm worker:test` (sandbox tests and seccomp derivation +on the musl binary), `pnpm worker:image` started once with `--read-only +--cap-drop ALL`. The local Supabase stack is not required; `pnpm db:test` +is the real-stack complement after a change under `supabase/`. + +Workflow files, each mapping onto those stages so a future runner calls the +same scripts: + +- `ci.yml` (push on main/develop, pull requests): job `rust` (builder image, + `scripts/rust-check.sh`, types drift, `scripts/wasm-build.sh`), job `check` + (module built the same way, then the Node pipeline), job `docker` + (application image, worker `test` stage, worker runtime started read-only), + job `supabase` (real stack, pgTAP, generated types diff). - `rust-advisories.yml`: nightly and manual `cargo deny check advisories`, out of the blocking path on purpose. - `release-images.yml`: manual only; the dispatch input is a typed choice re-validated through an environment variable; the RustSec and npm - (`pnpm audit --prod`) advisories gates run first; a named ephemeral - `buildx` builder (`--builder`, no silent fallback) with its own cache - builds the images; the registry token reaches the login step only, into - a per-job `DOCKER_CONFIG` under `RUNNER_TEMP` created before any step and - removed by a literal path; digests read from the build metadata; - `actions/checkout` pinned to a commit. The `if: github.ref` line is not a - boundary (push access and the token's scope are; a dedicated release - runner label is the Step 27 operation). It pushes `faunosteo_app:` - and `faunosteo_worker:` and prints their digests. Deploys nothing + (`pnpm audit --prod`) advisories gates run first in a job without + secrets; a named ephemeral `buildx` builder (`--builder`, no silent + fallback) with its own cache builds the images; the registry token + reaches the login step only, into a per-job `DOCKER_CONFIG` under + `RUNNER_TEMP` created before any step and removed by a literal path; + digests read from the build metadata; `actions/checkout` pinned to a + commit. The `if: github.ref` line is not a boundary (push access and the + token's scope are; a dedicated release runner label is the Step 27 + operation). It would push `faunosteo_app:` and + `faunosteo_worker:` and print their digests. Deploys nothing (DEPLOY.md "Image deploy by digest"). -Not measured here: the forge runner's cache behaviour and cold build times -(the runner was not available from this host). The builder image build took -about 6 minutes cold on the development host; both image builds reuse -BuildKit cache mounts (`faunosteo_cargo_registry_musl`, +Target of the future CI/CD step (not implemented, recorded so the scripts +stay compatible): tests on every push and pull request, Rust and +TypeScript checks, PostgreSQL/RLS tests, Docker builds, security checks, +branch verification before a merge, image publication only in an +explicitly authorised workflow. + +Not measured: the forge runner's cache behaviour, cold build times and the +workflow syntax (job-level `env`, `GITHUB_ENV` path) have never run. The +builder image build took about 6 minutes cold on the development host; both +image builds reuse BuildKit cache mounts (`faunosteo_cargo_registry_musl`, `faunosteo_target_musl`). ## 5. R0 proofs (2026-10-12, development host, Docker 29.8 rootless, kernel 7.0) @@ -267,14 +343,43 @@ builder image cold build about 6 minutes; worker test stage cold about 2 minutes, warm under 30 s with the BuildKit cache mounts; the wasm build under 10 s warm. The first run on the forge records its own numbers here. +### 5.8 Native toolchain run (2026-10-13, Ubuntu 26.04.1, rustup 1.29.1, 16 cores) + +Same proofs, native toolchain (`rustc 1.99.0 (b940084d7 2026-09-28)`, +`cargo 1.99.0`, `clippy 0.1.99`, `rustfmt 1.10.0-stable`, `wasm-bindgen +0.2.129`, `cargo-deny 0.20.2`), compared with the builder image runs above: + +| Measure | Native | Builder image (R0) | +|---|---|---| +| cold `cargo build --workspace --all-targets` (libwebp compiled by gcc 15) | 31.5 s | not timed at R0 | +| cold wasm release build | 4.3 s | under 10 s warm | +| `scripts/rust-check.sh` warm (fmt, clippy, tests, ts export, deny) | 32 s | 2.2 s warm volumes (same script, same results) | +| cargo tests | core 10 + 2, worker unit 9, core with `ts` 14 + 2, all green | identical counts | +| committed ts-rs types | unchanged after the export | unchanged | +| `faunosteo_wasm_bg.wasm` raw | 224 777 B | 224 713 B | +| gzip -9 | 84 924 B (`gzip`), 85 390 B (the vitest measure) | 84 927 B / 85 396 B | +| sha256 | `2d559a6b…bb3a` | `451d49c1…a752` | +| core-wasm proofs | 14 passed, instantiate 3.9 ms | 14 passed, 2.7 ms | + +The 64-byte difference and the distinct hashes come from the source paths +embedded in the panic locations (`/home//Documents/faunosteo/…` +natively, `/w/…` in the builder): the module is not bit-reproducible across +build roots without `--remap-path-prefix`, which R0 does not set. The +builder image is the reference when two builds must match byte for byte. +`cargo deny` prints the same warnings in both environments +(`license-not-encountered` for six allowed licences, duplicate `miniz_oxide` +and `syn`): policy `warn`, not failures. + ## 6. Reproduction ```bash -pnpm rust:builder && pnpm rust:check # §2, §5.2 (cargo side) -pnpm wasm:build && pnpm --filter @faunosteo/core-wasm test:run # §5.1, §5.2, §5.6 -pnpm worker:test # §5.3, §5.4 (tests + derivation in the build log) +pnpm rust:check # §2, §5.2 (cargo side), native +pnpm wasm:build && pnpm --filter @faunosteo/core-wasm test:run # §5.1, §5.2, §5.6, native module +pnpm rust:builder && pnpm rust:check:docker && pnpm wasm:build:docker # the same, reproducible in the builder +pnpm worker:test # §5.3, §5.4 (tests + derivation in the build log) pnpm worker:image && docker run --rm --read-only --cap-drop ALL faunosteo_worker:dev # §3 -CHROME= bash docker/csp-proof/run.sh # §5.5 (playwright-core on NODE_PATH) +CHROME= bash docker/csp-proof/run.sh # §5.5 (playwright-core on NODE_PATH) +pnpm check:all # everything above but the CSP proof, in one command ``` ## 7. What stays hypothetical after R0 @@ -291,7 +396,9 @@ CHROME= bash docker/csp-proof/run.sh # §5.5 (playwright-core built: R2 builds and tests it. - HTTPS from the worker (CA bundle of distroless, `webpki-roots`): no HTTP client in the R0 binary; R1 adds `reqwest` and the CI HTTPS call. -- The forge runner's cache and timings (§5.7). +- The forge runner's cache and timings (§5.7), and the workflow files + themselves: no runner exists, none of the three workflows has ever run + (§4). - The allowlist is x86_64; an aarch64 host would need its own derivation (`target_arch` selects the seccomp architecture, the syscall numbers come from `libc`). diff --git a/docs/development/RUST_ARCHITECTURE_REVIEW.md b/docs/development/RUST_ARCHITECTURE_REVIEW.md index 4b665065..aa1806cd 100644 --- a/docs/development/RUST_ARCHITECTURE_REVIEW.md +++ b/docs/development/RUST_ARCHITECTURE_REVIEW.md @@ -985,3 +985,35 @@ frontend to load the module; no anatomy, PDF or business validation migration (R2–R5); no avatar processing (R1); everything dockerised, the production image produced in CI and pulled by the VPS; the live deployment untouched; no tag or merge into main. + +### C.1 Addendum 2026-10-13: development environment and CI status (owner brief after R0) + +R0 delivered and validated on the architecture. The owner corrected the +development organisation before closing R0; these points amend Annex B §B.1 +and VIII.4 and are implemented on `feature/rust-r0-foundation`: + +- Three environments, three roles. The development machine is a self-hosted + Ubuntu server (native, not WSL, not the VPS) with Docker rootless: Rust is + developed and compiled **natively** there (user-level `rustup`, channel of + `rust-toolchain.toml`, `wasm32-unknown-unknown`, `wasm-bindgen-cli` and + `cargo-deny` at the pinned versions; `cargo check / test / build / clippy / + fmt` and the browser module build run without Docker). Docker on that + machine runs the Faunosteo services (Supabase), the worker in its hardened + container, the integration tests, the sandbox and seccomp proofs, and + builds the deployment images (multi-stage). The production VPS is separate + and never compiles. +- The R0 builder image is kept as the reproducible alternative + (`pnpm rust:check:docker`, `pnpm wasm:build:docker`) and as the reference + for a future runner; it is no longer the arbiter of daily work and is not + required for a compile or a test. The worker's isolation is unchanged. +- No CI/CD is operational on the forge (no runner, secret or registry). The + workflow files are prepared material for a separate CI/CD decision; a + workflow is never reported as passed before it has run on the forge. The + pre-merge gate is local: `pnpm check:all` (`scripts/check-all.sh`), which a + future runner calls as is. +- Target of that future step (not implemented): tests on every push and pull + request, Rust and TypeScript checks, PostgreSQL/RLS tests, Docker builds, + security checks, branch verification before a merge, image publication + only in an explicitly authorised workflow. +- R1 (the Step 8 worker) waits for the owner's final validation of this + correction; no deployment, no release 0.2.0. diff --git a/docs/development/TEST_STRATEGY.md b/docs/development/TEST_STRATEGY.md index a8421800..eaffd0a8 100644 --- a/docs/development/TEST_STRATEGY.md +++ b/docs/development/TEST_STRATEGY.md @@ -17,9 +17,9 @@ Status: Step 0 deliverable, 2026-10-08. Current state: 32 files under | Migration | `supabase db reset` from zero + seed + regression fixtures | CI | every PR touching `supabase/` | | PDF regression | vitest golden tests: text extraction of generated PDFs, attachment roundtrip, raster hash on fixed inputs | `packages/pdf/tests` | CI | | SQL/TS contract | generated types diff check | CI | every PR touching `supabase/` | -| Rust core and facade | `cargo test` (unit + fixture tests reading the repository JSON in place), `clippy -D warnings`, `cargo deny`; ts-rs types drift check | `crates/*`, CI job `rust` through the builder image | CI on every PR | -| Rust/TS contract | vitest proofs on the built module: byte equality with `JSON.stringify(…, null, 2)` on the real documents, optional/nullable shape read by the current Zod schemas, panic and stack traps, served size budget | `packages/core-wasm/tests` (skipped with a notice when `pkg/` is absent) | CI `check` after `wasm:build` | -| Worker sandbox | integration tests against the musl binary that ships: fixture corpus through the full sandbox, refusals, SIGSYS on a denied syscall, output cap, `RLIMIT_AS` abort, `RLIMIT_CPU`, deadline; seccomp allowlist derived by `strace` and compared | `crates/worker/tests`, Dockerfile stage `test` | CI `docker` job | +| Rust core and facade | `cargo test` (unit + fixture tests reading the repository JSON in place), `clippy -D warnings`, `cargo deny`; ts-rs types drift check | `crates/*`, `pnpm rust:check` (native, `scripts/rust-check.sh`; same file in the builder image), `pnpm check:all` | before every Rust commit and every merge; CI job `rust` prepared, no runner yet | +| Rust/TS contract | vitest proofs on the built module: byte equality with `JSON.stringify(…, null, 2)` on the real documents, optional/nullable shape read by the current Zod schemas, panic and stack traps, served size budget | `packages/core-wasm/tests` (skipped with a notice when `pkg/` is absent) | `pnpm check:all` (after the native `wasm:build`); CI `check` prepared | +| Worker sandbox | integration tests against the musl binary that ships: fixture corpus through the full sandbox, refusals, SIGSYS on a denied syscall, output cap, `RLIMIT_AS` abort, `RLIMIT_CPU`, deadline; seccomp allowlist derived by `strace` and compared | `crates/worker/tests`, Dockerfile stage `test` (`pnpm worker:test`, Docker only, never native) | every `crates/worker` commit, `pnpm check:all`; CI `docker` job prepared | Browser-only code (`svgToPng`, `DOMParser` in the importer) runs under happy-dom where sufficient and under Playwright when real rendering matters diff --git a/docs/development/WORKSPACE.md b/docs/development/WORKSPACE.md index 0748e98d..c25a8168 100644 --- a/docs/development/WORKSPACE.md +++ b/docs/development/WORKSPACE.md @@ -40,7 +40,9 @@ Future units, created only by the step that needs them: Rust workspace (plan R0, `docs/development/RUST.md`): `Cargo.toml` at the root with `crates/core` (library), `crates/wasm` (browser facade, built into `packages/core-wasm/pkg`) and `crates/worker` (binary). Every cargo command -runs through the builder image (`pnpm rust:check`, `pnpm wasm:build`); the +runs on the native toolchain of the development machine (`pnpm rust:check`, +`pnpm wasm:build`; user-level rustup, channel of `rust-toolchain.toml`) or, +for a reproducible run, through the builder image (`:docker` variants); the pnpm workspace gains `packages/core-wasm` (loader, generated types, proofs), whose tests skip with a notice when the module has not been built. | `supabase/` | migrations, functions, seed, tests | plan Step 3 |