diff --git a/.claude/memory/decisions.md b/.claude/memory/decisions.md index 1c045504..b3781f40 100644 --- a/.claude/memory/decisions.md +++ b/.claude/memory/decisions.md @@ -360,3 +360,8 @@ Status: closed; Step 7 finished into develop. Owner rule: Rust preferred wherever equivalent or better; "already TS", "more work", "no perf gain" never reasons for C. Kept: React/TS UI + DOM + stores + vendor glue, Tailwind, SQL + RLS, Supabase Auth/Storage, Vite tooling. Study `docs/development/RUST_ARCHITECTURE_REVIEW.md` rev 2 (61 challenger findings folded). Classification: A = domain contracts/rules, sheet handle + index data, reducers, data-building component rules, publish gate, layer renderer + compose, one SVG sanitiser, PDF reader + writer, CLI tools, curation pipeline + importer, upload script, Step 8 worker, every future backend/rule mirror; B = none; C = React render glue + display helpers, Zustand/storage adapters, supabase-js session + error shapes (no Rust GoTrue PKCE client), UI form schemas + dictionaries, clock/crypto/Intl screen dates, browser rasteriser (DOM), Vite tooling, SQL/RLS, public web (Astro, global rule). Target: 3 crates (`core` feature-gated lib + tool bins, `wasm` facade, `worker` bin), JSON strings across the boundary + `ts-rs` types (no serde-wasm-bindgen: writes absent optionals as null), `SheetHandle` built once, closed error kinds, ids/time injected, panic=abort handled TS-side (poison + reinstantiate), lazy init + `core_unavailable`, reader in terminated Web Worker. Worker: Rust binary, decode child = same binary dispatched before any thread, empty env, RLIMIT_AS/CPU, traced seccomp allowlist (Landlock dropped: redundant), pure-Rust decoders (`image`/zune-jpeg/png/image-webp) + `moxcms` ICC, lossy WebP via `libwebp-sys` ≥ 1.6 in the child only, parent parses RIFF header only (never decodes), 512 KiB output cap, per-stage deadlines; service key kept (dedicated role = Step 27 option). Roadmap: R0 workspace + worker image inside Step 8, R1 = Step 8 worker, R2 domain core before Step 9, R3 anatomy, R4 pdf reader + writer (Std14 AFM metrics, draw-log golden frozen first), R5 pipeline + importer + tools, all before Step 13; TTF fonts separate later change. Deploy prerequisites: CSP `'wasm-unsafe-eval'`, gzip wasm, images built in CI pulled by digest (VPS 1 GB never compiles), N-1 reader test, draft quarantine. Alt rejected: single `core` crate without features (worker would carry pdf deps) → features chosen; 6 crates (rev 1) → too many; resvg in browser (949 KB gz, visual change); wasm-pack (one maintainer, CLI lock); Edge Functions (limits, hand deploy); Prettier-chained sheet identity (hash only compared at runtime → one-time rebase accepted); parent re-decode of child output (breaks keyless-decoder invariant). Status: active, awaiting owner validation of the study + 13 decisions D-R1..D-R14 (D-R6 withdrawn); nothing implemented; release 0.2.0 untouched. + +## BDR-044 | 2026-10-12 | Rust orientation validated by the owner; detailed decisions D-R1..D-R14 pending + +Owner validated the study's orientation + 10 principles: Rust worker at Step 8; Cargo workspace; native + WASM libs; progressive anatomy migration; PDF migration under complete compatibility proofs; React + PostgreSQL/Supabase kept; testable increments with rollback; no premature deletion of TS still used; fully dockerised, production builds outside the small VPS; WASM vs production CSP proven before activation. Requests: expanded decision table (Annex A), R0 scope with build tools reproducible in Docker and no Rust on the VPS (Annex B), serde/WASM/panic/musl hypotheses verified experimentally when they condition a choice (R0 proofs as permanent tests). Release 0.2.0 still waiting. R0 and Step 8 implementation NOT started until the 13 decisions are validated. +Status: active; awaiting D-R1..D-R14 validation. diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index 0d613fc4..89400a6f 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -227,3 +227,4 @@ User: Step 7 closed; Step 8 = files foundation + Storage, analysis before implem ## 2026-10-12 (Rust-by-default review) User: new permanent rule, Rust preferred for everything equivalent or better, existing code included; study + roadmap before Step 8 or any migration; nothing implemented. On chore/step-08-files-analysis: 4 analyzers + 3 researchers (scratchpad reports) → `docs/development/RUST_ARCHITECTURE_REVIEW.md` rev 1 (6a078da) → 3 plan-challengers (61 findings, 2 blockers: deletion schedule vs surviving TS consumers; CSP/VPS deployment facts) → rev 2 folded. Outcome: A/B/C classification (B empty), 3-crate workspace, Rust worker design (keyless parent, traced seccomp, libwebp in child), roadmap R0–R5 with end criteria + rollback rows, 13 decisions D-R1..D-R14. Memory: BDR-043, LRN-048, EVAL-016; CLAUDE.md "Rust by default" section; TODO Rust block. No cargo on the dev host (owner installs rustup at R0). Awaiting owner validation; release 0.2.0 untouched. +- Same day: owner validated the Rust orientation (10 principles, BDR-044), asked the expanded decision table + R0 scope before validating D-R1..D-R14 → Annex A (13 decisions: recommendation, alternatives, pros/cons, risks, consequence, impact tags) and Annex B (R0: Docker builder image as the reference toolchain, rustup optional locally, workspace skeleton, worker image, CI job, 7 experimental proofs kept as tests, exit criteria) appended to the review. Nothing started; awaiting the 13 validations. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 47711a04..b09f3a57 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -134,8 +134,8 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [ ] 8.4 app: avatar block in Mon compte (picker, canvas ≤1024 JPEG transport, preview, save, processing, supersede, delete), UserBadge photo, top bar avatar (D3), authStore blob URL, i18n, tests; design toolchain + impeccable detect - [ ] 8.5 docs: STORAGE.md (model, keys, statuses, purge/grace rules, reconciliation query, backup ordering + invariant, managed migration runbook), DEPLOY.md (worker, env file, network, CSP connect-src), SUPABASE.md, CLAUDE.md, CHANGELOG, plan note (worker early, SVG processor → Step 13); CI jobs - [ ] 8.6 verify: root check, PGlite + real stack pgTAP, types, docker builds (app + worker), real-stack e2e (sign allowed/denied), gates verifier + security; memory; stop for user validation -- [ ] Rust by default (rule adopted 2026-10-12, BDR-043; study `docs/development/RUST_ARCHITECTURE_REVIEW.md` rev 2 on chore/step-08-files-analysis, 3 challengers folded; AWAITING OWNER VALIDATION + decisions D-R1..D-R14; nothing implemented). Roadmap once validated (each phase: oracles frozen with the old code, consumer-driven deletion, rollback row, gates): - - [ ] R0 foundation (inside Step 8): Cargo workspace (`crates/core`, `wasm`, `worker`), toolchain pin, deny.toml, lints, CI job `rust`, worker image, `docs/development/RUST.md`, DEPLOY.md (images pulled by digest, CSP `'wasm-unsafe-eval'`, gzip wasm, VPS headroom); owner installs rustup; verify panic strategy on wasm32 +- [ ] Rust by default (rule adopted 2026-10-12, BDR-043; orientation + 10 principles VALIDATED by the owner 2026-10-12, BDR-044; study `docs/development/RUST_ARCHITECTURE_REVIEW.md` rev 2 + Annex A decision table + Annex B R0 scope on chore/step-08-files-analysis; AWAITING the owner's validation of decisions D-R1..D-R14 (D-R6 withdrawn); R0 and Step 8 NOT started until then). Roadmap (each phase: oracles frozen with the old code, consumer-driven deletion, rollback row, gates): + - [ ] R0 foundation (inside Step 8, Annex B): Docker builder image = reference toolchain (rustup optional locally, never on the VPS), Cargo workspace (`crates/core`, `wasm`, `worker` skeletons), toolchain pin, deny.toml, lints, CI job `rust`, worker image started once against the local stack, app wasm stage built but not loaded, `docs/development/RUST.md`, DEPLOY.md (images pulled by digest, CSP `'wasm-unsafe-eval'`, gzip wasm, VPS headroom, rollback = previous digest); 7 experimental proofs kept as tests (wasm panic/re-instantiate, js_number + optional/nullable vs Zod, musl + libwebp build, traced seccomp child, CSP headless, size/instantiate, runner caches) - [ ] R1 = Step 8 worker in Rust (replaces 8.2 below): decode child dispatched before any thread, empty env, RLIMIT_AS/CPU, traced seccomp allowlist, `image` + `moxcms` + `libwebp-sys` in the child, parent RIFF-header check only, 512 KiB output cap, per-stage deadlines, service key - [ ] R2 domain core (before Step 9): sheet/report/archive contracts, `SheetHandle` + `SheetIndexData`, reducers, Finding builder, vertebra selection, publish gate; wasm facade (JSON strings, ts-rs types, poison + reinstantiate, lazy init); app switched; 3 envelopes frozen first; N-1 reader test; draft quarantine; conformance fixtures + generated pgTAP; `packages/domain` frozen for engine/pipeline - [ ] R3 anatomy core: tree, one sanitiser, serialiser, layer renderer, `compose_view_svg`; app canvas switched; composed views frozen as trees first diff --git a/docs/development/RUST_ARCHITECTURE_REVIEW.md b/docs/development/RUST_ARCHITECTURE_REVIEW.md index 1d4c8e0e..8fe2923b 100644 --- a/docs/development/RUST_ARCHITECTURE_REVIEW.md +++ b/docs/development/RUST_ARCHITECTURE_REVIEW.md @@ -849,3 +849,103 @@ Rust worker" from this review on; the worker credential stays the service key, the dedicated role being a Step 27 option (IV.4). D-R6 of revision 1 (that role) is therefore no longer a decision of this study, which is why the numbering skips it. + +--- + +## Annex A. Decision table (expanded for the owner's validation, 2026-10-12) + +Orientation validated by the owner on 2026-10-12 with ten principles (Rust +worker at Step 8, Cargo workspace, native and WASM libraries, progressive +anatomy migration, PDF migration under complete compatibility proofs, React +and PostgreSQL/Supabase kept, testable increments with rollback, no premature +deletion of TypeScript still in use, fully dockerised infrastructure with +production builds outside the small VPS, WASM compatibility with the +production CSP proven before activation). The detailed decisions below await +validation. Impact tags: SEC security, HIST historical data, PERF browser +performance, UI interfaces, DEPLOY deployment. + +| Id, subject | Recommendation | Relevant alternatives | Advantages / drawbacks | Risks | If validated | Impacts | +|---|---|---|---|---|---|---| +| D-R1 workspace layout | Three crates: `core` (feature-gated library, tool binaries behind `cli`), `wasm` (cdylib facade), `worker` (binary) | Six crates per boundary (rev 1); one crate holding the facade too (wasm-bindgen would pollute native builds); one crate per module | One dependency graph and one test run; features keep the worker lean / feature combinations must be tested in CI (`--no-default-features`, `--all-features`); core compile time grows with modules | Accidental I/O or clock in core (lint `disallowed_methods`); feature creep | R0 creates the three crates and the lint set | none | +| D-R2 PDF engine | Reader and writer ported together at R4 on `pdf-writer` with the Standard 14 AFM metrics (kerning-inclusive widths); a draw-log golden for lines, rectangles and colours frozen with the TS engine before the port; TTF fonts as a separate later change | Reader first, writer later (two phases, longer hybrid); writer on `krilla` + TTF at once (no Standard 14 page text, goldens regenerated blind); keep pdf-lib for writing (unmaintained, contradicts the rule) | The 39 goldens stay the exact oracle; `packages/pdf` deleted in one phase; no object streams by construction / largest single phase; AFM files vendored under the APAFML notice | Kerning, rounding, ICU dates or `f32` coordinates moving a run (caught by goldens and the `lopdf` read-back); viewer compatibility of classic xref (standard) | Golden regeneration decision for the additive draw-log golden; the gated functions port (D-R12); rollback note in DEPLOY.md as for Step 6 | HIST, DEPLOY | +| D-R3 rasteriser | Browser canvas stays the app's rasteriser between the two core calls; `resvg` native only in tools and worker if a server-side rendering need appears; nothing resvg in the browser bundle | `resvg` lazy-loaded in the browser (949 KB gzip, anti-aliasing change, new raster goldens); `resvg` everywhere for identical bytes server and browser | No bundle growth, no visual change, no golden change / browser and future server renderings would differ (only matters if server rendering appears) | Browser rasteriser limits unmeasured (true today already) | Export becomes `compose_export_views` → rasterise → `write_archive`; `browserRasterizer.ts` kept as DOM adapter | PERF, UI | +| D-R4 lossy WebP encoder | `libwebp-sys` 0.14 (libwebp ≥ 1.6.0, C) confined to the sandboxed child; output checked by the parent at container level only | Lossless WebP via `image-webp` (pure Rust; avatars 5 to 10× larger); AVIF via `ravif` (pure Rust, slower encode, changes D4's format); wait for a pure-Rust lossy encoder (`oxideav-webp`, months old) | Standard output, small files, mature encoder / C compiled in the worker build (Alpine `build-base`), vendored version invisible to RustSec (logged at start, checked in CI) | libwebp CVEs (historically decoder side); mitigation = process confinement, rlimits, seccomp, version check | R0 spike proves the musl static build with `cc`; R1 implements | SEC, DEPLOY | +| D-R5 sheet identity at R5 | `documentSha256` = sha256 of the bytes the Rust pipeline emits; generated dirs leave Prettier's scope; the one-time hash change makes older archives and drafts rebase once with a visible message and 0 unresolved items | Keep a Node Prettier step to preserve the bytes (rev 1; chains the identity to a JS formatter); hash of canonical JSON (RFC 8785, `serde_jcs`): identity independent of formatting forever, one more concept | Simplest rule, matches today's semantics (the file is the identity) / one import notice per old archive after R5 | None on data (rebase keeps everything); user confusion if the message is not worded for "identical content" | R5 regenerates the seed golden knowingly; CHANGELOG entry; the import and restore messages re-read for the 0-unresolved case | HIST, UI | +| D-R7 form validation | Stays TypeScript (React-bound); a separate chore gives the constraints one home (hand-written constants module checked by a PGlite test against `pg_constraint` and `config.toml`, counting rule recorded) | Rust validation through the wasm per field (render-path calls on every keystroke); constants generated from the SQL migrations (rev 1: a DDL interpreter for eleven numbers) | Forms stay responsive and simple; the drift found by the inventory is closed / constants hand-written (kept honest by the test) | low | A small chore branch, any time; no UI change | UI | +| D-R8 public web (Step 22) | Keep Astro (global SEO/GEO rule); record the Rust exception | Zola (Rust static generator, no islands, no React); Leptos/Dioxus SSR (immature SEO tooling, contradicts "React stays"); Next.js (global rule names Astro) | SEO/GEO tooling and islands / a TypeScript app in a Rust-by-default repository | none now | Exception recorded; Step 22 brainstorm re-checks per the global rule | DEPLOY | +| D-R9 Edge Functions | Retired from the plan; webhooks and OAuth callbacks served by the worker's `serve` subcommand behind the reverse proxy | Edge Functions (Deno can run the wasm core; limits, hand deployment on self-hosted); a separate HTTP container; `pg_net`/`pg_cron` for some jobs | One runtime, one image, Rust / the worker gains an inbound route at Steps 21 and 23 and uptime expectations | Inbound surface (signature checks, proxy rate limits, monitoring at Step 27) | `BACKEND_ARCHITECTURE.md` §6 updated; Steps 21 and 23 designs change | SEC, DEPLOY | +| D-R10 order | R0 + R1 within Step 8; R2 before Step 9; R3, R4, R5 before Step 13, interleaved with Steps 9 to 12 | R2 after Steps 9 to 12 (more TypeScript rules to migrate twice); R3 to R5 after Step 13 (impossible: the Rust worker needs the Rust sanitiser at Step 13) | Every later step writes its rules in Rust once / Steps 9 to 12 slowed by the migrations | Velocity of the functional steps | TODO order; Step 9 starts after R2 | DEPLOY | +| D-R11 archive attachment bytes | New contract adopted at R2: declared field order (struct order), `js_number` floats, optional absent, nullable `null`; every previous reader keeps parsing it (permanent N-1 reader test); existing PDFs untouched. At R2 the frozen TS engine still writes the attachment from the core's objects (order changes, floats printed by JS); at R4 the Rust writer serialises fully | Reproduce the insertion order of the TS store exactly (fragile mirror); keep TS serialisation of the attachment until R4 (same end state, same key-order change) | Declared and testable order / byte diff against pre-R2 exports (no reader depends on order; Zod ignores it) | A reader depending on key order (none known) | Struct field orders documented; N-1 reader test added at R2 and kept | HIST | +| D-R12 gated functions | Explicit go to port `readArchiveDocument`, `readReportArchive`, `readAttachedDocument` and `rebaseReportOnSheet` with behaviour preserved (CLAUDE.md gate) | Keep the TypeScript readers and rebase permanently (two implementations, contradicts the rule) | One reader for browser, worker and tools / the highest-stakes port (user data) | Behaviour drift on archives; mitigation = exact-object oracles, all fixtures, hostile corpus through the wasm build, N-1 test | The port follows the oracle suites; any behaviour change needs its own decision | SEC, HIST | +| D-R13 serving the wasm | CSP `script-src 'self' 'wasm-unsafe-eval'` in both `DEPLOY.md` examples; gzip for `.wasm`; images built in CI and pulled by digest (no compile on the VPS); served module ≤ 600 KB compressed enforced in CI; registry = the forge's Gitea container registry (recommended; Scaleway Container Registry as alternative) | `'unsafe-eval'` (broader, rejected); inline the module as base64 (CSP still applies, bigger); Brotli (nginx module needed); build on the VPS (rejected) | Minimal CSP relaxation specific to WebAssembly; deploy and rollback never compile / `'wasm-unsafe-eval'` needs Chrome 97+, Firefox 102+, Safari 16+ (Vite 7 baseline); an older browser lands in `core_unavailable` | Registry credentials on the VPS; CSP forgotten on one proxy (sanity-check row) | DEPLOY.md rewritten at R0 (procedure prepared, applied at the first image deployed: worker at R1); the deploy procedure change is validated separately | SEC, PERF, DEPLOY | +| D-R14 BLK-009 and the reader | R4 runs the reader in a terminated Web Worker and records the `PDF_IMPORT_HARDENING.md` §5 browser measurements (Chromium, Firefox, Safari, one low-end phone); the external-user import gate (BDR-040) stays open until every §5 item is done | Main-thread synchronous reader (memory never shrinks, UI blocked); server-side import only (Step 15) | Bounded blast radius, measurable, import off the main thread / a second instantiation per import; a browser runner (Playwright) introduced for the measurements | Safari worker + wasm specifics; measurement effort | R4 includes the browser measurement campaign; the gate stays open | SEC, PERF | + +## Annex B. R0 scope (owner's clarification request, 2026-10-12) + +R0 is the foundation phase inside Step 8. It changes no production +artifact, implements no feature and starts only on the owner's go. + +### B.1 Toolchain, reproducible in Docker + +- Reference build environment = a Docker builder image + (`docker/rust-builder.Dockerfile`, `FROM rust:1.99.0-bookworm`, plus + `wasm32-unknown-unknown`, `wasm-bindgen-cli` at the version in + `Cargo.lock`, `cargo-deny`, `binaryen` for `wasm-opt`), used by CI and by + the application and worker image builds. The worker's own builder stays + `rust:1.99.0-alpine3.24` (musl, `build-base` for libwebp). +- The development host may install `rustup` for the inner loop (same + `rust-toolchain.toml`, so the same compiler); the containerised build is + the arbiter. Root scripts: `pnpm rust:check` (fmt, clippy, test, deny) + and `pnpm wasm:build`, each runnable locally or through the builder image + (`docker run --rm -v "$PWD":/w`). +- The production VPS never gets Rust: it pulls images by digest. + +### B.2 Workspace + +- `Cargo.toml` (workspace, `[workspace.dependencies]`, `[workspace.lints]`: + `unwrap_used`, `expect_used`, `disallowed_methods` for `std::time`, + `std::fs`, `std::env` in core), `rust-toolchain.toml`, `rustfmt.toml`, + `clippy.toml`, `deny.toml`, `.cargo/config.toml` (wasm stack size, + `-C link-arg=-zstack-size`). +- `crates/core` (modules declared, `SheetRef` + `Locale` types, the + `js_number` formatter, the conformance fixture loader), `crates/wasm` + (facade exporting `core_version()` and one JSON round-trip call), + `crates/worker` (binary with `run` printing its configuration and + `decode` wired to the sandbox spike, no job logic). +- `packages/core-wasm/package.json` committed, `pkg/` ignored. + +### B.3 Docker and CI + +- Worker image built from the skeleton (Alpine musl builder → + `distroless/static:nonroot`), started once in CI against the local stack + with no job (proves transport configuration, CA bundle, non-root, + read-only). +- Application image: the wasm stage added and exercised in CI, its output + not loaded by the application until R2. +- CI job `rust` before `check`: fmt, clippy, test, `cargo deny check + licenses bans sources`, wasm build artifact; `docker` job builds both + images; registry push by tag and digest prepared (registry decision in + D-R13); nightly advisories job; CI duration and runner cache behaviour + recorded in `RUST.md`. + +### B.4 Feasibility proofs (experimental, each one a permanent test) + +| Proof | Why it conditions a choice | How | Lives on as | +|---|---|---|---| +| Panic strategy on wasm32 | III.3 rule 5 (poison and re-instantiate) | a facade call that panics; observe `WebAssembly.RuntimeError`; re-instantiate; call again; try a deep recursion at the configured stack size | vitest in `packages/core-wasm` | +| Serialisation contract | D-R11, I.2 (`js_number`, optional vs nullable) | parse `report-v3.demo.json` and `document.json` with serde, re-emit through the custom formatter, compare byte for byte with `JSON.stringify(JSON.parse(x), null, 2)`; a struct with optional and nullable fields parsed by the current Zod schemas | cargo test + vitest N-1 reader seed | +| musl static binary with libwebp (`cc`) | D-R4 | build the worker skeleton with `libwebp-sys` on the Alpine builder; log `WebPGetEncoderVersion` | the worker image build in CI | +| seccomp allowlist derivation and application | Part IV (traced allowlist, `Threads: 1`, TSYNC) | `decode` spike: minimal `image` decode of one JPEG, one PNG, one WebP at the caps, traced with `strace -f` in the production image under Docker rootless; filter installed before any thread; disallowed syscall kills the child; environment empty | the sandbox test of R1 | +| CSP with `'wasm-unsafe-eval'` | D-R13, owner principle "proven before activation" | serve the wasm artifact under the production nginx config with the DEPLOY.md headers in headless Chromium: instantiation succeeds with the keyword, fails without it; result written in DEPLOY.md | CI headless check | +| Served size and instantiate time | D-R13 budget | measure raw and gzip bytes of the skeleton module and its instantiate time; set the budget check | CI size check | +| Runner caches and build times | VIII.3 | time the cold and warm `rust` job with and without `Swatinem/rust-cache` on the Gitea runner | `RUST.md` numbers | + +### B.5 R0 exit criteria + +`cargo test`, `clippy`, `deny` green in CI through the builder image; worker +image < 30 MB, non-root, starts against the local stack; the seven proofs +recorded in `RUST.md` with their numbers; `DEPLOY.md` rewritten (pull by +digest, CSP line, gzip, VPS headroom, rollback = previous digest) without +touching the live deployment; `pnpm check` unchanged; no production +artifact changed. Owner validation closes R0; R1 (the Step 8 worker) starts +on a separate go.