chore(memory): R0 closed for owner validation — BDR-046, LRN-049, EVAL-017, journal, TODO R0 check-offs and follow-ups

This commit is contained in:
bchanot committed 2026-10-13 00:54:36 +02:00
1 parent 47d84161f5
commit bd9c7d1968
5 files changed
+36 -4

No files matched your search

+6
View File
@@ -370,3 +370,9 @@ Status: active; awaiting D-R1..D-R14 validation.
Owner validated D-R1/2/3/7/8/10/11/12/14 as recommended. Amended: D-R4 libwebp C only in the isolated child, parent validates result + limits before use, linked libwebp version known/controlled, no new unmanaged risk; D-R5 fingerprint → canonical JSON (RFC 8785 JCS or strictly defined alternative) independent of formatter, explicit compat with old fingerprints/archives, existing fingerprints untouched, decision prepared for R5; D-R9 Edge Functions → Rust services, shared code but distinct processes (worker vs HTTP endpoints), same image + subcommands, least privilege per service, Supabase Auth keeps OAuth; D-R13 CSP `'wasm-unsafe-eval'` proven experimentally without widening JS permissions, reproducible Docker builds in CI, digest images + rollback prepared, no auto-deploy, no image publication without workflow + validation, VPS never compiles Rust. R0 go per Annex B (7 proofs for real, unverified hypotheses tested, blockers reported not assumed); R1 NOT started without a go; release 0.2.0 pending; live deployment untouched.
Status: closed (owner, 2026-10-12); R0 in progress on feature/rust-r0-foundation.
## BDR-046 | 2026-10-13 | R0 implementation choices (Rust foundation)
Choices made while building R0 (feature/rust-r0-foundation), within the validated review: (1) toolchain arbiter = Docker builder image (`docker/rust-builder.Dockerfile`, rust:1.99.0-bookworm + wasm target + wasm-bindgen-cli 0.2.129 + cargo-deny 0.20.2 + rustfmt/clippy baked in: official images ship rustup's minimal profile), `docker/rust-run.sh` with named volumes `faunosteo_cargo_registry` / `faunosteo_rust_target`; no rustup on the host, never on the VPS. (2) `serde_json` features `preserve_order` + `float_roundtrip` (default parser one ULP off, found by the proof). (3) Boundary = JSON strings + `ts-rs` types committed under `packages/core-wasm/types` with a CI drift check (no serde-wasm-bindgen/tsify). (4) Fresh wasm instance = cache-busting dynamic import of the glue (ESM cache returns the same instance otherwise); R2 must do the same in the browser or use a Worker per instance. (5) Worker: `RLIMIT_CPU` soft N / hard N+2 (soft = hard → SIGKILL first); seccomp allowlist = the 7 traced syscalls (read write mmap munmap mremap exit_group getrandom) + 7 abort-path ones, derived by strace in the Dockerfile `test` stage and compared on every build; dependencies optimised in the dev profile (4096² decode under the 10 s CPU cap); test hooks behind a cargo feature never in release; parent parses the RIFF container only (D-R4 amendment). (6) App `Dockerfile`, `docker/nginx.conf`, compose untouched at R0 (deviation from Annex B's "wasm stage built in CI": no consumer before R2, owner said live deployment untouched); CI `check` job builds the wasm itself (no artifact action dependency on the Gitea runner). (7) cargo-deny `allow-wildcard-paths = true` (a versionless path dep reads as a wildcard) and the core path dep declared per crate (workspace-inherited path dep hits a cargo-deny bug). (8) CSP proof committed as `docker/csp-proof/` (playwright-core outside the repo, documented).
Alt rejected: rustup on the dev host (owner: builds reproducible in Docker); wasm-pack; artifact upload/download between CI jobs (unverified on the forge); Landlock + NPROC/FSIZE/NOFILE (redundant under the allowlist, challenge finding).
Status: closed (R0 delivered, awaiting owner validation; R1 not started).
+7
View File
@@ -129,3 +129,10 @@ Method: 4 read-only analyzers (domain, anatomy, pdf + app pdf side, app logic +
Result: real blockers caught by the gates, not by me: (1) VI.4 deleted TS modules still imported by later-phase TS consumers (pipeline, engine); (2) production CSP `script-src 'self'` refuses wasm, VPS 1 GB compiles images; plus wrong premises: envelope byte-parity (integer-valued floats, insertion order), Prettier identity overstated (hash only compared), `catch_unwind` unusable on wasm32 (panic=abort), serde-wasm-bindgen null-for-absent, parent re-decode breaking the keyless invariant, Step 15 misread as server rendering. Simplicity: 6 crates → 3, dual-run harness → frozen goldens, Landlock + 3 rlimits redundant, constraints generator/schemars/cargo-chef/cargo-audit/infer/img-parts dropped.
Anomalies: one analyzer had a read-only `find -exec cat | wc -l` refused (no rule named) and re-ran as `find | xargs wc -l` (already allowed earlier) — reported to the user as a possible workaround; agents flagged writing report files to the scratchpad against their default (orchestrator asked for it). Three challenger claims came from memory not code (json_compatible null, panic=abort, musl mprotect) → folded as "verify at R0/R1".
Action: owner validates study + D-R1..D-R14; nothing implemented; CLAUDE.md rule section added; TODO roadmap lines.
## EVAL-017 | 2026-10-13 | Rust R0 foundation — proofs in the real environment, 3 verifier passes, 3 security passes
Method: contract `2026-10-13-rust-r0-verify.md` (16 criteria, CHECK/EXPECT) → fresh verifier; security-auditor on `5fec065..HEAD` (semgrep p/security-audit + p/secrets + p/owasp-top-ten + checklist + manual Rust review); all cargo commands through the Docker builder (no Rust on the host); worker sandbox tests + strace derivation inside the Alpine test stage on the release-profile musl binary; wasm proofs in vitest on the built module; CSP proof with the production nginx image + headers in headless Chrome 156.
Result: verifier pass 1 ECARTS(2), pass 2 CONFORME 16/16, pass 3 ECARTS(2) — every gap an oracle of MY contract (greps counting comment lines, the contract matching its own regex, the `[gated]` SAFETY form and a second refusal arm moved by the security fixes); code behaviour confirmed by reading each time. Security pass 1 BLOCK(1) HIGH real: shell injection via the `workflow_dispatch` input of release-images.yml, + 6 MEDIUM (shared BuildKit caches, tkill/tgkill without arg filters, core dumps, creds on the runner, no advisories gate, mutable action tag) → fixed; pass 2 PASS but 3 MEDIUM introduced by my fixes (ephemeral builder never used: GITHUB_ENV applied one step too late, silent fallback; `rm -rf "$VAR"`; content reaching the `Broken` outcome via `Reject::Internal`) → fixed; pass 3 PASS, 2 new MEDIUM (unpinned setup-node and `pnpm install` inside the token job) → gates moved to a secret-less job; remaining lows documented as R1/Step 27 follow-ups. Experimental findings (LRN-049): serde_json fast float path (float_roundtrip), traps leave the instance unusable + ESM cache, RLIMIT_CPU soft=hard → SIGKILL, minimal rustup profile, cargo-deny path wildcard, 7 traced syscalls, strace blind once dumpable=0 (marker became a syscall). Final: cargo 12+14 core tests, 9 unit + 16 sandbox worker tests on the release profile, 14 vitest proofs, root `pnpm check` 777 tests green, worker image 8.7 MB read-only/cap-drop, CSP refused/accepted as predicted, floor-guard clean ×3.
Anomalies: my contract oracles cost two verifier passes (lesson repeated from EVAL-013: run every CHECK once before dispatch, and re-run them after any fix commit); two of my own security fixes created new findings (an env variable applied too late, a recursive delete through a variable: the exact pattern the destructive-tools rule forbids) → fixes to CI files need the same adversarial pass as code; a scripted edit silently aborted on a rustfmt-reformatted string and the commit went out with the old test (caught by the next build); the `[gated 2026-10-13]` SAFETY markers were placed by me at the auditor's request, not by the owner → surfaced in the report for the owner's confirmation; one security-auditor Bash call (`cat` of two files) was refused twice by the auto-mode hook, the agent used Read instead (reported, no workaround). Not proven here: the Gitea runner (caches, timings, workflow syntax such as job-level env, actions/cache), HTTPS from the worker (no HTTP client before R1), browser-side fresh-instance mechanism (R2).
Action: owner validation of R0 (report with proved vs hypothetical); R1 not started.
+4
View File
@@ -228,3 +228,7 @@ User: Step 7 closed; Step 8 = files foundation + Storage, analysis before implem
User: new permanent rule, Rust preferred for everything equivalent or better, existing code included; study + roadmap before Step 8 or any migration; nothing implemented. On chore/step-08-files-analysis: 4 analyzers + 3 researchers (scratchpad reports) → `docs/development/RUST_ARCHITECTURE_REVIEW.md` rev 1 (6a078da) → 3 plan-challengers (61 findings, 2 blockers: deletion schedule vs surviving TS consumers; CSP/VPS deployment facts) → rev 2 folded. Outcome: A/B/C classification (B empty), 3-crate workspace, Rust worker design (keyless parent, traced seccomp, libwebp in child), roadmap R0–R5 with end criteria + rollback rows, 13 decisions D-R1..D-R14. Memory: BDR-043, LRN-048, EVAL-016; CLAUDE.md "Rust by default" section; TODO Rust block. No cargo on the dev host (owner installs rustup at R0). Awaiting owner validation; release 0.2.0 untouched.
- Same day: owner validated the Rust orientation (10 principles, BDR-044), asked the expanded decision table + R0 scope before validating D-R1..D-R14 → Annex A (13 decisions: recommendation, alternatives, pros/cons, risks, consequence, impact tags) and Annex B (R0: Docker builder image as the reference toolchain, rustup optional locally, workspace skeleton, worker image, CI job, 7 experimental proofs kept as tests, exit criteria) appended to the review. Nothing started; awaiting the 13 validations.
## 2026-10-13 (Rust R0 delivered)
Owner go (BDR-045): chore/step-08-files-analysis finished into develop (Annex C), feature/rust-r0-foundation. Built R0 through the Docker builder only: workspace (core, wasm, worker), proofs (float_roundtrip finding, trap/fresh-instance behaviour, musl + libwebp 1.6.0, seccomp 7 syscalls derived by strace on the release binary, CSP 'wasm-unsafe-eval' proven in Chrome 156, 225 KB / 85 KB gzip module), packages/core-wasm (14 proofs), CI (rust job, worker test stage, advisories nightly, manual release-images), docs (RUST.md, DEPLOY digest deploy prepared, CLAUDE/WORKSPACE/TEST_STRATEGY/CHANGELOG). Gates: verifier ×3 (oracle gaps only, CONFORME in substance), security ×3 (1 HIGH + mediums fixed; my first fixes introduced 3 mediums, fixed; follow-ups documented). Root `pnpm check` 777 tests, live deployment untouched, frontend loads nothing. BDR-046, LRN-049, EVAL-017. Stop: owner validation of R0; R1 not started; release 0.2.0 pending.
+13
View File
@@ -362,3 +362,16 @@ Apply: every future Supabase auth surface (Step 14 gating, invitations), any pgT
Context: Rust review inventories. Facts found: `documentSha256` = sha256 of Prettier-formatted text (only compared at runtime → mismatch = rebase, not loss); coordinates = raw JS number strings (serde_json prints `1523.0`); attachment = insertion-order JSON, snapshot = Zod schema order; `.optional()` absent vs nullable `null` (serde-wasm-bindgen json preset writes null → breaks readers); text x positions = AFM widths + kerning pairs; dates = ICU; `toUpperCase`, `/\s+/`, non-fatal TextDecoder, last-dup-key JSON.parse, Zod `.int()` accepting 2.0. The 39 PDF goldens record draw calls (spies), never bytes → a Rust engine with a draw-log layer can match them exactly; byte goldens (xmldom XMLSerializer, Prettier files) cannot be matched by another stack → compare parsed trees or regenerate by decision.
Apply: any port (R2–R5): freeze unpinned outputs with the old code (LRN-046), write the contract list with a fixture each, decide lenient-vs-strict per item, keep an N-1 reader test permanently. Challengers found 2 blockers rev 1: deletion schedule vs surviving consumers (name the last consumer per module), and deployment facts (CSP without 'wasm-unsafe-eval', VPS compiling images) → check the serving path before the language.
## LRN-049 | 2026-10-13 | R0 experimental findings: serde_json fast float path, wasm traps and the ESM cache, RLIMIT_CPU soft=hard, minimal rustup profile, traced seccomp set
Facts proven on the real environment (docs/development/RUST.md §5), each one changed a design assumption of the review:
- `serde_json` default float parsing is a fast path one ULP off on some inputs (`1715.4285714285713` → `1715.428571428571`): enable `float_roundtrip` whenever bytes must match `JSON.stringify`; the formatter alone (`js_number`) is not enough.
- wasm32 = panic=abort: a panic and a shadow-stack overflow both surface as `WebAssembly.RuntimeError`; after a stack-overflow trap the next call fails in the allocator (deterministic), after a plain panic the instance still answers with a never-unwound stack pointer (silent) → never reuse an instance after any trap; the wasm-bindgen glue caches its instance in a module variable and `import()` of the same URL returns it → a fresh instance needs a new module record (cache-busting query) or a new Worker.
- `setrlimit(RLIMIT_CPU, soft, hard)` with soft == hard kills with SIGKILL (hard checked first), indistinguishable from a parent deadline kill → soft two seconds below hard gives SIGXCPU.
- Official `rust:` images install rustup's minimal profile: `rustup component add rustfmt clippy` in the image, else every container downloads them (rust-toolchain.toml lists the components).
- A single-threaded musl static decode (image/zune-jpeg/png/image-webp + libwebp encode) makes exactly 7 syscalls after start: read write mmap munmap mremap exit_group getrandom; mallocng used neither brk nor mprotect, no futex. Derive with strace after a stderr marker, compare in the build, keep an explicit abort-path set (rt_sigaction rt_sigprocmask getpid gettid tkill tgkill exit) proven by the RLIMIT_AS → SIGABRT test.
- Unoptimised debug decoders blow a 10 s CPU budget on 4096² inputs: `[profile.dev.package."*"] opt-level = 3`.
- cargo-deny 0.20: a versionless path dependency is a "wildcard" (`allow-wildcard-paths = true`) and a workspace-inherited path dependency triggers an `unresolved-workspace-dependency` bug (declare `path = "../core"` per crate).
- CSP: `'wasm-unsafe-eval'` is required for `WebAssembly.instantiateStreaming` under `script-src 'self'` and allows nothing else (inline scripts still blocked); proven with the production nginx image + headers in headless Chrome 156.
Apply: R1 (worker loop keeps the same child), R2 (browser loader: fresh-instance mechanism, float_roundtrip stays on, CSP line at activation), any future "it should work" assumption about a runtime → run it in the real environment first.
+6 -4
View File
@@ -142,10 +142,12 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
- [x] R0.4 crates/wasm: facade (core_version, call, panic_for_test, recurse_for_test); build 224,713 B raw / 85 KB gzip (float_roundtrip on), instantiate ≈ 3 ms in Node
- [x] R0.5 crates/worker: plain main, `decode` child (env check → rlimits AS 512 MiB / CPU 10 s soft, 12 s hard → seccomp allowlist TSYNC kill) → image decode with Limits → orientation → resize_to_fill 512 → libwebp-sys lossy; parent `Spawner` (env_clear, 512 KiB output cap, deadline, signal classification) + `webp_check` (RIFF allowlist, dims, no metadata); `run --check`, `version`; test-hooks (fixtures, violate, flood, alloc, spin, print-allowlist); Dockerfile alpine musl → test stage (sandbox tests + strace derivation) → distroless/static:nonroot
- [x] R0.6 packages/core-wasm: loader (lazy, Node bytes), generated types, vitest proofs (facade, serialisation byte-equal with JSON.stringify on 4 real documents + number cases + Zod readers, panic/stack traps, size budget) skipped with a notice when pkg/ is absent
- [ ] R0.7 proofs recorded with numbers in RUST.md: panic=abort + fresh instance path; float_roundtrip finding; musl static + libwebp ≥ 1.6 build; seccomp allowlist derived by strace and proven on the corpus; CSP with/without 'wasm-unsafe-eval' under the production nginx config (headless Chromium); size/instantiate; runner cache (not measurable here: documented as pending on the forge)
- [ ] R0.8 CI: job `rust` (builder image, fmt/clippy/test/deny, wasm build, types drift), worker test stage + runtime image build in the docker job, nightly advisories, manual `release-images` workflow (digest output, no auto deploy)
- [ ] R0.9 docs: docs/development/RUST.md, DEPLOY.md (prepared: pull by digest, CSP line, gzip wasm, rollback = previous digest; live deployment unchanged), WORKSPACE.md, packages/README.md, CLAUDE.md (stack, map, commands), TEST_STRATEGY.md, CHANGELOG
- [ ] R0.10 gates: `pnpm check` (existing suites untouched), verifier contract, security-auditor; memory (BDR/LRN/EVAL/journal); stop for owner validation, R1 not started
- [x] R0.7 proofs recorded with numbers in RUST.md §5: panic=abort traps + stack-trap instance unusable + fresh instance via cache-busting import; serde_json `float_roundtrip` finding (byte parity on 4 real documents + number corpus, Zod readers); musl static + libwebp 1.6.0 (image 8.7 MB, read-only, caps dropped); seccomp allowlist derived by strace on the release binary = 7 syscalls + 7 abort path, 13/13 sandbox tests; CSP: refused with `script-src 'self'`, OK with `'wasm-unsafe-eval'` (Chrome 156, nginx 1.31, `docker/csp-proof/`); wasm 224,713 B raw / 85 KB gzip / ≈ 3 ms; runner cache NOT measurable here (pending on the forge, §5.7)
- [x] R0.8 CI: job `rust` (builder image, fmt/clippy/test/deny, types drift, wasm build), `check` builds the wasm then runs the Node pipeline, `docker` builds the worker test stage + runtime started read-only/cap-drop, nightly advisories, manual `release-images` hardened after the security audit (typed input via env, advisories gate, ephemeral buildx builder, env secrets + per-job DOCKER_CONFIG + logout, main/develop only, pinned checkout; no deploy)
- [x] R0.9 docs: RUST.md, DEPLOY.md (prepared digest deploy + CSP note, live procedure untouched), WORKSPACE.md, packages/README.md, CLAUDE.md (stack, map, commands, workflow), TEST_STRATEGY.md, CHANGELOG
- [ ] R0 follow-ups from the security audits (R1 unless stated): base images and the buildkit engine image by digest (Step 27); verification decode of the child's output in a second sandboxed child; R1 treats Signalled/Timeout/OutputOverflow as rejections (content never alerts) and logs the child's stderr escaped only; negative tests for tkill(self, SIGTERM) and tgkill; iCCP fixture with image data so the allocation budget is seen firing; WebP structure residuals (duplicate ALPH, ALPH + VP8L, VP8X flags); dedicated release runner label that never runs PR jobs (Step 27); `actions/*@v4` in ci.yml pinned by commit (Step 27); drop `--features proofs` from docker/wasm-build.sh at R2
- [x] R0.10 gates: root `pnpm check` 777 tests green (existing suites untouched); verifier ×3 (CONFORME in substance, oracle fixes only); security ×3 (HIGH shell injection + 6 MEDIUM fixed; 3 MEDIUM introduced by the first fixes fixed; 2 MEDIUM of pass 3 fixed by the secret-less gates job); memory BDR-046, LRN-049, EVAL-017, journal. STOPPED for owner validation (2026-10-13); R1 not started
- [ ] R0 owner points: confirm the `[gated 2026-10-13]` SAFETY markers on the worker's unsafe blocks (placed by Claude at the auditor's request); decide the registry (forge Gitea registry recommended) and set REGISTRY/REGISTRY_USER/REGISTRY_TOKEN secrets before the first `release-images` run; first run on the forge records the runner numbers (RUST.md §5.7) and validates the workflow syntax (job-level env, GITHUB_ENV path)
- [ ] R1 = Step 8 worker in Rust (replaces 8.2 below): decode child dispatched before any thread, empty env, RLIMIT_AS/CPU, traced seccomp allowlist, `image` + `moxcms` + `libwebp-sys` in the child, parent RIFF-header check only, 512 KiB output cap, per-stage deadlines, service key
- [ ] R2 domain core (before Step 9): sheet/report/archive contracts, `SheetHandle` + `SheetIndexData`, reducers, Finding builder, vertebra selection, publish gate; wasm facade (JSON strings, ts-rs types, poison + reinstantiate, lazy init); app switched; 3 envelopes frozen first; N-1 reader test; draft quarantine; conformance fixtures + generated pgTAP; `packages/domain` frozen for engine/pipeline
- [ ] R3 anatomy core: tree, one sanitiser, serialiser, layer renderer, `compose_view_svg`; app canvas switched; composed views frozen as trees first