From f41dd59b2c74d8307a65776236b6d8a4d0cea9a2 Mon Sep 17 00:00:00 2001 From: bchanot Date: Sun, 18 Oct 2026 17:41:12 +0200 Subject: [PATCH] chore(memory): BDR-058 R4.2 validated and R4.3 gates; LRN-061, LRN-062; R4.3 subtasks --- .claude/memory/decisions.md | 7 +++++++ .claude/memory/journal.md | 1 + .claude/memory/learnings.md | 12 ++++++++++++ .claude/tasks/TODO.md | 7 ++++++- 4 files changed, 26 insertions(+), 1 deletion(-) diff --git a/.claude/memory/decisions.md b/.claude/memory/decisions.md index fc434640..c1dd564c 100644 --- a/.claude/memory/decisions.md +++ b/.claude/memory/decisions.md @@ -443,3 +443,10 @@ Context: R4.0 plan rev 2 (`.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md`: Decision: OD-1 module workers everywhere; import needs Firefox 114 (today it already needs DecompressionStream: Firefox 113, Safari 16.4); import support documented per browser apart from app support, from real checks. OD-2 incremental ExportSession in main instance: PDF + one decode ≈ 0.37 MB/view vs TS ≈ 3.8 MB/view (RGB planes held until save) vs two calls ≈ 0.91; offline export kept; cleanup on error/cancel. OD-3 JSON value budget in linear pre-scan + module `--max-memory`, thresholds from R4.2 reproducible measurements; not shown → stop before R4.3 import switch. OD-4 overwritten duplicate key holding lone surrogate refused, file untouched. OD-5 PDF crash/timeout `error_generic`, copy crash/timeout `error_core_unavailable`, kinds keep technical / resource / invalid apart. OD-6 J1 matrix exact, unrun device never reported validated. OD-7 ~5 MB fixtures, no real data in git. Alt rejected: classic iife worker for Firefox 102-113 (dev/prod split for dead versions); two-call export (all PNGs in linear memory); disposable export worker (asset fetched at export breaks offline, protocol cost); throwaway main-thread instance (glue retains instances, LRN-060); own lossless JSON reader (R2 rejection holds). Status: active. Order: plan merge → Safari download bugfix (owner validation) → feature/rust-r4-pdf-core → R4.1. + +## BDR-058 | 2026-10-18 | R4.2 validated: OD-3 thresholds fixed, R4.3 switch gated on large-file memory and leak series + +Context: R4.2 report (RUST.md §10.3, §10.4): O7 144/144, O9 38/38, O8 47 + 4 declared, browsers 666/678; security gate BLOCK(1) fixed (prescan count past depth cap) → PASS. Owner validation 2026-10-18. +Decision: `MAX_JSON_VALUES` = 400,000 (10 × CAP envelope 39,652) and `--max-memory` 1 GiB final for R4; existing depth / size / complexity / inflation caps kept, no raise without measurement. 1 GiB = safety net, never a tab memory guarantee. Leak criterion: first import = init cost, counted apart; then ≥ 5 consecutive imports + idle, recurring growth ≤ 10 MiB, in Chromium + Firefox + WebKit, GC fluctuations repeated and explained; global peaks kept beside. Three distinct subjects: no leak / low peak / ceiling guarantee. Before the app import switch: 20–50 MiB files (WebKit 324 MiB vs TS 76) must be fixed (fewer ArrayBuffer copies, fewer linear copies, early release, no retained refs, transfer / streaming when it pays) and re-measured on 3 engines, or an explicit size limit with controlled refusal proposed with numbers; no switch before that is solved or decided. OD-4 refusals accepted (L1); no legitimate corpus file may be refused; never rewrite source files or drop bilan data on error; oracles unchanged. Worker technical error / resource overflow / timeout never presented as proof of a corrupt PDF. TS reader deleted only after parity + non-regression. WebKit one-off worker `error`: not blocking, keep trace, repetition tests, never mask `error` events. J1 on real devices announced with "TEST FONCTIONNEL DISPONIBLE"; headless ≠ Safari/iPhone validation. +Alt rejected: 1,000,000 values (25 × CAP; more headroom, weaker bound). Switch import before the large-file fix. +Status: active. R4.3 started 2026-10-18 on feature/rust-r4-pdf-core; no R4 merge into develop, no R5, no 0.2.0, no deploy. diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index 8f2e2f1e..0cbe1087 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -285,3 +285,4 @@ Owner validated R4.0 plan with conditions (BDR-057, plan §A.6, LRN-059/060). R3 - 2026-10-18 R4.1 freeze done on feature/rust-r4-pdf-core (eff5405..4439fd2): O1-O13 generators + oracles committed, ledger 299 rows (150 ported / 28 replaced / 88 kept / 33 deleted), browser harness scripts/browser (Chromium 156, Firefox 157, WebKit 27.2, 4 GiB RSS watchdog = not guaranteed limit). TS baselines: CAP export stopped by ceiling in all 3 engines, L peak content 0.8-1.35 GB, import 226 files same outcome everywhere; O9 browsers = Node on 26 rows. Repo +5.92 MB stored vs ~5 MB approved. Frozen step6/r2/r3 goldens untouched. .gitleaks.toml narrow allowlist for compare.json. Next: checkpoint report to owner, then R4.2. - 2026-10-18 owner validated R4.1 (fixtures 5.92 MB kept incl. real raster; gitleaks exception OK if narrow; decoy-first OD-4 kept in register, risk to explain before R4.3; UPNG/text oracles kept; headless OK for R4.1). Go R4.2 (reader, inflate, JSON pre-scan, UTF-8 parity, wasm facade, memory protections). Hard stop end of R4.2 before any import switch; budgets whole-browser; gains compared on identical perimeters. No merge, no R5, no release/prod. - 2026-10-18 R4.2 done on feature/rust-r4-pdf-core (21d34fb..): pdf::read (scan, strings, stamp, inflate, probe), json_js, import_pdf, SheetHandle.import_pdf; O7 144/144, O9 38/38, O8 51 (4 declared: 3 OD-4 lone surrogate in overwritten dup value, 1 OD-3 values-1m), reference PDFs all import, through wasm too. OD-3: MAX_JSON_VALUES 400,000 (10 × CAP envelope 39,652), rescue copy guarded, --max-memory 1 GiB trap proven; hostile 16 MiB: 784 MiB → 16 MiB linear. Security gate BLOCK(1) (count stopped past depth cap → bypass) fixed 593893a, re-gate PASS. Browser run 3 engines: 666/678 agree, W4 max 68 MiB, W2 46 ms; open: after-GC residue 11.7 MiB first import (one-time), worker holds file twice on 50 MiB inputs (WebKit 324 MiB). Module 422 KB gzip. STOP before R4.3: owner to confirm thresholds. +- 2026-10-18 owner validated R4.2 (BDR-058): thresholds 400,000 / 1 GiB final; R4.3 go on feature branch; app import switch gated on the 20–50 MiB file memory fix (WebKit 324 vs 76 MiB) re-measured on 3 engines, and leak series (init apart, ≥ 5 imports, ≤ 10 MiB recurring). LRN-061, LRN-062 written. Next: R4.3a bytes API + streaming read, worker protocol, HomeRoute, CSP proof, series measurements. diff --git a/.claude/memory/learnings.md b/.claude/memory/learnings.md index ac0d7a1d..7195ea68 100644 --- a/.claude/memory/learnings.md +++ b/.claude/memory/learnings.md @@ -466,3 +466,15 @@ Apply: Rust ports of JS readers of untrusted text (R4 PDF reader, Step 13 upload - `--target web` glue keeps the instance in module scope (`let wasm`, `__wbg_finalize_init`); ES module records are never unloaded, so each fresh glue import (`?instance=n`) keeps its old instance and memory (BLK-012: dead instances retained until reload). Linear memory never shrinks. - Memory-heavy one-shot work: reuse the long-lived instance (high-water kept until reload) or run in a Worker terminated after (everything freed). A "disposable instance" on the main thread is a leak, not a release. Apply: R4 import worker and export placement, any future in-browser wasm batch job. + +## LRN-061 | 2026-10-18 | A multi-budget pre-scan must keep every budget sound, even once another check already refuses + +- R4.2 `json_js::prescan` returned early at depth > 64 with the partial value count; the PDF probe checked both verdicts, the rescue copy checked values only → a 16 MiB copy nested 65 deep reported 66 values, passed the budget, built 792 MiB (security gate BLOCK, fixed 593893a: count runs to the end, depth verdict kept). +- Rule: one linear pass serving several guards finishes every measure; an early return for one verdict silently disarms the callers that rely on the others. Controls must never neutralise each other; a refusal path stays bounded in time and memory. +Apply: any shared pre-scan / lexer feeding several caps (R4.3 worker, R5 pipeline, Step 13 uploads); test each guard alone with the other verdict already failing. + +## LRN-062 | 2026-10-18 | Non-finite numbers: follow the JSON actually serialised, not the JS type before serialisation + +- `parseVersioned` tested `typeof version === 'number'` on `JSON.parse` output: `1e999` → `Infinity`, a number, so the probe accepted; `JSON.stringify` then wrote `null`, so the core saw no version → `no_attachment`. A Rust port that rewrote overflow → `null` before the probe gave `attachment_stripped` (3 O8 rows). +- Fix: probe on a copy where overflowing tokens stay finite numbers (`rewrite_overflow_numbers_as_finite`, `0`), hand the `null` form to the core. Same family as LRN-059 (overflow, lone surrogates): every JS parse → stringify hop is a transformation to reproduce step by step, in order. +Apply: porting any JS reader whose acceptance test ran between parse and stringify (R4 writer inputs, R5 pipeline, server validation). diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index b0042f39..62498937 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -223,7 +223,12 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] R4.2e (Node part done: cap 400,000 = 10 × CAP envelope 39,652 values, rescue copy guarded, 1 GiB ceiling proven; browser run pending) OD-3: measurement harness (fresh wasm instance per case, time, linear memory; Node) on the hostile classes (16 MiB token-dense attachment, 16 MiB rescue copy, 50 MiB file, 16 MiB inflation, 32 candidates) + browser harness (worker page, content-process RSS, same perimeter as the TS import baseline); value budget = 10 × largest legitimate archive; `--max-memory`; W1 module size; thresholds table for the owner - [ ] R4.2e follow-ups for R4.3: worker holds the File ArrayBuffer and its linear copy together (50 MiB file: WebKit content peak 324 MiB vs 76 TS) → release the buffer after the glue copy or read in chunks; after-GC residue 11.7 MiB first import (one-time, +1.9 then −0.2 on repeats) → owner decides the §B.3 criterion; `scripts/browser/lib/import-inputs.mjs` shortcut: copy of import-baseline's listing, fold into one module when import-baseline is retired (R4.8) - [x] R4.2f (done 2026-10-18; RUST.md §10.3 + §10.4; STOP for the owner before R4.3) record: RUST.md §10.3 (reader, parity, budgets, thresholds, residual risks), decoy-first (OD-4) risk explanation + proof it blocks no legitimate Faunosteo document, ledger `ported` targets exist, suites green (rust:check, wasm:build, check); STOP for the owner before R4.3 - - [ ] R4.3 import worker + rescue copy (B.1.5, OD-1 applied, browser proof per slice from here) + - [ ] R4.3 import worker + rescue copy (B.1.5, OD-1 applied, browser proof per slice from here); owner go 2026-10-18 (BDR-058): switch gated on large-file memory fix + leak series + - [ ] R4.3a bytes API: `PdfBytes` sink (reserve ≤ 50 MiB, push chunks) + `SheetHandle.import_pdf_bytes` / `restore_draft_bytes` (strict UTF-8, one BOM, OD-3 guard) so the worker streams `file.stream()` into linear memory, one copy, no JS ArrayBuffer of the file, no JSON wrapping of the copy; Node + browser re-measure of 20–50 MiB files on 3 engines (same perimeter as R4.2e) + - [ ] R4.3b worker + client (`apps/app/src/core/worker/`): protocol v1 (B.1.5 items 1–7), controller module-level, timers (start, task 20 s paused when hidden), settle-once, terminate always, failure classification (worker_unavailable / crashed / timeout / read), compiled Module posted, static glue, prefetch after editor load; vitest in-process via spawn seam; no `error` event masked + - [ ] R4.3c HomeRoute on the controller (routing on first KiB + size pre-checks unchanged), rescue copy through the worker, B.1.8 messages (OD-5), hydrateImport from the core output; route tests per throwable and phase, settle-once, result after timeout, error after result, unmount, second pick, main runtime untouched; message test over O2/O3/O8/hostile; 612 leaves through the worker; v2/v3 imports preserved + - [ ] R4.3d browser proofs on the production build (SHEET_BUNDLE=draft dist, serve.mjs CSP): import-baseline on the app (3 engines) vs TS baseline; CSP with and without 'wasm-unsafe-eval'; leak series per owner §2 (init cost apart, ≥ 5 imports + idle, ≤ 10 MiB recurring, GC fluctuations repeated); peaks kept; WebKit repetition on dense documents (error events captured) + - [ ] R4.3e record RUST.md §10.5, DEPLOY note (worker CSP, Firefox 114 line per OD-1), PDF_IMPORT_HARDENING §5 results, J1 procedure; announce "TEST FONCTIONNEL DISPONIBLE" with the 7-point checkpoint (BDR-058) - [ ] J1 owner checkpoint (plan A.4) - [ ] R4.4 layout → draw log - [ ] R4.5 serialiser (pdf-writer, self-check, from_envelope)