From 0d2c687b795c837451baf69719b0c4d0ae8aa742 Mon Sep 17 00:00:00 2001 From: bchanot Date: Sun, 18 Oct 2026 16:16:31 +0200 Subject: [PATCH] =?UTF-8?q?docs(docs):=20RUST.md=20=C2=A710.3=20R4.2=20rea?= =?UTF-8?q?der=20record=20(parity,=20OD-4=20explanation,=20OD-3=20threshol?= =?UTF-8?q?ds,=20Node=20budgets)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/tasks/TODO.md | 10 +-- docs/development/RUST.md | 134 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 139 insertions(+), 5 deletions(-) diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index fca890ec..d9905374 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -216,11 +216,11 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] R4.1e browser harness (scripts/browser, Chromium + Firefox + WebKit, 4 GiB RSS watchdog): O9 browser verdicts, O12 TS baselines (export demo/L/CAP, import O2 + hostile) - [x] R4.1f O10 report:compare outputs, test ledger (crates/core/tests/pdf/LEDGER.tsv), freeze record (RUST.md §10 draft, fixtures README) - [ ] R4.2 reader (cargo + wasm, differential, OD-3 thresholds confirmed with owner before R4.3); owner go 2026-10-18 (R4.1 validated: fixtures kept, gitleaks exception accepted if narrow, decoy-first kept in register + risk explained before R4.3, headless accepted; stop at end of R4.2, budgets = whole browser, same perimeters as TS when a gain is claimed) - - [ ] R4.2a `crates/core/src/pdf/read/{scan,strings,stamp}.rs`: scan.ts port (header, typed dicts, stream bounds, Filespec names, /Keywords stamp, decodePdfString; JS regex semantics explicit); cargo tests on O7 findEmbeddedFiles / hasArchiveStamp / decodePdfString rows (155 inputs incl. semantics rows) - - [ ] R4.2b `pdf/read/inflate.rs` (miniz_oxide, zlib framing, DecompressionStream verdicts): cargo tests on O9 (26 rows) + O7 flate inputs; Node verdict pinned, browsers agree (inflate.browsers.json) - - [ ] R4.2c `pdf/json_js.rs`: `{` probe after BOM/whitespace, depth 64 + value-count pre-scan (OD-3 budget hook), lossy UTF-8 + one BOM (TextDecoder parity), overflow → null rewrite, lone-surrogate-neutralised probe through parse_json, versioned check; unit tests + text parity cases - - [ ] R4.2d `pdf/read/probe.rs` locate_attachment (shared 16 MiB budget, `break` on oversized, cancelled inflation not charged), `pdf/import.rs` (read → session::import_archive), op `pdf.import` + `SheetHandle.importPdf(bytes)` on the wasm facade, ts-rs `PdfImportKind`; cargo tests on O7 locate/readAttachedDocument/readReportArchive rows, O8 differential (51), Node tests through the wasm build on O2/O3/O8/hostile - - [ ] R4.2e OD-3: measurement harness (fresh wasm instance per case, time, linear memory; Node) on the hostile classes (16 MiB token-dense attachment, 16 MiB rescue copy, 50 MiB file, 16 MiB inflation, 32 candidates) + browser harness (worker page, content-process RSS, same perimeter as the TS import baseline); value budget = 10 × largest legitimate archive; `--max-memory`; W1 module size; thresholds table for the owner + - [x] R4.2a `crates/core/src/pdf/read/{scan,strings,stamp}.rs`: scan.ts port (header, typed dicts, stream bounds, Filespec names, /Keywords stamp, decodePdfString; JS regex semantics explicit); cargo tests on O7 findEmbeddedFiles / hasArchiveStamp / decodePdfString rows (155 inputs incl. semantics rows) + - [x] R4.2b `pdf/read/inflate.rs` (miniz_oxide, zlib framing, DecompressionStream verdicts): cargo tests on O9 (26 rows) + O7 flate inputs; Node verdict pinned, browsers agree (inflate.browsers.json) + - [x] R4.2c `pdf/json_js.rs`: `{` probe after BOM/whitespace, depth 64 + value-count pre-scan (OD-3 budget hook), lossy UTF-8 + one BOM (TextDecoder parity), overflow → null rewrite, lone-surrogate-neutralised probe through parse_json, versioned check; unit tests + text parity cases + - [x] R4.2d `pdf/read/probe.rs` locate_attachment (shared 16 MiB budget, `break` on oversized, cancelled inflation not charged), `pdf/import.rs` (read → session::import_archive), op `pdf.import` + `SheetHandle.importPdf(bytes)` on the wasm facade, ts-rs `PdfImportKind`; cargo tests on O7 locate/readAttachedDocument/readReportArchive rows, O8 differential (51), Node tests through the wasm build on O2/O3/O8/hostile + - [~] R4.2e (Node part done: cap 400,000 = 10 × CAP envelope 39,652 values, rescue copy guarded, 1 GiB ceiling proven; browser run pending) OD-3: measurement harness (fresh wasm instance per case, time, linear memory; Node) on the hostile classes (16 MiB token-dense attachment, 16 MiB rescue copy, 50 MiB file, 16 MiB inflation, 32 candidates) + browser harness (worker page, content-process RSS, same perimeter as the TS import baseline); value budget = 10 × largest legitimate archive; `--max-memory`; W1 module size; thresholds table for the owner - [ ] R4.2f record: RUST.md §10.3 (reader, parity, budgets, thresholds, residual risks), decoy-first (OD-4) risk explanation + proof it blocks no legitimate Faunosteo document, ledger `ported` targets exist, suites green (rust:check, wasm:build, check); STOP for the owner before R4.3 - [ ] R4.3 import worker + rescue copy (B.1.5, OD-1 applied, browser proof per slice from here) - [ ] J1 owner checkpoint (plan A.4) diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 50f8e5dc..16ad2a7a 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -1047,3 +1047,137 @@ ArrayBuffer memory); memory after an import settles (the §B.3 check belongs to the R4 worker proof); real browsers and phones (J1 and the owner's iPhone run). Seen on every page: the known `img-src` refusal of the footer's external credit logo; no script or wasm violation. + +### 10.3 R4.2: the reader in Rust (2026-10-18, no import switched yet) + +What runs in Rust (`crates/core/src/pdf`, feature `pdf-read`, enabled by the wasm +facade, never by the worker): the byte scan of `scan.ts` (`read/scan.rs`, +`strings.rs`, `stamp.rs`, the JavaScript regex semantics reproduced by hand +and pinned by the `semantics.*` rows of O7), the inflation on miniz_oxide +(`read/inflate.rs`, non-wrapping output buffer growing to the limit plus one, +trailing bytes refused, Adler-32 checked: the verdicts of +`DecompressionStream`), the JavaScript-path side effects of the probe +(`json_js.rs`: TextDecoder parity with one BOM dropped, overflow to `null`, +the lone surrogate neutralised for the probe only, the depth and value +pre-scan), the probe itself (`read/probe.rs`, one 16 MiB budget shared by the +candidates, a cancelled inflation not charged) and `pdf::import::import_pdf`, +which hands the text to the unchanged R2 session import. On the facade: +`SheetHandle.import_pdf(bytes)` with the kinds of `PdfImportKind` (ts-rs). +Nothing in the application calls it yet (R4.3). + +Parity (every suite green on the committed oracles, `cargo test -p +faunosteo-core --features pdf-read` and the core-wasm Node suites): + +| Oracle | Rust check | Result | +|---|---|---| +| O7 `findEmbeddedFiles`, `hasArchiveStamp`, `decodePdfString` | `pdf_scan` | 144 PDF inputs, 178 candidates, 19 string rows, all equal | +| O9 and the O7 flate rows | `pdf_inflate` | 26 + 12 verdicts equal to Node; the browser columns agree on every row | +| TextDecoder lossy decode | `pdf_json_js` on `fixtures/pdf/r4/utf8-lossy.json` (400 cases, `scripts/r4-utf8-oracle.mjs`) | 0 difference | +| O7 `locateAttachment`, `readAttachedDocument`, `readReportArchive` | `pdf_import` | 144 inputs equal (candidate, name, offset, bytes digest, parsed-object digest; origin, counters, snapshot rejection, report digest) | +| O8, the full R3 import path, 51 PDFs | `pdf_import` natively and `pdf-import.test.ts` through the shipped module | 47 equal, 4 declared differences below | +| Reference PDFs (Step 5, Step 6, R3, O2) | `pdf-import.test.ts` | every one imports | + +Declared differences, all on hostile files (plan §A.3): + +- OD-4, three O8 cases (`surrogate-content-overwritten`, + `surrogate-version-overwritten`, `two-files-decoy-first`): the text holds a + lone UTF-16 surrogate escape in a value that a later duplicate key + overwrote. `JSON.parse` kept the last value, so the stringified object the + core received no longer held the escape and R3 imported the file; the core + refuses the text whole (`lone_surrogate`), nothing altered. Why the refusal + is right: a Rust string cannot carry a lone surrogate, so reading such a + text means either altering it silently (L1 forbids it) or reading only the + parts that happen to survive duplicate-key resolution, which is reading a + document other than the one in the file. In `two-files-decoy-first` R3 did + not import the real envelope at all: it imported the decoy (conclusion + « Leurre ») that sorted first. No legitimate Faunosteo document is + affected: every PDF the product wrote comes from `JSON.stringify` of one + object, which never emits a duplicate key, and a lone surrogate in a kept + value is already refused by R3 (`error_lone_surrogate`); the proof is the + O2, O3, O7 and Step 5 reference set above, imported identically, and the + O8 corpus where the only lone-surrogate cases that import in R3 are the + three duplicate-key constructions. +- OD-3, one O8 case (`values-1m`): over the value cap (below). It is skipped + like a candidate nested too deep, so the kind follows the stamp + (`attachment_stripped` here, as `depth-65` already answers in R3). +- The `archiveSha256` column of the O7 `readReportArchive` rows is not + compared: the frozen TS domain stringified its Zod output, keys in schema + order, where the core keeps the document's order (R2); the R3 application + stored the core's form, which O8 pins and matches. + +Memory protections (OD-3), measured in the shipped module, one fresh +instance per case (`packages/core-wasm/tests/pdf-import.budget.test.ts`, +`R4_BUDGET_ALL=1` for the slow classes; values from `cargo test --test +pdf_budget -- --nocapture`): + +| Document | Bytes | Values (`json_js::prescan`) | +|---|---|---| +| demo fr logos (R3 export) | 305,938 | 2,559 | +| v2full fr logos (R3 export) | 298,035 | 2,232 | +| 612 leaves (O7 export) | 739,316 | 23,803 | +| L envelope (60 observations, 120 views, logos) | 403,016 | 6,740 | +| CAP envelope (500 observations, 1,000 views, logos) | 1,164,658 | 39,652 | +| hostile `values-1m.pdf` | 2,006,973 | 1,000,364 | + +The parser's tree costs about 100 bytes of linear memory per value (the +final `Vec` doubling included). Cap off, a 16 MiB token-dense document +(8.4 million values) takes 784 MiB of linear memory and 0.9 s in Node, what +BLK-012 observed; every shape tried (numbers, empty strings, nesting at the +depth cap, the rescue copy) lands at the same 784 MiB, under the 1 GiB +ceiling, so the cap is the guard and the ceiling the backstop. + +Thresholds set (to confirm before R4.3): + +- `json_js::MAX_JSON_VALUES = 400,000`, ten times the CAP envelope (the plan's + rule), 17 times the 612-leaf export; applied in the attachment probe and in + `draft.restore` (the rescue copy and the tab's draft: an over-budget text is + `invalid`, set aside verbatim, never dropped). Worst case left: about 40 MiB + of tree on top of the file copy. +- `--max-memory` 1 GiB on the module (`.cargo/config.toml`): an allocation + past it traps (`allocate_for_test` on the proofs build: 256 MiB returns, + 1,100 MiB throws `RuntimeError`), and `CoreRuntime` poisons and re-creates + the instance as for any trap. The ceiling applies to the export too; the + R4.6 measurements must stay well below it (CAP export estimated at about + 0.4 GB in §B.4). + +Per class, cap on (Node, i7-11700B, shipped module; "linear" is the +instance's growth, which never shrinks; "RSS" the Node process): + +| Case | Outcome | ms | Linear MiB | Node RSS MiB | +|---|---|---|---|---| +| L envelope (394 KB) | ok | 11 | 0.6 | 0.4 | +| CAP envelope (1,138 KB) | ok | 40 | 10.0 | 14.9 | +| demo fr logos (R3 export, 237 KB) | ok | 7 | 0.0 | 1.3 | +| 612 leaves (O7 export) | ok | 23 | 2.8 | 5.4 | +| values-1m (O8 hostile, 2 MB attachment) | attachment_stripped | 11 | 3.1 | 3.0 | +| 16 MiB dense numbers (8.4 M values) | attachment_stripped | 103 | 16.1 | 16.0 | +| 16 MiB dense strings (5.6 M values) | attachment_stripped | 85 | 16.1 | 16.0 | +| 16 MiB at depth 64 | attachment_stripped | 92 | 16.1 | 16.0 | +| 16 MiB dense numbers as a rescue copy | invalid | 152 | 48.1 | 80.1 | +| 16 MiB inflation (zeros, not JSON) | attachment_stripped | 14 | 31.3 | 31.0 | +| 16 MiB + 1 inflation (bomb, refused) | attachment_stripped | 19 | 63.4 | 47.0 | +| 32 bombs (17 MiB each) before the attachment | attachment_stripped | 181 | 63.9 | 47.5 | +| 50 MiB file, attachment at the end | no_attachment | 73 | 50.0 | 50.0 | + +Same classes, cap off: values-1m 109 ms and 98 MiB; the three 16 MiB +documents 0.6 to 0.9 s and 784 MiB; the rescue copy 0.4 s and 792 MiB. W4 +(≤ 256 MiB linear on every hostile class) holds with the cap on; the 16 MiB +file copy and the inflation buffer dominate what is left. + +Module: 1,251,849 B raw, 421,940 B gzip -9 (W1: +26 KB over R3, budget +600 KB, report line 550 KB). + +Residual risks and open points for the owner (R4.2 checkpoint): + +- The thresholds above are set from fixtures at the collection caps; a report + far denser than CAP per observation (hundreds of findings on each of 500 + observations) would be refused as a hostile file. 400,000 values is 17 + times the densest export seen (612 leaves on one observation). +- The ceiling is a module-wide constant: it bounds the import worker and the + main instance alike. If an R4.6 export measurement approaches it, the value + must be revisited, never the cap. +- The Node numbers measure the module, not a browser: the browser run of the + same classes (content-process RSS, worker terminated, GC) is in 10.4. +- The probe parses the candidate twice (version check, then the session), + as the TypeScript did; the peak is one tree since the first is dropped. A + single parse would need the gated session reader to take a value.