docs(docs): RUST.md §10.3 R4.2 reader record (parity, OD-4 explanation, OD-3 thresholds, Node budgets)

This commit is contained in:
bchanot committed 2026-10-18 16:16:31 +02:00
1 parent ebcae06ac1
commit 0d2c687b79
2 files changed
+139 -5

No files matched your search

+5 -5
View File
@@ -216,11 +216,11 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
- [x] R4.1e browser harness (scripts/browser, Chromium + Firefox + WebKit, 4 GiB RSS watchdog): O9 browser verdicts, O12 TS baselines (export demo/L/CAP, import O2 + hostile)
- [x] R4.1f O10 report:compare outputs, test ledger (crates/core/tests/pdf/LEDGER.tsv), freeze record (RUST.md §10 draft, fixtures README)
- [ ] R4.2 reader (cargo + wasm, differential, OD-3 thresholds confirmed with owner before R4.3); owner go 2026-10-18 (R4.1 validated: fixtures kept, gitleaks exception accepted if narrow, decoy-first kept in register + risk explained before R4.3, headless accepted; stop at end of R4.2, budgets = whole browser, same perimeters as TS when a gain is claimed)
- [ ] R4.2a `crates/core/src/pdf/read/{scan,strings,stamp}.rs`: scan.ts port (header, typed dicts, stream bounds, Filespec names, /Keywords stamp, decodePdfString; JS regex semantics explicit); cargo tests on O7 findEmbeddedFiles / hasArchiveStamp / decodePdfString rows (155 inputs incl. semantics rows)
- [ ] R4.2b `pdf/read/inflate.rs` (miniz_oxide, zlib framing, DecompressionStream verdicts): cargo tests on O9 (26 rows) + O7 flate inputs; Node verdict pinned, browsers agree (inflate.browsers.json)
- [ ] R4.2c `pdf/json_js.rs`: `{` probe after BOM/whitespace, depth 64 + value-count pre-scan (OD-3 budget hook), lossy UTF-8 + one BOM (TextDecoder parity), overflow → null rewrite, lone-surrogate-neutralised probe through parse_json, versioned check; unit tests + text parity cases
- [ ] R4.2d `pdf/read/probe.rs` locate_attachment (shared 16 MiB budget, `break` on oversized, cancelled inflation not charged), `pdf/import.rs` (read → session::import_archive), op `pdf.import` + `SheetHandle.importPdf(bytes)` on the wasm facade, ts-rs `PdfImportKind`; cargo tests on O7 locate/readAttachedDocument/readReportArchive rows, O8 differential (51), Node tests through the wasm build on O2/O3/O8/hostile
- [ ] R4.2e OD-3: measurement harness (fresh wasm instance per case, time, linear memory; Node) on the hostile classes (16 MiB token-dense attachment, 16 MiB rescue copy, 50 MiB file, 16 MiB inflation, 32 candidates) + browser harness (worker page, content-process RSS, same perimeter as the TS import baseline); value budget = 10 × largest legitimate archive; `--max-memory`; W1 module size; thresholds table for the owner
- [x] R4.2a `crates/core/src/pdf/read/{scan,strings,stamp}.rs`: scan.ts port (header, typed dicts, stream bounds, Filespec names, /Keywords stamp, decodePdfString; JS regex semantics explicit); cargo tests on O7 findEmbeddedFiles / hasArchiveStamp / decodePdfString rows (155 inputs incl. semantics rows)
- [x] R4.2b `pdf/read/inflate.rs` (miniz_oxide, zlib framing, DecompressionStream verdicts): cargo tests on O9 (26 rows) + O7 flate inputs; Node verdict pinned, browsers agree (inflate.browsers.json)
- [x] R4.2c `pdf/json_js.rs`: `{` probe after BOM/whitespace, depth 64 + value-count pre-scan (OD-3 budget hook), lossy UTF-8 + one BOM (TextDecoder parity), overflow → null rewrite, lone-surrogate-neutralised probe through parse_json, versioned check; unit tests + text parity cases
- [x] R4.2d `pdf/read/probe.rs` locate_attachment (shared 16 MiB budget, `break` on oversized, cancelled inflation not charged), `pdf/import.rs` (read → session::import_archive), op `pdf.import` + `SheetHandle.importPdf(bytes)` on the wasm facade, ts-rs `PdfImportKind`; cargo tests on O7 locate/readAttachedDocument/readReportArchive rows, O8 differential (51), Node tests through the wasm build on O2/O3/O8/hostile
- [~] R4.2e (Node part done: cap 400,000 = 10 × CAP envelope 39,652 values, rescue copy guarded, 1 GiB ceiling proven; browser run pending) OD-3: measurement harness (fresh wasm instance per case, time, linear memory; Node) on the hostile classes (16 MiB token-dense attachment, 16 MiB rescue copy, 50 MiB file, 16 MiB inflation, 32 candidates) + browser harness (worker page, content-process RSS, same perimeter as the TS import baseline); value budget = 10 × largest legitimate archive; `--max-memory`; W1 module size; thresholds table for the owner
- [ ] R4.2f record: RUST.md §10.3 (reader, parity, budgets, thresholds, residual risks), decoy-first (OD-4) risk explanation + proof it blocks no legitimate Faunosteo document, ledger `ported` targets exist, suites green (rust:check, wasm:build, check); STOP for the owner before R4.3
- [ ] R4.3 import worker + rescue copy (B.1.5, OD-1 applied, browser proof per slice from here)
- [ ] J1 owner checkpoint (plan A.4)
+134
View File
@@ -1047,3 +1047,137 @@ ArrayBuffer memory); memory after an import settles (the §B.3 check belongs
to the R4 worker proof); real browsers and phones (J1 and the owner's
iPhone run). Seen on every page: the known `img-src` refusal of the
footer's external credit logo; no script or wasm violation.
### 10.3 R4.2: the reader in Rust (2026-10-18, no import switched yet)
What runs in Rust (`crates/core/src/pdf`, feature `pdf-read`, enabled by the wasm
facade, never by the worker): the byte scan of `scan.ts` (`read/scan.rs`,
`strings.rs`, `stamp.rs`, the JavaScript regex semantics reproduced by hand
and pinned by the `semantics.*` rows of O7), the inflation on miniz_oxide
(`read/inflate.rs`, non-wrapping output buffer growing to the limit plus one,
trailing bytes refused, Adler-32 checked: the verdicts of
`DecompressionStream`), the JavaScript-path side effects of the probe
(`json_js.rs`: TextDecoder parity with one BOM dropped, overflow to `null`,
the lone surrogate neutralised for the probe only, the depth and value
pre-scan), the probe itself (`read/probe.rs`, one 16 MiB budget shared by the
candidates, a cancelled inflation not charged) and `pdf::import::import_pdf`,
which hands the text to the unchanged R2 session import. On the facade:
`SheetHandle.import_pdf(bytes)` with the kinds of `PdfImportKind` (ts-rs).
Nothing in the application calls it yet (R4.3).
Parity (every suite green on the committed oracles, `cargo test -p
faunosteo-core --features pdf-read` and the core-wasm Node suites):
| Oracle | Rust check | Result |
|---|---|---|
| O7 `findEmbeddedFiles`, `hasArchiveStamp`, `decodePdfString` | `pdf_scan` | 144 PDF inputs, 178 candidates, 19 string rows, all equal |
| O9 and the O7 flate rows | `pdf_inflate` | 26 + 12 verdicts equal to Node; the browser columns agree on every row |
| TextDecoder lossy decode | `pdf_json_js` on `fixtures/pdf/r4/utf8-lossy.json` (400 cases, `scripts/r4-utf8-oracle.mjs`) | 0 difference |
| O7 `locateAttachment`, `readAttachedDocument`, `readReportArchive` | `pdf_import` | 144 inputs equal (candidate, name, offset, bytes digest, parsed-object digest; origin, counters, snapshot rejection, report digest) |
| O8, the full R3 import path, 51 PDFs | `pdf_import` natively and `pdf-import.test.ts` through the shipped module | 47 equal, 4 declared differences below |
| Reference PDFs (Step 5, Step 6, R3, O2) | `pdf-import.test.ts` | every one imports |
Declared differences, all on hostile files (plan §A.3):
- OD-4, three O8 cases (`surrogate-content-overwritten`,
`surrogate-version-overwritten`, `two-files-decoy-first`): the text holds a
lone UTF-16 surrogate escape in a value that a later duplicate key
overwrote. `JSON.parse` kept the last value, so the stringified object the
core received no longer held the escape and R3 imported the file; the core
refuses the text whole (`lone_surrogate`), nothing altered. Why the refusal
is right: a Rust string cannot carry a lone surrogate, so reading such a
text means either altering it silently (L1 forbids it) or reading only the
parts that happen to survive duplicate-key resolution, which is reading a
document other than the one in the file. In `two-files-decoy-first` R3 did
not import the real envelope at all: it imported the decoy (conclusion
« Leurre ») that sorted first. No legitimate Faunosteo document is
affected: every PDF the product wrote comes from `JSON.stringify` of one
object, which never emits a duplicate key, and a lone surrogate in a kept
value is already refused by R3 (`error_lone_surrogate`); the proof is the
O2, O3, O7 and Step 5 reference set above, imported identically, and the
O8 corpus where the only lone-surrogate cases that import in R3 are the
three duplicate-key constructions.
- OD-3, one O8 case (`values-1m`): over the value cap (below). It is skipped
like a candidate nested too deep, so the kind follows the stamp
(`attachment_stripped` here, as `depth-65` already answers in R3).
- The `archiveSha256` column of the O7 `readReportArchive` rows is not
compared: the frozen TS domain stringified its Zod output, keys in schema
order, where the core keeps the document's order (R2); the R3 application
stored the core's form, which O8 pins and matches.
Memory protections (OD-3), measured in the shipped module, one fresh
instance per case (`packages/core-wasm/tests/pdf-import.budget.test.ts`,
`R4_BUDGET_ALL=1` for the slow classes; values from `cargo test --test
pdf_budget -- --nocapture`):
| Document | Bytes | Values (`json_js::prescan`) |
|---|---|---|
| demo fr logos (R3 export) | 305,938 | 2,559 |
| v2full fr logos (R3 export) | 298,035 | 2,232 |
| 612 leaves (O7 export) | 739,316 | 23,803 |
| L envelope (60 observations, 120 views, logos) | 403,016 | 6,740 |
| CAP envelope (500 observations, 1,000 views, logos) | 1,164,658 | 39,652 |
| hostile `values-1m.pdf` | 2,006,973 | 1,000,364 |
The parser's tree costs about 100 bytes of linear memory per value (the
final `Vec` doubling included). Cap off, a 16 MiB token-dense document
(8.4 million values) takes 784 MiB of linear memory and 0.9 s in Node, what
BLK-012 observed; every shape tried (numbers, empty strings, nesting at the
depth cap, the rescue copy) lands at the same 784 MiB, under the 1 GiB
ceiling, so the cap is the guard and the ceiling the backstop.
Thresholds set (to confirm before R4.3):
- `json_js::MAX_JSON_VALUES = 400,000`, ten times the CAP envelope (the plan's
rule), 17 times the 612-leaf export; applied in the attachment probe and in
`draft.restore` (the rescue copy and the tab's draft: an over-budget text is
`invalid`, set aside verbatim, never dropped). Worst case left: about 40 MiB
of tree on top of the file copy.
- `--max-memory` 1 GiB on the module (`.cargo/config.toml`): an allocation
past it traps (`allocate_for_test` on the proofs build: 256 MiB returns,
1,100 MiB throws `RuntimeError`), and `CoreRuntime` poisons and re-creates
the instance as for any trap. The ceiling applies to the export too; the
R4.6 measurements must stay well below it (CAP export estimated at about
0.4 GB in §B.4).
Per class, cap on (Node, i7-11700B, shipped module; "linear" is the
instance's growth, which never shrinks; "RSS" the Node process):
| Case | Outcome | ms | Linear MiB | Node RSS MiB |
|---|---|---|---|---|
| L envelope (394 KB) | ok | 11 | 0.6 | 0.4 |
| CAP envelope (1,138 KB) | ok | 40 | 10.0 | 14.9 |
| demo fr logos (R3 export, 237 KB) | ok | 7 | 0.0 | 1.3 |
| 612 leaves (O7 export) | ok | 23 | 2.8 | 5.4 |
| values-1m (O8 hostile, 2 MB attachment) | attachment_stripped | 11 | 3.1 | 3.0 |
| 16 MiB dense numbers (8.4 M values) | attachment_stripped | 103 | 16.1 | 16.0 |
| 16 MiB dense strings (5.6 M values) | attachment_stripped | 85 | 16.1 | 16.0 |
| 16 MiB at depth 64 | attachment_stripped | 92 | 16.1 | 16.0 |
| 16 MiB dense numbers as a rescue copy | invalid | 152 | 48.1 | 80.1 |
| 16 MiB inflation (zeros, not JSON) | attachment_stripped | 14 | 31.3 | 31.0 |
| 16 MiB + 1 inflation (bomb, refused) | attachment_stripped | 19 | 63.4 | 47.0 |
| 32 bombs (17 MiB each) before the attachment | attachment_stripped | 181 | 63.9 | 47.5 |
| 50 MiB file, attachment at the end | no_attachment | 73 | 50.0 | 50.0 |
Same classes, cap off: values-1m 109 ms and 98 MiB; the three 16 MiB
documents 0.6 to 0.9 s and 784 MiB; the rescue copy 0.4 s and 792 MiB. W4
(≤ 256 MiB linear on every hostile class) holds with the cap on; the 16 MiB
file copy and the inflation buffer dominate what is left.
Module: 1,251,849 B raw, 421,940 B gzip -9 (W1: +26 KB over R3, budget
600 KB, report line 550 KB).
Residual risks and open points for the owner (R4.2 checkpoint):
- The thresholds above are set from fixtures at the collection caps; a report
far denser than CAP per observation (hundreds of findings on each of 500
observations) would be refused as a hostile file. 400,000 values is 17
times the densest export seen (612 leaves on one observation).
- The ceiling is a module-wide constant: it bounds the import worker and the
main instance alike. If an R4.6 export measurement approaches it, the value
must be revisited, never the cap.
- The Node numbers measure the module, not a browser: the browser run of the
same classes (content-process RSS, worker terminated, GC) is in 10.4.
- The probe parses the candidate twice (version check, then the session),
as the TypeScript did; the peak is one tree since the first is dropped. A
single parse would need the gated session reader to take a value.