docs(docs): RUST.md §10.3 R4.2 reader record (parity, OD-4 explanation, OD-3 thresholds, Node budgets)
This commit is contained in:
1 parent
ebcae06ac1
commit
0d2c687b79
2 files changed
+139
-5
No files matched your search
@@ -216,11 +216,11 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
|
||||
- [x] R4.1e browser harness (scripts/browser, Chromium + Firefox + WebKit, 4 GiB RSS watchdog): O9 browser verdicts, O12 TS baselines (export demo/L/CAP, import O2 + hostile)
|
||||
- [x] R4.1f O10 report:compare outputs, test ledger (crates/core/tests/pdf/LEDGER.tsv), freeze record (RUST.md §10 draft, fixtures README)
|
||||
- [ ] R4.2 reader (cargo + wasm, differential, OD-3 thresholds confirmed with owner before R4.3); owner go 2026-10-18 (R4.1 validated: fixtures kept, gitleaks exception accepted if narrow, decoy-first kept in register + risk explained before R4.3, headless accepted; stop at end of R4.2, budgets = whole browser, same perimeters as TS when a gain is claimed)
|
||||
- [ ] R4.2a `crates/core/src/pdf/read/{scan,strings,stamp}.rs`: scan.ts port (header, typed dicts, stream bounds, Filespec names, /Keywords stamp, decodePdfString; JS regex semantics explicit); cargo tests on O7 findEmbeddedFiles / hasArchiveStamp / decodePdfString rows (155 inputs incl. semantics rows)
|
||||
- [ ] R4.2b `pdf/read/inflate.rs` (miniz_oxide, zlib framing, DecompressionStream verdicts): cargo tests on O9 (26 rows) + O7 flate inputs; Node verdict pinned, browsers agree (inflate.browsers.json)
|
||||
- [ ] R4.2c `pdf/json_js.rs`: `{` probe after BOM/whitespace, depth 64 + value-count pre-scan (OD-3 budget hook), lossy UTF-8 + one BOM (TextDecoder parity), overflow → null rewrite, lone-surrogate-neutralised probe through parse_json, versioned check; unit tests + text parity cases
|
||||
- [ ] R4.2d `pdf/read/probe.rs` locate_attachment (shared 16 MiB budget, `break` on oversized, cancelled inflation not charged), `pdf/import.rs` (read → session::import_archive), op `pdf.import` + `SheetHandle.importPdf(bytes)` on the wasm facade, ts-rs `PdfImportKind`; cargo tests on O7 locate/readAttachedDocument/readReportArchive rows, O8 differential (51), Node tests through the wasm build on O2/O3/O8/hostile
|
||||
- [ ] R4.2e OD-3: measurement harness (fresh wasm instance per case, time, linear memory; Node) on the hostile classes (16 MiB token-dense attachment, 16 MiB rescue copy, 50 MiB file, 16 MiB inflation, 32 candidates) + browser harness (worker page, content-process RSS, same perimeter as the TS import baseline); value budget = 10 × largest legitimate archive; `--max-memory`; W1 module size; thresholds table for the owner
|
||||
- [x] R4.2a `crates/core/src/pdf/read/{scan,strings,stamp}.rs`: scan.ts port (header, typed dicts, stream bounds, Filespec names, /Keywords stamp, decodePdfString; JS regex semantics explicit); cargo tests on O7 findEmbeddedFiles / hasArchiveStamp / decodePdfString rows (155 inputs incl. semantics rows)
|
||||
- [x] R4.2b `pdf/read/inflate.rs` (miniz_oxide, zlib framing, DecompressionStream verdicts): cargo tests on O9 (26 rows) + O7 flate inputs; Node verdict pinned, browsers agree (inflate.browsers.json)
|
||||
- [x] R4.2c `pdf/json_js.rs`: `{` probe after BOM/whitespace, depth 64 + value-count pre-scan (OD-3 budget hook), lossy UTF-8 + one BOM (TextDecoder parity), overflow → null rewrite, lone-surrogate-neutralised probe through parse_json, versioned check; unit tests + text parity cases
|
||||
- [x] R4.2d `pdf/read/probe.rs` locate_attachment (shared 16 MiB budget, `break` on oversized, cancelled inflation not charged), `pdf/import.rs` (read → session::import_archive), op `pdf.import` + `SheetHandle.importPdf(bytes)` on the wasm facade, ts-rs `PdfImportKind`; cargo tests on O7 locate/readAttachedDocument/readReportArchive rows, O8 differential (51), Node tests through the wasm build on O2/O3/O8/hostile
|
||||
- [~] R4.2e (Node part done: cap 400,000 = 10 × CAP envelope 39,652 values, rescue copy guarded, 1 GiB ceiling proven; browser run pending) OD-3: measurement harness (fresh wasm instance per case, time, linear memory; Node) on the hostile classes (16 MiB token-dense attachment, 16 MiB rescue copy, 50 MiB file, 16 MiB inflation, 32 candidates) + browser harness (worker page, content-process RSS, same perimeter as the TS import baseline); value budget = 10 × largest legitimate archive; `--max-memory`; W1 module size; thresholds table for the owner
|
||||
- [ ] R4.2f record: RUST.md §10.3 (reader, parity, budgets, thresholds, residual risks), decoy-first (OD-4) risk explanation + proof it blocks no legitimate Faunosteo document, ledger `ported` targets exist, suites green (rust:check, wasm:build, check); STOP for the owner before R4.3
|
||||
- [ ] R4.3 import worker + rescue copy (B.1.5, OD-1 applied, browser proof per slice from here)
|
||||
- [ ] J1 owner checkpoint (plan A.4)
|
||||
|
||||
@@ -1047,3 +1047,137 @@ ArrayBuffer memory); memory after an import settles (the §B.3 check belongs
|
||||
to the R4 worker proof); real browsers and phones (J1 and the owner's
|
||||
iPhone run). Seen on every page: the known `img-src` refusal of the
|
||||
footer's external credit logo; no script or wasm violation.
|
||||
|
||||
### 10.3 R4.2: the reader in Rust (2026-10-18, no import switched yet)
|
||||
|
||||
What runs in Rust (`crates/core/src/pdf`, feature `pdf-read`, enabled by the wasm
|
||||
facade, never by the worker): the byte scan of `scan.ts` (`read/scan.rs`,
|
||||
`strings.rs`, `stamp.rs`, the JavaScript regex semantics reproduced by hand
|
||||
and pinned by the `semantics.*` rows of O7), the inflation on miniz_oxide
|
||||
(`read/inflate.rs`, non-wrapping output buffer growing to the limit plus one,
|
||||
trailing bytes refused, Adler-32 checked: the verdicts of
|
||||
`DecompressionStream`), the JavaScript-path side effects of the probe
|
||||
(`json_js.rs`: TextDecoder parity with one BOM dropped, overflow to `null`,
|
||||
the lone surrogate neutralised for the probe only, the depth and value
|
||||
pre-scan), the probe itself (`read/probe.rs`, one 16 MiB budget shared by the
|
||||
candidates, a cancelled inflation not charged) and `pdf::import::import_pdf`,
|
||||
which hands the text to the unchanged R2 session import. On the facade:
|
||||
`SheetHandle.import_pdf(bytes)` with the kinds of `PdfImportKind` (ts-rs).
|
||||
Nothing in the application calls it yet (R4.3).
|
||||
|
||||
Parity (every suite green on the committed oracles, `cargo test -p
|
||||
faunosteo-core --features pdf-read` and the core-wasm Node suites):
|
||||
|
||||
| Oracle | Rust check | Result |
|
||||
|---|---|---|
|
||||
| O7 `findEmbeddedFiles`, `hasArchiveStamp`, `decodePdfString` | `pdf_scan` | 144 PDF inputs, 178 candidates, 19 string rows, all equal |
|
||||
| O9 and the O7 flate rows | `pdf_inflate` | 26 + 12 verdicts equal to Node; the browser columns agree on every row |
|
||||
| TextDecoder lossy decode | `pdf_json_js` on `fixtures/pdf/r4/utf8-lossy.json` (400 cases, `scripts/r4-utf8-oracle.mjs`) | 0 difference |
|
||||
| O7 `locateAttachment`, `readAttachedDocument`, `readReportArchive` | `pdf_import` | 144 inputs equal (candidate, name, offset, bytes digest, parsed-object digest; origin, counters, snapshot rejection, report digest) |
|
||||
| O8, the full R3 import path, 51 PDFs | `pdf_import` natively and `pdf-import.test.ts` through the shipped module | 47 equal, 4 declared differences below |
|
||||
| Reference PDFs (Step 5, Step 6, R3, O2) | `pdf-import.test.ts` | every one imports |
|
||||
|
||||
Declared differences, all on hostile files (plan §A.3):
|
||||
|
||||
- OD-4, three O8 cases (`surrogate-content-overwritten`,
|
||||
`surrogate-version-overwritten`, `two-files-decoy-first`): the text holds a
|
||||
lone UTF-16 surrogate escape in a value that a later duplicate key
|
||||
overwrote. `JSON.parse` kept the last value, so the stringified object the
|
||||
core received no longer held the escape and R3 imported the file; the core
|
||||
refuses the text whole (`lone_surrogate`), nothing altered. Why the refusal
|
||||
is right: a Rust string cannot carry a lone surrogate, so reading such a
|
||||
text means either altering it silently (L1 forbids it) or reading only the
|
||||
parts that happen to survive duplicate-key resolution, which is reading a
|
||||
document other than the one in the file. In `two-files-decoy-first` R3 did
|
||||
not import the real envelope at all: it imported the decoy (conclusion
|
||||
« Leurre ») that sorted first. No legitimate Faunosteo document is
|
||||
affected: every PDF the product wrote comes from `JSON.stringify` of one
|
||||
object, which never emits a duplicate key, and a lone surrogate in a kept
|
||||
value is already refused by R3 (`error_lone_surrogate`); the proof is the
|
||||
O2, O3, O7 and Step 5 reference set above, imported identically, and the
|
||||
O8 corpus where the only lone-surrogate cases that import in R3 are the
|
||||
three duplicate-key constructions.
|
||||
- OD-3, one O8 case (`values-1m`): over the value cap (below). It is skipped
|
||||
like a candidate nested too deep, so the kind follows the stamp
|
||||
(`attachment_stripped` here, as `depth-65` already answers in R3).
|
||||
- The `archiveSha256` column of the O7 `readReportArchive` rows is not
|
||||
compared: the frozen TS domain stringified its Zod output, keys in schema
|
||||
order, where the core keeps the document's order (R2); the R3 application
|
||||
stored the core's form, which O8 pins and matches.
|
||||
|
||||
Memory protections (OD-3), measured in the shipped module, one fresh
|
||||
instance per case (`packages/core-wasm/tests/pdf-import.budget.test.ts`,
|
||||
`R4_BUDGET_ALL=1` for the slow classes; values from `cargo test --test
|
||||
pdf_budget -- --nocapture`):
|
||||
|
||||
| Document | Bytes | Values (`json_js::prescan`) |
|
||||
|---|---|---|
|
||||
| demo fr logos (R3 export) | 305,938 | 2,559 |
|
||||
| v2full fr logos (R3 export) | 298,035 | 2,232 |
|
||||
| 612 leaves (O7 export) | 739,316 | 23,803 |
|
||||
| L envelope (60 observations, 120 views, logos) | 403,016 | 6,740 |
|
||||
| CAP envelope (500 observations, 1,000 views, logos) | 1,164,658 | 39,652 |
|
||||
| hostile `values-1m.pdf` | 2,006,973 | 1,000,364 |
|
||||
|
||||
The parser's tree costs about 100 bytes of linear memory per value (the
|
||||
final `Vec` doubling included). Cap off, a 16 MiB token-dense document
|
||||
(8.4 million values) takes 784 MiB of linear memory and 0.9 s in Node, what
|
||||
BLK-012 observed; every shape tried (numbers, empty strings, nesting at the
|
||||
depth cap, the rescue copy) lands at the same 784 MiB, under the 1 GiB
|
||||
ceiling, so the cap is the guard and the ceiling the backstop.
|
||||
|
||||
Thresholds set (to confirm before R4.3):
|
||||
|
||||
- `json_js::MAX_JSON_VALUES = 400,000`, ten times the CAP envelope (the plan's
|
||||
rule), 17 times the 612-leaf export; applied in the attachment probe and in
|
||||
`draft.restore` (the rescue copy and the tab's draft: an over-budget text is
|
||||
`invalid`, set aside verbatim, never dropped). Worst case left: about 40 MiB
|
||||
of tree on top of the file copy.
|
||||
- `--max-memory` 1 GiB on the module (`.cargo/config.toml`): an allocation
|
||||
past it traps (`allocate_for_test` on the proofs build: 256 MiB returns,
|
||||
1,100 MiB throws `RuntimeError`), and `CoreRuntime` poisons and re-creates
|
||||
the instance as for any trap. The ceiling applies to the export too; the
|
||||
R4.6 measurements must stay well below it (CAP export estimated at about
|
||||
0.4 GB in §B.4).
|
||||
|
||||
Per class, cap on (Node, i7-11700B, shipped module; "linear" is the
|
||||
instance's growth, which never shrinks; "RSS" the Node process):
|
||||
|
||||
| Case | Outcome | ms | Linear MiB | Node RSS MiB |
|
||||
|---|---|---|---|---|
|
||||
| L envelope (394 KB) | ok | 11 | 0.6 | 0.4 |
|
||||
| CAP envelope (1,138 KB) | ok | 40 | 10.0 | 14.9 |
|
||||
| demo fr logos (R3 export, 237 KB) | ok | 7 | 0.0 | 1.3 |
|
||||
| 612 leaves (O7 export) | ok | 23 | 2.8 | 5.4 |
|
||||
| values-1m (O8 hostile, 2 MB attachment) | attachment_stripped | 11 | 3.1 | 3.0 |
|
||||
| 16 MiB dense numbers (8.4 M values) | attachment_stripped | 103 | 16.1 | 16.0 |
|
||||
| 16 MiB dense strings (5.6 M values) | attachment_stripped | 85 | 16.1 | 16.0 |
|
||||
| 16 MiB at depth 64 | attachment_stripped | 92 | 16.1 | 16.0 |
|
||||
| 16 MiB dense numbers as a rescue copy | invalid | 152 | 48.1 | 80.1 |
|
||||
| 16 MiB inflation (zeros, not JSON) | attachment_stripped | 14 | 31.3 | 31.0 |
|
||||
| 16 MiB + 1 inflation (bomb, refused) | attachment_stripped | 19 | 63.4 | 47.0 |
|
||||
| 32 bombs (17 MiB each) before the attachment | attachment_stripped | 181 | 63.9 | 47.5 |
|
||||
| 50 MiB file, attachment at the end | no_attachment | 73 | 50.0 | 50.0 |
|
||||
|
||||
Same classes, cap off: values-1m 109 ms and 98 MiB; the three 16 MiB
|
||||
documents 0.6 to 0.9 s and 784 MiB; the rescue copy 0.4 s and 792 MiB. W4
|
||||
(≤ 256 MiB linear on every hostile class) holds with the cap on; the 16 MiB
|
||||
file copy and the inflation buffer dominate what is left.
|
||||
|
||||
Module: 1,251,849 B raw, 421,940 B gzip -9 (W1: +26 KB over R3, budget
|
||||
600 KB, report line 550 KB).
|
||||
|
||||
Residual risks and open points for the owner (R4.2 checkpoint):
|
||||
|
||||
- The thresholds above are set from fixtures at the collection caps; a report
|
||||
far denser than CAP per observation (hundreds of findings on each of 500
|
||||
observations) would be refused as a hostile file. 400,000 values is 17
|
||||
times the densest export seen (612 leaves on one observation).
|
||||
- The ceiling is a module-wide constant: it bounds the import worker and the
|
||||
main instance alike. If an R4.6 export measurement approaches it, the value
|
||||
must be revisited, never the cap.
|
||||
- The Node numbers measure the module, not a browser: the browser run of the
|
||||
same classes (content-process RSS, worker terminated, GC) is in 10.4.
|
||||
- The probe parses the candidate twice (version check, then the session),
|
||||
as the TypeScript did; the peak is one tree since the first is dropped. A
|
||||
single parse would need the gated session reader to take a value.
|
||||
Reference in new issue
Block a user