docs(docs): RUST.md §10.8 R4.6 record (buffers, memory, parity, E3/E4 three engines, release series, W1, risks); ledger targets filled; compare-exports script; roundtrip raster-failure case; TODO R4.6 closed; memory BDR-062, LRN-063/064, journal

This commit is contained in:
bchanot committed 2026-10-19 02:49:42 +02:00
1 parent cdcdda8e1c
commit 264c5fa3fc
14 files changed
+5230 -38

No files matched your search

+6
View File
@@ -469,3 +469,9 @@ Status: active.
Context: R4.4 report (RUST.md §10.6: O1 12 runs 2,124 calls, O4/O5/O6 exact). Owner decision 2026-10-18.
Decision: R4.5 on the branch: content emitter + numberToString, pdf-writer, PNG gate and decode, links, Info, attachment, self-check, from_envelope. Gates: parity with O2, O4, O6 and applicable goldens (declared differences kept, oracles never edited to hide a gap); `qpdf --check`, poppler 150 dpi 0 px vs R3, O11 reads; restoration compat: R4 PDF read by Rust reader + kept TS reader, envelope/N-1 copies, no bilan data loss on export → import; PNG and PDF structure protections before large allocations (plan caps), invalid cases and messages checked; measure performance, allocations, real wasm size with the writer linked (W1 600 KB per plan), compared with TS baselines on same files/conditions; R4.3 import and worker guarantees untouched; WebKit reserves and iOS recette still blocking merge; gates green; RUST.md documented. Stop at R4.5 checkpoint: components, oracle/golden results, compat, perf/memory, wasm size, gaps/risks, R4.6 recommendation.
Status: active.
## BDR-062 | 2026-10-19 | R4.5 validated; R4.6 session export authorised; chunked file, decode on arrival, catalog last
Context: owner decision 2026-10-19 on the R4.5 report (RUST.md §10.7). R4.6 brief: ExportSession + app driver, rasters as bytes one by one, OD-2 buffers documented, `view_raster_failed` on raster failure, parity + restoration (Rust → Rust, Rust → TS reader, N-1), E3/E4 three engines same data, shipped W1 measured, TS engine kept until R4.8, no merge, stop at R4.6.
Decision (design, measured): session in main instance per OD-2. File written in chunks never reallocated (target 1 MiB doubling to 8 MiB), image object written at `add_raster` (PNG + planes freed in the call), logos kept until branding batch completes (slot order as engine), catalog + page tree objects written last, file handed to JS in parts → `new Blob(parts)`, self-check scans attachment chunk behind header. R4 digests (own bytes, not TS oracle) re-frozen once for it; O2 suite + qpdf + poppler re-run green. Hex strings upper-case as pdf-lib (B4). Alternatives measured and dropped: single `Vec` doubling (125 MiB for 44 MiB file, realloc spike), two buffers (2 × file at finish), 1 MiB fixed chunks (0.12 MB fragmentation per view).
Status: active; owner checkpoint R4.6 pending.
+1
View File
@@ -292,3 +292,4 @@ Owner validated R4.0 plan with conditions (BDR-057, plan §A.6, LRN-059/060). R3
- 2026-10-18 R4.4 done (70007e0, 1550d21): pdf::write text/afm_data/dates (O4 55,992 widths bit-exact, 1,168 wraps, safeText every code point, upper 1,580 = Rust to_uppercase, O5 tables, O6 6,468 cells), input checks, layouts → DrawLog; O1 12 runs 2,124 calls equal, step6 text/images 24 derived. Layout L 3 ms, CAP 24 ms release. Nothing wired, module size unchanged. STOP before R4.5.
- 2026-10-18 owner validated R4.4; R4.5 go (BDR-061). Subtasks R4.5a-f in TODO. Tooling present: qpdf 12.3.2, poppler 26.01 (pdftoppm, pdfdetach), crates.io reachable.
- 2026-10-18 R4.5 done (d420f67..b44dd80): content emitter (60 streams byte-equal, numberToString 703 + V8 pow10 quirk), PNG gate + UPNG decoder (O6 108/108), own object writer (pdf-writer dropped: numberToString everywhere, no dep), serialiser, export + self-check, from_envelope; O2 14 runs equal (streams after B3, planes, links, Info, attachment), qpdf clean, poppler 0 px, O11 + N-1 + TS reader read R4 PDFs, wasm = native digests; Node TS→Rust demo 198→36 ms, L 772→57, CAP 5.7 s→0.22 s; writer linked 531.6 KB gzip (proofs build), shipped 426 KB. Gate PASS 4 LOW → hardenings. Open: one-shot path holds PNGs twice (R4.6 session must stream), L real rasters 5 s native (deflate 6), E3 judged at R4.6. STOP before R4.6.
- 2026-10-19 owner validated R4.5; R4.6 go (BDR-062). R4.6 done (9e688b6..): ExportSession (decode on arrival, chunked never-realloc file 1→8 MiB, catalog last, parts → Blob, self-check on attachment chunk), views.rs (compose_highlight requests, raster_size_of O4 57/57), wasm ExportSession + CoreExportSession, app export.ts on session (abort on unmount, User Timing), labels.ts; app suites read content streams (pdfRead.ts), app-path bytes = R4 digests (re-frozen: object order B2, hex upper-case B4), Step 6 goldens via app path, 14 O2 runs through module = digests, request SVG = TS compose text, export Rust → import Rust / TS reader / N-1 ok. E3 Chromium demo 0.67, L 0.74 (Firefox 0.66/0.67, WebKit 0.91/0.95); E4 CAP completes 3 engines (TS stopped by ceiling); peaks lower (L Chromium 805→470 MiB). Linear memory L real 62 MiB flat over 7 exports; Chromium per-export RSS growth ≈ 20 MiB on L same as TS control, plateau after 8. Shipped 519.7 KB gzip (W1). rust-check 0, pnpm check 1,099 tests, ledger 11 pending (R4.7). J2 kit + procedure. STOP before R4.7.
+13
View File
@@ -478,3 +478,16 @@ Apply: any shared pre-scan / lexer feeding several caps (R4.3 worker, R5 pipelin
- `parseVersioned` tested `typeof version === 'number'` on `JSON.parse` output: `1e999` → `Infinity`, a number, so the probe accepted; `JSON.stringify` then wrote `null`, so the core saw no version → `no_attachment`. A Rust port that rewrote overflow → `null` before the probe gave `attachment_stripped` (3 O8 rows).
- Fix: probe on a copy where overflowing tokens stay finite numbers (`rewrite_overflow_numbers_as_finite`, `0`), hand the `null` form to the core. Same family as LRN-059 (overflow, lone surrogates): every JS parse → stringify hop is a transformation to reproduce step by step, in order.
Apply: porting any JS reader whose acceptance test ran between parse and stringify (R4 writer inputs, R5 pipeline, server validation).
## LRN-063 | 2026-10-19 | wasm linear memory: never grow one big Vec; long-lived chunks big enough not to interleave with transients
- Growing `Vec<u8>` for a 44 MiB file in wasm (dlmalloc) peaked 125 MiB: realloc keeps old + new while copying; decode transients (≈ 10 MB per PNG) sit between file and heap top so no in-place growth. Linear memory never shrinks → spike is the high-water for the page.
- Fixed chunks (never realloc) remove the spike. 1 MiB chunks still cost 0.12 MB per view (long-lived alloc every other raster interleaved with transients → fragmentation); 8 MiB chunks cost 0. Adaptive target 1 → 8 MiB: small files pay one small chunk, big files few long-lived allocs. `shrink_to_fit` on seal gives slack back.
- Hand chunks to JS one by one (`take_part`, freed in module), `new Blob(parts)`: file never held twice across heaps.
Apply: any wasm producer of large outputs (R5 pipeline, future image work): measure stage by stage with `memory.buffer.byteLength`, never trust the estimate.
## LRN-064 | 2026-10-19 | Test facts from the produced PDF, not from draw-call spies; recorder order ≠ file order
- Step 6 goldens were recorded by spying pdf-lib `drawText`/`drawImage`: footers recorded after every page's content (global call order). Read back from content streams, facts come page by page. Same set, same within-page order: compare against the golden stably sorted by page.
- `tests/helpers/pdfRead.ts`: pdf-lib loads the file, inflates each content stream, parses `Tf/Tm/Tj/ET` and `cm … Do`, resolves `/F1` through page resources, WinAnsi = windows-1252. Lets 16 roundtrip + 7 goldens cases run unchanged on the Rust output.
Apply: when an implementation is swapped under a test that spied the old one, read the artefact instead of mocking the new one; keep the old generator pinned to the old engine (r4.writer-oracles → `exportReportArchive`).