docs(docs): RUST.md §10.8 R4.6 record (buffers, memory, parity, E3/E4 three engines, release series, W1, risks); ledger targets filled; compare-exports script; roundtrip raster-failure case; TODO R4.6 closed; memory BDR-062, LRN-063/064, journal
This commit is contained in:
1 parent
cdcdda8e1c
commit
264c5fa3fc
14 files changed
+5230
-38
No files matched your search
+269
-1
@@ -905,7 +905,7 @@ imports it and restyles `data-state` in place as before.
|
||||
needs it.
|
||||
- WASM budget: 396 KB of 600 KB used; R4 adds the PDF reader and writer.
|
||||
|
||||
## 10. R4: the PDF core in Rust (in progress; R4.1 freeze done 2026-10-18)
|
||||
## 10. R4: the PDF core in Rust (in progress; R4.1 freeze 2026-10-18, export switched 2026-10-19)
|
||||
|
||||
Plan and contract: `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md`
|
||||
(revision 2, validated by the owner 2026-10-18 with the conditions of its
|
||||
@@ -1612,3 +1612,271 @@ Residual risks and open points (R4.5):
|
||||
- `MAX_ATTACHMENT_BYTES` keeps the 16 MiB envelope cap; an `exportedAt` that
|
||||
is not an ISO UTC timestamp is a new closed kind `invalid_input`, never
|
||||
reachable from the application.
|
||||
|
||||
### 10.8 R4.6: the application export on the core's session (2026-10-19, export switched)
|
||||
|
||||
What runs where. The core (`crates/core/src/pdf/write/session.rs`, `views.rs`)
|
||||
holds an `ExportSession`: `new(input)` runs the R4.4 checks, builds the
|
||||
envelope under 16 MiB and opens the file (header, Info, the three fonts);
|
||||
`next_step()` answers the rasters the next step needs (`RasterRequest { key,
|
||||
svg, widthPx, heightPx }`: the SVG logos in slot order, then each observation's
|
||||
views, main view first, composed by R3's `compose_highlight` at the size
|
||||
`rasterSizeOf` gives, the O4 cases pinned) or `done`; `add_raster(key, png)`
|
||||
and `raster_failed(key)` answer one key each; `finish()` lays out the
|
||||
conclusion page and the footers, writes the pages and the attachment, runs
|
||||
the self-check, and hands the file out in parts (`take_part()`). The refusals
|
||||
are closed kinds (`unknown_key`, `duplicate_key`, `out_of_order`), never a
|
||||
trap. The wasm facade exposes the same struct (`crates/wasm`: `ExportSession`,
|
||||
outcome envelopes); `packages/core-wasm` wraps it (`CoreExportSession`,
|
||||
`openExportSession`): every call goes through `CoreRuntime.invoke`, so a trap
|
||||
poisons the instance as any core call, and a session of a dropped instance
|
||||
answers `core_unavailable` without touching the dead handle. The application
|
||||
(`apps/app/src/pdf/export.ts`) drives it: the snapshot from the sheet handle,
|
||||
the bundled branding, the labels (`labels.ts`, moved verbatim from the engine,
|
||||
same key order), the clock, then the loop (`browserRasterizer` under
|
||||
`Promise.all` per batch, `rasterFailed` as each raster rejects, `addRaster` in
|
||||
request order once the batch settled, as the engine embedded), `finish`, a
|
||||
`Blob` of the parts, `free()` in `finally`; leaving the editor aborts the
|
||||
export (the session freed, no file). `EditorRoute` and its alerts are
|
||||
unchanged. `exportReportToPdf` keeps its signature but returns the `Blob`.
|
||||
|
||||
Buffers, ownership and lives (OD-2, the owner's R4.6 item 2):
|
||||
|
||||
| Buffer | Owner | Lives | Size |
|
||||
|---|---|---|---|
|
||||
| the `ExportInput` JSON | the page (a string), copied into linear memory by the glue for `new()` | the call | report + snapshot + branding + labels (demo 338 KB) |
|
||||
| `CheckedInput` (report, snapshot with its asset texts, index, the sanitised asset trees, branding, labels) | the session | until `free()` | demo: 3.7 MiB of linear memory after `new()` |
|
||||
| the envelope text | the session | until the self-check | the attachment's size (demo 55 KB) |
|
||||
| the file being written | the session's `Objects`: chunks never reallocated, the target 1 MiB doubling to 8 MiB per chunk, an object larger than the target in a chunk of its own, a sealed chunk shrunk to its bytes | until `take_part()` moves each one out (copied into the page, freed in the module) | the file |
|
||||
| a request's SVG text | the core builds it, the page receives it as JSON | the `next_step()` call (module side), the batch (page side) | 143 KB per view |
|
||||
| a raster PNG | the page's `Uint8Array`, copied into linear memory by the glue for `add_raster` | the call: decoded (RGBA, raw scanlines, RGB and alpha planes), deflated, written, then freed before the call returns | 0.5 MB (PNG), about 10 MB of transients for a 1200 × 943 raster |
|
||||
| a logo raster | the session, until its batch completes (the branding embeds in slot order, as the engine did) | the branding step | the four logos |
|
||||
| the draw log | the session | until `finish()` | the text calls (KB) |
|
||||
|
||||
So the module holds, at its highest, the file so far plus one decode plus the
|
||||
parsed input plus the open chunk's slack; nothing of the rasters once their
|
||||
object is written, nothing of the file once the parts are out. Measured on a
|
||||
fresh instance per run, the shipped module in Node, the growth of linear
|
||||
memory from before `new()` to after `free()` (linear memory never shrinks:
|
||||
the high-water mark), file in parts:
|
||||
|
||||
| Run | views | PDF bytes | parts | linear before | linear after | growth / PDF | ms |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| demo, 2 × 1 stub | 7 | 235,948 | 1 | 1.25 MiB | 5.9 MiB | 20.6 | 52 |
|
||||
| L, stub | 120 | 386,414 | 1 | 1.25 MiB | 6.6 MiB | 14.6 | 179 |
|
||||
| CAP, stub | 1,000 | 1,599,445 | 2 | 1.25 MiB | 13.7 MiB | 8.2 | 1,195 |
|
||||
| demo, real 1200 × 943 | 7 | 3,282,540 | 3 | 1.25 MiB | 18.0 MiB | 5.4 | 499 |
|
||||
| L, real | 120 | 46,464,887 | 8 | 1.25 MiB | 62.3 MiB | 1.38 | 7,062 |
|
||||
|
||||
L with real rasters: 62 MiB held for a 44 MiB file, where the 120 PNGs alone
|
||||
weigh 64 MB and the engine held every decoded plane until `save()` (plan
|
||||
§B.4: about 0.45 GB); the plan's estimate for the session was 53 MB (file +
|
||||
9 MB). What the module holds beyond the file is bounded (the test asserts it
|
||||
under 24 MiB): the parsed input and trees (3.7 MiB), one decode (about 10
|
||||
MiB), the open chunk's slack (up to 8 MiB). Stage by stage on L real: 1.3 MiB
|
||||
at start, 5.0 after `new()`, 15.5 after the second raster (the first decode),
|
||||
then 0.34 MB per raster for 0.364 MB of file per raster, 62.3 before
|
||||
`finish()`, 62.3 after, 62.3 after the parts are out and the session freed.
|
||||
|
||||
Two findings on the way, measured and kept:
|
||||
|
||||
- A single growing `Vec` for the file (amortised doubling) spiked to 125 MiB
|
||||
on L real: the reallocation holds the old and the new buffer while it
|
||||
copies, and the decode transients sit between the file and the top of the
|
||||
heap, so dlmalloc could not grow it in place. Chunks never reallocated
|
||||
remove the spike.
|
||||
- Chunks of 1 MiB cost 0.12 MB of fragmentation per view (a long-lived
|
||||
allocation interleaved with 10 MB of transients every other raster: 74 MiB
|
||||
for the same file); 8 MiB chunks cost none (62.3 MiB). The target starts at
|
||||
1 MiB and doubles per chunk up to 8 MiB, so a small file pays little slack
|
||||
(demo: one chunk).
|
||||
|
||||
Declared differences, each one a B2 or B4 class of plan §B.2.3, every O2
|
||||
comparison and the poppler render re-run unchanged:
|
||||
|
||||
- The catalog and the page tree (objects 1 and 2) are written last (the xref
|
||||
maps them); R4.5 wrote them first. The R4 digests (`goldens/r4/pdf.*.sha256`,
|
||||
the writer's own bytes, not a TypeScript oracle) were rewritten once for
|
||||
it, with the O2 suite, qpdf, pdfdetach and poppler (0 pixel) green on the
|
||||
new layout.
|
||||
- UTF-16BE hex strings are printed in upper case, as pdf-lib printed them
|
||||
(R4.5 printed lower case: B4, the same decoded value).
|
||||
- pdf-writer stays unused (R4.5).
|
||||
- A view raster the decoder refuses is that view's `view_raster_failed`
|
||||
warning, as `embedPng` throwing was in the engine; R4.5's one-shot path
|
||||
made it `self_check_failed`.
|
||||
- The self-check scans the bytes from the attachment stream object to the end
|
||||
of its chunk, behind the file header, with the R4 reader: the reader finds
|
||||
the attachment by scanning its bytes, never by the xref, and the file is
|
||||
never held whole in the module. The bytes scanned are the file's.
|
||||
|
||||
Parity and restoration (cargo `pdf_session`, `pdf_serialize`, `pdf_layout`;
|
||||
Node `pdf-export.session.test.ts`; the app suites):
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| the session driven key by key = `export_pdf` one-shot (bytes, envelope, warnings); the file grows as the rasters arrive | demo, equal |
|
||||
| the 14 O2 runs through the shipped module and `CoreExportSession` | the native digests |
|
||||
| every view request = the TypeScript `composeHighlightedViewSvg` text of that (observation, view); every (observation, view) the snapshot holds requested once | 14 runs, all views |
|
||||
| the real-raster run: the two SVGs the R4.1 rasteriser drew (`manifest.json` `svgSha256`, 1200 × 263 and 1200 × 943) | equal, sizes equal |
|
||||
| `rasterSizeOf`, the O4 cases (JS `Number()` on the viewBox parts, the clamps) | 57 cases equal |
|
||||
| warnings: a raster that fails before any embed, then a PNG the decoder refuses, view order; a logo warning at its slot whatever the arrival | as the engine recorded them |
|
||||
| refusals: before `next()`, a key not in the batch, answered twice, `next()` with keys unanswered, `finish()` early or twice, after `free()`; garbage, empty and truncated PNGs | closed kinds, no trap |
|
||||
| a trap mid-session (proofs build) | the instance poisoned, the session answers `core_unavailable`, `free()` safe, a new session on the recovered instance writes the digest |
|
||||
| export Rust → import Rust (the module's `import_pdf`, the bundled sheet) | 14 runs: `envelope_v1`, the report equal, not rebased |
|
||||
| export Rust → the kept TypeScript reader (`importReportFromPdf`, the app suites) → the core | the roundtrip, leaves, N-1 and envelope suites (27 cases): equal reports, the 612 leaves (29 unresolved) kept |
|
||||
| N-1 (O11 scanner and the Zod copy: the R3 build and 0.2.0) | every R4 PDF of the O2 runs read; a stripped one `attachment_stripped` (R4.5 suite, unchanged) |
|
||||
| the app path (`exportReportToPdf`, stub rasters, the clock fixed at `FIXED_EXPORTED_AT`) | the 12 R4 digests (3 reports × 2 locales × 2 logo modes) |
|
||||
| the Step 6 goldens (text with page, position, size and font; images; links) read back from the app path's file | 12 runs equal, in page order (the recorder listed every footer last; the file lists each page's footer after its content) |
|
||||
| `pdf.roundtrip` (16), `pdf.goldens` (13), `pdf.export.app` (2), `routes.EditorRoute.export` (6, abort on unmount added), `routes.coreStopped` (15), `state.coreFailures` (5), `migration.leaves` (2), `n-1.reader` (6), `archive.envelopes` (3) | green on the Rust export; the suites read the file's content streams (`tests/helpers/pdfRead.ts`) where they spied pdf-lib |
|
||||
| the oracle generator `r4.writer-oracles` (17) | still verifies the frozen O1 / O2 oracles against the TypeScript engine, called directly with the application's inputs |
|
||||
|
||||
The engine leaves the application's export path: `apps/app/src` imports
|
||||
`@faunosteo/pdf` for the kept reader only (`pdf/import.ts`); pdf-lib is no
|
||||
longer in the application bundle (index chunk 922 KB). `packages/pdf` stays
|
||||
until R4.8 as the R3 reference of the oracle suites and the tests' reader.
|
||||
|
||||
E3 and E4 in the browsers (`scripts/browser/export-baseline.mjs`, 2026-10-19,
|
||||
`results/2026-10-19-ts-export.json` on the build of the last TypeScript-export
|
||||
commit, `results/2026-10-19-r4-export.json` on the R4.6 build; both under the
|
||||
same harness, the same host, the same evening, demo and L three runs each, CAP
|
||||
once; the production CSP; `compare-exports.mjs` prints the tables). Time is
|
||||
the page clock from the click to the PDF blob, the median of the runs:
|
||||
|
||||
| Engine | Report | TS | Rust | Rust / TS | TS file | Rust file |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Chromium 156 | demo | 1,372 ms | 916 ms | 0.67 | 2,852,004 | 2,823,275 |
|
||||
| Chromium | L | 19,286 ms | 14,218 ms | 0.74 | 43,893,082 | 43,536,994 |
|
||||
| Chromium | CAP | stopped by the 4 GiB ceiling after 818 PNGs | 117.6 s | | | 360,935,689 |
|
||||
| Firefox 157 | demo | 1,559 ms | 1,035 ms | 0.66 | 2,872,189 | 2,843,130 |
|
||||
| Firefox | L | 24,559 ms | 16,574 ms | 0.67 | 44,105,143 | 43,743,780 |
|
||||
| Firefox | CAP | stopped after 565 PNGs | 138.0 s | | | 362,665,313 |
|
||||
| WebKit 27.2 | demo | 2,626 ms | 2,386 ms | 0.91 | 2,493,691 | 2,473,027 |
|
||||
| WebKit | L | 52,251 ms | 49,538 ms | 0.95 | 38,557,714 | 38,286,426 |
|
||||
| WebKit | CAP | stopped after 962 PNGs | 412.2 s | | | 317,323,081 |
|
||||
|
||||
E3 (Rust ≤ 1.2 × the TypeScript on demo and L in Chromium): 0.67 and 0.74,
|
||||
met; Firefox 0.66 and 0.67, WebKit 0.91 and 0.95. E4: L completes on the
|
||||
three engines (it did with the engine); CAP completes on the three (1,001
|
||||
rasters, a 361 MB file) where the engine was stopped by the harness ceiling
|
||||
on every engine, as at R4.1. The phone part of E4 is the owner's J2.6.
|
||||
|
||||
Where the time goes (medians; head = click to the first raster, raster phase
|
||||
= first `toBlob` start to last `toBlob` end, tail = last raster to the PDF
|
||||
blob; the Rust driver's User Timing adds the sum of its core calls and of its
|
||||
rasteriser awaits):
|
||||
|
||||
| Engine | Report | TS head / raster phase / tail | Rust head / raster phase / tail | Rust core calls | Rust raster awaits |
|
||||
|---|---|---|---|---|---|
|
||||
| Chromium | demo | 62 / 794 / 518 | 27 / 803 / 80 | 487 | 415 |
|
||||
| Chromium | L | 66 / 11,880 / 7,381 | 27 / 14,014 / 178 | 7,153 | 7,066 |
|
||||
| Chromium | CAP | | 57 / 117,093 / 495 | 58,970 | 58,521 |
|
||||
| Firefox | demo | 62 / 841 / 650 | 20 / 942 / 74 | 455 | 573 |
|
||||
| Firefox | L | 67 / 14,762 / 9,731 | 25 / 16,382 / 173 | 6,684 | 9,862 |
|
||||
| Firefox | CAP | | 60 / 137,464 / 474 | 55,024 | 82,803 |
|
||||
| WebKit | demo | 63 / 2,193 / 370 | 20 / 2,283 / 81 | 455 | 1,918 |
|
||||
| WebKit | L | 70 / 47,227 / 4,957 | 25 / 49,260 / 251 | 6,733 | 42,693 |
|
||||
| WebKit | CAP | | 54 / 411,048 / 1,091 | 55,372 | 356,032 |
|
||||
|
||||
The engine's tail (its `save()`: every plane deflated at the end) is gone:
|
||||
the Rust tail is the conclusion page, the pages, the attachment and the
|
||||
self-check (80 to 250 ms on demo and L). The core's own work (decode, deflate
|
||||
at level 6, write) is 7 s on L and 59 s on CAP in Chromium, interleaved with
|
||||
the browser's rasterisation, which is the other half (7 s on L in Chromium,
|
||||
43 s in WebKit); the export is bound by those two, in sequence per batch.
|
||||
|
||||
Memory in the browsers (content process RSS, MiB; growth = settled after −
|
||||
settled before, forced GC in Chromium, idle elsewhere):
|
||||
|
||||
| Engine | Report | TS peak | Rust peak | TS growth | Rust growth |
|
||||
|---|---|---|---|---|---|
|
||||
| Chromium | demo | 307 | 264 | 17 | 28 |
|
||||
| Chromium | L | 805 | 470 | 43 | 92 |
|
||||
| Chromium | CAP | 3,739 (stopped) | 1,787 | | 560 |
|
||||
| Firefox | demo | 378 | 291 | 77 | 78 |
|
||||
| Firefox | L | 1,397 | 819 | 183 | 264 |
|
||||
| Firefox | CAP | 3,487 (stopped) | 1,914 | | 826 |
|
||||
| WebKit | demo | 658 | 593 | 277 | 144 |
|
||||
| WebKit | L | 1,191 | 848 | 210 | 218 |
|
||||
| WebKit | CAP | 3,871 (stopped) | 2,633 | | 552 |
|
||||
|
||||
The peak falls everywhere (L: 805 → 470 MiB in Chromium, 1,397 → 819 in
|
||||
Firefox, 1,191 → 848 in WebKit). The settled growth after the export is
|
||||
higher on L and CAP: it is the module's linear memory, which never shrinks
|
||||
(plan §B.4 "the same high-water, until reload"): the file was held once in
|
||||
the module (44 MB on L, 361 MB on CAP) plus the transients, and that memory
|
||||
stays reserved for the page's next export instead of being returned. The
|
||||
engine's growth was JavaScript heap, collected. On CAP the page keeps about
|
||||
560 to 830 MiB after the export; CAP is the cap, not a bilan, and the engine
|
||||
never finished it.
|
||||
|
||||
Release across exports (`scripts/browser/r4-export-series.mjs`,
|
||||
`results/2026-10-19-r4-export-series.json`, `…-ts-export-series-chromium.json`,
|
||||
`…-r4-export-series-chromium-12.json`): one editor tab, the start level, a
|
||||
first export (init) and six more, the content RSS read after each (forced GC
|
||||
in Chromium; idle elsewhere, indicative), the import series' criterion of
|
||||
10 MiB over the last five:
|
||||
|
||||
| Engine | Report | Init | Per export | Last 5 | Peak content |
|
||||
|---|---|---|---|---|---|
|
||||
| Chromium, Rust | demo | 28.0 MiB | 1.5 / 2.5 / 2.8 / 2.5 / 3.4 / 1.2 | 12.3 MiB | 287 MiB |
|
||||
| Chromium, Rust | L | 92.2 | 25.5 / 21.1 / 20.1 / 13.3 / 23.9 / 22.4 | 100.8 | 643 |
|
||||
| Chromium, TypeScript (control) | demo | 16.1 | 3.1 / 3.3 / 2.1 / 1.6 / 2.2 / 1.6 | 10.9 | 330 |
|
||||
| Chromium, TypeScript (control) | L | 44.6 | 20.7 / 14.8 / 19.6 / 18.3 / 22.1 / 22.7 | 97.6 | 957 |
|
||||
| Chromium, Rust, 12 exports | L | 95.2 | 19.7 / 27.3 / 11.1 / 29.5 / 14.0 / 28.4 / 16.7 / 22.6 / 4.4 / 1.5 / 3.4 / −4.3 | 27.6 (the last five) | 686 |
|
||||
| Firefox, Rust | demo | 95.1 | 93.9 / −6.7 / 103.3 / −50.4 / 96.5 / −41.8 | 100.8 | 596 |
|
||||
| Firefox, Rust | L | 263.8 | 156.8 / 0.7 / 86.9 / −196.3 / 58.7 / 74.1 | 24.0 | 1,024 |
|
||||
| WebKit, Rust | demo | 139.8 | 91.5 / 225.9 / −216.8 / 108.3 / −90.4 / −0.3 | 26.9 | 946 |
|
||||
| WebKit, Rust | L | 326.7 | 153.7 / 14.7 / 1.2 / 1.1 / −0.4 / 2.3 | 19.0 | 1,992 |
|
||||
|
||||
Reading: the per-export growth of about 20 MiB on L in Chromium is the
|
||||
browser's, not the session's: the TypeScript build shows the same figures on
|
||||
the same page (control), and the module's linear memory, read in Node over
|
||||
seven L exports with real rasters on one instance, stays at 62.3 MiB from
|
||||
the first export to the last (the session frees everything at `free()` and
|
||||
the next export reuses it). Over twelve exports Chromium's growth stops
|
||||
after the eighth (the last five: 27.6 MiB, then a plateau), which points at
|
||||
a bounded cache (canvas backing stores or the blob URLs kept 30 s by
|
||||
`downloadBlob`), not at a leak that grows with use. Firefox and WebKit
|
||||
sawtooth without a forced GC, as in the import series; WebKit L settles
|
||||
after the second export. The criterion as written (10 MiB over five) is not
|
||||
met by either build on L in Chromium; the behaviour is the engine's, the
|
||||
module flat, the plateau reached. Kept as an observation for the owner
|
||||
(BDR-058's reserves extend to it), with the phone series J2.7 due.
|
||||
|
||||
Release on the other paths (cargo `pdf_session`, Node
|
||||
`pdf-export.session.test.ts`, the route test): success (every part out,
|
||||
`free()`), a refused input (no session, the runtime stays `ready`), a raster
|
||||
that fails (its warning, the export continues), a trap mid-session (the
|
||||
instance poisoned and recovered, the session answers `core_unavailable`,
|
||||
`free()` a no-op on the dead handle), a cancelled export (the editor left
|
||||
mid-way: the signal aborts the driver, `free()` in `finally`, no download,
|
||||
no alert). Dropping the session, finished or not, frees everything it holds
|
||||
(Rust ownership); nothing of the file survives `free()` in the module.
|
||||
|
||||
Module size (W1): the shipped module with the writer and the session linked,
|
||||
1,535,289 B raw, 519,730 B gzip -9 (the proofs build 531.6 KB at R4.5 held the
|
||||
trap and allocation proofs too). Under the 550 KB report line and the 600 KB
|
||||
budget; from 426 KB at R4.3, +94 KB for the whole writer. `DEPLOY.md`'s
|
||||
served-size note is updated at R4.8 with the final module.
|
||||
|
||||
Residual risks and open points (R4.6):
|
||||
|
||||
- The settled growth after a large export is the module's linear memory
|
||||
(the file once, the transients, the chunk slack), kept until reload: 92
|
||||
MiB after L and 560 MiB after CAP in Chromium. Within the plan (OD-2: "the
|
||||
same high-water, until reload"); the disposable export worker (S2) stays
|
||||
the fallback if a device fails L (J2.6).
|
||||
- The export is bound by the browser's rasterisation and the core's decode
|
||||
and deflate (level 6) in sequence per batch: L takes 14 s in Chromium and
|
||||
50 s in WebKit (the engine: 19 and 52). A lower deflate level or
|
||||
overlapping the next batch's rasterisation with the core's writing would
|
||||
cut it, at the price of E2 or of a second copy in flight; not done.
|
||||
- The per-export RSS growth in Chromium is the browser's and plateaus; the
|
||||
10 MiB criterion is not met by either build on L. Owner decision.
|
||||
- `view_raster_failed` now also covers a PNG the decoder refuses and a view
|
||||
that cannot be composed; a raster that never settles still hangs the
|
||||
export (the engine's residual, unchanged; the abort on leaving the editor
|
||||
frees the session).
|
||||
- The headless runs validate nothing on a device: J2 (`R4_J2.md`) on the
|
||||
owner's computer and iPhone, with the J1 import recette still due.
|
||||
Reference in new issue
Block a user