docs(docs): RUST.md §10.7 R4.5 serialiser record; TODO R4.5a-e ticked

This commit is contained in:
bchanot committed 2026-10-18 23:29:43 +02:00
1 parent babe7b76d2
commit d2376a4574
2 files changed
+100 -5

No files matched your search

+5 -5
View File
@@ -235,11 +235,11 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
- [x] R4.4b `pdf/write/{mod,input,draw_log,layout/{header,footer,observation,conclusion}}.rs`: ExportInput (ts-rs) checks (branding/labels, assert_snapshot + read_svg per asset, sheet mismatch, caps), rasters by key + PNG dims (IHDR only; gate at R4.5), DrawLog ops at full precision in TS call order (footers after all pages), links, embeds, warnings; cargo O1 12 logs exact, step6 text/images goldens (24) derived
- [x] R4.4c (done 2026-10-18, STOP before R4.5) record RUST.md §10.6 (modules, parity, declared differences, performance), ledger targets, suites green, STOP before R4.5
- [ ] R4.5 serialiser (pdf-writer, self-check, from_envelope) (owner go 2026-10-18, BDR-061; stop before R4.6)
- [ ] R4.5a `pdf/write/content.rs`: pdf-lib operator sequences for text/image/line/rectangle + `numberToString` (O4 cases, exponent quirks); `pdf/write/png.rs`: branding PNG gate (`pngIssue` port) + UPNG-rule decode to RGBA8 (O6 108 cases: gate verdicts + plane digests), alpha plane only when an alpha < 255; caps before allocation
- [ ] R4.5b `pdf/write/serialize.rs` on pdf-writer: catalog, pages, 3 Type1 WinAnsi fonts, image XObjects (DeviceRGB + DeviceGray SMask Decode [0 1]), Link annots (Border [0 0 0], URI hex ASCII), Info (Producer/Creator Faunosteo, Title/Author/Subject/Keywords UTF-16BE hex, dates literal), attachment (EmbeddedFile Flate + Params, Filespec, Names, AF), classic xref; `export.rs` (checks → layout → serialise), `self_check.rs` (re-read with R4 reader, attachment == envelope), `from_envelope.rs`; errors `self_check_failed`
- [ ] R4.5c cargo O2: 12 R3 PDFs + real raster + store path: streams equal after B3 name resolution (lopdf dev-dep), image planes equal, links equal, Info equal but Producer/Creator, attachment bytes equal (store-path: D-R11 order only); goldens/r4/pdf.*.sha256 of the R4 outputs; export-side security cases (PNG budget 28, SVG hostile, URL injection, size E2, reexport)
- [ ] R4.5d external checks: `qpdf --check` clean, `pdfdetach -list` osteo-report.json, `pdftoppm -r 150` R3 vs R4 0 px (12 + real raster), O11 frozen scanner + N-1 Zod copy read every R4 PDF, stripped R4 PDF → attachment_stripped; TS reader (packages/pdf) reads R4 PDFs; R4 PDF → Rust import → same report (no data loss)
- [ ] R4.5e wasm: proofs export `export_pdf_for_test` (input JSON + rasters) → wasm digests == native digests; module size with the writer linked (W1 ≤ 600 KB gzip); Node timings/allocations on demo, L, CAP vs TS baseline (same files)
- [x] R4.5a `pdf/write/content.rs`: pdf-lib operator sequences for text/image/line/rectangle + `numberToString` (O4 cases, exponent quirks); `pdf/write/png.rs`: branding PNG gate (`pngIssue` port) + UPNG-rule decode to RGBA8 (O6 108 cases: gate verdicts + plane digests), alpha plane only when an alpha < 255; caps before allocation
- [x] R4.5b `pdf/write/serialize.rs` on pdf-writer: catalog, pages, 3 Type1 WinAnsi fonts, image XObjects (DeviceRGB + DeviceGray SMask Decode [0 1]), Link annots (Border [0 0 0], URI hex ASCII), Info (Producer/Creator Faunosteo, Title/Author/Subject/Keywords UTF-16BE hex, dates literal), attachment (EmbeddedFile Flate + Params, Filespec, Names, AF), classic xref; `export.rs` (checks → layout → serialise), `self_check.rs` (re-read with R4 reader, attachment == envelope), `from_envelope.rs`; errors `self_check_failed`
- [x] R4.5c cargo O2: 12 R3 PDFs + real raster + store path: streams equal after B3 name resolution (lopdf dev-dep), image planes equal, links equal, Info equal but Producer/Creator, attachment bytes equal (store-path: D-R11 order only); goldens/r4/pdf.*.sha256 of the R4 outputs; export-side security cases (PNG budget 28, SVG hostile, URL injection, size E2, reexport)
- [x] R4.5d external checks: `qpdf --check` clean, `pdfdetach -list` osteo-report.json, `pdftoppm -r 150` R3 vs R4 0 px (12 + real raster), O11 frozen scanner + N-1 Zod copy read every R4 PDF, stripped R4 PDF → attachment_stripped; TS reader (packages/pdf) reads R4 PDFs; R4 PDF → Rust import → same report (no data loss)
- [x] R4.5e wasm: proofs export `export_pdf_for_test` (input JSON + rasters) → wasm digests == native digests; module size with the writer linked (W1 ≤ 600 KB gzip); Node timings/allocations on demo, L, CAP vs TS baseline (same files)
- [ ] R4.5f record RUST.md §10.7, ledger targets, suites green; STOP before R4.6
- [ ] R4.6 app export (session in main instance, owner phone check of L)
- [ ] R4.7 report-compare in Rust
+95
View File
@@ -1501,3 +1501,98 @@ Residual risks and open points (R4.4):
- The module size impact of the writer is measured when R4.5 links it
(W1, the `afm_data` tables are 82 KB of source, the compiled tables about
12 KB).
### 10.7 R4.5: the serialiser in Rust (2026-10-18, nothing wired yet)
What runs in Rust (`crates/core/src/pdf/write`, with R4.4's layout): the
content-stream emitter (`content.rs`: pdf-lib 1.17.1's operator sequences
for the four calls, its `cleanText`, `24 TL`, upper-case hex text, the
doubled `m` of a line, `h f` of a filled rectangle; `numberToString` with
its `e-N` / `e+N` branches and V8's `Math.pow(10, k)` one-ULP quirk at k = 23
and 210; WinAnsi encoding), the PNG side (`png.rs`: the branding gate
`pngIssue` in its order, and a decoder that reproduces UPNG's `decode` and
`toRGBA8` with the deviations the O6 oracle pinned: the Average first-row
pre-pass at absolute offsets, the 16-bit grey tRNS key at `off + x`, the
16-bit high byte, palette tRNS by declared length; pdf-lib's alpha drop;
caps checked before any allocation; crafted files refused where UPNG spun or
threw), a minimal object writer (`objects.rs`: numbered objects, streams,
a classic xref, pdf-lib's string forms: UTF-16BE hex with a BOM, `D:` dates,
hex ASCII URIs), the serialiser (`serialize.rs`: catalog with the names tree
and `/AF`, the page tree, three Type1 `WinAnsiEncoding` fonts, one image
XObject per embed with a DeviceGray `/SMask` `/Decode [0 1]` when an alpha
is below 255, Link annotations `/Border [0 0 0]`, the Info dictionary with
Producer and Creator `Faunosteo`, the Flate `EmbeddedFile` with `Params` and
its Filespec in clear, per-page resources listing only the images the page
draws) and the driver (`export.rs`: the R4.4 checks, the envelope as
`JSON.stringify(_, null, 2)` under 16 MiB, the layout, the file, then the
self-check: the file re-read by the R4 reader must carry the envelope it was
written with; `export_from_envelope`). The facade gets `export_pdf_for_test`
on the proofs build only; the application's session is R4.6, so the shipped
module is unchanged (426 KB gzip) until then.
Deviation from the plan, declared: pdf-writer is not used. The document has
a dozen object kinds; a writer of 150 lines spells every number through
`numberToString` (the B5 f32 difference the plan allowed never arises), keeps
the string forms and key order under our control and adds no dependency to
the module.
Parity (cargo `pdf_content`, `pdf_png`, `pdf_serialize`; Node
`pdf-export.test.ts`, `pdf-export.ts-engine.test.ts`):
| Check | Result |
|---|---|
| Content streams of the 12 R3 PDFs (60 pages, 224,534 bytes) from the O1 logs | equal byte for byte, resource names renamed only |
| O4 `numberToString` 703 cases | equal |
| O6 PNG: 108 rows (gate verdicts, decode planes, error rows) | equal; the 50 hostile files UPNG never ran return in under 20 ms |
| O2, 14 runs (12 stub-raster, the real-raster `v2full-obs1`, the store path): content after B3 name resolution, image planes and SMasks, links (rect spellings and URIs), Info but Producer/Creator, attachment bytes (store path: values; D-R11 order) | equal |
| `qpdf --check`, `pdfdetach -list` | clean, `osteo-report.json` listed, 14 files |
| poppler 150 dpi, R3 against R4, every page | 0 of 94,781,000 pixels differ (14 runs) |
| R4 PDF → Rust import = R3 PDF → Rust import; report value equal to the envelope's | 14 runs |
| re-export from the envelope | byte-identical to the first export |
| wasm (proofs build) output | the native digests, 14 runs (`goldens/r4/pdf.*.sha256`) |
| frozen O11 scanner and N-1 Zod readers (the R3 build and 0.2.0) | read every R4 PDF; a stripped R4 PDF answers `attachment_stripped` |
| the TypeScript engine's reader | the R4 PDFs give the R3 outcome, 12 runs |
| E2 (file ≤ R3 + 5 %) | 12 runs: smaller every time |
| export-side security | hostile SVG asset `snapshot_invalid`, URL injection refused by the schema and hex-only sink, a refused PNG logo fails alone with `logo_failed` |
Cost, Node, stub rasters, same inputs for both (TS engine → Rust through the
module): demo 198 → 36 ms, 29 → 3 MiB of RSS growth, 243 → 236 KB; L 772 →
57 ms, 50 → 1 MiB, 463 → 386 KB; CAP 5.7 s → 0.22 s, 155 → 9 MiB, 2.24 →
1.60 MB (the per-page resources: listing every embed on every page had cost
9 MB on CAP, fixed). Native release, L with a real 1200 × 943 raster on each
of its 120 views: 46.5 MB file, 5.2 s (120 PNG decodes and deflates at
pako's level 6; `--ignored` timing test).
Memory: linear memory of the proofs module after a one-shot export, stub
rasters: demo 3.1 MiB, L 4.2, CAP 14.1. With the real raster on L, the
one-shot proofs path reaches 524 MiB and 7.6 s: it receives the 120 PNGs as
one JSON of base64 (85 MB as text, parsed to a value, decoded to 63 MB), the
layout holds every PNG until the serialiser decodes it, and the file is
assembled from the object bodies. That path is the test's, not the
application's: R4.6's session (OD-2) must take each raster as bytes, decode
and write its XObject on arrival and drop the PNG, so that the high-water
mark is the file so far plus one decode (plan §B.4: ≈ 0.37 MB per view plus
9 MB). The serialiser is ready for that order (an image is written as its own
object when given); the driver that interleaves the calls is R4.6.
Module size (W1): the writer linked (proofs build, which also holds the trap
and allocation proofs) 1,652,719 B raw, 531,573 B gzip -9, +105 KB over the
shipped module, inside the plan's estimate (+100 to 150 KB) and under 600 KB;
above 550 KB it is to be reported, so R4.6 measures the shipped module again
once the session links the writer.
Residual risks and open points (R4.5):
- The one-shot export of L with real rasters is slow (5 s native, dominated
by deflate level 6 of 400 MB of planes); E3 (≤ 1.2 × the TS export time in
Chromium) is judged at R4.6 on the whole export; a lower deflate level
would trade E2 (size) for time.
- The decoder reproduces UPNG's accepted deviations on the oracle corpus; on
files outside it (unusual interlace or tRNS combinations) UPNG's behaviour
is reproduced by the code's structure, not pinned.
- The self-check proves the file carries its envelope and that the R4 reader
reads it; it does not render the pages. poppler's 0-pixel result covers
the corpus only.
- `MAX_ATTACHMENT_BYTES` keeps the 16 MiB envelope cap; an `exportedAt` that
is not an ISO UTC timestamp is a new closed kind `invalid_input`, never
reachable from the application.