diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 1c4fe117..247c6a07 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -235,11 +235,11 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] R4.4b `pdf/write/{mod,input,draw_log,layout/{header,footer,observation,conclusion}}.rs`: ExportInput (ts-rs) checks (branding/labels, assert_snapshot + read_svg per asset, sheet mismatch, caps), rasters by key + PNG dims (IHDR only; gate at R4.5), DrawLog ops at full precision in TS call order (footers after all pages), links, embeds, warnings; cargo O1 12 logs exact, step6 text/images goldens (24) derived - [x] R4.4c (done 2026-10-18, STOP before R4.5) record RUST.md §10.6 (modules, parity, declared differences, performance), ledger targets, suites green, STOP before R4.5 - [ ] R4.5 serialiser (pdf-writer, self-check, from_envelope) (owner go 2026-10-18, BDR-061; stop before R4.6) - - [ ] R4.5a `pdf/write/content.rs`: pdf-lib operator sequences for text/image/line/rectangle + `numberToString` (O4 cases, exponent quirks); `pdf/write/png.rs`: branding PNG gate (`pngIssue` port) + UPNG-rule decode to RGBA8 (O6 108 cases: gate verdicts + plane digests), alpha plane only when an alpha < 255; caps before allocation - - [ ] R4.5b `pdf/write/serialize.rs` on pdf-writer: catalog, pages, 3 Type1 WinAnsi fonts, image XObjects (DeviceRGB + DeviceGray SMask Decode [0 1]), Link annots (Border [0 0 0], URI hex ASCII), Info (Producer/Creator Faunosteo, Title/Author/Subject/Keywords UTF-16BE hex, dates literal), attachment (EmbeddedFile Flate + Params, Filespec, Names, AF), classic xref; `export.rs` (checks → layout → serialise), `self_check.rs` (re-read with R4 reader, attachment == envelope), `from_envelope.rs`; errors `self_check_failed` - - [ ] R4.5c cargo O2: 12 R3 PDFs + real raster + store path: streams equal after B3 name resolution (lopdf dev-dep), image planes equal, links equal, Info equal but Producer/Creator, attachment bytes equal (store-path: D-R11 order only); goldens/r4/pdf.*.sha256 of the R4 outputs; export-side security cases (PNG budget 28, SVG hostile, URL injection, size E2, reexport) - - [ ] R4.5d external checks: `qpdf --check` clean, `pdfdetach -list` osteo-report.json, `pdftoppm -r 150` R3 vs R4 0 px (12 + real raster), O11 frozen scanner + N-1 Zod copy read every R4 PDF, stripped R4 PDF → attachment_stripped; TS reader (packages/pdf) reads R4 PDFs; R4 PDF → Rust import → same report (no data loss) - - [ ] R4.5e wasm: proofs export `export_pdf_for_test` (input JSON + rasters) → wasm digests == native digests; module size with the writer linked (W1 ≤ 600 KB gzip); Node timings/allocations on demo, L, CAP vs TS baseline (same files) + - [x] R4.5a `pdf/write/content.rs`: pdf-lib operator sequences for text/image/line/rectangle + `numberToString` (O4 cases, exponent quirks); `pdf/write/png.rs`: branding PNG gate (`pngIssue` port) + UPNG-rule decode to RGBA8 (O6 108 cases: gate verdicts + plane digests), alpha plane only when an alpha < 255; caps before allocation + - [x] R4.5b `pdf/write/serialize.rs` on pdf-writer: catalog, pages, 3 Type1 WinAnsi fonts, image XObjects (DeviceRGB + DeviceGray SMask Decode [0 1]), Link annots (Border [0 0 0], URI hex ASCII), Info (Producer/Creator Faunosteo, Title/Author/Subject/Keywords UTF-16BE hex, dates literal), attachment (EmbeddedFile Flate + Params, Filespec, Names, AF), classic xref; `export.rs` (checks → layout → serialise), `self_check.rs` (re-read with R4 reader, attachment == envelope), `from_envelope.rs`; errors `self_check_failed` + - [x] R4.5c cargo O2: 12 R3 PDFs + real raster + store path: streams equal after B3 name resolution (lopdf dev-dep), image planes equal, links equal, Info equal but Producer/Creator, attachment bytes equal (store-path: D-R11 order only); goldens/r4/pdf.*.sha256 of the R4 outputs; export-side security cases (PNG budget 28, SVG hostile, URL injection, size E2, reexport) + - [x] R4.5d external checks: `qpdf --check` clean, `pdfdetach -list` osteo-report.json, `pdftoppm -r 150` R3 vs R4 0 px (12 + real raster), O11 frozen scanner + N-1 Zod copy read every R4 PDF, stripped R4 PDF → attachment_stripped; TS reader (packages/pdf) reads R4 PDFs; R4 PDF → Rust import → same report (no data loss) + - [x] R4.5e wasm: proofs export `export_pdf_for_test` (input JSON + rasters) → wasm digests == native digests; module size with the writer linked (W1 ≤ 600 KB gzip); Node timings/allocations on demo, L, CAP vs TS baseline (same files) - [ ] R4.5f record RUST.md §10.7, ledger targets, suites green; STOP before R4.6 - [ ] R4.6 app export (session in main instance, owner phone check of L) - [ ] R4.7 report-compare in Rust diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 84b91470..e2645a53 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -1501,3 +1501,98 @@ Residual risks and open points (R4.4): - The module size impact of the writer is measured when R4.5 links it (W1, the `afm_data` tables are 82 KB of source, the compiled tables about 12 KB). + +### 10.7 R4.5: the serialiser in Rust (2026-10-18, nothing wired yet) + +What runs in Rust (`crates/core/src/pdf/write`, with R4.4's layout): the +content-stream emitter (`content.rs`: pdf-lib 1.17.1's operator sequences +for the four calls, its `cleanText`, `24 TL`, upper-case hex text, the +doubled `m` of a line, `h f` of a filled rectangle; `numberToString` with +its `e-N` / `e+N` branches and V8's `Math.pow(10, k)` one-ULP quirk at k = 23 +and 210; WinAnsi encoding), the PNG side (`png.rs`: the branding gate +`pngIssue` in its order, and a decoder that reproduces UPNG's `decode` and +`toRGBA8` with the deviations the O6 oracle pinned: the Average first-row +pre-pass at absolute offsets, the 16-bit grey tRNS key at `off + x`, the +16-bit high byte, palette tRNS by declared length; pdf-lib's alpha drop; +caps checked before any allocation; crafted files refused where UPNG spun or +threw), a minimal object writer (`objects.rs`: numbered objects, streams, +a classic xref, pdf-lib's string forms: UTF-16BE hex with a BOM, `D:` dates, +hex ASCII URIs), the serialiser (`serialize.rs`: catalog with the names tree +and `/AF`, the page tree, three Type1 `WinAnsiEncoding` fonts, one image +XObject per embed with a DeviceGray `/SMask` `/Decode [0 1]` when an alpha +is below 255, Link annotations `/Border [0 0 0]`, the Info dictionary with +Producer and Creator `Faunosteo`, the Flate `EmbeddedFile` with `Params` and +its Filespec in clear, per-page resources listing only the images the page +draws) and the driver (`export.rs`: the R4.4 checks, the envelope as +`JSON.stringify(_, null, 2)` under 16 MiB, the layout, the file, then the +self-check: the file re-read by the R4 reader must carry the envelope it was +written with; `export_from_envelope`). The facade gets `export_pdf_for_test` +on the proofs build only; the application's session is R4.6, so the shipped +module is unchanged (426 KB gzip) until then. + +Deviation from the plan, declared: pdf-writer is not used. The document has +a dozen object kinds; a writer of 150 lines spells every number through +`numberToString` (the B5 f32 difference the plan allowed never arises), keeps +the string forms and key order under our control and adds no dependency to +the module. + +Parity (cargo `pdf_content`, `pdf_png`, `pdf_serialize`; Node +`pdf-export.test.ts`, `pdf-export.ts-engine.test.ts`): + +| Check | Result | +|---|---| +| Content streams of the 12 R3 PDFs (60 pages, 224,534 bytes) from the O1 logs | equal byte for byte, resource names renamed only | +| O4 `numberToString` 703 cases | equal | +| O6 PNG: 108 rows (gate verdicts, decode planes, error rows) | equal; the 50 hostile files UPNG never ran return in under 20 ms | +| O2, 14 runs (12 stub-raster, the real-raster `v2full-obs1`, the store path): content after B3 name resolution, image planes and SMasks, links (rect spellings and URIs), Info but Producer/Creator, attachment bytes (store path: values; D-R11 order) | equal | +| `qpdf --check`, `pdfdetach -list` | clean, `osteo-report.json` listed, 14 files | +| poppler 150 dpi, R3 against R4, every page | 0 of 94,781,000 pixels differ (14 runs) | +| R4 PDF → Rust import = R3 PDF → Rust import; report value equal to the envelope's | 14 runs | +| re-export from the envelope | byte-identical to the first export | +| wasm (proofs build) output | the native digests, 14 runs (`goldens/r4/pdf.*.sha256`) | +| frozen O11 scanner and N-1 Zod readers (the R3 build and 0.2.0) | read every R4 PDF; a stripped R4 PDF answers `attachment_stripped` | +| the TypeScript engine's reader | the R4 PDFs give the R3 outcome, 12 runs | +| E2 (file ≤ R3 + 5 %) | 12 runs: smaller every time | +| export-side security | hostile SVG asset `snapshot_invalid`, URL injection refused by the schema and hex-only sink, a refused PNG logo fails alone with `logo_failed` | + +Cost, Node, stub rasters, same inputs for both (TS engine → Rust through the +module): demo 198 → 36 ms, 29 → 3 MiB of RSS growth, 243 → 236 KB; L 772 → +57 ms, 50 → 1 MiB, 463 → 386 KB; CAP 5.7 s → 0.22 s, 155 → 9 MiB, 2.24 → +1.60 MB (the per-page resources: listing every embed on every page had cost +9 MB on CAP, fixed). Native release, L with a real 1200 × 943 raster on each +of its 120 views: 46.5 MB file, 5.2 s (120 PNG decodes and deflates at +pako's level 6; `--ignored` timing test). + +Memory: linear memory of the proofs module after a one-shot export, stub +rasters: demo 3.1 MiB, L 4.2, CAP 14.1. With the real raster on L, the +one-shot proofs path reaches 524 MiB and 7.6 s: it receives the 120 PNGs as +one JSON of base64 (85 MB as text, parsed to a value, decoded to 63 MB), the +layout holds every PNG until the serialiser decodes it, and the file is +assembled from the object bodies. That path is the test's, not the +application's: R4.6's session (OD-2) must take each raster as bytes, decode +and write its XObject on arrival and drop the PNG, so that the high-water +mark is the file so far plus one decode (plan §B.4: ≈ 0.37 MB per view plus +9 MB). The serialiser is ready for that order (an image is written as its own +object when given); the driver that interleaves the calls is R4.6. + +Module size (W1): the writer linked (proofs build, which also holds the trap +and allocation proofs) 1,652,719 B raw, 531,573 B gzip -9, +105 KB over the +shipped module, inside the plan's estimate (+100 to 150 KB) and under 600 KB; +above 550 KB it is to be reported, so R4.6 measures the shipped module again +once the session links the writer. + +Residual risks and open points (R4.5): + +- The one-shot export of L with real rasters is slow (5 s native, dominated + by deflate level 6 of 400 MB of planes); E3 (≤ 1.2 × the TS export time in + Chromium) is judged at R4.6 on the whole export; a lower deflate level + would trade E2 (size) for time. +- The decoder reproduces UPNG's accepted deviations on the oracle corpus; on + files outside it (unusual interlace or tRNS combinations) UPNG's behaviour + is reproduced by the code's structure, not pinned. +- The self-check proves the file carries its envelope and that the R4 reader + reads it; it does not render the pages. poppler's 0-pixel result covers + the corpus only. +- `MAX_ATTACHMENT_BYTES` keeps the 16 MiB envelope cap; an `exportedAt` that + is not an ISO UTC timestamp is a new closed kind `invalid_input`, never + reachable from the application.