From 2f08dad1d7e066d3ecf3fa268f3378cfd4b76f99 Mon Sep 17 00:00:00 2001 From: bchanot Date: Sat, 17 Oct 2026 18:57:29 +0200 Subject: [PATCH 1/3] chore(chore): R4 plan rev 1 --- .../plans/2026-10-17-rust-r4-pdf-core.md | 365 ++++++++++++++++++ 1 file changed, 365 insertions(+) create mode 100644 .claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md diff --git a/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md b/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md new file mode 100644 index 00000000..6f6f1914 --- /dev/null +++ b/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md @@ -0,0 +1,365 @@ +# PLAN: Rust R4, PDF core (reader in a disposable worker, writer, report-compare), `packages/pdf` deleted + +- date: 2026-10-17 | rev 1 | branch: `chore/rust-r4-plan` (plan only); code later on `feature/rust-r4-pdf-core`, one merge +- spec: owner briefs 2026-10-17 "Préparation de R4" and "Autorisation de préparation R4.0" (13 arbitrations, A1 to A13 below); `docs/development/RUST_ARCHITECTURE_REVIEW.md` I.2, II.2 rows "pdf reader", "pdf writer", "report-compare", III.3, VI.1 to VI.4, D-R2, D-R3, D-R12, D-R13, D-R14; `RUST.md` §8.7, §8.8, §9.4, §9.7; `PDF_IMPORT_HARDENING.md` §5; CLAUDE.md "PDF", "Schemas", workflow gates +- status: R4.0 only. No code and no fixture generated; R4.1 starts on the owner's explicit go. +- not in scope: TTF fonts, `scope_disclaimer`, collection caps on read, opening the import to external users (BDR-040 gate stays open), R5 (pipeline, `packages/anatomy`, `packages/domain`, `documentSha256`), the Safari download fix (separate bugfix), release 0.2.0, any deployment, any visible change + +## 0. Facts this plan rests on (develop ae845c6, inventory 2026-10-17) + +Engine, consumers, tests +- `packages/pdf/src` 3,122 lines: reader (`attachment/scan.ts` 549, `inflate.ts` 83, `read.ts`), writer (`export.ts` 228, `render/*` about 1,250 incl. `observationPage.ts` 634), `branding.ts` 291 (PNG gate), `svg/*` (TS sanitiser and view composition), `text.ts`, `render/dates.ts`, `i18n` (25 labels). Tests: 11 files, 140 cases (scan 39, inflate 9, read 13, security 55, text 5, size 2, goldens 11, raster 2, reexport 1, R3 oracle generators 3). +- App consumers: `src/pdf/{import,export,browserRasterizer}.ts`, `EditorRoute` (type only), `scripts/compare-reports.ts` + `compare/*` (589 lines, 11 cases). PDF-related app suites: pdf.goldens 13, pdf.roundtrip 16, pdf.import 5, archive.envelopes 3, n-1.reader 6, pdf.export.app 2, migration.leaves 2, pdf.browserRasterizer 2, scripts.compare 11, plus route and store suites that mock `@/pdf/*`. Only pdf.goldens, pdf.import and pdf.roundtrip load `pdf-lib`; `apps/app/src` never does. +- `release/0.2.0` and develop carry the same `packages/pdf/src`, and the same archive, report and sheet schemas as the N-1 Zod copy (`git diff release/0.2.0 1a7ee7d` on those directories: empty). One frozen TS scanner plus the N-1 copy therefore stand for the R3 build and for 0.2.0. +- Import today: whole file read on the main thread, TS byte scan, `JSON.parse(new TextDecoder().decode(data))` (scan.ts:498, non-fatal, one BOM stripped), candidate kept if it is an object with a numeric `version` or `envelopeVersion`, then `JSON.stringify` of that object into the core's `archive.import` (sheetOps.ts:33). A trap there poisons the main instance and spends one of its 3 recoveries. +- Rescue copy today: routed on the first KiB (`%PDF-` goes the PDF path), ≤ 16 MiB, `TextDecoder('utf-8', { fatal: true })`, core `draft.restore` on the main instance. + +Writer (pdf-lib 1.17.1 as used) +- Fonts Helvetica, Helvetica-Bold, Times-Roman, WinAnsi, not embedded. Width = Σ(AFM width, 250 when absent, + KPX kerning) × size / 1000; glyphs drawn unkerned. +- Operators: text `q BT rg Tf TL Tm ( Tj T*)+ ET Q` (TL 24, one `Tj T*` per line); image `q cm cm cm cm Do Q`; line `q RG w d m m l S Q`; rectangle `q rg w d cm cm cm m l l l h f Q` (no `re`). Operands printed by `numberToString` (JS shortest round trip with exponent expansion). No opacity, no ExtGState. Resource names `Tag-<10 digits>` from a seeded RNG (deterministic). Streams Flate (pako level 6). Classic xref (`useObjectStreams: false`), no `/ID`. +- Images: PNG fully decoded (UPNG, RGBA8), RGB plane plus a DeviceGray SMask only when an alpha < 255, re-deflated; no dedup. +- Info: Title, Author, Subject, Keywords (`faunosteo-archive/1`), Producer and Creator (`pdf-lib (https://github.com/Hopding/pdf-lib)`) as UTF-16BE hex with BOM; CreationDate and ModDate = `exportedAt` as literal `D:YYYYMMDDHHMMSSZ`. +- Attachment: `JSON.stringify(envelope, null, 2)` in UTF-8, ≤ 16 MiB; EmbeddedFile `/Subtype /application#2Fjson`, Params Size, CreationDate, ModDate; Filespec `/F (osteo-report.json)`, `/UF` and `/Desc` hex UTF-16BE, `/AFRelationship /Data`; catalog `Names/EmbeddedFiles` (key hex UTF-16BE) and `/AF`. +- Links: low-level dict, `Border [0 0 0]`, URI action holding the hex of the ASCII bytes. +- Dates: `Intl.DateTimeFormat` fr-FR and en-US, long month, UTC; input must match `^\d{4}-\d{2}-\d{2}$`, else printed unchanged; `Date.UTC` maps years 0 to 99 to 1900 to 1999 (`0050-03-04` → "4 mars 1950"); a rolled-over date is printed unchanged. +- Text: `safeText` removes `[\p{Cc}\p{Cf}\p{Zl}\p{Zp}]` and replaces non-WinAnsi code points with `?`; `wrapLines` splits on JS `/\s+/`; `toUpperCase` on severity, view and zone names. +- Caps: 500 observations × 8 distinct views (`PAGE_CAPS`); rasters 1200 px wide, aspect clamped to 0.2 to 2.0; the views of one observation rasterised in parallel (`Promise.all`), embedded in order. + +Measurements (Node 24, TS engine, resvg test rasteriser, fr) + +| Report | Views rasterised | PNG in | PDF | Stored attachment | +|---|---|---|---|---| +| v2full nologos | 3 | 1.63 MB | 1.17 MB | 54 KB | +| demo nologos | 7 | 3.75 MB | 2.64 MB | 55 KB | +| demo logos | 8 | 3.79 MB | 2.82 MB | not measured | +| v2unresolved logos | 1 (SVG logo) | 0.03 MB | 0.26 MB | not measured | + +Per view: PNG 526 to 553 KB, decoded RGBA 4.4 to 4.5 MB, about 0.37 MB of image stream in the PDF. Browser canvas PNG sizes: not measured yet (R4.1 baseline). + +Reader experiments (scratch, 2026-10-17) +- `String::from_utf8_lossy` after stripping one leading BOM equals `new TextDecoder().decode` on 200,015 random and edge byte strings (overlongs, UTF-8-encoded surrogates, truncated sequences, double BOM): 0 differences. +- serde_json refuses `1e400` ("number out of range") where `JSON.parse` gives `Infinity`, which `JSON.stringify` then writes `null`. `1e-400`, `-0`, integers beyond u64, long decimals and duplicate keys come out as in the JS path once printed by `js_number`. A lone surrogate escape fails in serde_json (classified `lone_surrogate` since R2). + +Browsers and tooling +- Module workers: Chrome 80, Firefox 114, Safari 15. Documented floor for `'wasm-unsafe-eval'`: Chrome 97, Firefox 102, Safari 16 (DEPLOY.md, which calls it "the Vite 7 baseline"); the Vite 7.3.6 default build target is chrome107, firefox104, safari16. +- Vite 7.3.6: production workers are `iife` by default, asset URLs resolved against `self.location.href`; in dev a classic worker gets `importScripts` while its code keeps ESM imports, so dev needs a module worker. +- CSP (DEPLOY.md, sent `always` by the proxy; `docker/nginx.conf` sends none): `script-src 'self' 'wasm-unsafe-eval'`, no `worker-src`, which falls back to `script-src`: same-origin worker files allowed, `blob:` workers refused. +- Playwright (playwright-core 1.64.0, outside the repository as for the CSP proofs): Chromium present; Firefox 157 installed 2026-10-17 and proven headless (classic and module workers instantiate wasm); WebKit 27.2 downloaded, starts only once `libevent-2.1-7t64` and `libmanette-0.2-0` are installed (sudo, owner). +- WASM: 395.6 KB gzip of the 600 KB budget; linear memory after the editor path 4.1 MiB; sheet bundle text about 522 KB, handle 8 to 10 ms. +- R2 and R3 kept no per-test traceability table (prose census only). + +## 1. Outcome + +The application reads and writes archives with Rust only. An import, PDF or rescue copy, runs in a Web Worker created for it, with its own wasm instance, terminated after the answer, a timeout or a crash: the main instance never sees the file's bytes. The export composes the views in Rust, rasterises them with the existing canvas adapter and writes the PDF in Rust through an incremental session, placed in the main instance or in a disposable worker by the measured rule of §6. Old PDFs read exactly as before. New PDFs render pixel-identically to R3 for the same inputs; the one decided metadata change is Producer and Creator `Faunosteo`. `pnpm report:compare` keeps its name on a Rust binary. `packages/pdf` and `pdf-lib` leave the repository, and every TS test case is traced in a ledger. + +## 2. Owner arbitrations and plan decisions + +Owner, 2026-10-17 +- A1 one feature branch, one final merge, J1 stop once the import is wired. +- A2 old PDFs behave as before; UTF-8 differences documented; any break presented before it is applied. +- A3 years 0 to 99 unchanged (printed 19xx). +- A4 Producer and Creator `Faunosteo`. +- A5 new tracing goldens in `goldens/r4/`; frozen goldens untouched. +- A6 zero render difference; any difference explained, measured, submitted. +- A7 existing user message for import failures; timeout, worker crash and read error told apart technically; 20 s is a hypothesis to measure. +- A8 rescue copies restored through the import worker, same isolation and limits. +- A9 Firefox and WebKit for Playwright; real Safari still required; module workers vs the Firefox 102 floor handled by options (§11), policy unchanged unless the owner decides. +- A10 `pnpm report:compare` kept. +- A11 out of R4: TTF, `scope_disclaimer`, collection caps; the import stays closed to external users. +- A12 WASM budget exceeded: stop and options. +- A13 real anonymised PDFs join the corpus when provided (§7.2). + +Plan decisions (internal; trade-offs stated) +- P1 Replicate the reader, never tighten it. Lossy UTF-8 identical to TextDecoder (measured), one BOM stripped, and the two side effects of the JS path reproduced: a number overflowing f64 becomes `null` before the core reads the text (what `JSON.stringify` did with ±Infinity); a candidate holding a lone surrogate escape is still probed for its numeric version (JSON.parse accepted it) and is then refused by `import_archive` as `lone_surrogate`, as today. `session::import_archive` and the layered readers are not modified (D-R12): R4 adds the byte layer in front of them. Any other difference found by the differential corpus is a stop (A2). +- P2 Content streams come from our own emitter, which reproduces pdf-lib's operator sequences and `numberToString`; `pdf-writer` writes objects, dictionaries, strings, xref and trailer (D-R2). Same operators and operands make the render identical by construction, and give a complete oracle (normalised stream equality) on top of the draw log. +- P3 The writer is an incremental session: begin, raster requests per observation, add rasters, finish. The review's two-call shape holds every PNG in linear memory at once (§6); the session holds one decode at a time and keeps the TS engine's per-observation loop, concurrency and warnings. Image objects are written into the output buffer as soon as their PNG is decoded, so nothing is held twice. Placement by §6.4. +- P4 One worker script for core tasks, one task per worker instance (`import.pdf`, `import.copy`, and `export.pdf` if §6.4 selects it), terminated in a `finally`. +- P5 The default PDF labels (25 keys, two locales) move verbatim from `packages/pdf/src/i18n` to `apps/app/src/pdf/labels.ts` and stay an explicit, archived input (dictionaries stay TypeScript; same key order, so the attachment bytes do not move). +- P6 AFM tables for the three fonts are generated once from the frozen pdf-lib metrics (Adobe Core 14 AFM, APAFML notice committed) and checked against that oracle. +- P7 Cargo tests feed frozen PNGs to the writer; no rasteriser in the workspace (D-R3). +- P8 `exportFromEnvelope` is ported (a bilan reproduced from its own snapshot, branding and labels); no UI (BDR-040). +- P9 New crates: `pdf-writer` (MIT or Apache-2.0), `miniz_oxide` (inflate and deflate, already locked through `image`; one version for the core), `png` (0.18.1, already locked) under decoding limits; `lopdf` as a dev-dependency only. No new npm dependency. + +## 3. Architecture and contracts + +### 3.1 Modules (`crates/core`, features `pdf-read`, `pdf-write`, `cli`; the wasm facade enables both pdf features, the worker crate neither) + +``` +src/pdf/ + read/{header,scan,strings,inflate,probe,stamp}.rs bytes → Attachment | ReadKind + json_js.rs lossy decode, overflow → null, lone-surrogate-tolerant probe (one linear pass with the depth pre-scan) + import.rs import_pdf(sheet, bytes) = read → JS-faithful text → session::import_archive (unchanged) + write/ + input.rs ExportInput + checks (branding and labels schemas, assert_snapshot, read_svg per asset, + sheet ref match, caps), envelope written as JSON.stringify(_, null, 2) with js_number + session.rs ExportSession state machine + views.rs raster requests: compose_highlight (R3) per (observation, view), raster_size_of + layout/{observation,conclusion,header,footer}.rs → DrawLog + text.rs, afm/, dates.rs, png.rs (gate, decode, alpha split), content.rs (emitter, number_to_string), + serialize.rs (pdf-writer) +src/bin/report_compare.rs feature cli, the only file allowed std::fs +``` + +### 3.2 Reader contract (identical to scan.ts, pinned by O8 to O10) + +File ≤ 50 MiB and `%PDF-` within the first 1,024 bytes, else `invalid_pdf`. At most 32 `/Type /EmbeddedFile` dictionaries, each bounded to 4,096 bytes; object header from a 48-byte window; `stream` then `\r\n` or `\n`; a direct `/Length` when it lands on `endstream`, else the length measured up to the first `endstream` within 16 MiB + 16 bytes (one trailing `\n`, then one `\r`, dropped); `/FlateDecode` alone or first in an array, else raw bytes. Names from Filespec dictionaries in clear (`/F`, else `/UF`); `osteo-report*` first (stable), then data offset. One 16 MiB budget shared by the candidates; a cancelled inflation is not charged. A candidate opens on `{` after an optional BOM and JSON whitespace, nests at most 64 levels (raw bytes, strings skipped, a backslash skips one byte), then lossy decode, JS-faithful probe, and the chosen text goes to `import_archive`. No candidate: `attachment_stripped` when a `/Keywords` string (≤ 32 occurrences, 4,096-byte window, hex or literal with the same limited unescape, UTF-16BE with BOM else Latin-1) contains `faunosteo-archive`, else `no_attachment`. Inflation: zlib framing, accepted and refused like `DecompressionStream('deflate')` on O10 (truncated, bad Adler-32, trailing bytes, preset dictionary), output capped by the remaining budget, cancelled beyond it. + +### 3.3 Writer contract + +Input `ExportInput { report, sheetSnapshot, branding, labels, locale, exportedAt }` (ts-rs; `exportedAt` an ISO string from the caller). Errors before any page, with today's kinds: `branding_invalid`, `snapshot_invalid`, `snapshot_too_large`, `sheet_mismatch`, `too_many_pages`. Warnings unchanged: `view_raster_failed { observationId, viewId }`, `logo_failed { slot }`. Output: PDF bytes and warnings. The layout produces a `DrawLog` (per page, in call order: text, image, line, rectangle, link, every argument at full precision); `content.rs` emits pdf-lib's operator sequences; `serialize.rs` writes the catalog (Pages, Names/EmbeddedFiles, AF), A4 pages (595.28 × 841.89), three Type1 fonts with WinAnsiEncoding, image XObjects (DeviceRGB 8 bits, DeviceGray SMask with `Decode [0 1]` when needed), Link annotations, Info (Producer and Creator `Faunosteo`, the rest as today in the same string forms), the attachment objects of §0, a classic xref, no object stream, no `/ID`. + +### 3.4 Facade (`crates/wasm`), JSON strings as before, bytes as `Uint8Array` + +- `SheetHandle.importPdf(bytes) → string`: `{ ok: true, value: ArchiveImport } | { ok: false, kind, detail }`, kinds `invalid_pdf`, `no_attachment`, `attachment_stripped`, `incompatible_version`, `lone_surrogate`, `invalid`, `internal`. The rescue copy keeps `SheetHandle.call('draft.restore', …)`. +- `ExportSession` (wasm-bindgen struct): `new(inputJson)` (throws `": "` like `SheetHandle`), `next() → { kind: 'rasters', requests: [{ key, svg, widthPx, heightPx }] } | { kind: 'done' }`, `addRaster(key, png)`, `rasterFailed(key)`, `finish() → Uint8Array`, `warnings() → string`; `ExportSession.fromEnvelope(envelopeJson, locale, exportedAt)` for P8. +- Linear memory is read from the instance's exported `memory` for diagnostics; no new export. +- ts-rs types committed under `packages/core-wasm/types`: `ExportInput`, `RasterRequest`, `ExportStep`, `ExportWarning`, `PdfImportKind`. + +### 3.5 Core worker protocol (`apps/app/src/core/worker/`, messages tagged `v: 1`) + +1. The main thread creates the worker from a same-origin file (classic in the production build, module in dev: OD-1) and arms the task timer. +2. The worker fetches the shipped module (the main runtime's Vite asset URL), compiles (`compileStreaming`, fallback `compile`), instantiates, posts `{ type: 'ready' }`. A failure before `ready` is `worker_unavailable`. +3. The main thread posts `{ v: 1, task, file, sheet, limits }`: the `File` itself (structured clone of a Blob, nothing read on the main thread) and the bundle text the main runtime already builds (about 522 KB). +4. The worker re-checks `file.size` (50 MiB PDF, 16 MiB copy), reads `file.arrayBuffer()` once, builds a `SheetHandle`, runs `importPdf`, or the strict decode then `draft.restore`, and posts `{ type: 'result', out, diag: { ms, linearBytes } }`. A `WebAssembly.RuntimeError` is caught and posted as `{ type: 'crashed', detail }`. +5. The main thread: `result` → parse → the existing hydrate path; `crashed`, `error` or `messageerror` after `ready` → `crashed`; timer → `timeout`. `terminate()` in every case, timer cleared, references dropped. One task at a time (the Home busy flag already serialises imports). `diag` is printed in DEV only and never sent anywhere. +6. The main runtime is untouched by worker failures (no recovery spent, no poisoning). When it is `stopped`, the import is refused up front with today's stopped text, because the editor could not open the result. +7. Tests: vitest (happy-dom has no `Worker`) runs the worker entry in-process through the client's spawn seam; the real worker runs in the browser harness. + +### 3.6 Export driver + +The main thread computes `archive.snapshot` (needs the sheet, as today), builds `ExportInput` (branding from `loadBranding()`, labels from P5, `new Date()`), then loops: `next()`, rasterise the batch with `browserRasterizer` (`Promise.all`, as today), `addRaster` or `rasterFailed` per request, until `done`; `finish()`, download, warnings alert unchanged. In the worker placement the same loop runs over messages: SVG strings to the main thread, PNGs transferred to the worker, the PDF transferred back. A trap during an export in the main instance spends a recovery as any core call today; in the worker it does not. + +### 3.7 `report-compare` + +A `crates/core` binary behind `cli`, same arguments, output lines and exit codes as the TS CLI (O11, LRN-044 escaping). Root script `report:compare` → `cargo run --release --locked -q -p faunosteo-core --features cli --bin report-compare --` (the development machine's toolchain; the first run compiles). + +### 3.8 Import outcomes and messages (existing keys only) + +| Outcome | Raised by | Message | +|---|---|---| +| file > 50 MiB, no `%PDF-` | main pre-check, worker | `home.import.error_invalid_pdf` | +| `no_attachment` | reader | `error_no_attachment` | +| `attachment_stripped` | reader | `error_attachment_stripped` | +| `incompatible_version` | core | `error_incompatible_version` (version shown) | +| `lone_surrogate` | core | `error_lone_surrogate` | +| `invalid` (rebase refused) | core | `error_generic` (as today) | +| copy > 16 MiB, invalid UTF-8, refused by `draft.restore` | main pre-check, worker | `error_invalid_copy` | +| no bundled sheet; worker cannot start (CSP, script, module, no `Worker`) | main | `error_core_unavailable` (as a failed engine load today) | +| `timeout` or `crashed`, PDF | main | `error_generic` « Erreur de lecture du PDF. » (A7; today a trap showed `error_core_unavailable`) | +| `timeout` or `crashed`, copy | main | OD-3 | +| main runtime stopped | main | `core.stopped` or `core.stopped_unsaved` (unchanged) | +| held session | store | `session.held_blocked*` (unchanged) | + +The technical kind stays in the outcome and in the DEV console. + +## 4. PDF compatibility + +### 4.1 Historical behaviours preserved (each pinned by O8 or O9 rows) + +- H1 bare v2 MVP PDFs with object streams (`report-v2.full.pdf`, `report-v2.minimal.pdf`): migrated through the map. +- H2 bare v3 from Step 5 (`report-v3.bare.pdf`): read as is, rebased when the sheet differs. +- H3 envelopes v1 from Step 6 (pre-R2 key order, O3) and from R2 and R3 (O2): layered reading, report kept when the snapshot is rejected. +- H4 stamped PDFs with object streams: a stripped attachment still reads `no_attachment` (known trade-off). +- H5 several embedded files: `osteo-report*` first, then data order; decoys skipped. +- H6 indirect, wrong or missing `/Length`; `\r\n` after `stream`; `/Type/EmbeddedFile` without a space; hex and literal names; unfiltered streams. +- H7 caps: 50 MiB file, 32 candidates, 4 KiB dictionaries, 16 MiB shared budget, depth 64; bombs refused without memory growth. +- H8 invalid UTF-8 becomes U+FFFD, one BOM stripped (same decoder, measured). +- H9 JSON: a duplicate key keeps the last value at the first position; an overflowing number becomes `null`; a lone surrogate is refused (`lone_surrogate`, R2); numbers are f64. +- H10 encrypted PDF: `no_attachment`. +- H11 every §3.8 message for the same file. +- H12 rescue copy: BOM accepted, strict UTF-8, ≤ 16 MiB, `draft.restore`, `editor.draft_rebased` when rebased. +- Declared, decided at R3 (RUST.md §9.4) and now applied to the PDF path: `exportFromEnvelope` on a foreign archive whose snapshot SVG the R3 reader refuses answers `snapshot_invalid` where the TS sanitised and drew; no product drawing is affected (R3 proof), no interface reaches it. + +### 4.2 Readers that must read R4 PDFs (N-1, review VI.3) + +The frozen TS scanner (O12) and the N-1 Zod copy, standing for the R3 build and 0.2.0 (§0), read every R4 fixture PDF and app-path export; the R3 build, run side by side, reads them at the final acceptance. The live site (pre-Step 6) reads no envelope today; R4 changes nothing there. The DEPLOY.md rollback note says that R3 and 0.2.0 read R4 PDFs. + +### 4.3 Difference classes (R3 build vs R4 build, same inputs) + +Acceptable, no validation needed, each checked by a test: +- B1 Flate bytes of every stream (miniz_oxide vs pako): the decompressed bytes are compared. +- B2 object numbers, order and offsets, header binary comment, xref and trailer layout (both classic xref, no object stream). +- B3 resource names (`/Helvetica-7036340225` → `/F1`, `/Image-…` → `/Im1`): streams compared after normalising names. +- B4 string forms with the same decoded value (literal vs hex) where pdf-writer chooses, such as the link URI. Info, `/UF`, `/Desc` and the names-tree key keep pdf-lib's UTF-16BE hex form, checked by the frozen TS scanner. +- B5 numbers in dictionaries no content depends on, at pdf-writer's `f32` precision (link `/Rect`, under 0.0001 pt), if pdf-writer has no exact primitive; MediaBox expected to print the same (checked). +- B6 dictionary key order, whitespace, structural keys without effect (each listed when found). + +Decided (A4): Producer and Creator `Faunosteo`. Every other Info value identical: Title, Author, Subject, Keywords, CreationDate, ModDate, no `/ID`. + +Not acceptable without the owner (stop, measure, explain, submit): +- a pixel difference in the poppler render (150 dpi on every page; 300 dpi on the real-raster reference); +- a content-stream difference after B3 normalisation; +- a decoded image difference (RGB, alpha, size, bit depth, colour space, SMask presence); +- a draw-log difference (text, font, size, colour, position, line, rectangle); +- the set of links, a URI, or a rect beyond 0.01 pt; page count, size or order; +- the attachment: filename, subtype, Params, Desc, AFRelationship, names tree, `/AF`, or the decompressed JSON bytes (target: byte-identical to the R3 output for the same input; a key-order or formatting change is a contract change, presented with its N-1 proof); +- an import outcome difference on any file of O2, O3, O8, O9, the hostile set or the owner's real PDFs. + +## 5. Import and memory + +- 5.1 Isolation. Untrusted bytes (PDF or copy) are read only inside the worker created for that import. The main instance receives the core's canonical output (validated report and envelope), as it does today after `archive.import`. This closes the R3 residual (16 MiB parsed in the long-lived instance, RUST.md §8.8) and two BLK-012 residuals (a trapping copy spends a main recovery; memory exhaustion near 16 MiB in the main instance). +- 5.2 Transfer. The `File` is posted, never read on the main thread; the worker checks the size, reads once into an ArrayBuffer, copies once into linear memory. The sheet travels per import as the bundle text, the worker builds its own `SheetHandle`; nothing is cached across imports. The result comes back as one JSON string (report and envelope); its cost is measured. +- 5.3 Time. One constant `IMPORT_TIMEOUT_MS` (20 s hypothesis) from creation to answer. R4.3 measures per browser the worker start (cached fetch, compile, instantiate, handle), the largest legitimate archive (demo with logos) and the slowest hostile case; J1 repeats on the owner's devices, the low-end phone included. Rule proposed: timeout ≥ 10 × the slowest legitimate import on the slowest device, and ≥ 2 × the slowest hostile case that completes. The owner sets the value at J1 if the rule moves it away from 20 s. +- 5.4 Destruction. `terminate()` after a result, a crash or a timeout. The harness checks that the content process comes back within 10 MiB of its pre-import level after a forced GC (Chromium CDP `HeapProfiler.collectGarbage`; Firefox `about:memory` minimise at J1). +- 5.5 Errors and recovery. The §3.8 table; no automatic retry (the practitioner picks the file again); the file and the session are never modified; held sessions and the replace dialog behave as today. +- 5.6 Out of memory. A worker bounds the blast radius, not the peak: it shares the tab's content process on every engine, and iOS kills the whole tab beyond its limit. The harness measures the worker peak on the classes that grow memory most: a 16 MiB token-dense JSON attachment (serde_json `Value` expansion, estimated 8 to 30 × in linear memory), the 50 MiB file, the 16 MiB inflation, 32 candidates. Budget W4 below. Options when a class breaks it, each a behaviour change on hostile files only, presented at J1, none applied by default: (a) link the module with `--max-memory` so growth fails as a trap before the OS limit (it also caps the main instance, so it sits above the largest export need of §6); (b) a value-count budget in the existing linear pre-scan; (c) an attachment cap below 16 MiB (real archives are under 0.4 MB); (d) typed deserialisation without the intermediate `Value` (changes the R2 reader, gated). +- 5.7 Browsers: §11. +- 5.8 Rescue copy. Same worker, timeout and termination; the strict UTF-8 decode stays in TypeScript inside the worker (same decoder), `draft.restore` runs in the worker's instance; the routing on the first KiB stays on the main thread (five bytes and the first non-blank byte). + +## 6. Export memory: PNG accumulation in linear memory + +### 6.1 Where the bytes live + +From the measured sizes, a report with V rasterised views takes about 0.54 V MB of PNG input (resvg; browser PNGs measured at R4.1), 4.5 MB of decoded RGBA per view at a time, and yields about 0.08 + 0.37 V MB of PDF. The image streams are our own deflate of the RGB plane, so their size does not depend on the browser. + +| Design | Peak linear memory | Kept by a long-lived instance afterwards | +|---|---|---| +| Two calls (review sketch): every PNG in, then write | ΣPNG + PDF + one decode ≈ 0.91 V + 4.5 MB | the same high-water (linear memory never shrinks) | +| Session in the main instance | PDF + one decode ≈ 0.37 V + 4.5 MB | the same | +| Session in a disposable worker | as above, inside the worker | nothing (worker terminated) | +| TypeScript today (pdf-lib, JS heap) | ≈ PDF + up to 8 PNGs and their decodes | nothing (garbage-collected) | + +Examples, two calls / session: V = 7 (demo) 11 / 7 MB; V = 120 (60 observations, 2 views each) 114 / 49 MB; V = 500: 460 / 190 MB. The cap (500 × 8 = 4,000 views, about 1.5 GB of PDF) is not exportable today either; R4 must not fail earlier than the TS engine does (baseline O13). + +### 6.2 Strategies + +- S1 incremental session (P3), chosen by construction: it takes ΣPNG out of linear memory at no cost on the JS side, which already rasterises per observation. +- S2 disposable export worker: frees the high-water after each export. +- S3 chunked output: image streams handed to JS as Blob parts as they are written, the PDF assembled as a `Blob`; only if S2 still fails on a device. +- S4 visible or behavioural measures (lower raster resolution, a smaller cap): never without the owner. + +### 6.3 Measurement + +Harness in Chromium and Firefox (WebKit once it starts): linear memory after each session step, content-process RSS sampled every 50 ms from `/proc`, JS heap from CDP; reports demo, L (V = 120) and XL (V = 500) built from the demo's observations (O14), and a doubling probe up to failure, for the TS baseline (R4.1) and for R4 (R4.6). Phones: L at the final acceptance. + +### 6.4 Decision rule, fixed now + +Keep the session in the main instance if, for L, the main instance keeps at most 32 MiB more linear memory than before the export and L completes on every test device; otherwise run it in the disposable worker (S2). The sizes above predict S2 (about 49 MB kept for L): the plan budgets it, the measurement confirms or refutes it before R4.6 wires the application. If S2 fails L on a test device: stop and present S3 and S4. + +## 7. Fixtures and oracles + +### 7.1 Frozen with the TS code at R4.1, before any Rust (generators committed behind `MAKE_R4_ORACLES=1`; never regenerated without the owner) + +| Id | Content | Location (planned) | Used by | +|---|---|---|---| +| O1 | draw log, 3 reports × fr/en × logos/nologos = 12 files: every drawText, drawImage, drawLine, drawRectangle and link call in order with all arguments at full precision (the step6 fixed rasterisers and FIXED_EXPORTED_AT) | `apps/app/tests/fixtures/goldens/r4/draw.*.json` | R4.4 | +| O2 | the 12 matching R3-build PDFs, plus one real-raster PDF (v2full fr logos) with its 3 resvg view PNGs and its SVG-logo PNG | `apps/app/tests/fixtures/pdf/r3/` (about 5 MB) | R4.2, R4.5 (normalised streams, image planes, links, Info, attachment), N-1 | +| O3 | pre-R2 envelope PDFs: the three frozen `goldens/r2/envelope.*.json` attached byte for byte by pdf-lib 1.17.1 (Step 6 layout) | `apps/app/tests/fixtures/pdf/step6/` | reader (H3), N-1 | +| O4 | text: widths (strings × sizes × 3 fonts), `wrapLines`, `safeText` over every code point class, JS `toUpperCase` over all code points (differences only), JS `\s`, `Math.round`, `numberToString` (random and edge values) | `crates/core/tests/oracles/r4/text.json` | R4.4 | +| O5 | pdf-lib metrics of the 3 fonts (widths, KPX, WinAnsi map) | `oracles/r4/afm.json` | P6 | +| O6 | `formatDate` fr and en over years 0000, 0001, 0050, 0099, 0100, 0999, 1000, 1900, 1904, 1999, 2000, 2024, 2026, 9999 × months 00 to 13 × days 00 to 32, plus non-matching strings | `oracles/r4/dates.json` | R4.4 | +| O7 | PNG: `pngIssue` verdict for every security-suite PNG, the brand logos and a variant set (palette, palette + tRNS, grey, grey + alpha, 16-bit, RGB, RGBA); for accepted ones, sha256 of pdf-lib's RGB and alpha planes | `oracles/r4/png.json` + small PNGs | R4.5 | +| O8 | reader outcomes for every input of scan, inflate, read and the reader part of security (inputs ≤ 64 KiB committed; bombs as deterministic generators ported to Rust and checked by the sha256 of the generated bytes): kind, chosen candidate, attachment and object digests | `oracles/r4/read.json`, `fixtures/pdf/r4/read/` | R4.2 | +| O9 | differential corpus: UTF-8 classes, BOM, double BOM, overflowing numbers (in `version` and in content), lone surrogate (probe and content), duplicate keys (incl. `version`), integers beyond 2^53 and u64, `-0`, `__proto__`, depth 64 and 65, a first candidate only JSON.parse accepts; outcome of the full R3 path (TS scanner, JSON.parse and stringify, core `archive.import` in wasm) | `fixtures/pdf/r4/tolerance/`, `oracles/r4/tolerance.json` | R4.2 (a difference is a stop, A2) | +| O10 | inflate: valid, empty, truncated, bad Adler-32, trailing bytes, raw deflate without zlib header, preset dictionary, exact cap, cap + 1: `DecompressionStream` verdict and output digest | `oracles/r4/inflate.json` | R4.2 | +| O11 | `report:compare` stdout and exit code on every fixture PDF and the 11 scenarios | `oracles/r4/compare/` | R4.7 | +| O12 | frozen TS scanner: verbatim copy of R3's `attachment/{scan,inflate,read}.ts` and their error module, never edited: the permanent N-1 PDF reader | `packages/core-wasm/tests/n-1/scanner/` | N-1 tests | +| O13 | TS baseline in Chromium and Firefox with the real `browserRasterizer`: PNG bytes per view, export time, RSS peak and after, JS heap, PDF size (demo, L, XL, doubling probe); TS import time and memory on O2 and the hostile set | table in RUST.md §10 (draft) | §5, §6, §10 | +| O14 | synthetic reports L (60 observations × 2 views) and XL (250 × 2) built from the demo | `apps/app/tests/fixtures/r4/reports/` | O13, R4.6 | + +The step6, r2 and r3 goldens and the R3 oracles stay untouched: `git diff --stat` on them is empty at every slice. + +### 7.2 Real PDFs (A13) + +Kept out of git on the development machine (`$FAUNOSTEO_FIELD_CORPUS`), read by an opt-in test; only a table without personal data is committed (sha256, producing build when known, expected outcome, observation count). A file enters git only after the owner reviews its anonymised form (attachment strings and page text replaced by same-length placeholders, structure kept), never automatically. + +## 8. Test traceability + +`crates/core/tests/pdf/LEDGER.tsv`: one row per TS case at the freeze commit, with the TS file, the full case title (describe path, each-row label), a disposition (`ported`, `replaced`, `kept`, `deleted`), the target (cargo `file::test` or vitest `file > title`), the oracle id and a note. Built at R4.1 from vitest's JSON reporter: the 140 engine cases, the PDF-related app cases of §0 and the 11 CLI cases. A census check fails when a frozen case has no row, when a `ported` target is missing from `cargo test -- --list` or the vitest list, or when the counts do not reconcile; the verifier runs it at R4.9. Rust test files keep a header naming their TS source, as at R2 and R3. + +## 9. Slices + +One branch, `feature/rust-r4-pdf-core`, cut from develop once this plan is merged. Every slice ends green on `pnpm rust:check`, `pnpm wasm:build` and `pnpm check`, its proof recorded in the TODO and in RUST.md §10. + +| Slice | Deliverable | Proof | +|---|---|---| +| R4.1 Freeze | O1 to O14, ledger, browser harness (`scripts/browser/`, Playwright kept outside the repository) | generators committed, frozen goldens untouched, suites green, baseline table; no Rust | +| R4.2 Reader | `pdf::read`, `json_js`, `pdf::import`, `SheetHandle.importPdf`; scan, inflate, read and reader-security suites in cargo on O8 and O10; O9 differential; hostile corpus through the wasm build (Node, fresh instance per case, time and linear memory) | 0 oracle difference, 0 differential difference (else stop, A2), no trap outside declared memory cases, W1 | +| R4.3 Import worker and rescue copy | §3.5 client and entry; HomeRoute on the worker (routing and pre-checks unchanged); OD-1 applied; client tests (ready, timeout, crash, messageerror, unavailable, terminate always, main runtime untouched); a message-table test over O2, O3, O9 and the hostile set; 612 leaves through the worker (export still TS); CSP proof on the app image with and without `'wasm-unsafe-eval'` in Chromium and Firefox (WebKit when it starts); time and memory measured | same messages as R3 on every file, tab responsive during hostile imports (W5), memory back to baseline (W4), P-W1 | +| J1 Owner checkpoint, no merge | production build (`SHEET_BUNDLE=draft`) behind the DEPLOY CSP on the DEV origin (port agreed with the owner, ENVIRONMENTS.md); a French checklist; old PDFs (v2 MVP, v3 Step 5, Step 6 and R3 envelopes), O2, a hostile folder with the expected message per file, rescue copies (valid, BOM, invalid UTF-8, oversized, refused), engine unavailable during an import; memory and time per device | owner OK; timeout value; OD-1 confirmed on real browsers; 5.6 options decided if needed; results in PDF_IMPORT_HARDENING §5 (gate still open) | +| R4.4 Layout | text, AFM, dates, upper-casing, `safeText`, `wrapLines`, the four layouts → DrawLog | O1 (12) exact at full precision, step6 text and images (24) exact, O4 to O6 exact | +| R4.5 Serialiser | emitter, pdf-writer, PNG gate and decode, links, Info, attachment, `exportFromEnvelope`; export-side security, size, reexport and text suites in cargo | against O2: normalised streams equal, image planes equal, links (lopdf) and step6 links (12) equal, Info equal except Producer and Creator, attachment bytes equal (else declared, A6); `qpdf --check` clean; `pdfdetach -list` shows `osteo-report.json`; poppler diff 0 px; O12 and the N-1 copy read every R4 PDF; E2; digests of the R4 outputs in `goldens/r4/pdf.*.sha256` (wasm output = native output) | +| R4.6 App export | session driver, placement by §6.4 measured first, `labels.ts` (P5), `export.ts` on the session; alerts unchanged | app suites; app-path bytes = native digests; step6 views via the app path; browser rasters of the Rust- and TS-composed views pixel-identical (Chromium, Firefox, WebKit when it starts; the TS composition still exists here); E3, E4; 612 leaves exported and re-imported on Rust | +| R4.7 report-compare | Rust binary, `pnpm report:compare` unchanged | O11 and the 11 cases in cargo | +| R4.8 Deletions and docs | `packages/pdf` (with the R3 oracle generators; the frozen oracle files stay), `@faunosteo/pdf` and `pdf-lib` in apps/app, the TS CLI, the pdf-lib-based tests rewritten per the ledger; N-1 test on O12; RUST.md §10, PDF-FORMAT, PDF_IMPORT_HARDENING §5, DEPLOY (rollback note, worker CSP note, floor line), CHANGELOG, CLAUDE.md, TEST_STRATEGY, WORKSPACE, packages/README, fixtures README (R4 freeze section) | `pnpm check:all` green, ledger census green, D1 | +| R4.9 Gates and acceptance | verify contract, fresh verifier, fresh security audit; owner acceptance (§12), after the separate Safari download fix has merged | verifier CONFORME, security PASS, owner's explicit merge signal | + +## 10. Budgets (measured, recorded in RUST.md §10) + +| Id | Budget | Measured by | +|---|---|---| +| W1 | served module ≤ 600 KB gzip -9; reported at 550 KB; above 600: stop (A12) | `size.test.ts`, every slice | +| W2 | worker start + largest legitimate import ≤ 1 s on the development machine (Chromium, CPU × 4); recorded per J1 device | harness, J1 | +| W3 | timeout value per the 5.3 rule | harness, J1 | +| W4 | worker linear memory ≤ 256 MiB on every hostile class; the tab survives on every device; the content process back within 10 MiB after GC | harness, J1 | +| W5 | main-thread long tasks during an import ≤ 50 ms, hydrate and navigation excluded | Chromium trace | +| E1 | render difference 0 px (poppler 150 dpi; 300 dpi on the real raster) | cargo + poppler | +| E2 | PDF size ≤ R3 + 5 %; stored attachment < 250 KB (demo with logos) | cargo | +| E3 | export time ≤ 1.2 × the TS baseline (demo, L) in Chromium | harness | +| E4 | §6.4 rule for L; XL completes on the development machine; the doubling probe fails no earlier than the TS baseline | harness | +| T1 | every frozen case dispositioned in the ledger | census | +| D1 | no `pdf-lib` or `@pdf-lib/*` in the lockfile; no `@xmldom/xmldom` in apps/app's closure (kept by `packages/anatomy` until R5) | lockfile, `pnpm why` | + +## 11. Browser compatibility + +- Floor as documented: Chrome 97, Firefox 102, Safari 16 for the keyword. The bundle's syntax target is already Firefox 104 (Vite 7 default): a doc inaccuracy that R4.8 corrects in DEPLOY.md whatever OD-1 decides. +- Firefox 102 to 113 lacks module workers. OD-1 options: + - A, recommended: a classic worker in the production build (Vite's default `iife` format, constructor without `type: 'module'` in the build), a module worker in dev only. Keeps the floor and one production path. Proof P-W1 at R4.3: the built worker runs under the production CSP in Chromium, Firefox 157, WebKit (once it starts), and Firefox ESR 102 and 115 binaries downloaded to a scratch directory (Playwright cannot drive a stock Firefox: a test page posts its result to the harness). Cost: worker type differs between dev and production, so J1 runs on the production build. + - B: raise the documented floor to Firefox 115 ESR (module workers everywhere, one worker type in dev and build). A policy change, the owner's decision; DEPLOY.md updated. + - C: feature detection with a main-thread fallback: two import paths, isolation lost where it applies. Not recommended. + - D: refuse the import without module workers: a regression on old Firefox. Not recommended. +- CSP: the worker is a same-origin hashed file under `/assets/`, allowed by `script-src 'self'`; its response carries the proxy's CSP, `'wasm-unsafe-eval'` included; no `blob:` or inline worker. No CSP change is needed; DEPLOY.md gains a note. Without the keyword the import answers `error_core_unavailable`, with no fallback. +- Safari: automated only once WebKit starts (OD-7); real Safari (macOS when available, iOS) at J1 and at the final acceptance. The PDF download revokes its URL at once (`EditorRoute.tsx:245`), which WebKit may cancel: separate bugfix before the final iPhone acceptance, outside R4. +- APIs in the worker: `Blob.arrayBuffer` (Firefox 69, Safari 14), `WebAssembly.compileStreaming` with the main runtime's fallback, `TextDecoder`, `postMessage` with transfer: all under the floor. + +## 12. Checkpoints, stop rules, exit criteria + +- Checkpoints: (1) this plan, validated by the owner, then merged (OD-2); (2) J1 after R4.3; (3) the final acceptance after R4.9. +- Stop rules (stop, measure, present): W1 above 600 KB; a reader or differential difference (A2); a §4.3 "not acceptable" difference; a visible change found necessary; S2 failing §6.4; a W4 class breaking its budget; a failed proof (P-W1, CSP); any change needed inside the gated readers (`import_archive`, `read_archive_document`, `rebase_report_on_sheet`; D-R12). +- J1 exit: the import runs on Rust in the worker; the §3.8 messages match R3 on every file; measurements recorded in PDF_IMPORT_HARDENING.md §5; OD-1, the timeout and any 5.6 option decided. +- Final acceptance (owner, LAN, production build behind the DEPLOY CSP): FR and EN exports of the three reference reports and of a new bilan, on computer and phone; viewers (Chrome, Firefox, Acrobat, Preview or Files, Android): pages, attachment listed, properties (Producer Faunosteo), footer links; a side-by-side R3 vs R4 page with per-page differences, preceded by an R3 vs R3 run that measures the browser's own noise; 400 % zoom; print preview and a paper print; accents in pages and properties; R4 PDFs imported into the R3 build run side by side; R4 PDFs re-imported (same bilan); engine unavailable during an import and an export. +- R4 exit (merge): the review's VI.2 R4 criteria as amended here, the §4.3 proofs, the ledger census, the budgets, J1 and the final acceptance, verifier CONFORME, security PASS, `pnpm check:all` green, the owner's explicit merge signal. + +## 13. Risks + +| Risk | Mitigation | Detection | +|---|---|---| +| Writer exactness: float order, `Math.round`, kerning, upper-casing, the Cf table, dates | port expression by expression; full-precision O1; O4 to O6 | O1, normalised streams | +| pdf-writer's `f32` or string forms break a form the N-1 scanner needs (Keywords, Filespec) | UTF-16BE hex written as pdf-lib does; B5 only for numbers nothing reads | O12 read-back, byte test | +| Reader divergence on foreign or old PDFs | O8, O9, the owner's real PDFs | stop rule A2 | +| miniz_oxide and DecompressionStream accept different streams | O10 | cargo | +| Worker type, CSP or wasm in workers on Firefox and Safari | OD-1 and P-W1, CSP proof on three engines, J1 on real devices | R4.3, J1 | +| Memory exhaustion despite the worker (iOS kills the tab) | W4 classes measured, 5.6 options | harness, J1 | +| Export memory | session by construction, §6.4 rule | harness | +| WASM size (+100 to 150 KB estimated) | three fonts only, features, measured per slice | W1 | +| A test lost in the migration | ledger census | R4.8, verifier | +| Browser canvas noise hides or invents differences in the final comparison | R3 vs R3 noise run first; automated proofs use frozen PNGs | final acceptance | +| Largest phase of the roadmap | J1 split, per-slice proofs, stop rules | slice reports | +| Two PDF stacks coexist until R4.8 | the TS stays frozen (no new logic), deleted in one slice | grep census | + +## 14. Kept and deleted (consumers named) + +- Deleted at R4.8: `packages/pdf/**` (last consumers: app import at R4.3, app export at R4.6, CLI at R4.7, tests per the ledger), `apps/app/scripts/compare-reports.ts` and `compare/*`, `@faunosteo/pdf` and `pdf-lib` in apps/app, `@resvg/resvg-js` and `@pdf-lib/*` with packages/pdf. +- Kept: `browserRasterizer.ts` (DOM adapter), `loadBranding` and `src/assets/brand`, `packages/anatomy/render` and `packages/domain` (pipeline, R5), the frozen R3 oracle files (their generators go with packages/pdf; never regenerated), the N-1 Zod copy, O12. + +## 15. Open decisions for the owner + +| # | Question | Recommendation | +|---|---|---| +| OD-1 | Module workers vs the Firefox 102 floor | A, classic worker in production, proven by P-W1; if P-W1 fails, B (floor Firefox 115 ESR) for your decision | +| OD-2 | Branches: this plan on `chore/rust-r4-plan`, merged at validation; then the Safari fix off develop; then `feature/rust-r4-pdf-core` cut from develop (one feature branch, one feature merge, no sync of develop into it) | yes | +| OD-3 | Message when a rescue copy times out or crashes the worker | `error_invalid_copy` (accurate: refused, file untouched); alternative `error_generic` | +| OD-4 | A PDF that crashes the reader moves from « Le moteur de lecture n'a pas pu être chargé. Réessayez l'import. » (today's trap mapping) to « Erreur de lecture du PDF. » (A7) | confirm | +| OD-5 | A worker that cannot start keeps « Le moteur de lecture n'a pas pu être chargé. Réessayez l'import. » | yes | +| OD-6 | J1 devices and browsers (computer, iPhone, low-end Android) and the LAN port of the J1 build | your list | +| OD-7 | WebKit system packages: `sudo apt-get install libevent-2.1-7t64 libmanette-0.2-0` | install before R4.3 | +| OD-8 | Slot of the Safari download fix | right after this plan's validation, before the feature branch | +| OD-9 | Repository weight of O2 (about 5 MB of PDFs and PNGs) | accept | + +Scheduled later: the timeout value and the 5.6 options (J1), the §6.4 outcome (reported at R4.6), any §4.3 "not acceptable" difference (stop rule). + +## 16. Tracked outside R4 + +- Safari PDF download fix (separate bugfix, before the final iPhone acceptance). +- The worker container ignores SIGTERM: Docker kills it after 10 s, the job resumes after its lease expires (chore, for example `init: true` in compose). +- README test counter stale (367; 1,015 at the R3 merge). From 2c8ac2ebe692cd693c0045e0d9906d804e4a6d38 Mon Sep 17 00:00:00 2001 From: bchanot Date: Sat, 17 Oct 2026 19:33:47 +0200 Subject: [PATCH 2/3] chore(chore): R4 plan rev 2 after three challenges; TODO, journal --- .claude/memory/journal.md | 4 + .claude/tasks/TODO.md | 13 + .../plans/2026-10-17-rust-r4-pdf-core.md | 519 +++++++++--------- 3 files changed, 264 insertions(+), 272 deletions(-) diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index 1816d16e..ab7252e2 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -273,3 +273,7 @@ feature/rust-r3-anatomy-core: plan rev 2 after 3 challengers (32 findings; block Owner brief: 5 checks before R3 close. Copy round trip proven, no in-app restore → Home import reads copy via draft.restore. Loop: fuzz 37k 0 traps (OOM ~50 MB only) → defended: trapping pair held in panel, re-read only on Retry. Work loss: guard app-wide, replace dialog on New report / Import. GC finalizer harmless, bounded, recorded. 9bd9aca; verifier 10/10, security PASS, check:all green. BDR-056, BLK-012, LRN-058, EVAL-024. R3 not merged (LAN pending), no R4, no release, no deploy. - 2026-10-17 owner LAN acceptance R3: all functional + graphical tests passed, go to merge. Docs: RUST.md status (R2, R3 validated), §9 status, CLAUDE.md, CHANGELOG heading, README rescue copy bullet (copy ≠ archive of record) → db8c0ef. `gitflow finish` → develop 8df964b (pushed, branch deleted local + origin). check:all green on develop (Rust, wasm, Node 1015 tests incl. app 405, build, Docker app + worker). Residuals stay in BLK-011/BLK-012. R4 not started; release 0.2.0 untouched; no deploy. Waiting owner instructions. + +## 2026-10-17 (R4.0 plan) + +Owner go R4.0 (A1-A13: one branch + J1, old PDFs preserved, years 0-99 kept, Producer/Creator Faunosteo, goldens/r4, 0 px diff, import message kept + kinds split, copy via worker, Firefox/WebKit, report:compare name, TTF/scope_disclaimer/caps out, WASM stop, real PDFs). chore/rust-r4-plan: plan rev 1 → 3 challengers (19 major, 0 fatal) → rev 2. Facts: Rust lossy UTF-8 = TextDecoder (200,015 cases, 0 diff); serde_json refuses 1e400 (JS → null) → rewrite; TS embedPng holds RGB planes until save (≈ 3.8 MB/view) → session ≈ 0.37 MB/view; import already needs DecompressionStream (Firefox 113, Safari 16.4); glue module record retains instances (no throwaway instance). Firefox 157 Playwright OK, WebKit needs 2 apt libs. No code. Stop: owner validation, OD-1..OD-7. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 2198f219..83191a9b 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -204,6 +204,19 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] R3 final checks 2026-10-17 (owner brief 2026-10-16): rescue copy importable from Home, trapping draft held (no reload loop), replace confirmation, guard app-wide → 9bd9aca (BDR-056, BLK-012); verifier CONFORME 10/10, security PASS, check:all green; residuals in BLK-012 - [ ] R3 follow-ups kept by the owner (2026-10-15): worker TLS exercised against a real HTTPS endpoint before the deployment that needs it; roxmltree limits + SVG prescan (pin the iterative release when published); Firefox/WebKit check; SVG references across drawings scoped per view before community uploads (Step 13); WASM size margin (396 of 600 KB) before R4; TypeScript PDF sanitiser still present until R4 - [ ] R4 pdf core: reader (Web Worker, BLK-009 browser measurements) + writer (`pdf-writer`, Std14 AFM, draw log; lines/rects/colours golden frozen with the TS engine first, D-R2) + `report-compare` tool; `packages/pdf` deleted; N-1 scanner reads R4 PDFs + - [x] R4.0 plan (owner go 2026-10-17, arbitrations A1-A13): `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md` rev 2 after 3 challengers (correctness 6 major, robustness 10 major, simplicity 3 major; none fatal) on chore/rust-r4-plan; Playwright Firefox 157 installed + proven, WebKit 27.2 downloaded (owner: `sudo apt-get install libevent-2.1-7t64 libmanette-0.2-0`); awaiting owner validation + OD-1..OD-7; no code + - [ ] after validation: finish chore/rust-r4-plan into develop (owner signal), Safari PDF download fix (separate bugfix, owner go), then cut feature/rust-r4-pdf-core + - [ ] R4.1 freeze (O1-O13, ledger, browser harness, TS baselines) + - [ ] R4.2 reader (cargo + wasm, differential, OD-3 thresholds confirmed with owner before R4.3) + - [ ] R4.3 import worker + rescue copy (B.1.5, OD-1 applied, browser proof per slice from here) + - [ ] J1 owner checkpoint (plan A.4) + - [ ] R4.4 layout → draw log + - [ ] R4.5 serialiser (pdf-writer, self-check, from_envelope) + - [ ] R4.6 app export (session in main instance, owner phone check of L) + - [ ] R4.7 report-compare in Rust + - [ ] R4.8 deletions + docs + ledger census + - [ ] R4.9 verifier + security + owner final acceptance → merge on owner signal + - [ ] outside R4 (owner 2026-10-17, non-blocking): worker container ignores SIGTERM (Docker kill after 10 s; chore, e.g. `init: true`); README test counter stale (367 vs 1,015) - [ ] R5 curation pipeline + importer + upload tool: seed golden regenerated once (new `documentSha256`, D-R5), `packages/anatomy` + `packages/domain` deleted, docs map; all before Step 13 - [ ] separate chore (D-R7): form constraints module + PGlite `pg_constraint`/config.toml parity test - [ ] Release 0.2.0 (branch release/0.2.0 = Steps 3–6, version.txt + package versions 0.2.0, CHANGELOG [0.2.0] 2026-10-11): deploy-trigger check done 2026-10-12 (CI = branches main/develop only, no tag trigger, no deploy job, docker job builds `faunosteo:ci` locally; hooks push only; DEPLOY.md deploy = manual on the VPS). Default 0.2.0 build carries NO sheet (SHEET_BUNDLE=none) → the live local tool must not be redeployed from main without `--build-arg SHEET_BUNDLE=draft`. Finish via /release-candidate (finish + tag span, tag push on explicit go); CHANGELOG back-merge conflict expected (Unreleased Step 7 above [0.2.0]). Awaiting the user's go. diff --git a/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md b/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md index 6f6f1914..514cc32a 100644 --- a/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md +++ b/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md @@ -1,365 +1,340 @@ # PLAN: Rust R4, PDF core (reader in a disposable worker, writer, report-compare), `packages/pdf` deleted -- date: 2026-10-17 | rev 1 | branch: `chore/rust-r4-plan` (plan only); code later on `feature/rust-r4-pdf-core`, one merge -- spec: owner briefs 2026-10-17 "Préparation de R4" and "Autorisation de préparation R4.0" (13 arbitrations, A1 to A13 below); `docs/development/RUST_ARCHITECTURE_REVIEW.md` I.2, II.2 rows "pdf reader", "pdf writer", "report-compare", III.3, VI.1 to VI.4, D-R2, D-R3, D-R12, D-R13, D-R14; `RUST.md` §8.7, §8.8, §9.4, §9.7; `PDF_IMPORT_HARDENING.md` §5; CLAUDE.md "PDF", "Schemas", workflow gates +- date: 2026-10-17 | rev 2 (after 3 plan challenges: correctness CONCERNS (6 major, 8 minor), robustness CONCERNS (10 major, 12 minor), simplicity CONCERNS (3 major, 9 minor); log in §B.12) | branch: `chore/rust-r4-plan` (plan only); code later on `feature/rust-r4-pdf-core`, one merge +- spec: owner briefs 2026-10-17 "Préparation de R4" and "Autorisation de préparation R4.0" (arbitrations A1 to A13, §A.0); `docs/development/RUST_ARCHITECTURE_REVIEW.md` I.2, II.2 rows "pdf reader", "pdf writer", "report-compare", III.3, VI.1 to VI.4, D-R2, D-R3, D-R11, D-R12, D-R14; `RUST.md` §8.3, §8.7, §8.8, §9.4, §9.7; `PDF_IMPORT_HARDENING.md` §5; BLK-009, BLK-012; CLAUDE.md "PDF", "Schemas", archive-reader gate +- facts: Annex (inventory, measurements and experiments of 2026-10-17) - status: R4.0 only. No code and no fixture generated; R4.1 starts on the owner's explicit go. -- not in scope: TTF fonts, `scope_disclaimer`, collection caps on read, opening the import to external users (BDR-040 gate stays open), R5 (pipeline, `packages/anatomy`, `packages/domain`, `documentSha256`), the Safari download fix (separate bugfix), release 0.2.0, any deployment, any visible change +- not in scope: TTF fonts, `scope_disclaimer`, the planned collection caps, opening the import to external users (BDR-040 gate stays open), R5 (pipeline, `packages/anatomy`, `packages/domain`, `documentSha256`), the Safari download fix (separate bugfix), release 0.2.0, any deployment, any visible change on the documented browsers. -## 0. Facts this plan rests on (develop ae845c6, inventory 2026-10-17) +# Part A. For the owner -Engine, consumers, tests -- `packages/pdf/src` 3,122 lines: reader (`attachment/scan.ts` 549, `inflate.ts` 83, `read.ts`), writer (`export.ts` 228, `render/*` about 1,250 incl. `observationPage.ts` 634), `branding.ts` 291 (PNG gate), `svg/*` (TS sanitiser and view composition), `text.ts`, `render/dates.ts`, `i18n` (25 labels). Tests: 11 files, 140 cases (scan 39, inflate 9, read 13, security 55, text 5, size 2, goldens 11, raster 2, reexport 1, R3 oracle generators 3). -- App consumers: `src/pdf/{import,export,browserRasterizer}.ts`, `EditorRoute` (type only), `scripts/compare-reports.ts` + `compare/*` (589 lines, 11 cases). PDF-related app suites: pdf.goldens 13, pdf.roundtrip 16, pdf.import 5, archive.envelopes 3, n-1.reader 6, pdf.export.app 2, migration.leaves 2, pdf.browserRasterizer 2, scripts.compare 11, plus route and store suites that mock `@/pdf/*`. Only pdf.goldens, pdf.import and pdf.roundtrip load `pdf-lib`; `apps/app/src` never does. -- `release/0.2.0` and develop carry the same `packages/pdf/src`, and the same archive, report and sheet schemas as the N-1 Zod copy (`git diff release/0.2.0 1a7ee7d` on those directories: empty). One frozen TS scanner plus the N-1 copy therefore stand for the R3 build and for 0.2.0. -- Import today: whole file read on the main thread, TS byte scan, `JSON.parse(new TextDecoder().decode(data))` (scan.ts:498, non-fatal, one BOM stripped), candidate kept if it is an object with a numeric `version` or `envelopeVersion`, then `JSON.stringify` of that object into the core's `archive.import` (sheetOps.ts:33). A trap there poisons the main instance and spends one of its 3 recoveries. -- Rescue copy today: routed on the first KiB (`%PDF-` goes the PDF path), ≤ 16 MiB, `TextDecoder('utf-8', { fatal: true })`, core `draft.restore` on the main instance. +## A.0 Your arbitrations (2026-10-17), applied as written -Writer (pdf-lib 1.17.1 as used) -- Fonts Helvetica, Helvetica-Bold, Times-Roman, WinAnsi, not embedded. Width = Σ(AFM width, 250 when absent, + KPX kerning) × size / 1000; glyphs drawn unkerned. -- Operators: text `q BT rg Tf TL Tm ( Tj T*)+ ET Q` (TL 24, one `Tj T*` per line); image `q cm cm cm cm Do Q`; line `q RG w d m m l S Q`; rectangle `q rg w d cm cm cm m l l l h f Q` (no `re`). Operands printed by `numberToString` (JS shortest round trip with exponent expansion). No opacity, no ExtGState. Resource names `Tag-<10 digits>` from a seeded RNG (deterministic). Streams Flate (pako level 6). Classic xref (`useObjectStreams: false`), no `/ID`. -- Images: PNG fully decoded (UPNG, RGBA8), RGB plane plus a DeviceGray SMask only when an alpha < 255, re-deflated; no dedup. -- Info: Title, Author, Subject, Keywords (`faunosteo-archive/1`), Producer and Creator (`pdf-lib (https://github.com/Hopding/pdf-lib)`) as UTF-16BE hex with BOM; CreationDate and ModDate = `exportedAt` as literal `D:YYYYMMDDHHMMSSZ`. -- Attachment: `JSON.stringify(envelope, null, 2)` in UTF-8, ≤ 16 MiB; EmbeddedFile `/Subtype /application#2Fjson`, Params Size, CreationDate, ModDate; Filespec `/F (osteo-report.json)`, `/UF` and `/Desc` hex UTF-16BE, `/AFRelationship /Data`; catalog `Names/EmbeddedFiles` (key hex UTF-16BE) and `/AF`. -- Links: low-level dict, `Border [0 0 0]`, URI action holding the hex of the ASCII bytes. -- Dates: `Intl.DateTimeFormat` fr-FR and en-US, long month, UTC; input must match `^\d{4}-\d{2}-\d{2}$`, else printed unchanged; `Date.UTC` maps years 0 to 99 to 1900 to 1999 (`0050-03-04` → "4 mars 1950"); a rolled-over date is printed unchanged. -- Text: `safeText` removes `[\p{Cc}\p{Cf}\p{Zl}\p{Zp}]` and replaces non-WinAnsi code points with `?`; `wrapLines` splits on JS `/\s+/`; `toUpperCase` on severity, view and zone names. -- Caps: 500 observations × 8 distinct views (`PAGE_CAPS`); rasters 1200 px wide, aspect clamped to 0.2 to 2.0; the views of one observation rasterised in parallel (`Promise.all`), embedded in order. +A1 one feature branch, one final merge, J1 stop once the import is wired. A2 old PDFs behave as before; differences documented; any break presented before it is applied. A3 years 0 to 99 unchanged (printed 19xx). A4 Producer and Creator `Faunosteo`. A5 new tracing goldens in `goldens/r4/`, frozen goldens untouched. A6 zero render difference; any difference explained, measured, submitted. A7 existing import message; timeout, worker crash and read error told apart technically; 20 s is a hypothesis to measure. A8 rescue copies through the import worker. A9 Firefox and WebKit for Playwright; real Safari still required; module workers vs the Firefox 102 floor: options, policy unchanged unless you decide. A10 `pnpm report:compare` kept. A11 out of R4: TTF, `scope_disclaimer`, collection caps; import closed to external users. A12 WASM budget exceeded: stop and options. A13 real anonymised PDFs join the corpus when you provide them. -Measurements (Node 24, TS engine, resvg test rasteriser, fr) +## A.1 What R4 delivers -| Report | Views rasterised | PNG in | PDF | Stored attachment | -|---|---|---|---|---| -| v2full nologos | 3 | 1.63 MB | 1.17 MB | 54 KB | -| demo nologos | 7 | 3.75 MB | 2.64 MB | 55 KB | -| demo logos | 8 | 3.79 MB | 2.82 MB | not measured | -| v2unresolved logos | 1 (SVG logo) | 0.03 MB | 0.26 MB | not measured | +The application reads and writes archives with Rust only. An import, PDF or rescue copy, runs in a Web Worker created for it and terminated after the answer, a timeout or a crash: the main instance never parses the file's bytes. The export composes the views in Rust, rasterises them with the existing canvas adapter, and writes the PDF in Rust through an incremental session. Old PDFs read as before; new PDFs render pixel-identically to R3 for the same inputs. `pnpm report:compare` keeps its name on a Rust binary. `packages/pdf` and `pdf-lib` leave the repository, and every TS test case is traced in a ledger. -Per view: PNG 526 to 553 KB, decoded RGBA 4.4 to 4.5 MB, about 0.37 MB of image stream in the PDF. Browser canvas PNG sizes: not measured yet (R4.1 baseline). +## A.2 Decisions to take now -Reader experiments (scratch, 2026-10-17) -- `String::from_utf8_lossy` after stripping one leading BOM equals `new TextDecoder().decode` on 200,015 random and edge byte strings (overlongs, UTF-8-encoded surrogates, truncated sequences, double BOM): 0 differences. -- serde_json refuses `1e400` ("number out of range") where `JSON.parse` gives `Infinity`, which `JSON.stringify` then writes `null`. `1e-400`, `-0`, integers beyond u64, long decimals and duplicate keys come out as in the JS path once printed by `js_number`. A lone surrogate escape fails in serde_json (classified `lone_surrogate` since R2). +| # | Question | Options | Recommendation | +|---|---|---|---| +| OD-1 | Import worker vs the Firefox 102 floor (A9). Fact found by the challenges: today the import already fails below Firefox 113 and Safari 16.4, because every attachment is Flate-compressed and the TS inflater needs `DecompressionStream` (it answers `attachment_stripped` there); the PNG gate fails the same way (`logo_failed`). | B': module worker everywhere, floor line unchanged, DEPLOY.md states "import needs Firefox 114 or later"; Firefox 113 is the only version that imports today and would not (it would show `error_core_unavailable`), Safari 16.0 to 16.3 and every logo gain. A: classic worker in the production build, module worker in dev; the import works from the documented floor up (a gain on Firefox 102 to 113), at the cost of a dev/production difference, a static check of the worker chunk and a proof on a Firefox 113 binary. B: raise the floor to Firefox 115 ESR (same code as B', policy change). | B' (one worker type, no policy change, one end-of-life version affected) | +| OD-2 | Export memory design (the brief allows leaving the review's two calls only on evidence) | Two calls hold every PNG and the PDF at once in linear memory: ≈ 0.91 MB per view (118 MB for 120 views). The TS engine today holds every decoded RGB plane until `save()`: ≈ 3.8 MB per view (≈ 0.45 GB for 120 views). An incremental session holds the PDF plus one decode: ≈ 0.37 MB per view + 9 MB (53 MB for 120 views), in the main instance, which keeps that high-water until reload (linear memory never shrinks). A disposable export worker frees it but needs a message protocol and an asset fetched at export time (offline export, which works today, would then need a fallback). | Session in the main instance (§B.4); worker only if a device fails (stop rule) | +| OD-3 | Memory guard for large hostile JSON (the worker bounds the blast radius, not the peak; BLK-012 measured 0.4 to 0.8 GB of linear memory for a 5 MB worst-case copy, so a 16 MiB file can exhaust any phone today and after R4) | (b) a JSON value budget in the existing linear pre-scan (attachment probe and copy), set at 10 × the largest legitimate archive measured at R4.2, refusal with today's messages (`error_no_attachment` as an oversized attachment today, `error_invalid_copy`); (a) a module memory maximum (`--max-memory`, above the largest export need) so a runaway allocation traps instead of the OS killing the tab; (c) an attachment cap below 16 MiB; (d) typed parsing without the intermediate `Value` (changes the R2 reader, gated) | (b) + (a), thresholds confirmed by the R4.2 measurements before R4.3 wires the import; both act on files no Faunosteo build writes | +| OD-4 | One JS side effect cannot be reproduced without a JSON reader of our own: a duplicate key whose overwritten value holds a lone surrogate escape imports today (JSON.stringify dropped it) and R4 refuses it (`error_lone_surrogate`) | accept this break (no Faunosteo build writes duplicate keys); or write a lossless JSON round-trip reader (R2 rejected it for the same reason) | accept | +| OD-5 | Messages (existing keys only) | PDF timeout or crash: « Erreur de lecture du PDF. » (A7; today a reader trap showed « Le moteur de lecture n'a pas pu être chargé. Réessayez l'import. »). Rescue copy timeout or crash: `error_core_unavailable` (the same text, which invites a retry) rather than `error_invalid_copy`, which would call a valid copy unreadable | as stated | +| OD-6 | J1 devices and browsers (computer, iPhone, low-end Android), LAN port of the J1 build | | your list | +| OD-7 | Repository weight of the new fixtures (about 5 MB of PDFs, PNGs and compressed bombs) | | accept | -Browsers and tooling -- Module workers: Chrome 80, Firefox 114, Safari 15. Documented floor for `'wasm-unsafe-eval'`: Chrome 97, Firefox 102, Safari 16 (DEPLOY.md, which calls it "the Vite 7 baseline"); the Vite 7.3.6 default build target is chrome107, firefox104, safari16. -- Vite 7.3.6: production workers are `iife` by default, asset URLs resolved against `self.location.href`; in dev a classic worker gets `importScripts` while its code keeps ESM imports, so dev needs a module worker. -- CSP (DEPLOY.md, sent `always` by the proxy; `docker/nginx.conf` sends none): `script-src 'self' 'wasm-unsafe-eval'`, no `worker-src`, which falls back to `script-src`: same-origin worker files allowed, `blob:` workers refused. -- Playwright (playwright-core 1.64.0, outside the repository as for the CSP proofs): Chromium present; Firefox 157 installed 2026-10-17 and proven headless (classic and module workers instantiate wasm); WebKit 27.2 downloaded, starts only once `libevent-2.1-7t64` and `libmanette-0.2-0` are installed (sudo, owner). -- WASM: 395.6 KB gzip of the 600 KB budget; linear memory after the editor path 4.1 MiB; sheet bundle text about 522 KB, handle 8 to 10 ms. -- R2 and R3 kept no per-test traceability table (prose census only). +## A.3 Behaviour that changes, declared -## 1. Outcome +- Producer and Creator: `Faunosteo` (A4). +- Messages: OD-5. A foreign PDF picked while the reading engine cannot start now shows the engine message instead of « Ce PDF n'a pas été généré par Faunosteo » (the scan runs in the worker). +- Old browsers gain: logos are drawn without `DecompressionStream` (Firefox 102 to 112, Safari 16.0 to 16.3), and the import works there too (Safari 16.0 to 16.3 in every option, Firefox 102 to 112 under option A only). Visible only on those versions. +- Attachment bytes: the report inside the attachment is written in contract order (D-R11), also when the editor reordered the store object (next appointment set, practitioner edited): values identical, N-1 proof. +- OD-3 and OD-4 refusals, on hostile files only. +- Declared at R3 (RUST.md §9.4), now on the PDF path, no interface reaches it: a foreign archive whose snapshot SVG or branding SVG the R3 reader refuses gets `snapshot_invalid` or `logo_failed` where the TS sanitised and drew. +- `report:compare` prints closed wording where the TS printed a ZodError message, kept a lone surrogate or read a version as `Infinity`. +- Kept as is (A2): invalid UTF-8 becomes U+FFFD (Rust decoder measured identical to TextDecoder on 200,015 byte strings), one BOM stripped, an overflowing number becomes `null`, years 0 to 99 print 19xx. -The application reads and writes archives with Rust only. An import, PDF or rescue copy, runs in a Web Worker created for it, with its own wasm instance, terminated after the answer, a timeout or a crash: the main instance never sees the file's bytes. The export composes the views in Rust, rasterises them with the existing canvas adapter and writes the PDF in Rust through an incremental session, placed in the main instance or in a disposable worker by the measured rule of §6. Old PDFs read exactly as before. New PDFs render pixel-identically to R3 for the same inputs; the one decided metadata change is Producer and Creator `Faunosteo`. `pnpm report:compare` keeps its name on a Rust binary. `packages/pdf` and `pdf-lib` leave the repository, and every TS test case is traced in a ledger. +## A.4 Checkpoints and stop rules -## 2. Owner arbitrations and plan decisions +- Checkpoints: (1) this plan; once validated, the plan branch merges, the separate Safari download fix lands, then the feature branch is cut (one develop merge into the feature branch allowed later only with your OK); (2) J1 after R4.3; (3) the final acceptance after R4.9, before any merge. +- J1, on the production build behind the DEPLOY CSP, on your devices: old PDFs (v2 MVP, v3 Step 5, Step 6 and R3 envelopes), the reference PDFs, a hostile folder with the expected message per file, rescue copies (valid, BOM, invalid UTF-8, oversized, refused), an import while the engine is unavailable and after its recoveries, an import interrupted by leaving Home, memory and time readings per device, and the TS export of the large report L on the phones (baseline for OD-2). +- Final acceptance: FR and EN exports of the three reference reports and of a new bilan on computer and phone; each viewer (Chrome, Firefox, Acrobat, Preview or Files, Android) opened once: pages, attachment listed, properties (Producer Faunosteo), footer links; a side-by-side R3 vs R4 page per report, passing when the differences stay within an R3 vs R3 run on the same device (outside private browsing); 400 % zoom and one print as quick confirmations; R4 PDFs imported into the R3 build run side by side; R4 PDFs re-imported (same bilan); engine unavailable during an import and an export; an offline export. +- I stop and come back to you when: the module exceeds 600 KB; any reader or differential difference outside A.3 appears; any §B.2.3 "not acceptable" difference appears; a visible change looks necessary; an export of L fails on a test device; a hostile class breaks its memory budget; a browser proof fails; a gated reader would need a change (`import_archive`, `read_archive_document`, `rebase_report_on_sheet`, `pdf::read`, `json_js`). -Owner, 2026-10-17 -- A1 one feature branch, one final merge, J1 stop once the import is wired. -- A2 old PDFs behave as before; UTF-8 differences documented; any break presented before it is applied. -- A3 years 0 to 99 unchanged (printed 19xx). -- A4 Producer and Creator `Faunosteo`. -- A5 new tracing goldens in `goldens/r4/`; frozen goldens untouched. -- A6 zero render difference; any difference explained, measured, submitted. -- A7 existing user message for import failures; timeout, worker crash and read error told apart technically; 20 s is a hypothesis to measure. -- A8 rescue copies restored through the import worker, same isolation and limits. -- A9 Firefox and WebKit for Playwright; real Safari still required; module workers vs the Firefox 102 floor handled by options (§11), policy unchanged unless the owner decides. -- A10 `pnpm report:compare` kept. -- A11 out of R4: TTF, `scope_disclaimer`, collection caps; the import stays closed to external users. -- A12 WASM budget exceeded: stop and options. -- A13 real anonymised PDFs join the corpus when provided (§7.2). +## A.5 Your actions -Plan decisions (internal; trade-offs stated) -- P1 Replicate the reader, never tighten it. Lossy UTF-8 identical to TextDecoder (measured), one BOM stripped, and the two side effects of the JS path reproduced: a number overflowing f64 becomes `null` before the core reads the text (what `JSON.stringify` did with ±Infinity); a candidate holding a lone surrogate escape is still probed for its numeric version (JSON.parse accepted it) and is then refused by `import_archive` as `lone_surrogate`, as today. `session::import_archive` and the layered readers are not modified (D-R12): R4 adds the byte layer in front of them. Any other difference found by the differential corpus is a stop (A2). -- P2 Content streams come from our own emitter, which reproduces pdf-lib's operator sequences and `numberToString`; `pdf-writer` writes objects, dictionaries, strings, xref and trailer (D-R2). Same operators and operands make the render identical by construction, and give a complete oracle (normalised stream equality) on top of the draw log. -- P3 The writer is an incremental session: begin, raster requests per observation, add rasters, finish. The review's two-call shape holds every PNG in linear memory at once (§6); the session holds one decode at a time and keeps the TS engine's per-observation loop, concurrency and warnings. Image objects are written into the output buffer as soon as their PNG is decoded, so nothing is held twice. Placement by §6.4. -- P4 One worker script for core tasks, one task per worker instance (`import.pdf`, `import.copy`, and `export.pdf` if §6.4 selects it), terminated in a `finally`. -- P5 The default PDF labels (25 keys, two locales) move verbatim from `packages/pdf/src/i18n` to `apps/app/src/pdf/labels.ts` and stay an explicit, archived input (dictionaries stay TypeScript; same key order, so the attachment bytes do not move). -- P6 AFM tables for the three fonts are generated once from the frozen pdf-lib metrics (Adobe Core 14 AFM, APAFML notice committed) and checked against that oracle. -- P7 Cargo tests feed frozen PNGs to the writer; no rasteriser in the workspace (D-R3). -- P8 `exportFromEnvelope` is ported (a bilan reproduced from its own snapshot, branding and labels); no UI (BDR-040). -- P9 New crates: `pdf-writer` (MIT or Apache-2.0), `miniz_oxide` (inflate and deflate, already locked through `image`; one version for the core), `png` (0.18.1, already locked) under decoding limits; `lopdf` as a dev-dependency only. No new npm dependency. +WebKit on this machine: `sudo apt-get install libevent-2.1-7t64 libmanette-0.2-0` (Firefox 157 and WebKit 27.2 for Playwright are downloaded; Firefox runs headless with workers and wasm). The go for the Safari download fix. Real PDFs, if any (§B.5.2). -## 3. Architecture and contracts +# Part B. The plan -### 3.1 Modules (`crates/core`, features `pdf-read`, `pdf-write`, `cli`; the wasm facade enables both pdf features, the worker crate neither) +## B.1 Architecture and contracts + +### B.1.1 Modules (`crates/core`, features `pdf-read`, `pdf-write`, `cli`; the wasm facade enables both pdf features, the worker crate neither) ``` src/pdf/ - read/{header,scan,strings,inflate,probe,stamp}.rs bytes → Attachment | ReadKind - json_js.rs lossy decode, overflow → null, lone-surrogate-tolerant probe (one linear pass with the depth pre-scan) - import.rs import_pdf(sheet, bytes) = read → JS-faithful text → session::import_archive (unchanged) + read/{header,scan,strings,inflate,probe,stamp}.rs bytes → Attachment | ReadKind (B.1.2) + json_js.rs JS-path side effects: overflow → null rewrite, value budget (OD-3), probe text + import.rs import_pdf(sheet, bytes) = read → text → session::import_archive (unchanged) write/ - input.rs ExportInput + checks (branding and labels schemas, assert_snapshot, read_svg per asset, - sheet ref match, caps), envelope written as JSON.stringify(_, null, 2) with js_number - session.rs ExportSession state machine + input.rs ExportInput checks (branding and labels schemas, assert_snapshot, read_svg per + asset, sheet ref match, caps), envelope as JSON.stringify(_, null, 2) + js_number + session.rs ExportSession state machine (B.1.4) views.rs raster requests: compose_highlight (R3) per (observation, view), raster_size_of layout/{observation,conclusion,header,footer}.rs → DrawLog - text.rs, afm/, dates.rs, png.rs (gate, decode, alpha split), content.rs (emitter, number_to_string), - serialize.rs (pdf-writer) + text.rs, afm/ (3 fonts), dates.rs, png.rs (branding gate, decode), content.rs (emitter, + number_to_string), serialize.rs (pdf-writer), self_check.rs (B.1.3) + from_envelope.rs exportFromEnvelope, cargo-tested, not on the facade src/bin/report_compare.rs feature cli, the only file allowed std::fs ``` -### 3.2 Reader contract (identical to scan.ts, pinned by O8 to O10) +New crates: `pdf-writer` (MIT or Apache-2.0), `miniz_oxide` (inflate and deflate, already locked through `image`; one version for the core), `png` (0.18.1, already locked); `lopdf` as a dev-dependency only. No new npm dependency. -File ≤ 50 MiB and `%PDF-` within the first 1,024 bytes, else `invalid_pdf`. At most 32 `/Type /EmbeddedFile` dictionaries, each bounded to 4,096 bytes; object header from a 48-byte window; `stream` then `\r\n` or `\n`; a direct `/Length` when it lands on `endstream`, else the length measured up to the first `endstream` within 16 MiB + 16 bytes (one trailing `\n`, then one `\r`, dropped); `/FlateDecode` alone or first in an array, else raw bytes. Names from Filespec dictionaries in clear (`/F`, else `/UF`); `osteo-report*` first (stable), then data offset. One 16 MiB budget shared by the candidates; a cancelled inflation is not charged. A candidate opens on `{` after an optional BOM and JSON whitespace, nests at most 64 levels (raw bytes, strings skipped, a backslash skips one byte), then lossy decode, JS-faithful probe, and the chosen text goes to `import_archive`. No candidate: `attachment_stripped` when a `/Keywords` string (≤ 32 occurrences, 4,096-byte window, hex or literal with the same limited unescape, UTF-16BE with BOM else Latin-1) contains `faunosteo-archive`, else `no_attachment`. Inflation: zlib framing, accepted and refused like `DecompressionStream('deflate')` on O10 (truncated, bad Adler-32, trailing bytes, preset dictionary), output capped by the remaining budget, cancelled beyond it. +### B.1.2 Reader (identical to scan.ts; every rule pinned by O7 to O9) -### 3.3 Writer contract +- File ≤ 50 MiB and `%PDF-` within the first 1,024 bytes, else `invalid_pdf`. +- At most 32 `/Type /EmbeddedFile` dictionaries, each bounded to 4,096 bytes; object header from a 48-byte window; `stream` then `\r\n` or `\n`; a direct `/Length` when it lands on `endstream`, else measured up to the first `endstream` within 16 MiB + 16 (one trailing `\n` then one `\r` dropped); `/FlateDecode` alone or first in an array, else raw. +- JS regex semantics reproduced explicitly: `\s` in the header, `/Length`, `/Filter`, `/EF` and `/F` patterns matches 0x0B and 0xA0 but not NUL, while `isWhitespace` matches NUL and neither of those; in the string token `.` excludes CR and LF; the `/EF` removal is lazy up to the first `>>`; a candidate whose stored length exceeds the remaining budget ends the search (`break`), it is not skipped. +- Names from Filespec dictionaries in clear (`/F`, else `/UF`), `osteo-report*` first (stable), then data offset. One 16 MiB budget shared by the candidates; a cancelled inflation is not charged. +- A candidate opens on `{` after an optional BOM and JSON whitespace and nests at most 64 levels (raw bytes, strings skipped, a backslash skips one byte); the same pass counts JSON values (OD-3 budget). Then lossy decode (`from_utf8_lossy` after one BOM), the overflow → null rewrite (number tokens that overflow f64, as JSON.stringify wrote ±Infinity), and the probe: a copy of the text with lone surrogate escapes neutralised goes through R2's `parse_json` (JS duplicate-key semantics) and must be an object with a numeric `version` or `envelopeVersion`. The accepted text, with its real escapes, goes to `import_archive`, which refuses a surviving lone surrogate as today. +- No candidate: `attachment_stripped` when a `/Keywords` string (≤ 32 occurrences, 4,096-byte window, hex, or literal with the same limited unescape, UTF-16BE with BOM else Latin-1) contains `faunosteo-archive`, else `no_attachment`. +- Inflation: zlib framing, verdicts of `DecompressionStream('deflate')` as recorded in Node, Chromium and Firefox (O9); where engines disagree, Node's verdict is pinned and the others are listed. Output capped by the remaining budget, cancelled beyond it. -Input `ExportInput { report, sheetSnapshot, branding, labels, locale, exportedAt }` (ts-rs; `exportedAt` an ISO string from the caller). Errors before any page, with today's kinds: `branding_invalid`, `snapshot_invalid`, `snapshot_too_large`, `sheet_mismatch`, `too_many_pages`. Warnings unchanged: `view_raster_failed { observationId, viewId }`, `logo_failed { slot }`. Output: PDF bytes and warnings. The layout produces a `DrawLog` (per page, in call order: text, image, line, rectangle, link, every argument at full precision); `content.rs` emits pdf-lib's operator sequences; `serialize.rs` writes the catalog (Pages, Names/EmbeddedFiles, AF), A4 pages (595.28 × 841.89), three Type1 fonts with WinAnsiEncoding, image XObjects (DeviceRGB 8 bits, DeviceGray SMask with `Decode [0 1]` when needed), Link annotations, Info (Producer and Creator `Faunosteo`, the rest as today in the same string forms), the attachment objects of §0, a classic xref, no object stream, no `/ID`. +### B.1.3 Writer -### 3.4 Facade (`crates/wasm`), JSON strings as before, bytes as `Uint8Array` +- Input `ExportInput { report, sheetSnapshot, branding, labels, locale, exportedAt }` (ts-rs; `exportedAt` an ISO string from the caller; labels from `apps/app/src/pdf/labels.ts`, moved verbatim from `packages/pdf/src/i18n`, same key order: the review's II.2 kept them with the engine, the app keeps them as data like the other dictionaries). +- Errors before any page, today's kinds: `branding_invalid`, `snapshot_invalid`, `snapshot_too_large`, `sheet_mismatch`, `too_many_pages`. Warnings unchanged: `view_raster_failed { observationId, viewId }`, `logo_failed { slot }`. +- Layout → `DrawLog` (per page, in call order: text, image, line, rectangle, link, every argument at full precision) → `content.rs` emits pdf-lib's operator sequences with `numberToString` reproduced, quirks included (the exponent branches multiply or divide) → `serialize.rs` with pdf-writer: catalog (Pages, Names/EmbeddedFiles, AF), A4 pages, three Type1 fonts with WinAnsiEncoding, image XObjects (DeviceRGB 8 bits, DeviceGray SMask with `Decode [0 1]` when an alpha < 255), Link annotations (`Border [0 0 0]`, URI action), Info (Producer and Creator `Faunosteo`, the rest as today in pdf-lib's string forms: UTF-16BE hex with BOM, dates literal), the attachment objects as today, classic xref, no object stream, no `/ID`. The output buffer is pre-sized; each image object is written as soon as its PNG is decoded, and the PNG and planes are dropped. +- Text rules: width = `total * (size / 1000)` with `total` the sum of AFM widths (`|| 250`, so a zero width counts 250) plus KPX kerning; `safeText` with the Cc, Cf, Zl and Zp sets frozen from V8 (O4); `wrapLines` on JS `\s`; JS `toUpperCase`; `Math.round` half up; `rasterSizeOf` parsing the viewBox with JS `Number()` semantics (hex, `Infinity`). +- PNG: the gate (`pngIssue` port) applies to the base64 branding PNGs only, as today; view rasters and the SVG-logo raster are decoded with UPNG's acceptance rules (no CRC check, ancillary chunks ignored); decode under limits (a gated 4096² logo is 64 MiB of RGBA), RGBA8 like `UPNG.toRGBA8`, alpha plane kept only when an alpha < 255. +- Self-check: `finish()` re-reads its own output with the R4 reader and compares the attachment with the envelope it wrote; a mismatch fails the export (`self_check_failed`, today's export alert). It costs one scan and one 55 KB inflation. -- `SheetHandle.importPdf(bytes) → string`: `{ ok: true, value: ArchiveImport } | { ok: false, kind, detail }`, kinds `invalid_pdf`, `no_attachment`, `attachment_stripped`, `incompatible_version`, `lone_surrogate`, `invalid`, `internal`. The rescue copy keeps `SheetHandle.call('draft.restore', …)`. -- `ExportSession` (wasm-bindgen struct): `new(inputJson)` (throws `": "` like `SheetHandle`), `next() → { kind: 'rasters', requests: [{ key, svg, widthPx, heightPx }] } | { kind: 'done' }`, `addRaster(key, png)`, `rasterFailed(key)`, `finish() → Uint8Array`, `warnings() → string`; `ExportSession.fromEnvelope(envelopeJson, locale, exportedAt)` for P8. -- Linear memory is read from the instance's exported `memory` for diagnostics; no new export. +### B.1.4 Facade (`crates/wasm`) + +- `SheetHandle.importPdf(bytes: Uint8Array) → string`: `{ ok: true, value: ArchiveImport } | { ok: false, kind, detail }`, kinds `invalid_pdf`, `no_attachment`, `attachment_stripped`, `incompatible_version`, `lone_surrogate`, `invalid`, `internal`. The rescue copy keeps `SheetHandle.call('draft.restore', …)` plus the OD-3 value budget. +- `ExportSession` (wasm-bindgen struct), every call made through `CoreRuntime.invoke` (a trap poisons and recovers as any core call), bound to the runtime generation (refused with `core_unavailable` after a recovery, like `SheetSession` handles), `free()` in the driver's `finally` and on EditorRoute unmount. + +| State | Call | Effect | +|---|---|---| +| none | `new(inputJson)` | validates; throws `": "` with the B.1.3 error kinds; → `branding` | +| `branding` | `next()` | requests for the SVG logos (`logo:`; PNG logos need none); none → first observation | +| `observation k` | `next()` | the views of observation k (`view::`), main view first as today | +| any batch | `addRaster(key, png)` / `rasterFailed(key)` | key of the current batch, answered once; else closed kinds `unknown_key`, `duplicate_key` (never a trap). A decode failure or `rasterFailed` records the warning for that key. When every key is answered, the logos are embedded or the page is laid out and written | +| any | `next()` with unanswered keys, `finish()` before `done` | closed kind `out_of_order` | +| `done` | `finish() → Uint8Array` | self-check, then the PDF; `warnings()` readable afterwards | + +- The driver keeps the TS ordering: `rasterFailed` is called as each raster rejects (rejection order, as the TS recorded them under `Promise.all`), `addRaster` in view order once the batch has settled (embedding order), so the warning list and the alert read the same. - ts-rs types committed under `packages/core-wasm/types`: `ExportInput`, `RasterRequest`, `ExportStep`, `ExportWarning`, `PdfImportKind`. -### 3.5 Core worker protocol (`apps/app/src/core/worker/`, messages tagged `v: 1`) +### B.1.5 Import worker (`apps/app/src/core/worker/`, messages tagged `v: 1`) -1. The main thread creates the worker from a same-origin file (classic in the production build, module in dev: OD-1) and arms the task timer. -2. The worker fetches the shipped module (the main runtime's Vite asset URL), compiles (`compileStreaming`, fallback `compile`), instantiates, posts `{ type: 'ready' }`. A failure before `ready` is `worker_unavailable`. -3. The main thread posts `{ v: 1, task, file, sheet, limits }`: the `File` itself (structured clone of a Blob, nothing read on the main thread) and the bundle text the main runtime already builds (about 522 KB). -4. The worker re-checks `file.size` (50 MiB PDF, 16 MiB copy), reads `file.arrayBuffer()` once, builds a `SheetHandle`, runs `importPdf`, or the strict decode then `draft.restore`, and posts `{ type: 'result', out, diag: { ms, linearBytes } }`. A `WebAssembly.RuntimeError` is caught and posted as `{ type: 'crashed', detail }`. -5. The main thread: `result` → parse → the existing hydrate path; `crashed`, `error` or `messageerror` after `ready` → `crashed`; timer → `timeout`. `terminate()` in every case, timer cleared, references dropped. One task at a time (the Home busy flag already serialises imports). `diag` is printed in DEV only and never sent anywhere. -6. The main runtime is untouched by worker failures (no recovery spent, no poisoning). When it is `stopped`, the import is refused up front with today's stopped text, because the editor could not open the result. -7. Tests: vitest (happy-dom has no `Worker`) runs the worker entry in-process through the client's spawn seam; the real worker runs in the browser harness. +1. The main thread refuses up front when its runtime is `stopped` (today's text). It creates the worker in a try/catch (`SecurityError` is synchronous), installs `onmessage`, `onerror` and `onmessageerror` first, and arms a start timer. +2. The worker entry installs its listener before any await, wraps its whole life in one try/catch, and adds `self.onunhandledrejection` as a backstop: every throwable becomes `{ type: 'failed', phase: 'start' | 'task', kind }` (`trap` for `RuntimeError`, `RangeError` and `InternalError` stack overflows, `read` for a file the browser cannot read, `alloc`, `sheet`, `other`). The glue is imported statically and bundled into the worker chunk (no `import()`, no recovery logic). The module is the compiled `WebAssembly.Module` posted by the main thread when its runtime holds one (structured clone; proven per engine), else fetched and compiled from the same asset URL. It instantiates once and posts `ready`. +3. On `ready`, the main thread clears the start timer, posts `{ v: 1, task, file, sheet, limits }` (the `File` itself, never read on the main thread; the bundle text the main runtime builds, about 522 KB) and arms the task timer (`IMPORT_TIMEOUT_MS`, 20 s hypothesis), paused while the page is hidden. +4. The worker re-checks `file.size` (50 MiB PDF, 16 MiB copy), reads `file.arrayBuffer()` once, builds a `SheetHandle`, runs `importPdf`, or the strict TextDecoder then `draft.restore`, and posts `{ type: 'result', out, diag: { ms, linearBytes } }` (`linearBytes` from the instance's exported `memory`; printed in DEV only, never sent anywhere). +5. The first settlement wins; anything after it is ignored. `terminate()` in every case, timers cleared, references dropped. Classification: failure before `ready` or start timeout → `worker_unavailable`; `trap` → `crashed`; task timer → `timeout`; `read` keeps today's mapping (pinned by a route test). +6. One module-level controller runs one import at a time across mounts; leaving Home cancels it (terminate, result dropped), and a result is dropped if the store's report changed since the pick. +7. When the main runtime finishes loading (editor), the worker script is prefetched so that a later offline import can start, as an offline import works today once the editor has loaded the engine. +8. vitest (happy-dom has no `Worker`) runs the worker entry in-process through the client's spawn seam; the real worker runs in the browser harness and in the per-slice browser proof (B.7). +9. The main instance receives the core's canonical output only; it loads its index after acceptance, for `hydrateImport`, as today. -### 3.6 Export driver +### B.1.6 Export driver -The main thread computes `archive.snapshot` (needs the sheet, as today), builds `ExportInput` (branding from `loadBranding()`, labels from P5, `new Date()`), then loops: `next()`, rasterise the batch with `browserRasterizer` (`Promise.all`, as today), `addRaster` or `rasterFailed` per request, until `done`; `finish()`, download, warnings alert unchanged. In the worker placement the same loop runs over messages: SVG strings to the main thread, PNGs transferred to the worker, the PDF transferred back. A trap during an export in the main instance spends a recovery as any core call today; in the worker it does not. +The main thread computes `archive.snapshot` (the sheet handle, as today), builds `ExportInput` (branding from `loadBranding()`, labels from `labels.ts`, `new Date()`), loops `next()`, rasterises each batch with `browserRasterizer` under `Promise.all` as today, answers with `addRaster` or `rasterFailed`, then `finish()`, download, the warnings alert unchanged. Any thrown error keeps today's alert (`editor.export_failed`, or the stopped text). A raster that never settles hangs the export as it does today (residual, unchanged). -### 3.7 `report-compare` +### B.1.7 `report-compare` -A `crates/core` binary behind `cli`, same arguments, output lines and exit codes as the TS CLI (O11, LRN-044 escaping). Root script `report:compare` → `cargo run --release --locked -q -p faunosteo-core --features cli --bin report-compare --` (the development machine's toolchain; the first run compiles). +A `crates/core` binary behind `cli`, same arguments, output lines and exit codes as the TS CLI (O10, LRN-044 escaping), except the lines listed in A.3. Root script `report:compare` → `cargo run --release --locked -q -p faunosteo-core --features cli --bin report-compare --` (development machine toolchain; the first run compiles). -### 3.8 Import outcomes and messages (existing keys only) +### B.1.8 Import outcomes and messages (existing keys; today's mapping kept) -| Outcome | Raised by | Message | -|---|---|---| -| file > 50 MiB, no `%PDF-` | main pre-check, worker | `home.import.error_invalid_pdf` | -| `no_attachment` | reader | `error_no_attachment` | -| `attachment_stripped` | reader | `error_attachment_stripped` | -| `incompatible_version` | core | `error_incompatible_version` (version shown) | -| `lone_surrogate` | core | `error_lone_surrogate` | -| `invalid` (rebase refused) | core | `error_generic` (as today) | -| copy > 16 MiB, invalid UTF-8, refused by `draft.restore` | main pre-check, worker | `error_invalid_copy` | -| no bundled sheet; worker cannot start (CSP, script, module, no `Worker`) | main | `error_core_unavailable` (as a failed engine load today) | -| `timeout` or `crashed`, PDF | main | `error_generic` « Erreur de lecture du PDF. » (A7; today a trap showed `error_core_unavailable`) | -| `timeout` or `crashed`, copy | main | OD-3 | -| main runtime stopped | main | `core.stopped` or `core.stopped_unsaved` (unchanged) | -| held session | store | `session.held_blocked*` (unchanged) | +Order: main pre-checks (routing on the first KiB, sizes), then the worker's reading (its messages come first, as the scan's did), then the main index for `hydrateImport`, then the held-session checks. + +| Outcome | Message | +|---|---| +| PDF > 50 MiB, no `%PDF-` | `home.import.error_invalid_pdf` | +| `no_attachment` (incl. an OD-3 refusal) | `error_no_attachment` | +| `attachment_stripped` | `error_attachment_stripped` | +| `incompatible_version` | `error_incompatible_version` | +| `lone_surrogate` | `error_lone_surrogate` | +| PDF `invalid` (rebase refused) | `error_generic` (today) | +| PDF non-trap `internal` or `core_unavailable` from the worker's core | `error_core_unavailable` (today) | +| PDF `crashed` or `timeout` | `error_generic` (A7, OD-5) | +| copy > 16 MiB, invalid UTF-8, `draft.restore` kinds `invalid`, `unsupported_version`, `lone_surrogate`, OD-3 refusal | `error_invalid_copy` (today) | +| copy other kinds (`rebase_failed`, `invalid_shape`, non-trap `internal`) | `error_core_unavailable` (today) | +| copy `crashed` or `timeout` | `error_core_unavailable` (OD-5) | +| `worker_unavailable`, no bundled sheet, main index failure | `error_core_unavailable` | +| main runtime stopped | `core.stopped` or `core.stopped_unsaved` | +| held session | `session.held_blocked*` | The technical kind stays in the outcome and in the DEV console. -## 4. PDF compatibility +## B.2 PDF compatibility -### 4.1 Historical behaviours preserved (each pinned by O8 or O9 rows) +### B.2.1 Behaviours preserved (pinned by O7 and O8 rows) -- H1 bare v2 MVP PDFs with object streams (`report-v2.full.pdf`, `report-v2.minimal.pdf`): migrated through the map. -- H2 bare v3 from Step 5 (`report-v3.bare.pdf`): read as is, rebased when the sheet differs. -- H3 envelopes v1 from Step 6 (pre-R2 key order, O3) and from R2 and R3 (O2): layered reading, report kept when the snapshot is rejected. -- H4 stamped PDFs with object streams: a stripped attachment still reads `no_attachment` (known trade-off). -- H5 several embedded files: `osteo-report*` first, then data order; decoys skipped. -- H6 indirect, wrong or missing `/Length`; `\r\n` after `stream`; `/Type/EmbeddedFile` without a space; hex and literal names; unfiltered streams. -- H7 caps: 50 MiB file, 32 candidates, 4 KiB dictionaries, 16 MiB shared budget, depth 64; bombs refused without memory growth. -- H8 invalid UTF-8 becomes U+FFFD, one BOM stripped (same decoder, measured). -- H9 JSON: a duplicate key keeps the last value at the first position; an overflowing number becomes `null`; a lone surrogate is refused (`lone_surrogate`, R2); numbers are f64. -- H10 encrypted PDF: `no_attachment`. -- H11 every §3.8 message for the same file. -- H12 rescue copy: BOM accepted, strict UTF-8, ≤ 16 MiB, `draft.restore`, `editor.draft_rebased` when rebased. -- Declared, decided at R3 (RUST.md §9.4) and now applied to the PDF path: `exportFromEnvelope` on a foreign archive whose snapshot SVG the R3 reader refuses answers `snapshot_invalid` where the TS sanitised and drew; no product drawing is affected (R3 proof), no interface reaches it. +H1 bare v2 MVP PDFs with object streams: migrated. H2 bare v3 from Step 5: read as is, rebased when the sheet differs. H3 envelopes from Step 6 (pre-R2 key order, O3) and from R2 and R3 (O2): layered reading, report kept when the snapshot is rejected. H4 stamped PDFs with object streams: a stripped attachment reads `no_attachment`. H5 several embedded files: `osteo-report*` first, then data order. H6 indirect, wrong or missing `/Length`, `\r\n` after `stream`, `/Type/EmbeddedFile` without a space, hex and literal names, unfiltered streams. H7 caps: 50 MiB, 32 candidates, 4 KiB dictionaries, 16 MiB shared budget, depth 64. H8 invalid UTF-8 → U+FFFD, one BOM stripped. H9 JSON: duplicate keys keep the last value at the first position, overflow → `null`, lone surrogate refused (R2), numbers as f64, integer-like keys (benign, typed outputs). H10 encrypted PDF: `no_attachment`. H11 every B.1.8 message. H12 rescue copy: BOM accepted, strict UTF-8, ≤ 16 MiB, `draft.restore`, `editor.draft_rebased`. -### 4.2 Readers that must read R4 PDFs (N-1, review VI.3) +### B.2.2 Readers that must read R4 PDFs (N-1) -The frozen TS scanner (O12) and the N-1 Zod copy, standing for the R3 build and 0.2.0 (§0), read every R4 fixture PDF and app-path export; the R3 build, run side by side, reads them at the final acceptance. The live site (pre-Step 6) reads no envelope today; R4 changes nothing there. The DEPLOY.md rollback note says that R3 and 0.2.0 read R4 PDFs. +The frozen TS scanner (O11) and the N-1 Zod copy, which stand for the R3 build and release 0.2.0 (Annex), read every R4 fixture PDF and app-path export; an R4 PDF with its EmbeddedFile stream removed must answer `attachment_stripped` through O11, and a byte test pins the `/Keywords` form. The R3 build, run side by side, reads them at the final acceptance. The live site (pre-Step 6) reads no envelope today. DEPLOY.md's rollback note says that R3 and 0.2.0 read R4 PDFs. -### 4.3 Difference classes (R3 build vs R4 build, same inputs) +### B.2.3 Difference classes (R3 build vs R4 build, same inputs) -Acceptable, no validation needed, each checked by a test: -- B1 Flate bytes of every stream (miniz_oxide vs pako): the decompressed bytes are compared. -- B2 object numbers, order and offsets, header binary comment, xref and trailer layout (both classic xref, no object stream). -- B3 resource names (`/Helvetica-7036340225` → `/F1`, `/Image-…` → `/Im1`): streams compared after normalising names. -- B4 string forms with the same decoded value (literal vs hex) where pdf-writer chooses, such as the link URI. Info, `/UF`, `/Desc` and the names-tree key keep pdf-lib's UTF-16BE hex form, checked by the frozen TS scanner. -- B5 numbers in dictionaries no content depends on, at pdf-writer's `f32` precision (link `/Rect`, under 0.0001 pt), if pdf-writer has no exact primitive; MediaBox expected to print the same (checked). -- B6 dictionary key order, whitespace, structural keys without effect (each listed when found). +Acceptable, each checked by a test: +- B1 Flate bytes of every stream: decompressed bytes compared. +- B2 object numbers, order, offsets, header binary comment, xref and trailer layout (both classic xref). +- B3 resource names: pdf-lib registers a new name per draw call; streams are compared after resolving each name through `/Resources` to its object. +- B4 string forms with the same decoded value where pdf-writer chooses (the link URI); Info, `/UF`, `/Desc` and the names key keep the UTF-16BE hex form. +- B5 dictionary numbers nothing reads at `f32` precision (link `/Rect`, under 0.0001 pt) if pdf-writer has no exact primitive; MediaBox checked to print the same. +- B6 key order, whitespace, structural keys without effect (listed when found). -Decided (A4): Producer and Creator `Faunosteo`. Every other Info value identical: Title, Author, Subject, Keywords, CreationDate, ModDate, no `/ID`. +Decided: Producer and Creator (A4); the attachment's report in contract order (D-R11, A.3). Every other Info value identical. -Not acceptable without the owner (stop, measure, explain, submit): -- a pixel difference in the poppler render (150 dpi on every page; 300 dpi on the real-raster reference); -- a content-stream difference after B3 normalisation; -- a decoded image difference (RGB, alpha, size, bit depth, colour space, SMask presence); -- a draw-log difference (text, font, size, colour, position, line, rectangle); -- the set of links, a URI, or a rect beyond 0.01 pt; page count, size or order; -- the attachment: filename, subtype, Params, Desc, AFRelationship, names tree, `/AF`, or the decompressed JSON bytes (target: byte-identical to the R3 output for the same input; a key-order or formatting change is a contract change, presented with its N-1 proof); -- an import outcome difference on any file of O2, O3, O8, O9, the hostile set or the owner's real PDFs. +Not acceptable without the owner: a pixel difference in the poppler render (150 dpi, every page); a content-stream difference after B3; a decoded image difference (planes, size, depth, colour space, SMask); a draw-log difference; links, page count, size or order; the attachment's filename, subtype, Params, Desc, AFRelationship, names tree, `/AF`, or JSON bytes beyond the D-R11 order; an import outcome difference on O2, O3, O7, O8, the hostile set or real PDFs, outside A.3. -## 5. Import and memory +## B.3 Import and memory -- 5.1 Isolation. Untrusted bytes (PDF or copy) are read only inside the worker created for that import. The main instance receives the core's canonical output (validated report and envelope), as it does today after `archive.import`. This closes the R3 residual (16 MiB parsed in the long-lived instance, RUST.md §8.8) and two BLK-012 residuals (a trapping copy spends a main recovery; memory exhaustion near 16 MiB in the main instance). -- 5.2 Transfer. The `File` is posted, never read on the main thread; the worker checks the size, reads once into an ArrayBuffer, copies once into linear memory. The sheet travels per import as the bundle text, the worker builds its own `SheetHandle`; nothing is cached across imports. The result comes back as one JSON string (report and envelope); its cost is measured. -- 5.3 Time. One constant `IMPORT_TIMEOUT_MS` (20 s hypothesis) from creation to answer. R4.3 measures per browser the worker start (cached fetch, compile, instantiate, handle), the largest legitimate archive (demo with logos) and the slowest hostile case; J1 repeats on the owner's devices, the low-end phone included. Rule proposed: timeout ≥ 10 × the slowest legitimate import on the slowest device, and ≥ 2 × the slowest hostile case that completes. The owner sets the value at J1 if the rule moves it away from 20 s. -- 5.4 Destruction. `terminate()` after a result, a crash or a timeout. The harness checks that the content process comes back within 10 MiB of its pre-import level after a forced GC (Chromium CDP `HeapProfiler.collectGarbage`; Firefox `about:memory` minimise at J1). -- 5.5 Errors and recovery. The §3.8 table; no automatic retry (the practitioner picks the file again); the file and the session are never modified; held sessions and the replace dialog behave as today. -- 5.6 Out of memory. A worker bounds the blast radius, not the peak: it shares the tab's content process on every engine, and iOS kills the whole tab beyond its limit. The harness measures the worker peak on the classes that grow memory most: a 16 MiB token-dense JSON attachment (serde_json `Value` expansion, estimated 8 to 30 × in linear memory), the 50 MiB file, the 16 MiB inflation, 32 candidates. Budget W4 below. Options when a class breaks it, each a behaviour change on hostile files only, presented at J1, none applied by default: (a) link the module with `--max-memory` so growth fails as a trap before the OS limit (it also caps the main instance, so it sits above the largest export need of §6); (b) a value-count budget in the existing linear pre-scan; (c) an attachment cap below 16 MiB (real archives are under 0.4 MB); (d) typed deserialisation without the intermediate `Value` (changes the R2 reader, gated). -- 5.7 Browsers: §11. -- 5.8 Rescue copy. Same worker, timeout and termination; the strict UTF-8 decode stays in TypeScript inside the worker (same decoder), `draft.restore` runs in the worker's instance; the routing on the first KiB stays on the main thread (five bytes and the first non-blank byte). +- Isolation: untrusted bytes are read only inside the worker created for that import. This closes the read residual of RUST.md §8.8 and the trapping-copy residual of BLK-012 (a trap no longer spends a main recovery). It does not close the accepted-content residual: an accepted report and envelope come back to the main thread and are parsed there again by later actions, export and reload; that stays until the collection caps (PDF_IMPORT_HARDENING §5) and is measured (class "largest accepted file": transfer, hydrate, one list action, export, reload; main linear memory and RSS). +- Transfer: the `File` is posted, read once in the worker, copied once into linear memory; the sheet travels per import as the bundle text; the result comes back as one JSON string; each step's cost measured. +- Time: a start timer until `ready` (worker_unavailable) and the task timer. Rule for the task timer: ≥ 10 × the slowest legitimate import (an L-size archive and copy) on the slowest device, ≥ 2 × the slowest hostile case that completes; you set the value at J1 if the rule moves it away from 20 s. CPU throttling of worker threads by CDP is unverified: the harness records unthrottled numbers and the throttle actually applied. +- Destruction: `terminate()` in every case; the harness checks the content process comes back within 10 MiB of its pre-import level after a forced GC (Chromium CDP), and Firefox `about:memory` at J1. +- Out of memory: a worker shares the tab's content process, and iOS kills the whole tab beyond its limit. Measured classes, in Node at R4.2 then in the browsers: a 16 MiB token-dense attachment, a 16 MiB rescue copy through `draft.restore` (its input nests the copy as a JSON string, one extra copy), the 50 MiB file, the 16 MiB inflation, 32 candidates. Budget W4. With BLK-012's 80 to 160 × ratio the first two break it today: OD-3 brings the guard before R4.3. +- Rescue copy: same worker, timers and termination; strict TextDecoder in the worker; routing on the first KiB stays on the main thread. -## 6. Export memory: PNG accumulation in linear memory +## B.4 Export memory: PNG accumulation -### 6.1 Where the bytes live - -From the measured sizes, a report with V rasterised views takes about 0.54 V MB of PNG input (resvg; browser PNGs measured at R4.1), 4.5 MB of decoded RGBA per view at a time, and yields about 0.08 + 0.37 V MB of PDF. The image streams are our own deflate of the RGB plane, so their size does not depend on the browser. - -| Design | Peak linear memory | Kept by a long-lived instance afterwards | +| Design (V rasterised views) | Peak | Kept after the export | |---|---|---| -| Two calls (review sketch): every PNG in, then write | ΣPNG + PDF + one decode ≈ 0.91 V + 4.5 MB | the same high-water (linear memory never shrinks) | -| Session in the main instance | PDF + one decode ≈ 0.37 V + 4.5 MB | the same | -| Session in a disposable worker | as above, inside the worker | nothing (worker terminated) | -| TypeScript today (pdf-lib, JS heap) | ≈ PDF + up to 8 PNGs and their decodes | nothing (garbage-collected) | +| TS today: every RGB plane from `embedPng` to `save()`, then the PDF | ≈ 3.4 V + 0.37 V MB + PNG transients | nothing (garbage-collected) | +| Two calls (review sketch) | ΣPNG + PDF + one decode ≈ 0.91 V + 9 MB | the same, by the main instance | +| Session in the main instance (OD-2) | PDF + one decode ≈ 0.37 V + 9 MB | the same high-water, until reload | +| Session in a disposable worker | as above, inside the worker | nothing | -Examples, two calls / session: V = 7 (demo) 11 / 7 MB; V = 120 (60 observations, 2 views each) 114 / 49 MB; V = 500: 460 / 190 MB. The cap (500 × 8 = 4,000 views, about 1.5 GB of PDF) is not exportable today either; R4 must not fail earlier than the TS engine does (baseline O13). +Per view: PNG ≈ 0.54 MB (resvg; browser PNGs measured at R4.1), decoded RGBA + RGB + alpha ≈ 9 MB transient, image stream ≈ 0.37 MB (our own deflate of the RGB plane, independent of the browser). The JS side adds the PDF twice at the end (`Uint8Array`, then the `Blob`). With the draft sheet's two views, the cap is 500 × 2 = 1,000 views. -### 6.2 Strategies +| Report | V | TS today | Two calls | Session | +|---|---|---|---|---| +| demo | 7 | ≈ 27 MB | ≈ 15 MB | ≈ 12 MB | +| L (60 obs × 2) | 120 | ≈ 0.45 GB | ≈ 118 MB | ≈ 53 MB | +| CAP (500 × 2) | 1,000 | ≈ 3.8 GB (fails) | ≈ 0.92 GB | ≈ 0.38 GB | -- S1 incremental session (P3), chosen by construction: it takes ΣPNG out of linear memory at no cost on the JS side, which already rasterises per observation. -- S2 disposable export worker: frees the high-water after each export. -- S3 chunked output: image streams handed to JS as Blob parts as they are written, the PDF assembled as a `Blob`; only if S2 still fails on a device. -- S4 visible or behavioural measures (lower raster resolution, a smaller cap): never without the owner. +Measurement (R4.1 baseline, R4.6 for R4): harness in Chromium and Firefox (WebKit once it starts), linear memory after each session step, content-process RSS sampled every 50 ms from `/proc`, JS heap from CDP; reports demo, L and CAP (O13). The harness enforces its own ceiling (it closes the browser when the browser's processes exceed 4 GiB RSS) so that the shared development host never reaches the kernel OOM killer; a run stopped by the ceiling is recorded as such. Phones: TS L at J1, R4 L after R4.6 (your check), before the gates. -### 6.3 Measurement +Strategies if a device fails L with the session: S2 the disposable export worker (with the main-instance session as its offline fallback), S3 chunked output (image streams handed out as Blob parts), S4 visible or behavioural measures (raster resolution, a smaller cap) never without you. -Harness in Chromium and Firefox (WebKit once it starts): linear memory after each session step, content-process RSS sampled every 50 ms from `/proc`, JS heap from CDP; reports demo, L (V = 120) and XL (V = 500) built from the demo's observations (O14), and a doubling probe up to failure, for the TS baseline (R4.1) and for R4 (R4.6). Phones: L at the final acceptance. +## B.5 Fixtures and oracles -### 6.4 Decision rule, fixed now - -Keep the session in the main instance if, for L, the main instance keeps at most 32 MiB more linear memory than before the export and L completes on every test device; otherwise run it in the disposable worker (S2). The sizes above predict S2 (about 49 MB kept for L): the plan budgets it, the measurement confirms or refutes it before R4.6 wires the application. If S2 fails L on a test device: stop and present S3 and S4. - -## 7. Fixtures and oracles - -### 7.1 Frozen with the TS code at R4.1, before any Rust (generators committed behind `MAKE_R4_ORACLES=1`; never regenerated without the owner) +### B.5.1 Frozen with the TS code at R4.1, before any Rust (generators behind `MAKE_R4_ORACLES=1`; never regenerated without the owner) | Id | Content | Location (planned) | Used by | |---|---|---|---| -| O1 | draw log, 3 reports × fr/en × logos/nologos = 12 files: every drawText, drawImage, drawLine, drawRectangle and link call in order with all arguments at full precision (the step6 fixed rasterisers and FIXED_EXPORTED_AT) | `apps/app/tests/fixtures/goldens/r4/draw.*.json` | R4.4 | -| O2 | the 12 matching R3-build PDFs, plus one real-raster PDF (v2full fr logos) with its 3 resvg view PNGs and its SVG-logo PNG | `apps/app/tests/fixtures/pdf/r3/` (about 5 MB) | R4.2, R4.5 (normalised streams, image planes, links, Info, attachment), N-1 | -| O3 | pre-R2 envelope PDFs: the three frozen `goldens/r2/envelope.*.json` attached byte for byte by pdf-lib 1.17.1 (Step 6 layout) | `apps/app/tests/fixtures/pdf/step6/` | reader (H3), N-1 | -| O4 | text: widths (strings × sizes × 3 fonts), `wrapLines`, `safeText` over every code point class, JS `toUpperCase` over all code points (differences only), JS `\s`, `Math.round`, `numberToString` (random and edge values) | `crates/core/tests/oracles/r4/text.json` | R4.4 | -| O5 | pdf-lib metrics of the 3 fonts (widths, KPX, WinAnsi map) | `oracles/r4/afm.json` | P6 | -| O6 | `formatDate` fr and en over years 0000, 0001, 0050, 0099, 0100, 0999, 1000, 1900, 1904, 1999, 2000, 2024, 2026, 9999 × months 00 to 13 × days 00 to 32, plus non-matching strings | `oracles/r4/dates.json` | R4.4 | -| O7 | PNG: `pngIssue` verdict for every security-suite PNG, the brand logos and a variant set (palette, palette + tRNS, grey, grey + alpha, 16-bit, RGB, RGBA); for accepted ones, sha256 of pdf-lib's RGB and alpha planes | `oracles/r4/png.json` + small PNGs | R4.5 | -| O8 | reader outcomes for every input of scan, inflate, read and the reader part of security (inputs ≤ 64 KiB committed; bombs as deterministic generators ported to Rust and checked by the sha256 of the generated bytes): kind, chosen candidate, attachment and object digests | `oracles/r4/read.json`, `fixtures/pdf/r4/read/` | R4.2 | -| O9 | differential corpus: UTF-8 classes, BOM, double BOM, overflowing numbers (in `version` and in content), lone surrogate (probe and content), duplicate keys (incl. `version`), integers beyond 2^53 and u64, `-0`, `__proto__`, depth 64 and 65, a first candidate only JSON.parse accepts; outcome of the full R3 path (TS scanner, JSON.parse and stringify, core `archive.import` in wasm) | `fixtures/pdf/r4/tolerance/`, `oracles/r4/tolerance.json` | R4.2 (a difference is a stop, A2) | -| O10 | inflate: valid, empty, truncated, bad Adler-32, trailing bytes, raw deflate without zlib header, preset dictionary, exact cap, cap + 1: `DecompressionStream` verdict and output digest | `oracles/r4/inflate.json` | R4.2 | -| O11 | `report:compare` stdout and exit code on every fixture PDF and the 11 scenarios | `oracles/r4/compare/` | R4.7 | -| O12 | frozen TS scanner: verbatim copy of R3's `attachment/{scan,inflate,read}.ts` and their error module, never edited: the permanent N-1 PDF reader | `packages/core-wasm/tests/n-1/scanner/` | N-1 tests | -| O13 | TS baseline in Chromium and Firefox with the real `browserRasterizer`: PNG bytes per view, export time, RSS peak and after, JS heap, PDF size (demo, L, XL, doubling probe); TS import time and memory on O2 and the hostile set | table in RUST.md §10 (draft) | §5, §6, §10 | -| O14 | synthetic reports L (60 observations × 2 views) and XL (250 × 2) built from the demo | `apps/app/tests/fixtures/r4/reports/` | O13, R4.6 | +| O1 | draw log, 3 reports × fr/en × logos/nologos = 12: every drawText, drawImage, drawLine, drawRectangle and link call in order, all arguments, full precision (step6 fixed rasterisers, FIXED_EXPORTED_AT) | `apps/app/tests/fixtures/goldens/r4/draw.*.json` | R4.4 | +| O2 | the 12 matching R3-build PDFs; one real-raster PDF (v2full fr logos) with its view PNGs and SVG-logo PNG; one PDF exported through the store path (next appointment set, practitioner edited) | `apps/app/tests/fixtures/pdf/r3/` | R4.2, R4.5, N-1 | +| O3 | pre-R2 envelope PDFs: the three frozen `goldens/r2/envelope.*.json` attached byte for byte by pdf-lib 1.17.1 | `apps/app/tests/fixtures/pdf/step6/` | H3, N-1 | +| O4 | text and numbers: widths (strings × sizes × 3 fonts, from the pdf-lib metrics, which also generate the Rust AFM tables once, APAFML notice committed), `wrapLines`, `safeText` over every code point class, `toUpperCase` differences over all code points, JS `\s`, `Math.round`, `numberToString` (random, edge and tiny-exponent values such as 1e-13), `rasterSizeOf` viewBox parsing | `crates/core/tests/oracles/r4/text.json`, `afm.json` | R4.4 | +| O5 | `formatDate` fr and en over years 0000, 0001, 0050, 0099, 0100, 0999, 1000, 1900, 1904, 1999, 2000, 2024, 2026, 9999 × months 00 to 13 × days 00 to 32, plus non-matching strings | `oracles/r4/dates.json` | R4.4 | +| O6 | PNG: gate verdicts (security-suite PNGs, brand logos, variants: palette, palette + tRNS, grey, grey + alpha, 16-bit, RGB, RGBA); UPNG decode of accepted and raster-path PNGs incl. bad-CRC and malformed-ancillary variants: sha256 of the RGB and alpha planes or the failure | `oracles/r4/png.json` + PNGs | R4.5 | +| O7 | reader outcomes for every input of scan, inflate, read and the reader part of security, plus rows for the B.1.2 regex semantics: kind, chosen candidate, digests. Inputs committed as bytes, the zlib bombs included (compressed, about 65 KB for 64 MiB) and the inputs cut from a demo export; generators only for uncompressed fillers | `oracles/r4/read.json`, `fixtures/pdf/r4/read/` | R4.2 | +| O8 | differential corpus: UTF-8 classes, BOM, double BOM, overflowing numbers (in `version` and in content, in a duplicate), lone surrogate (probe, content, overwritten duplicate: OD-4), duplicate keys incl. `version`, integers beyond 2^53 and u64, `-0`, `__proto__`, integer-like keys, depth 64 and 65, a first candidate only JSON.parse accepts, value-budget limits (OD-3); outcome of the full R3 path (TS scanner, JSON.parse and stringify, core `archive.import` in wasm) | `fixtures/pdf/r4/tolerance/`, `oracles/r4/tolerance.json` | R4.2 (a difference outside A.3 is a stop) | +| O9 | inflate: valid, empty, truncated, bad Adler-32, trailing bytes, raw deflate without zlib header, preset dictionary, exact cap, cap + 1: `DecompressionStream` verdict and digest in Node, Chromium and Firefox | `oracles/r4/inflate.json` | R4.2 | +| O10 | `report:compare` stdout and exit code on every fixture PDF and the 11 scenarios | `oracles/r4/compare/` | R4.7 | +| O11 | frozen TS scanner: verbatim copy of R3's `attachment/{scan,inflate,read}.ts` and their errors, never edited (the permanent N-1 PDF reader) | `packages/core-wasm/tests/n-1/scanner/` | N-1 | +| O12 | TS baseline in Chromium and Firefox with the real `browserRasterizer`: PNG bytes per view, export time, RSS peak and after, JS heap, PDF size (demo, L, CAP); TS import time and memory on O2 and the hostile set | RUST.md §10 (draft) | B.3, B.4, B.8 | +| O13 | synthetic reports L (60 × 2) and CAP (500 × 2) built from the demo | `apps/app/tests/fixtures/r4/reports/` | O12, R4.6 | -The step6, r2 and r3 goldens and the R3 oracles stay untouched: `git diff --stat` on them is empty at every slice. +The step6, r2 and r3 goldens and the R3 oracles stay untouched: `git diff --stat` on them is empty at every slice. (Oracle ids renumbered in rev 2; references below use these.) -### 7.2 Real PDFs (A13) +### B.5.2 Real PDFs (A13) -Kept out of git on the development machine (`$FAUNOSTEO_FIELD_CORPUS`), read by an opt-in test; only a table without personal data is committed (sha256, producing build when known, expected outcome, observation count). A file enters git only after the owner reviews its anonymised form (attachment strings and page text replaced by same-length placeholders, structure kept), never automatically. +Kept outside the worktree (`$FAUNOSTEO_FIELD_CORPUS`, a test refuses a path inside it), read by an opt-in test; only a table without personal data is committed (sha256, producing build when known, expected outcome, observation count); the oracle generators refuse to run while the variable is set. A file enters git only after you review its anonymised form: attachment strings, page text, Info (Title holds the animal name, Author the practitioner), Filespec and file names replaced, streams re-encoded with corrected `/Length`. -## 8. Test traceability +## B.6 Test traceability -`crates/core/tests/pdf/LEDGER.tsv`: one row per TS case at the freeze commit, with the TS file, the full case title (describe path, each-row label), a disposition (`ported`, `replaced`, `kept`, `deleted`), the target (cargo `file::test` or vitest `file > title`), the oracle id and a note. Built at R4.1 from vitest's JSON reporter: the 140 engine cases, the PDF-related app cases of §0 and the 11 CLI cases. A census check fails when a frozen case has no row, when a `ported` target is missing from `cargo test -- --list` or the vitest list, or when the counts do not reconcile; the verifier runs it at R4.9. Rust test files keep a header naming their TS source, as at R2 and R3. +`crates/core/tests/pdf/LEDGER.tsv`: one row per TS case at the freeze commit (TS file, full case title, disposition `ported` / `replaced` / `kept` / `deleted`, target cargo `file::test` or vitest `file > title`, oracle id, note). Scope, built from vitest's JSON reporter at R4.1: the 140 engine cases; the PDF app suites (pdf.goldens 13, pdf.roundtrip 16, pdf.import 5, archive.envelopes 3, n-1.reader 6, pdf.export.app 2, migration.leaves 2, pdf.browserRasterizer 2, scripts.compare 11); the suites that mock `@/pdf/*` and are rewritten at R4.3 or R4.6 (routes.HomeRoute.archive 5, routes.HomeRoute.import 11, routes.HomeRoute.rescueCopy 13, routes.EditorRoute.export 5, routes.coreStopped 15, state.coreFailures 5). A census check fails when a frozen case has no row, a `ported` target is missing from `cargo test -- --list` or the vitest list, or the counts do not reconcile; the verifier runs it. Rust test files keep a header naming their TS source. -## 9. Slices +## B.7 Slices -One branch, `feature/rust-r4-pdf-core`, cut from develop once this plan is merged. Every slice ends green on `pnpm rust:check`, `pnpm wasm:build` and `pnpm check`, its proof recorded in the TODO and in RUST.md §10. +One branch, `feature/rust-r4-pdf-core`, cut from develop after this plan and the Safari fix have merged. Every slice ends green on `pnpm rust:check`, `pnpm wasm:build` and `pnpm check`; from R4.3 on, also on the browser proof (the production image behind the DEPLOY CSP, an import of one reference PDF in Chromium and Firefox, WebKit once it starts). Proofs recorded in the TODO and RUST.md §10. | Slice | Deliverable | Proof | |---|---|---| -| R4.1 Freeze | O1 to O14, ledger, browser harness (`scripts/browser/`, Playwright kept outside the repository) | generators committed, frozen goldens untouched, suites green, baseline table; no Rust | -| R4.2 Reader | `pdf::read`, `json_js`, `pdf::import`, `SheetHandle.importPdf`; scan, inflate, read and reader-security suites in cargo on O8 and O10; O9 differential; hostile corpus through the wasm build (Node, fresh instance per case, time and linear memory) | 0 oracle difference, 0 differential difference (else stop, A2), no trap outside declared memory cases, W1 | -| R4.3 Import worker and rescue copy | §3.5 client and entry; HomeRoute on the worker (routing and pre-checks unchanged); OD-1 applied; client tests (ready, timeout, crash, messageerror, unavailable, terminate always, main runtime untouched); a message-table test over O2, O3, O9 and the hostile set; 612 leaves through the worker (export still TS); CSP proof on the app image with and without `'wasm-unsafe-eval'` in Chromium and Firefox (WebKit when it starts); time and memory measured | same messages as R3 on every file, tab responsive during hostile imports (W5), memory back to baseline (W4), P-W1 | -| J1 Owner checkpoint, no merge | production build (`SHEET_BUNDLE=draft`) behind the DEPLOY CSP on the DEV origin (port agreed with the owner, ENVIRONMENTS.md); a French checklist; old PDFs (v2 MVP, v3 Step 5, Step 6 and R3 envelopes), O2, a hostile folder with the expected message per file, rescue copies (valid, BOM, invalid UTF-8, oversized, refused), engine unavailable during an import; memory and time per device | owner OK; timeout value; OD-1 confirmed on real browsers; 5.6 options decided if needed; results in PDF_IMPORT_HARDENING §5 (gate still open) | -| R4.4 Layout | text, AFM, dates, upper-casing, `safeText`, `wrapLines`, the four layouts → DrawLog | O1 (12) exact at full precision, step6 text and images (24) exact, O4 to O6 exact | -| R4.5 Serialiser | emitter, pdf-writer, PNG gate and decode, links, Info, attachment, `exportFromEnvelope`; export-side security, size, reexport and text suites in cargo | against O2: normalised streams equal, image planes equal, links (lopdf) and step6 links (12) equal, Info equal except Producer and Creator, attachment bytes equal (else declared, A6); `qpdf --check` clean; `pdfdetach -list` shows `osteo-report.json`; poppler diff 0 px; O12 and the N-1 copy read every R4 PDF; E2; digests of the R4 outputs in `goldens/r4/pdf.*.sha256` (wasm output = native output) | -| R4.6 App export | session driver, placement by §6.4 measured first, `labels.ts` (P5), `export.ts` on the session; alerts unchanged | app suites; app-path bytes = native digests; step6 views via the app path; browser rasters of the Rust- and TS-composed views pixel-identical (Chromium, Firefox, WebKit when it starts; the TS composition still exists here); E3, E4; 612 leaves exported and re-imported on Rust | -| R4.7 report-compare | Rust binary, `pnpm report:compare` unchanged | O11 and the 11 cases in cargo | -| R4.8 Deletions and docs | `packages/pdf` (with the R3 oracle generators; the frozen oracle files stay), `@faunosteo/pdf` and `pdf-lib` in apps/app, the TS CLI, the pdf-lib-based tests rewritten per the ledger; N-1 test on O12; RUST.md §10, PDF-FORMAT, PDF_IMPORT_HARDENING §5, DEPLOY (rollback note, worker CSP note, floor line), CHANGELOG, CLAUDE.md, TEST_STRATEGY, WORKSPACE, packages/README, fixtures README (R4 freeze section) | `pnpm check:all` green, ledger census green, D1 | -| R4.9 Gates and acceptance | verify contract, fresh verifier, fresh security audit; owner acceptance (§12), after the separate Safari download fix has merged | verifier CONFORME, security PASS, owner's explicit merge signal | +| R4.1 Freeze | O1 to O13, ledger, browser harness (`scripts/browser/`, Playwright kept outside the repository) | generators committed, frozen goldens untouched, suites green, baseline table; no Rust | +| R4.2 Reader | `pdf::read`, `json_js`, `pdf::import`, `SheetHandle.importPdf`; reader suites in cargo on O7 and O9; O8 differential; hostile corpus and OD-3 classes through the wasm build in Node (fresh instance per case, time, linear memory) | 0 oracle difference, 0 differential difference outside A.3, OD-3 thresholds measured and confirmed with you, W1 | +| R4.3 Import worker, rescue copy | B.1.5; HomeRoute on the controller (routing and pre-checks unchanged); OD-1 applied; client tests per throwable and phase, settle-once, result after timeout, error after result, unmount, second pick, main runtime untouched; message test over O2, O3, O8 and the hostile set; 612 leaves through the worker (export still TS); CSP proof with and without `'wasm-unsafe-eval'`; time and memory | B.1.8 messages on every file, W2, W4, W5 | +| J1 | A.4; results in PDF_IMPORT_HARDENING §5 (gate still open) | your OK, timeout value, OD-1 confirmed on real browsers | +| R4.4 Layout | text, AFM, dates, upper-casing, `safeText`, `wrapLines`, the four layouts → DrawLog | O1 (12) exact at full precision, step6 text and images (24) exact, O4 and O5 exact | +| R4.5 Serialiser | emitter, pdf-writer, PNG gate and decode, links, Info, attachment, self-check, `from_envelope`; export-side security, size, reexport and text suites in cargo | against O2: streams equal after B3, image planes equal, links (lopdf) and step6 links (12) equal, Info equal but Producer and Creator, attachment bytes equal (the store-path PDF differs by D-R11 order only); `qpdf --check` clean; `pdfdetach -list` shows `osteo-report.json`; poppler 150 dpi diff 0 px; O11 and the N-1 copy read every R4 PDF, the stripped PDF answers `attachment_stripped`; E2; digests of R4 outputs in `goldens/r4/pdf.*.sha256` (wasm output = native output) | +| R4.6 App export | session driver, `labels.ts`, `export.ts` on the session; alerts unchanged | app suites; app-path bytes = native digests; step6 views via the app path; browser rasters of the Rust- and TS-composed views pixel-identical in Chromium and Firefox (WebKit once it starts); E3, E4; your phone check of L; 612 leaves exported and re-imported on Rust | +| R4.7 report-compare | Rust binary, `pnpm report:compare` unchanged | O10 and the 11 cases in cargo, A.3 lines declared | +| R4.8 Deletions, docs | `packages/pdf` (with the R3 oracle generators; the oracle files stay), `@faunosteo/pdf` and `pdf-lib` in apps/app, the TS CLI, the pdf-lib-based tests per the ledger; N-1 test on O11; RUST.md §10, PDF-FORMAT, PDF_IMPORT_HARDENING §5, DEPLOY (rollback, worker CSP note, floor line per OD-1), CHANGELOG, CLAUDE.md (map, PDF section, gated readers `pdf::read` and `json_js`), TEST_STRATEGY, WORKSPACE, packages/README, fixtures README | `pnpm check:all` green, ledger census green, D1 | +| R4.9 Gates, acceptance | verify contract, fresh verifier, fresh security audit, then your final acceptance (A.4) | verifier CONFORME, security PASS, your explicit merge signal | -## 10. Budgets (measured, recorded in RUST.md §10) +## B.8 Budgets (measured, recorded in RUST.md §10) -| Id | Budget | Measured by | -|---|---|---| -| W1 | served module ≤ 600 KB gzip -9; reported at 550 KB; above 600: stop (A12) | `size.test.ts`, every slice | -| W2 | worker start + largest legitimate import ≤ 1 s on the development machine (Chromium, CPU × 4); recorded per J1 device | harness, J1 | -| W3 | timeout value per the 5.3 rule | harness, J1 | -| W4 | worker linear memory ≤ 256 MiB on every hostile class; the tab survives on every device; the content process back within 10 MiB after GC | harness, J1 | -| W5 | main-thread long tasks during an import ≤ 50 ms, hydrate and navigation excluded | Chromium trace | -| E1 | render difference 0 px (poppler 150 dpi; 300 dpi on the real raster) | cargo + poppler | -| E2 | PDF size ≤ R3 + 5 %; stored attachment < 250 KB (demo with logos) | cargo | -| E3 | export time ≤ 1.2 × the TS baseline (demo, L) in Chromium | harness | -| E4 | §6.4 rule for L; XL completes on the development machine; the doubling probe fails no earlier than the TS baseline | harness | -| T1 | every frozen case dispositioned in the ledger | census | -| D1 | no `pdf-lib` or `@pdf-lib/*` in the lockfile; no `@xmldom/xmldom` in apps/app's closure (kept by `packages/anatomy` until R5) | lockfile, `pnpm why` | +| Id | Budget | +|---|---| +| W1 | served module ≤ 600 KB gzip -9, reported at 550 KB, stop above 600 (A12); estimated +100 to 150 KB | +| W2 | worker start + an L-size legitimate import ≤ 1 s on the development machine (Chromium); recorded per J1 device | +| W4 | worker linear memory ≤ 256 MiB on every hostile class once OD-3 applies; the tab survives on every device (per device: the tab survives, plus `diag.linearBytes`); content process back within 10 MiB after GC | +| W5 | main-thread long tasks during the worker's reading ≤ 50 ms | +| E2 | PDF size ≤ R3 + 5 %; stored attachment < 250 KB (demo with logos) | +| E3 | export time ≤ 1.2 × the TS baseline (demo, L) in Chromium | +| E4 | L completes on the development machine and on your phone; CAP completes on the development machine within the harness ceiling; R4 never fails where the TS completes | +| D1 | no `pdf-lib` or `@pdf-lib/*` left in the lockfile; no `@xmldom/xmldom` in apps/app's closure (kept by `packages/anatomy` until R5, as the review's criterion reads it for the apps/app importer) | -## 11. Browser compatibility +Render (B.2.3), timeout (B.3) and traceability (B.6) are rules, defined where they live. -- Floor as documented: Chrome 97, Firefox 102, Safari 16 for the keyword. The bundle's syntax target is already Firefox 104 (Vite 7 default): a doc inaccuracy that R4.8 corrects in DEPLOY.md whatever OD-1 decides. -- Firefox 102 to 113 lacks module workers. OD-1 options: - - A, recommended: a classic worker in the production build (Vite's default `iife` format, constructor without `type: 'module'` in the build), a module worker in dev only. Keeps the floor and one production path. Proof P-W1 at R4.3: the built worker runs under the production CSP in Chromium, Firefox 157, WebKit (once it starts), and Firefox ESR 102 and 115 binaries downloaded to a scratch directory (Playwright cannot drive a stock Firefox: a test page posts its result to the harness). Cost: worker type differs between dev and production, so J1 runs on the production build. - - B: raise the documented floor to Firefox 115 ESR (module workers everywhere, one worker type in dev and build). A policy change, the owner's decision; DEPLOY.md updated. - - C: feature detection with a main-thread fallback: two import paths, isolation lost where it applies. Not recommended. - - D: refuse the import without module workers: a regression on old Firefox. Not recommended. -- CSP: the worker is a same-origin hashed file under `/assets/`, allowed by `script-src 'self'`; its response carries the proxy's CSP, `'wasm-unsafe-eval'` included; no `blob:` or inline worker. No CSP change is needed; DEPLOY.md gains a note. Without the keyword the import answers `error_core_unavailable`, with no fallback. -- Safari: automated only once WebKit starts (OD-7); real Safari (macOS when available, iOS) at J1 and at the final acceptance. The PDF download revokes its URL at once (`EditorRoute.tsx:245`), which WebKit may cancel: separate bugfix before the final iPhone acceptance, outside R4. -- APIs in the worker: `Blob.arrayBuffer` (Firefox 69, Safari 14), `WebAssembly.compileStreaming` with the main runtime's fallback, `TextDecoder`, `postMessage` with transfer: all under the floor. +## B.9 Browser compatibility -## 12. Checkpoints, stop rules, exit criteria +- Floor documented for the keyword: Chrome 97, Firefox 102, Safari 16; the bundle's syntax target is Firefox 104 (Vite 7 default), and the import works today only from Firefox 113 and Safari 16.4 (`DecompressionStream`). The DEPLOY.md wording changes only per your OD-1 decision. +- Module workers: Chrome 80, Firefox 114, Safari 15; `import()` inside workers ships with them. The worker never uses `import()` (static glue), so option A stays possible. +- Proof P-W1 at R4.3 (B': Chromium, Firefox 157, WebKit once it starts; A: plus a static check that the production worker chunk holds no ESM syntax and is started without `type: 'module'`, and a run on a Firefox 113 binary in a throwaway, offline, loopback-only profile, checksum verified, failures classified as syntax, worker type or CSP): the built worker runs under the production CSP, receives the compiled module, imports a reference PDF and a copy. +- CSP: the worker is a same-origin hashed file under `/assets/`, allowed by `script-src 'self'`; its response carries the proxy's CSP with `'wasm-unsafe-eval'`; no `blob:` or inline worker; no CSP change (DEPLOY.md gains a note). Without the keyword the import answers `error_core_unavailable`, no fallback. +- Safari: automated once WebKit starts (A.5); real Safari (macOS when available, iOS) at J1 and at the final acceptance; the download fix lands separately before the iPhone acceptance. +- Worker APIs: `Blob.arrayBuffer` (Firefox 69, Safari 14), `WebAssembly.compileStreaming` with the main runtime's fallback, structured clone of `WebAssembly.Module` (proven per engine, fetch otherwise), `TextDecoder`, transfer lists. -- Checkpoints: (1) this plan, validated by the owner, then merged (OD-2); (2) J1 after R4.3; (3) the final acceptance after R4.9. -- Stop rules (stop, measure, present): W1 above 600 KB; a reader or differential difference (A2); a §4.3 "not acceptable" difference; a visible change found necessary; S2 failing §6.4; a W4 class breaking its budget; a failed proof (P-W1, CSP); any change needed inside the gated readers (`import_archive`, `read_archive_document`, `rebase_report_on_sheet`; D-R12). -- J1 exit: the import runs on Rust in the worker; the §3.8 messages match R3 on every file; measurements recorded in PDF_IMPORT_HARDENING.md §5; OD-1, the timeout and any 5.6 option decided. -- Final acceptance (owner, LAN, production build behind the DEPLOY CSP): FR and EN exports of the three reference reports and of a new bilan, on computer and phone; viewers (Chrome, Firefox, Acrobat, Preview or Files, Android): pages, attachment listed, properties (Producer Faunosteo), footer links; a side-by-side R3 vs R4 page with per-page differences, preceded by an R3 vs R3 run that measures the browser's own noise; 400 % zoom; print preview and a paper print; accents in pages and properties; R4 PDFs imported into the R3 build run side by side; R4 PDFs re-imported (same bilan); engine unavailable during an import and an export. -- R4 exit (merge): the review's VI.2 R4 criteria as amended here, the §4.3 proofs, the ledger census, the budgets, J1 and the final acceptance, verifier CONFORME, security PASS, `pnpm check:all` green, the owner's explicit merge signal. - -## 13. Risks +## B.10 Risks | Risk | Mitigation | Detection | |---|---|---| -| Writer exactness: float order, `Math.round`, kerning, upper-casing, the Cf table, dates | port expression by expression; full-precision O1; O4 to O6 | O1, normalised streams | -| pdf-writer's `f32` or string forms break a form the N-1 scanner needs (Keywords, Filespec) | UTF-16BE hex written as pdf-lib does; B5 only for numbers nothing reads | O12 read-back, byte test | -| Reader divergence on foreign or old PDFs | O8, O9, the owner's real PDFs | stop rule A2 | -| miniz_oxide and DecompressionStream accept different streams | O10 | cargo | -| Worker type, CSP or wasm in workers on Firefox and Safari | OD-1 and P-W1, CSP proof on three engines, J1 on real devices | R4.3, J1 | -| Memory exhaustion despite the worker (iOS kills the tab) | W4 classes measured, 5.6 options | harness, J1 | -| Export memory | session by construction, §6.4 rule | harness | -| WASM size (+100 to 150 KB estimated) | three fonts only, features, measured per slice | W1 | -| A test lost in the migration | ledger census | R4.8, verifier | -| Browser canvas noise hides or invents differences in the final comparison | R3 vs R3 noise run first; automated proofs use frozen PNGs | final acceptance | -| Largest phase of the roadmap | J1 split, per-slice proofs, stop rules | slice reports | -| Two PDF stacks coexist until R4.8 | the TS stays frozen (no new logic), deleted in one slice | grep census | +| Writer exactness (float order, `Math.round`, kerning, upper-casing, the Cf set, dates, `numberToString` quirks) | port expression by expression; full-precision O1; O4, O5 | O1, B3-normalised streams | +| pdf-writer's `f32` or string forms change a form the N-1 scanner reads | pdf-lib's string forms written explicitly; B5 limited to numbers nothing reads | O11 read-back, stripped PDF, byte test | +| Reader divergence on old or foreign PDFs | O7, O8, real PDFs | stop rule | +| Inflate verdicts differ per engine | O9 in three engines, Node pinned, others listed | R4.2 | +| Worker failures misclassified, late results, races | B.1.5 rules and their tests | R4.3, J1 | +| Memory exhaustion despite the worker | OD-3 before R4.3, W4 classes, accepted-content class | R4.2, harness, J1 | +| Export memory on phones | session, phone checks at J1 and after R4.6 | E4 | +| The measurement itself exhausts the shared host | harness ceiling (4 GiB RSS), no unbounded probe | harness log | +| WASM size | three fonts, features, measured per slice | W1 | +| A test lost in the migration | ledger census over every PDF-touching suite | verifier | +| Canvas noise hides or invents differences in the final comparison | R3 vs R3 run as the pass threshold; automated proofs use frozen PNGs | final acceptance | +| A production-only worker break between R4.3 and R4.9 | browser proof at every slice end | each slice | +| Largest phase of the roadmap | J1, per-slice proofs, stop rules | slice reports | -## 14. Kept and deleted (consumers named) +## B.11 Kept and deleted (consumers named) - Deleted at R4.8: `packages/pdf/**` (last consumers: app import at R4.3, app export at R4.6, CLI at R4.7, tests per the ledger), `apps/app/scripts/compare-reports.ts` and `compare/*`, `@faunosteo/pdf` and `pdf-lib` in apps/app, `@resvg/resvg-js` and `@pdf-lib/*` with packages/pdf. -- Kept: `browserRasterizer.ts` (DOM adapter), `loadBranding` and `src/assets/brand`, `packages/anatomy/render` and `packages/domain` (pipeline, R5), the frozen R3 oracle files (their generators go with packages/pdf; never regenerated), the N-1 Zod copy, O12. +- Kept: `browserRasterizer.ts` (DOM adapter), `loadBranding` and `src/assets/brand`, `packages/anatomy/render` and `packages/domain` (pipeline, R5), the frozen R3 oracle files (generators deleted with packages/pdf, never regenerated), the N-1 Zod copy, O11. +- Amendments to the review's VI.2 R4 row, listed: the export is an incremental session, not two calls (OD-2); the render threshold is 0 px (A6) with stream equality; N-1 = O11 + the N-1 Zod copy (R3 and 0.2.0); BLK-009 measurements per engine and per owner device; the rescue copy goes through the worker (A8); Producer and Creator (A4); labels move to the app; `exportFromEnvelope` is cargo-only; the lockfile criterion reads apps/app's importer. -## 15. Open decisions for the owner +## B.12 Revision 2 log -| # | Question | Recommendation | -|---|---|---| -| OD-1 | Module workers vs the Firefox 102 floor | A, classic worker in production, proven by P-W1; if P-W1 fails, B (floor Firefox 115 ESR) for your decision | -| OD-2 | Branches: this plan on `chore/rust-r4-plan`, merged at validation; then the Safari fix off develop; then `feature/rust-r4-pdf-core` cut from develop (one feature branch, one feature merge, no sync of develop into it) | yes | -| OD-3 | Message when a rescue copy times out or crashes the worker | `error_invalid_copy` (accurate: refused, file untouched); alternative `error_generic` | -| OD-4 | A PDF that crashes the reader moves from « Le moteur de lecture n'a pas pu être chargé. Réessayez l'import. » (today's trap mapping) to « Erreur de lecture du PDF. » (A7) | confirm | -| OD-5 | A worker that cannot start keeps « Le moteur de lecture n'a pas pu être chargé. Réessayez l'import. » | yes | -| OD-6 | J1 devices and browsers (computer, iPhone, low-end Android) and the LAN port of the J1 build | your list | -| OD-7 | WebKit system packages: `sudo apt-get install libevent-2.1-7t64 libmanette-0.2-0` | install before R4.3 | -| OD-8 | Slot of the Safari download fix | right after this plan's validation, before the feature branch | -| OD-9 | Repository weight of O2 (about 5 MB of PDFs and PNGs) | accept | +Folded (main points): worker failure classification with a top-level catch, unhandled rejections and synchronous `SecurityError`; settle-once and listener order; an import controller across mounts and dropped late results; start timer separate from the task timer, paused while hidden; copy timeout and crash on `error_core_unavailable`; message precedence and today's mapping for every kind; the third JS side effect (overwritten duplicate with a lone surrogate, OD-4) and the probe on R2's `parse_json` instead of a new grammar; `DecompressionStream` facts and their per-browser differences; static glue in the worker (no `import()`), compiled module posted by the main thread, worker script prefetched for offline imports; the TS memory row corrected (RGB planes held until `save()`), the 9 MB transient, the cap at 1,000 views for the draft sheet, L and CAP instead of XL and a doubling probe; OD-2 as a decision with the main-instance session under the runtime's poison and generation rules, offline-safe; OD-3 before R4.3 using BLK-012's measured ratio, the copy class and the accepted-content class; phone checks at J1 and after R4.6; the harness ceiling; the D-R11 order of the attachment's report and a store-path fixture; committed zlib bombs; JS regex semantics; PNG gate scope and UPNG acceptance; the stripped-PDF N-1 proof; `report:compare` lines declared; warning order; `rasterSizeOf` parsing; branding SVG through the R3 reader; ledger widened to the mocking suites; VI.2 amendments listed; `exportFromEnvelope` off the facade; AFM tables generated from the oracle that tests them; one poppler pass; the owner part first, facts in the annex, open decisions reduced to real ones; real-PDF anonymisation extended; gated readers extended; the branch cut after the Safari fix. -Scheduled later: the timeout value and the 5.6 options (J1), the §6.4 outcome (reported at R4.6), any §4.3 "not acceptable" difference (stop rule). +Not folded, with the reason: +- A throwaway main-thread instance per export (simplicity 1): the wasm-bindgen glue keeps its instance in a module variable and module records are never unloaded, so each throwaway instance would stay in memory until reload (measured at R3, BLK-012: dead instances retained). The finding's conclusion (no export worker by default) is adopted through OD-2. +- A raster timeout in the export (robustness 12): it would turn today's hang into a missing-image warning, a behaviour change; kept as today, recorded as a residual. +- Dropping the 400 % zoom and the print (simplicity 12): you asked to keep the manual checks; they stay as quick confirmations. +- A memory scope through `systemd-run` (robustness 10): refused by this session's permission rules; the harness enforces its own ceiling instead. -## 16. Tracked outside R4 +# Annex. Facts (develop ae845c6, 2026-10-17) -- Safari PDF download fix (separate bugfix, before the final iPhone acceptance). -- The worker container ignores SIGTERM: Docker kills it after 10 s, the job resumes after its lease expires (chore, for example `init: true` in compose). -- README test counter stale (367; 1,015 at the R3 merge). +Engine, consumers, tests +- `packages/pdf/src` 3,122 lines: reader (`attachment/scan.ts` 549, `inflate.ts` 83, `read.ts`), writer (`export.ts` 228, `render/*` about 1,250 incl. `observationPage.ts` 634), `branding.ts` 291, `svg/*`, `text.ts`, `render/dates.ts`, `i18n` (25 labels). Tests: 11 files, 140 cases (scan 39, inflate 9, read 13, security 55, text 5, size 2, goldens 11, raster 2, reexport 1, R3 oracle generators 3). +- App: `src/pdf/{import,export,browserRasterizer}.ts`, `EditorRoute` (type only), `scripts/compare-reports.ts` + `compare/*` (589 lines, 11 cases). Only pdf.goldens, pdf.import and pdf.roundtrip load `pdf-lib`; `apps/app/src` never does. +- `release/0.2.0` and develop carry the same `packages/pdf/src`, and the same archive, report and sheet schemas as the N-1 copy (`git diff release/0.2.0 1a7ee7d` on those directories: empty). +- Import today: whole file read on the main thread, TS scan, `JSON.parse(new TextDecoder().decode(data))` (scan.ts:498), candidate kept when it is an object with a numeric `version` or `envelopeVersion`, `JSON.stringify` of that object into the core's `archive.import` (sheetOps.ts:33). The byte scan runs before the core loads. A trap spends one of the main instance's 3 recoveries. +- Rescue copy today: routed on the first KiB, ≤ 16 MiB, `TextDecoder('utf-8', { fatal: true })`, `draft.restore` on the main instance. +- Export failures today: any throw → `editor.export_failed` or the stopped text (`EditorRoute.tsx:252-254`). + +pdf-lib 1.17.1 as used +- Fonts Helvetica, Helvetica-Bold, Times-Roman, WinAnsi, not embedded; width `total * (size / 1000)`, `widthOfGlyph || 250`, KPX kerning; glyphs drawn unkerned. +- Operators: text `q BT rg Tf TL Tm ( Tj T*)+ ET Q` (TL 24); image `q cm cm cm cm Do Q`; line `q RG w d m m l S Q`; rectangle `q rg w d cm cm cm m l l l h f Q`; operands by `numberToString`; no opacity, no ExtGState; a new resource name `Tag-<10 digits>` per draw call (seeded RNG); streams Flate (pako level 6); classic xref (`useObjectStreams: false`), no `/ID`. +- Images: `embedPng` decodes at call time (UPNG, RGBA8) and holds the RGB plane (and an alpha plane when an alpha < 255) until `save()` deflates them; no dedup. +- Info: Title, Author, Subject, Keywords (`faunosteo-archive/1`), Producer and Creator (`pdf-lib (https://github.com/Hopding/pdf-lib)`) as UTF-16BE hex with BOM; CreationDate and ModDate = `exportedAt`, literal `D:YYYYMMDDHHMMSSZ`. +- Attachment: `JSON.stringify(envelope, null, 2)`, report written as the store object (insertion order); EmbeddedFile `/Subtype /application#2Fjson`, Params; Filespec `/F (osteo-report.json)`, `/UF` and `/Desc` hex, `/AFRelationship /Data`; catalog `Names/EmbeddedFiles` (hex key) and `/AF`. +- Links: `Border [0 0 0]`, URI action holding the hex of the ASCII bytes. +- Dates: `Intl.DateTimeFormat` fr-FR and en-US, long month, UTC; `^\d{4}-\d{2}-\d{2}$` else unchanged; `Date.UTC` maps 0 to 99 to 1900 to 1999; rolled-over dates unchanged. +- Caps: 500 observations × 8 distinct views; the draft sheet has two views; rasters 1200 px wide, aspect clamped to 0.2 to 2.0; views of an observation rasterised under `Promise.all`, embedded in order; only the base64 branding PNGs pass the PNG gate. + +Measurements (Node 24, TS engine, resvg rasteriser, fr): v2full nologos 3 views, PNG 1.63 MB, PDF 1.17 MB, attachment stored 54 KB; demo nologos 7 views, 3.75 MB, 2.64 MB, 55 KB; demo logos 8 rasters, 3.79 MB, 2.82 MB; v2unresolved logos 1 raster, 0.26 MB. Per view: PNG 526 to 553 KB, decoded RGBA 4.4 to 4.5 MB, ≈ 0.37 MB of image stream. + +Experiments: `String::from_utf8_lossy` after one BOM = `new TextDecoder().decode` on 200,015 byte strings, 0 differences; serde_json refuses `1e400`, matches the JS path on `1e-400`, `-0`, big integers, long decimals and duplicates once printed by `js_number`; a lone surrogate escape fails in serde_json (`lone_surrogate` since R2). + +Browsers and tooling: module workers Chrome 80, Firefox 114, Safari 15; `DecompressionStream` Chrome 80, Firefox 113, Safari 16.4; Vite 7.3.6 default target chrome107, firefox104, safari16, production workers `iife` by default, dev classic workers keep ESM imports; CSP in DEPLOY.md `script-src 'self' 'wasm-unsafe-eval'`, no `worker-src`, `docker/nginx.conf` sends none; playwright-core 1.64.0 outside the repository, Chromium present, Firefox 157 installed and proven headless (classic and module workers with wasm), WebKit 27.2 downloaded and blocked by two missing system packages; WASM 395.6 KB gzip of 600 KB, linear memory 4.1 MiB after the editor path, sheet bundle text about 522 KB; the wasm-bindgen glue keeps its instance in a module variable (BLK-012: dead instances retained until reload); R2 and R3 kept no per-test traceability table. From ac2a3bc41bcdd2cb54b286c4035b531eec41cf35 Mon Sep 17 00:00:00 2001 From: bchanot Date: Sun, 18 Oct 2026 04:42:53 +0200 Subject: [PATCH 3/3] chore(memory): R4 plan validated by the owner (BDR-057, LRN-059, LRN-060) --- .claude/memory/decisions.md | 7 +++++++ .claude/memory/learnings.md | 12 ++++++++++++ .claude/tasks/TODO.md | 2 +- .../plans/2026-10-17-rust-r4-pdf-core.md | 19 ++++++++++++++++++- 4 files changed, 38 insertions(+), 2 deletions(-) diff --git a/.claude/memory/decisions.md b/.claude/memory/decisions.md index 3bfa4c10..fc434640 100644 --- a/.claude/memory/decisions.md +++ b/.claude/memory/decisions.md @@ -436,3 +436,10 @@ Status: closed; R3 still awaiting owner LAN confirmation, not merged, R4 not sta Owner brief 2026-10-16 + AskUserQuestion: (1) Home import also takes JSON rescue copy: sniff first KiB (`%PDF-` → PDF; `{` / .json → copy), cap MAX_ATTACHMENT_BYTES 16 MiB before read, fatal UTF-8, pure `readRescueCopy` → core `draft.restore` (no JS parse), then existing hydrateImport branch; UNREADABLE_DRAFT_KINDS → `invalid_copy`. (2) `internal` reading draft → held `engine_failure` row (+ archive companion), `TrapMemory` keyed on (draft, archive) pair: no re-read by mount / New report / Import, only panel Retry (`retryTrapped`); any discard clears; `held_blocked_engine` alert. SUPERSEDES BDR-055 "never held" for `internal` only (`core_unavailable` etc. stay FAILED). (3) ReplaceSessionDialog (Annuler / Ouvrir le bilan en cours / Remplacer, copy when unsaved) on New report submit + Home pick when report in memory or draft slot; Home picker clicked synchronously. (4) `useUnsavedGuard` once in App. Commit 9bd9aca. Alt rejected: new store action openCopy (dup of hydrateImport, session change before validation); JS JSON.parse of copy (depth; core reads raw text); copy cap MAX_PDF_BYTES (50 MiB file cap, not JSON bound); trapped memory on draft text only (archive parsed in same call); fixing GC finalizer via glue internals. Status: closed; R3 awaiting owner LAN validation, not merged, no R4, release 0.2.0 untouched. + +## BDR-057 | 2026-10-18 | R4 plan validated: module workers, incremental export in main instance, JSON memory guard gated at R4.2 + +Context: R4.0 plan rev 2 (`.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md`: options §A.2, conditions §A.6), 3 challengers, owner approval 2026-10-18. +Decision: OD-1 module workers everywhere; import needs Firefox 114 (today it already needs DecompressionStream: Firefox 113, Safari 16.4); import support documented per browser apart from app support, from real checks. OD-2 incremental ExportSession in main instance: PDF + one decode ≈ 0.37 MB/view vs TS ≈ 3.8 MB/view (RGB planes held until save) vs two calls ≈ 0.91; offline export kept; cleanup on error/cancel. OD-3 JSON value budget in linear pre-scan + module `--max-memory`, thresholds from R4.2 reproducible measurements; not shown → stop before R4.3 import switch. OD-4 overwritten duplicate key holding lone surrogate refused, file untouched. OD-5 PDF crash/timeout `error_generic`, copy crash/timeout `error_core_unavailable`, kinds keep technical / resource / invalid apart. OD-6 J1 matrix exact, unrun device never reported validated. OD-7 ~5 MB fixtures, no real data in git. +Alt rejected: classic iife worker for Firefox 102-113 (dev/prod split for dead versions); two-call export (all PNGs in linear memory); disposable export worker (asset fetched at export breaks offline, protocol cost); throwaway main-thread instance (glue retains instances, LRN-060); own lossless JSON reader (R2 rejection holds). +Status: active. Order: plan merge → Safari download bugfix (owner validation) → feature/rust-r4-pdf-core → R4.1. diff --git a/.claude/memory/learnings.md b/.claude/memory/learnings.md index ae4ed225..ac0d7a1d 100644 --- a/.claude/memory/learnings.md +++ b/.claude/memory/learnings.md @@ -454,3 +454,15 @@ Apply: R4 PDF worker failure states, Step 8 avatar upload errors, any capped ret - User text into i18n templates: `.replace(re, () => value)`; string replacement expands `$&`/`$'`. - Picker opened from a confirm dialog: call `input.click()` synchronously in the confirm handler (user activation, WebKit). Apply: any export/backup feature, R4 PDF reader worker failures, Step 13 uploads. + +## LRN-059 | 2026-10-17 | Rust `from_utf8_lossy` matches TextDecoder; serde_json differs from JSON.parse on overflow and surrogates + +- `String::from_utf8_lossy` after stripping one BOM = `new TextDecoder().decode()` (non-fatal): 200,015 random + edge byte strings (overlong, UTF-8 surrogates, truncated, double BOM), 0 diff. Parity holds on tested sequences, not proven in general. +- JSON.parse vs serde_json: `1e400` → JS Infinity (JSON.stringify writes `null`), serde_json "number out of range"; lone surrogate escape kept by JS, refused by serde_json; duplicate key last value at first position in both (preserve_order). A JS path that did parse → stringify has side effects (overflow → null, dropped duplicates): reproduce them or declare them. +Apply: Rust ports of JS readers of untrusted text (R4 PDF reader, Step 13 uploads, Step 15 server validation). + +## LRN-060 | 2026-10-17 | wasm-bindgen glue retains its instance until reload; only a terminated worker frees linear memory + +- `--target web` glue keeps the instance in module scope (`let wasm`, `__wbg_finalize_init`); ES module records are never unloaded, so each fresh glue import (`?instance=n`) keeps its old instance and memory (BLK-012: dead instances retained until reload). Linear memory never shrinks. +- Memory-heavy one-shot work: reuse the long-lived instance (high-water kept until reload) or run in a Worker terminated after (everything freed). A "disposable instance" on the main thread is a leak, not a release. +Apply: R4 import worker and export placement, any future in-browser wasm batch job. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 83191a9b..6269cea8 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -204,7 +204,7 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] R3 final checks 2026-10-17 (owner brief 2026-10-16): rescue copy importable from Home, trapping draft held (no reload loop), replace confirmation, guard app-wide → 9bd9aca (BDR-056, BLK-012); verifier CONFORME 10/10, security PASS, check:all green; residuals in BLK-012 - [ ] R3 follow-ups kept by the owner (2026-10-15): worker TLS exercised against a real HTTPS endpoint before the deployment that needs it; roxmltree limits + SVG prescan (pin the iterative release when published); Firefox/WebKit check; SVG references across drawings scoped per view before community uploads (Step 13); WASM size margin (396 of 600 KB) before R4; TypeScript PDF sanitiser still present until R4 - [ ] R4 pdf core: reader (Web Worker, BLK-009 browser measurements) + writer (`pdf-writer`, Std14 AFM, draw log; lines/rects/colours golden frozen with the TS engine first, D-R2) + `report-compare` tool; `packages/pdf` deleted; N-1 scanner reads R4 PDFs - - [x] R4.0 plan (owner go 2026-10-17, arbitrations A1-A13): `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md` rev 2 after 3 challengers (correctness 6 major, robustness 10 major, simplicity 3 major; none fatal) on chore/rust-r4-plan; Playwright Firefox 157 installed + proven, WebKit 27.2 downloaded (owner: `sudo apt-get install libevent-2.1-7t64 libmanette-0.2-0`); awaiting owner validation + OD-1..OD-7; no code + - [x] R4.0 plan (owner go 2026-10-17, arbitrations A1-A13): `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md` rev 2 after 3 challengers (correctness 6 major, robustness 10 major, simplicity 3 major; none fatal) on chore/rust-r4-plan; Playwright Firefox 157 installed + proven, WebKit 27.2 downloaded (owner: `sudo apt-get install libevent-2.1-7t64 libmanette-0.2-0`); validated by the owner 2026-10-18 with conditions (plan §A.6, BDR-057) - [ ] after validation: finish chore/rust-r4-plan into develop (owner signal), Safari PDF download fix (separate bugfix, owner go), then cut feature/rust-r4-pdf-core - [ ] R4.1 freeze (O1-O13, ledger, browser harness, TS baselines) - [ ] R4.2 reader (cargo + wasm, differential, OD-3 thresholds confirmed with owner before R4.3) diff --git a/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md b/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md index 514cc32a..508fb40f 100644 --- a/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md +++ b/.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md @@ -3,7 +3,7 @@ - date: 2026-10-17 | rev 2 (after 3 plan challenges: correctness CONCERNS (6 major, 8 minor), robustness CONCERNS (10 major, 12 minor), simplicity CONCERNS (3 major, 9 minor); log in §B.12) | branch: `chore/rust-r4-plan` (plan only); code later on `feature/rust-r4-pdf-core`, one merge - spec: owner briefs 2026-10-17 "Préparation de R4" and "Autorisation de préparation R4.0" (arbitrations A1 to A13, §A.0); `docs/development/RUST_ARCHITECTURE_REVIEW.md` I.2, II.2 rows "pdf reader", "pdf writer", "report-compare", III.3, VI.1 to VI.4, D-R2, D-R3, D-R11, D-R12, D-R14; `RUST.md` §8.3, §8.7, §8.8, §9.4, §9.7; `PDF_IMPORT_HARDENING.md` §5; BLK-009, BLK-012; CLAUDE.md "PDF", "Schemas", archive-reader gate - facts: Annex (inventory, measurements and experiments of 2026-10-17) -- status: R4.0 only. No code and no fixture generated; R4.1 starts on the owner's explicit go. +- status: validated by the owner 2026-10-18 with conditions (§A.6). R4.1 may be prepared once the plan branch is merged, the Safari fix has merged and `feature/rust-r4-pdf-core` is cut from the updated develop. - not in scope: TTF fonts, `scope_disclaimer`, the planned collection caps, opening the import to external users (BDR-040 gate stays open), R5 (pipeline, `packages/anatomy`, `packages/domain`, `documentSha256`), the Safari download fix (separate bugfix), release 0.2.0, any deployment, any visible change on the documented browsers. # Part A. For the owner @@ -50,6 +50,23 @@ The application reads and writes archives with Rust only. An import, PDF or resc WebKit on this machine: `sudo apt-get install libevent-2.1-7t64 libmanette-0.2-0` (Firefox 157 and WebKit 27.2 for Playwright are downloaded; Firefox runs headless with workers and wasm). The go for the Safari download fix. Real PDFs, if any (§B.5.2). +## A.6 Owner validation (2026-10-18) + +OD-1 to OD-7 approved as recommended in A.2 (BDR-057), with these conditions, binding on the slices: +- OD-1: module workers, no second implementation for old versions. The import's support per browser is documented apart from the application's support, from real capability checks (P-W1, J1). +- OD-2: incremental export with bounded memory, cleanup on error and cancellation, offline export kept; release proven at R4.6 on success, error, cancellation and recovery. +- OD-3 (conditional): R4.2 sets the resource budgets from reproducible measurements covering upstream allocations (file buffer, inflation, decoded text, value tree), wasm linear memory, JavaScript objects (transfers, result parse), decoded images and the browser's peak (content-process RSS). If a budget cannot be shown or the preventive caps are insufficient: stop before R4.3 wires the import and present the results. A resource error never alters the original PDF, the bilan or the archives. +- OD-4: the overwritten-duplicate lone-surrogate case is refused; no JSON reader of our own; the source file stays intact, nothing is dropped silently. +- OD-5: messages as proposed; the outcome kinds keep technical errors, resource exhaustion and invalid files apart. +- OD-6: the J1 matrix of A.4 exactly; a device or browser not actually run is reported as not run, never as validated. +- OD-7: about 5 MB of fixtures; no real professional or confidential data in git (B.5.2). +- The A.3 differences stay under the oracles and are reported explicitly at each checkpoint; no other silent difference. +- The harness's 4 GiB RSS watchdog is not a guaranteed memory limit: every result that relies on it says so. +- Proofs owed in the slices, never assumed: png crate tolerance on historical files (R4.5, O6), loading the module in the worker per engine (R4.3), memory on hostile files (R4.2, R4.3, J1), incremental export without image accumulation (R4.6), v2/v3 compatibility (R4.2, R4.5), visual rendering (R4.5, final acceptance), import and export on the J1 browsers (J1, final acceptance), data kept when the engine fails (R4.3, R4.6). +- Goldens stay references, never regenerated to hide a regression; the TS engine is deleted only after parity is proven and every consumer has switched (R4.8). +- Checkpoint reports give concrete results, not plan repetition, and list the functional and graphical validations available. Manual priorities: FR and EN export, anatomical views and markers, a v3 import, real historical v2 PDFs, attachments and kept data, wrong and large files, the targeted browsers and phones. +- Order: R3 closed (checked 2026-10-18: merge 8df964b on develop, branch deleted locally and on origin); this plan merged; the Safari fix on a short branch, tested, merged after the owner's validation; `feature/rust-r4-pdf-core` from the updated develop. WebKit packages installed by the owner, then workers, wasm, imports, exports and memory protections checked on WebKit; WebKit results never replace a manual iPhone run. LAN DEV access kept. Excluded: R5, any deployment or production change, release 0.2.0, any graphical redesign. + # Part B. The plan ## B.1 Architecture and contracts