docs(docs): RUST.md §10.10 R4.8 record (deleted, replaced, kept with reasons, census, non-regression table, risks, R5 prerequisites); CLAUDE.md, README, WORKSPACE, packages/README, TEST_STRATEGY, PDF-FORMAT, DEPLOY rollback R4 note and module size, hardening §5 pdf-lib closed, fixtures README, CHANGELOG R4 entry; TODO R4.8 closed; memory BDR-064, journal

This commit is contained in:
bchanot committed 2026-10-19 12:17:02 +02:00
1 parent e88769a707
commit f36dbdd4ae
13 files changed
+238 -62

No files matched your search

+27 -20
View File
@@ -1,9 +1,11 @@
# PDF format and report archive
A Faunosteo PDF is both a document for the reader and an archive of the
bilan. The engine lives in `packages/pdf` (`@faunosteo/pdf`) and renders from
explicit inputs only: no network, no browser DOM, no access to the current
catalog. The attachment carries everything a future reader needs to
bilan. The engine lives in the Rust core (`crates/core/src/pdf`, reached
through the wasm module in the browser and natively by the tools; the
TypeScript `packages/pdf` it reproduced byte for byte was deleted at R4.8,
`docs/development/RUST.md` §10) and renders from explicit inputs only: no
network, no browser DOM, no access to the current catalog. The attachment carries everything a future reader needs to
re-render the bilan when the anatomical sheet, the catalog, the labels or the
branding have changed since the export. Design record:
`docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md`.
@@ -19,8 +21,9 @@ branding have changed since the export. Design record:
asset), then zones, findings grouped by finding zone, free findings,
severity, comment.
- Conclusion page: conclusion, recommendations, next appointment.
- Fonts: pdf-lib StandardFonts Helvetica, Helvetica-Bold (body) and
Times-Roman (headings), WinAnsi encoding. Every printed string goes through
- Fonts: the standard Type1 fonts Helvetica, Helvetica-Bold (body) and
Times-Roman (headings), WinAnsi encoding, measured with the AFM widths
pdf-lib 1.17.1 used (`crates/core/src/pdf/write/afm_data.rs`). Every printed string goes through
`safeText`: control characters are stripped and any glyph WinAnsi cannot
encode becomes `?` (en and em dashes, accented letters, `•`, `·` are fine;
`⚠` is not, hence the `(!)` vet-referral marker).
@@ -44,7 +47,7 @@ branding have changed since the export. Design record:
One native embedded file (`EmbeddedFiles`, `AFRelationship /Data`) named
`osteo-report.json`, MIME `application/json`, description
`Faunosteo report archive v1`, Flate-compressed by pdf-lib. Pretty-printed
`Faunosteo report archive v1`, Flate-compressed (zlib level 6). Pretty-printed
JSON, camelCase keys (the sheet document inside keeps its own snake_case
schema):
@@ -227,15 +230,16 @@ The same caps are checked at export, so an export can never write a file its
own reader refuses.
The branding URL rule exists because a URL is written into every page's
link action: pdf-lib does not escape a literal string, so the engine writes
the URI as a hex string and the schema refuses the characters that could
close a literal. The export re-validates the branding and the labels it
link action: a literal string is not escaped (pdf-lib did not, the core
does not either), so the engine writes the URI as a hex string and the
schema refuses the characters that could close a literal. The export re-validates the branding and the labels it
receives (`ArchiveExportError('branding_invalid')` before any page is
drawn), even from an archive that passed the reader. The URI itself is the
hex form of the URL's ASCII bytes (`<68747470…>`), not UTF-16 with a BOM,
so readers that take `/URI` as raw bytes (poppler) follow it. A PNG logo is
checked before pdf-lib's decoder (UPNG) sees it, because that decoder trusts
its input: it sizes its bitmap from whatever IHDR claims, its `inflateRaw`
checked before the decoder sees it (the core's decoder reproduces UPNG's
acceptance, the decoder pdf-lib used, so that an archived logo decodes to
the same planes); the gate exists because that decoder trusted its input: it sizes its bitmap from whatever IHDR claims, its `inflateRaw`
never checks the input bound (a truncated or crafted IDAT stream makes it
read past the buffer and never return), and it scans text chunks for a NUL
byte that may be missing. The gate therefore validates, in order: the
@@ -247,10 +251,11 @@ sizes are not computed); an expected raw scanline size,
`(ceil(width × bits per pixel / 8) + 1) × height`, of at most 16 MiB, the
attachment cap the native inflater enforces (4096² at 8-bit RGBA is 67 MB
of scanlines and is refused; a 512 px wide logo is under 1 MB). That cap
bounds what is inflated, not what pdf-lib's decoder then allocates: the
decoder still expands any accepted image to a 32-bit RGBA buffer, so the
4096² pixel cap is what bounds memory (about 430 MB peak measured with four
4096² logos; real logos are 512 px wide). Then the chunk list, which must end on an empty `IEND`
bounds what is inflated, not what the decoder then allocates: the decoder
still expands any accepted image to a 32-bit RGBA buffer, so the 4096² pixel
cap is what bounds memory (about 430 MB peak measured with four 4096² logos
on the TypeScript engine; real logos are 512 px wide; the core frees the
planes as soon as the image object is written). Then the chunk list, which must end on an empty `IEND`
that closes the file and may hold only the chunk types the decoder reads by
fixed offsets or by declared length (`PLTE`, `tRNS`, `IDAT`, `IEND`,
`gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs`, `bKGD`): a second IHDR, an APNG
@@ -271,11 +276,13 @@ fallback text and the export reports `logo_failed`.
reader in a worker since R4.3) and the `report:compare` tool (the same
reader, natively, since R4.7) read the attachment by byte scan, so a
crafted object stream is never inflated (tested with a 64 MiB bomb:
milliseconds, no memory growth). pdf-lib stays on the export side, on trusted inputs, and
in tests that inspect exported files. What remains before external users
(`docs/development/PDF_IMPORT_HARDENING.md` §5): a Worker or server-side
budget for any path that still parses PDF objects, the pdf-lib dependency
decision, and read-side caps on report collections.
milliseconds, no memory growth). No PDF library remains in the repository
since R4.8: the core writes the files, the tests read them back with a
reader of the writer's own format (`apps/app/tests/helpers/pdfFile.ts`).
What remains before external users
(`docs/development/PDF_IMPORT_HARDENING.md` §5): the server-side budget
decision for any future path that parses PDF objects, and read-side caps
on report collections.
- The scanner reads what is in clear: an attachment whose stream dictionary
is malformed beyond `/Length` and `endstream`, or an encrypted file, yields
no candidate (`no_attachment`), as the previous reader did in practice.
+5 -3
View File
@@ -110,9 +110,11 @@ hides its `/Keywords`, so a stripped attachment on such a file reads as
brackets deeper than 64 levels (linear byte pre-scan, strings skipped).
- Option C or a server-side import (Step 14) with a hard memory and time
budget per file, for any path that still parses PDF objects.
- Dependency decision on pdf-lib (unmaintained since 2021): keep it for
generation only, or move to the maintained fork; recorded as a Step 14
question.
- Dependency decision on pdf-lib (unmaintained since 2021): closed at R4.8
(2026-10-19): pdf-lib left the repository with the TypeScript engine; the
core's writer (`RUST.md` §10.7, §10.8) generates the PDFs and its reader
reads them; no PDF library remains in the application, its tests or the
tools (`apps/app/tests/unit/r4.census.test.ts` fails on a return).
- Report collection caps on read (observations, zones, findings per
observation) so a hostile report cannot freeze the editor before the
export's own caps refuse it.
+131
View File
@@ -1960,3 +1960,134 @@ Residual risks and open points (R4.7):
- A PDF over 50 MiB is refused by the entry check (`file too large`) before
the reader, as before; the reader's own cap would say `not a readable
PDF`.
### 10.10 R4.8: the TypeScript PDF engine deleted (2026-10-19)
Audit before every deletion (owner's R4.8 brief §2): every importer of
`@faunosteo/pdf`, `pdf-lib`, `@pdf-lib/*`, `@resvg/resvg-js`, `@xmldom/xmldom`
and `@faunosteo/domain` was listed across the application, the glue package,
the tests, the scripts, the pipeline, the workspace files, the Dockerfile,
the CI files and the docs (`grep` census, now the `r4.census` suite). The
application's last consumer of the engine (the TypeScript import path,
tests only since R4.3) and the glue's (the session suite's compose check,
the engine comparison) were moved onto the core or deleted first; the
pipeline's consumers (`packages/anatomy/src/curation/app-review.ts` →
`render/compose.ts`, 22 files → `@faunosteo/domain`) stay until R5.
Deleted (70 files, 12,010 lines of TypeScript):
- `packages/pdf/**` (3,122 lines of engine, 8,211 of tests): the export,
the layered reader, the byte-scan attachment reader, the SVG sanitiser and
the view composition, the page renderers, the branding gate, the labels,
the 140 engine cases and the R3 and R4 oracle generators
(`r3.{highlight,sanitiser}-oracles`, `r4.{dates,inflate,png,reader,text}-oracles`)
with their helpers (`resvg`, the reader-input crafters).
- `apps/app/scripts/compare-reports.ts` and `compare/{report,sheet}.ts`
(589 lines, the TypeScript CLI replaced at R4.7), `tests/unit/scripts.compare.test.ts`.
- `apps/app/src/pdf/import.ts` (the TypeScript scanner behind
`importReportFromPdf`, tests only since R4.3) and `tests/unit/pdf.import.test.ts`.
- `tests/unit/pdf.export.app.test.ts` (T2 and scenario 3 through the
engine's `exportFromEnvelope`: the re-export from the envelope alone is
cargo's `export_from_envelope`, whose signature takes no sheet and no
branding, byte-equal to the R4 files on 13 runs).
- The application's generators `r4.{compare,tolerance,writer}-oracles` and
`tests/helpers/r4-draw.ts` (the pdf-lib draw recorder); the glue's
`pdf-export.ts-engine.test.ts` (the Node timings against the engine,
recorded in §10.7).
- Dependencies: `@faunosteo/pdf` (app, core-wasm), `pdf-lib`, `tsx` and
the app's `report:compare` script (app), `@faunosteo/domain` (app: its
two incidental test uses moved to `@faunosteo/core-wasm/types` and the
N-1 Zod copy), the Dockerfile's `COPY packages/pdf/package.json`.
`pnpm-lock.yaml` holds no `pdf-lib`, `@pdf-lib/*` or `@resvg/resvg-js`
any more; `@xmldom/xmldom` only under `packages/anatomy`.
Replaced in the tests (no case ignored, no golden regenerated):
| Before | After |
|---|---|
| pdf-lib parsed the app export's file (texts, images, page count, Info, Filespec) | `tests/helpers/pdfFile.ts`, a reader of the writer's own format (objects, dictionaries, Flate streams through `node:zlib`, pages in `/Kids` order, Info, annotations, XObjects); `pdfRead.ts` keeps its content-stream parser on it |
| pdf-lib read the links of the file | the same reader's annotations (`goldens.ts` `readLinks`) |
| pdf-lib built the foreign PDFs of the rejection cases | `buildPdfWithAttachment` (a minimal file with one embedded file in clear) and the frozen hostile fixtures |
| pdf-lib's `embedPng` checked the golden rasters | the XObject sizes read from the app export's file |
| the engine composed the views of the `views.*.json` goldens | the SVGs the core's session asks the rasteriser for, captured at the mocked rasteriser in request order |
| the engine's scanner read the attachment of the envelope and N-1 suites | the frozen O11 scanner, now exported by `@faunosteo/core-wasm/n-1` (`readAttachmentBytes`, `readAttachedDocument`): the R3 build's reader, as a rollback would read |
| `importReportFromPdf` (TypeScript scanner + core) | `tests/helpers/pdfImport.ts`: the module's `SheetHandle.import_pdf` on a handle of the bundled sheet, the envelope mapped by the worker's `pdfImportOf` (the same kinds HomeRoute shows); the main-thread session still never calls the reader (OD-1) |
| the session suite compared each request SVG with the engine's `composeHighlightedViewSvg` | structural facts per request (the view's frame, the highlight, no editor attribute) and the real-raster manifest digests; the text itself is cargo's (`pdf_session` against `compose_highlight`, `anatomy_compose` against the frozen R3 trees) |
Still TypeScript, with the reason (owner's R4.8 brief §6.4):
- `packages/anatomy` (the curation pipeline, the legacy importer, the frozen
renderer `render/` and `@xmldom/xmldom`): R5's scope, consumed by
`pnpm import:equine` and the seed golden; no app and no PDF consumer.
- `packages/domain`: the pipeline's schemas until R5; the application and the
glue no longer depend on it.
- `packages/core-wasm/tests/n-1/{domain,scanner}`: the frozen N-1 readers
(the R2 Zod schemas, the R3 byte-scan reader), never edited, permanent.
- `apps/app/src/pdf/{export,labels,branding,browserRasterizer}.ts`: the
application's side of the export (input assembly, the session driver, the
browser's rasteriser), UI glue by the review's boundary.
- `apps/app/src/core/worker/**`: the import worker (protocol, timers,
outcome mapping), browser glue.
- The test readers above (`pdfFile.ts`, `pdfRead.ts`, `pdfImport.ts`):
tooling on the writer's own files, never product logic.
- `scripts/r4-afm-data.mjs`: the generator of `afm_data.rs` from the frozen
O4/O5 oracles (no pdf-lib: reads `afm.json` and `text.json`), kept so the
tables stay reproducible.
Census and gates: `apps/app/tests/unit/r4.census.test.ts` (8 checks) fails
on a reintroduced import of the engine, pdf-lib, the TS domain or xmldom
under `apps/app/src` or `packages/core-wasm/src`, on a dependency added back,
on a lockfile holding pdf-lib, on an export that does not open the core's
session, on an import that does not go through the worker, on a root
`report:compare` that is not the binary, and on a missing N-1 reader or
oracle folder. The ledger (`scripts/r4-ledger.mjs`) now reads its 301 frozen
cases from the committed TSV (the deleted suites cannot run), re-derives the
dispositions and checks every `cargo` target against `cargo test --list`
and every `vitest (same case…)` target against the live report of the
surviving suites: 150 ported, 29 replaced, 89 kept, 33 deleted, 0 pending.
Test counts after R4.8: apps/app 57 files, 427 cases (460 before: 42 of the
deleted suites, 1 roundtrip case added, 8 census); packages/core-wasm 58
(60); packages/pdf 146 → 0; cargo unchanged (R4.7: the compare suite
included). `pnpm typecheck`, `pnpm -r lint`, `pnpm check` and
`pnpm rust:check` green; `pnpm check:all` (the app image, the worker test
stage and image) recorded below.
Non-regression (owner's R4.8 brief §3), each with its proof:
| Guarantee | Proof |
|---|---|
| the application imports through the core's reader | `r4.census` (HomeRoute → `core/worker`, `handler.ts` → `import_pdf_bytes`, no scanner identifier under `src`); the route and worker suites (`routes.HomeRoute.{import,archive,rescueCopy}`, `core.importWorker`) |
| the export runs on the core's session | `r4.census` (`export.ts` → `openExportSession`); `pdf.goldens`: app-path bytes = the R4 digests at the fixed time |
| `report:compare` is the binary | `r4.census` (root script → `scripts/report-compare.sh` → `--features cli --bin report-compare`); cargo `pdf_compare` spawns it |
| v2 and v3 archives stay compatible | `pdf.roundtrip` (the frozen v2 PDFs migrated, unresolved items flagged), `migration.leaves` (612 leaves exported and read back through the module, 29 unresolved kept), cargo `pdf_import` O7/O8 |
| historical PDFs stay readable | cargo `pdf_import` and `pdf_serialize::restoration_round_trip_loses_nothing` on the Step 6 and R3 fixtures; `pdf.goldens` on `report-v3.bare.pdf`; the N-1 suites on the R4 files |
| the validated renders stay conformant | `pdf.goldens` (Step 6 text, images, links read from the file; the R4 digests), cargo `pdf_layout` (O1), `pdf_serialize` (O2, poppler 0 pixel recorded at R4.5) |
| rescue copies and data-loss protections | `routes.HomeRoute.rescueCopy` (13), `routes.coreStopped` (15), `state.coreFailures` (5), the quarantine suites, unchanged |
| no TypeScript engine active in parallel | `r4.census`; the only TypeScript readers left are the frozen N-1 copies, read by tests only |
Residual risks and open points (R4.8):
- The test reader `pdfFile.ts` reads the writer's own format only (one
dictionary per line, direct `/Length`, classic xref): a change of the
writer's layout changes it too; it is tooling, pinned by the suites that
use it.
- `tests/helpers/pdfImport.ts` runs the module's reader on the test
process's main instance (a throwaway handle, freed): the application never
does (OD-1); the worker path keeps its own suites.
- A stale `packages/pdf/node_modules` folder (git-ignored symlinks of the
previous install) may linger on a development machine after the pull:
`rm -rf packages/pdf` by hand; the census checks the package files, not
the folder.
- The open points of §10.8 stand: the Chromium per-export growth (BDR-063,
an open reserve), J1 and J2 on real devices before any merge, the WebKit
reserves BDR-058/059, the hostile-file protections (`PDF_IMPORT_HARDENING`
§5 gate open), the module at 519,795 B gzip of a 600 KB budget. R4.8
measured nothing new on memory or time and claims nothing.
R5 prerequisites (the pipeline port): `packages/anatomy` (curation pipeline,
importer, `render/`, the review drawings) and `packages/domain` are the last
TypeScript business packages; their oracles (the seed golden, the 612-leaf
semantics, the determinism and count suites, the R3 layer oracle) are in
place; `documentSha256` will change once (D-R5) and every older archive will
rebase on import; `@xmldom/xmldom`, `tsx` and `prettier` on generated files
leave with them.
+5 -3
View File
@@ -15,7 +15,7 @@ Status: Step 0 deliverable, 2026-10-08. Current state: 32 files under
| RLS | pgTAP, per table, per role | `supabase/tests/rls` | CI, release gate |
| End-to-end | Playwright against `pnpm preview` + local Supabase | `apps/app/e2e` | CI nightly + release gate |
| Migration | `supabase db reset` from zero + seed + regression fixtures | CI | every PR touching `supabase/` |
| PDF regression | vitest golden tests: text extraction of generated PDFs, attachment roundtrip, raster hash on fixed inputs | `packages/pdf/tests` | CI |
| PDF regression | cargo: the R4 oracles (O1 draw logs, O2 PDFs, O4 to O10) and the R4 digests of the writer's own bytes; vitest: the Step 6 goldens read back from the app export's file (`tests/helpers/pdfFile.ts`, no pdf-lib), the attachment roundtrip through the module's reader, the N-1 readers | `crates/core/tests/pdf_*.rs`, `apps/app/tests/unit/pdf.*.test.ts`, `packages/core-wasm/tests/pdf-*.test.ts` | CI |
| SQL/TS contract | generated types diff check | CI | every PR touching `supabase/` |
| Rust core and facade | `cargo test` (unit + fixture tests reading the repository JSON in place), `clippy -D warnings`, `cargo deny`; ts-rs types drift check | `crates/*`, `pnpm rust:check` (native, `scripts/rust-check.sh`; same file in the builder image), `pnpm check:all` | before every Rust commit and every merge; CI job `rust` prepared, no runner yet |
| Rust/TS contract | vitest on the built module: byte equality with `JSON.stringify(…, null, 2)` on the real documents, ts-rs types against the Zod output types, the runtime (lazy load, trap → poison → offline re-instantiation, handle rebuild, cap, retry), panic and stack traps on `pkg-proofs`, served size budget and measurements | `packages/core-wasm/tests` (refused, never skipped, when `pkg/` or `pkg-proofs/` is missing or stale) | `pnpm check:all` (after the native `wasm:build`); CI `check` prepared |
@@ -78,8 +78,10 @@ Target after Step 1: roughly 230–250 cases, all current ones untouched.
- `supabase/seed/test/` provides two practitioners, one cabinet, two clients,
three animals, the official sheet, one personal sheet, reports in each
status. RLS tests reuse it.
- Golden PDFs are compared by extracted text and attachment JSON, not by
bytes (pdf-lib dates and ids vary); raster hash only on fixed-size inputs.
- Golden PDFs are compared by the text, images and links read back from the
file and by the attachment JSON; the writer's own bytes are pinned by the
R4 digests at a fixed export time (`goldens/r4/pdf.*.sha256`), rewritten
only for a declared structural change.
## 5. Gates before a merge, and the CI target
+1 -1
View File
@@ -35,7 +35,7 @@ Future units, created only by the step that needs them:
|---|---|---|
| `apps/public-web` | SSR public site (landing, practitioner profiles, search) | plan Step 22 |
| `crates/worker` | Rust job runner (avatars at R1, then emails, expiry, calendar sync, imports) — created at R0, replaces the planned `apps/worker` | R0 (2026-10-12) |
| `packages/*` | shared domain, anatomy, pdf, i18n, supabase, config, ui | see `packages/README.md` |
| `packages/*` | shared domain, anatomy, core-wasm, supabase (i18n, config, ui planned; `pdf` deleted at R4.8) | see `packages/README.md` |
Rust workspace (plan R0, `docs/development/RUST.md`): `Cargo.toml` at the
root with `crates/core` (library), `crates/wasm` (browser facade, built into