docs(docs): RUST.md §10.10 R4.8 record (deleted, replaced, kept with reasons, census, non-regression table, risks, R5 prerequisites); CLAUDE.md, README, WORKSPACE, packages/README, TEST_STRATEGY, PDF-FORMAT, DEPLOY rollback R4 note and module size, hardening §5 pdf-lib closed, fixtures README, CHANGELOG R4 entry; TODO R4.8 closed; memory BDR-064, journal

This commit is contained in:
bchanot committed 2026-10-19 12:17:02 +02:00
1 parent e88769a707
commit f36dbdd4ae
13 files changed
+238 -62

No files matched your search

+5 -3
View File
@@ -110,9 +110,11 @@ hides its `/Keywords`, so a stripped attachment on such a file reads as
brackets deeper than 64 levels (linear byte pre-scan, strings skipped).
- Option C or a server-side import (Step 14) with a hard memory and time
budget per file, for any path that still parses PDF objects.
- Dependency decision on pdf-lib (unmaintained since 2021): keep it for
generation only, or move to the maintained fork; recorded as a Step 14
question.
- Dependency decision on pdf-lib (unmaintained since 2021): closed at R4.8
(2026-10-19): pdf-lib left the repository with the TypeScript engine; the
core's writer (`RUST.md` §10.7, §10.8) generates the PDFs and its reader
reads them; no PDF library remains in the application, its tests or the
tools (`apps/app/tests/unit/r4.census.test.ts` fails on a return).
- Report collection caps on read (observations, zones, findings per
observation) so a hostile report cannot freeze the editor before the
export's own caps refuse it.
+131
View File
@@ -1960,3 +1960,134 @@ Residual risks and open points (R4.7):
- A PDF over 50 MiB is refused by the entry check (`file too large`) before
the reader, as before; the reader's own cap would say `not a readable
PDF`.
### 10.10 R4.8: the TypeScript PDF engine deleted (2026-10-19)
Audit before every deletion (owner's R4.8 brief §2): every importer of
`@faunosteo/pdf`, `pdf-lib`, `@pdf-lib/*`, `@resvg/resvg-js`, `@xmldom/xmldom`
and `@faunosteo/domain` was listed across the application, the glue package,
the tests, the scripts, the pipeline, the workspace files, the Dockerfile,
the CI files and the docs (`grep` census, now the `r4.census` suite). The
application's last consumer of the engine (the TypeScript import path,
tests only since R4.3) and the glue's (the session suite's compose check,
the engine comparison) were moved onto the core or deleted first; the
pipeline's consumers (`packages/anatomy/src/curation/app-review.ts` →
`render/compose.ts`, 22 files → `@faunosteo/domain`) stay until R5.
Deleted (70 files, 12,010 lines of TypeScript):
- `packages/pdf/**` (3,122 lines of engine, 8,211 of tests): the export,
the layered reader, the byte-scan attachment reader, the SVG sanitiser and
the view composition, the page renderers, the branding gate, the labels,
the 140 engine cases and the R3 and R4 oracle generators
(`r3.{highlight,sanitiser}-oracles`, `r4.{dates,inflate,png,reader,text}-oracles`)
with their helpers (`resvg`, the reader-input crafters).
- `apps/app/scripts/compare-reports.ts` and `compare/{report,sheet}.ts`
(589 lines, the TypeScript CLI replaced at R4.7), `tests/unit/scripts.compare.test.ts`.
- `apps/app/src/pdf/import.ts` (the TypeScript scanner behind
`importReportFromPdf`, tests only since R4.3) and `tests/unit/pdf.import.test.ts`.
- `tests/unit/pdf.export.app.test.ts` (T2 and scenario 3 through the
engine's `exportFromEnvelope`: the re-export from the envelope alone is
cargo's `export_from_envelope`, whose signature takes no sheet and no
branding, byte-equal to the R4 files on 13 runs).
- The application's generators `r4.{compare,tolerance,writer}-oracles` and
`tests/helpers/r4-draw.ts` (the pdf-lib draw recorder); the glue's
`pdf-export.ts-engine.test.ts` (the Node timings against the engine,
recorded in §10.7).
- Dependencies: `@faunosteo/pdf` (app, core-wasm), `pdf-lib`, `tsx` and
the app's `report:compare` script (app), `@faunosteo/domain` (app: its
two incidental test uses moved to `@faunosteo/core-wasm/types` and the
N-1 Zod copy), the Dockerfile's `COPY packages/pdf/package.json`.
`pnpm-lock.yaml` holds no `pdf-lib`, `@pdf-lib/*` or `@resvg/resvg-js`
any more; `@xmldom/xmldom` only under `packages/anatomy`.
Replaced in the tests (no case ignored, no golden regenerated):
| Before | After |
|---|---|
| pdf-lib parsed the app export's file (texts, images, page count, Info, Filespec) | `tests/helpers/pdfFile.ts`, a reader of the writer's own format (objects, dictionaries, Flate streams through `node:zlib`, pages in `/Kids` order, Info, annotations, XObjects); `pdfRead.ts` keeps its content-stream parser on it |
| pdf-lib read the links of the file | the same reader's annotations (`goldens.ts` `readLinks`) |
| pdf-lib built the foreign PDFs of the rejection cases | `buildPdfWithAttachment` (a minimal file with one embedded file in clear) and the frozen hostile fixtures |
| pdf-lib's `embedPng` checked the golden rasters | the XObject sizes read from the app export's file |
| the engine composed the views of the `views.*.json` goldens | the SVGs the core's session asks the rasteriser for, captured at the mocked rasteriser in request order |
| the engine's scanner read the attachment of the envelope and N-1 suites | the frozen O11 scanner, now exported by `@faunosteo/core-wasm/n-1` (`readAttachmentBytes`, `readAttachedDocument`): the R3 build's reader, as a rollback would read |
| `importReportFromPdf` (TypeScript scanner + core) | `tests/helpers/pdfImport.ts`: the module's `SheetHandle.import_pdf` on a handle of the bundled sheet, the envelope mapped by the worker's `pdfImportOf` (the same kinds HomeRoute shows); the main-thread session still never calls the reader (OD-1) |
| the session suite compared each request SVG with the engine's `composeHighlightedViewSvg` | structural facts per request (the view's frame, the highlight, no editor attribute) and the real-raster manifest digests; the text itself is cargo's (`pdf_session` against `compose_highlight`, `anatomy_compose` against the frozen R3 trees) |
Still TypeScript, with the reason (owner's R4.8 brief §6.4):
- `packages/anatomy` (the curation pipeline, the legacy importer, the frozen
renderer `render/` and `@xmldom/xmldom`): R5's scope, consumed by
`pnpm import:equine` and the seed golden; no app and no PDF consumer.
- `packages/domain`: the pipeline's schemas until R5; the application and the
glue no longer depend on it.
- `packages/core-wasm/tests/n-1/{domain,scanner}`: the frozen N-1 readers
(the R2 Zod schemas, the R3 byte-scan reader), never edited, permanent.
- `apps/app/src/pdf/{export,labels,branding,browserRasterizer}.ts`: the
application's side of the export (input assembly, the session driver, the
browser's rasteriser), UI glue by the review's boundary.
- `apps/app/src/core/worker/**`: the import worker (protocol, timers,
outcome mapping), browser glue.
- The test readers above (`pdfFile.ts`, `pdfRead.ts`, `pdfImport.ts`):
tooling on the writer's own files, never product logic.
- `scripts/r4-afm-data.mjs`: the generator of `afm_data.rs` from the frozen
O4/O5 oracles (no pdf-lib: reads `afm.json` and `text.json`), kept so the
tables stay reproducible.
Census and gates: `apps/app/tests/unit/r4.census.test.ts` (8 checks) fails
on a reintroduced import of the engine, pdf-lib, the TS domain or xmldom
under `apps/app/src` or `packages/core-wasm/src`, on a dependency added back,
on a lockfile holding pdf-lib, on an export that does not open the core's
session, on an import that does not go through the worker, on a root
`report:compare` that is not the binary, and on a missing N-1 reader or
oracle folder. The ledger (`scripts/r4-ledger.mjs`) now reads its 301 frozen
cases from the committed TSV (the deleted suites cannot run), re-derives the
dispositions and checks every `cargo` target against `cargo test --list`
and every `vitest (same case…)` target against the live report of the
surviving suites: 150 ported, 29 replaced, 89 kept, 33 deleted, 0 pending.
Test counts after R4.8: apps/app 57 files, 427 cases (460 before: 42 of the
deleted suites, 1 roundtrip case added, 8 census); packages/core-wasm 58
(60); packages/pdf 146 → 0; cargo unchanged (R4.7: the compare suite
included). `pnpm typecheck`, `pnpm -r lint`, `pnpm check` and
`pnpm rust:check` green; `pnpm check:all` (the app image, the worker test
stage and image) recorded below.
Non-regression (owner's R4.8 brief §3), each with its proof:
| Guarantee | Proof |
|---|---|
| the application imports through the core's reader | `r4.census` (HomeRoute → `core/worker`, `handler.ts` → `import_pdf_bytes`, no scanner identifier under `src`); the route and worker suites (`routes.HomeRoute.{import,archive,rescueCopy}`, `core.importWorker`) |
| the export runs on the core's session | `r4.census` (`export.ts` → `openExportSession`); `pdf.goldens`: app-path bytes = the R4 digests at the fixed time |
| `report:compare` is the binary | `r4.census` (root script → `scripts/report-compare.sh` → `--features cli --bin report-compare`); cargo `pdf_compare` spawns it |
| v2 and v3 archives stay compatible | `pdf.roundtrip` (the frozen v2 PDFs migrated, unresolved items flagged), `migration.leaves` (612 leaves exported and read back through the module, 29 unresolved kept), cargo `pdf_import` O7/O8 |
| historical PDFs stay readable | cargo `pdf_import` and `pdf_serialize::restoration_round_trip_loses_nothing` on the Step 6 and R3 fixtures; `pdf.goldens` on `report-v3.bare.pdf`; the N-1 suites on the R4 files |
| the validated renders stay conformant | `pdf.goldens` (Step 6 text, images, links read from the file; the R4 digests), cargo `pdf_layout` (O1), `pdf_serialize` (O2, poppler 0 pixel recorded at R4.5) |
| rescue copies and data-loss protections | `routes.HomeRoute.rescueCopy` (13), `routes.coreStopped` (15), `state.coreFailures` (5), the quarantine suites, unchanged |
| no TypeScript engine active in parallel | `r4.census`; the only TypeScript readers left are the frozen N-1 copies, read by tests only |
Residual risks and open points (R4.8):
- The test reader `pdfFile.ts` reads the writer's own format only (one
dictionary per line, direct `/Length`, classic xref): a change of the
writer's layout changes it too; it is tooling, pinned by the suites that
use it.
- `tests/helpers/pdfImport.ts` runs the module's reader on the test
process's main instance (a throwaway handle, freed): the application never
does (OD-1); the worker path keeps its own suites.
- A stale `packages/pdf/node_modules` folder (git-ignored symlinks of the
previous install) may linger on a development machine after the pull:
`rm -rf packages/pdf` by hand; the census checks the package files, not
the folder.
- The open points of §10.8 stand: the Chromium per-export growth (BDR-063,
an open reserve), J1 and J2 on real devices before any merge, the WebKit
reserves BDR-058/059, the hostile-file protections (`PDF_IMPORT_HARDENING`
§5 gate open), the module at 519,795 B gzip of a 600 KB budget. R4.8
measured nothing new on memory or time and claims nothing.
R5 prerequisites (the pipeline port): `packages/anatomy` (curation pipeline,
importer, `render/`, the review drawings) and `packages/domain` are the last
TypeScript business packages; their oracles (the seed golden, the 612-leaf
semantics, the determinism and count suites, the R3 layer oracle) are in
place; `documentSha256` will change once (D-R5) and every older archive will
rebase on import; `@xmldom/xmldom`, `tsx` and `prettier` on generated files
leave with them.
+5 -3
View File
@@ -15,7 +15,7 @@ Status: Step 0 deliverable, 2026-10-08. Current state: 32 files under
| RLS | pgTAP, per table, per role | `supabase/tests/rls` | CI, release gate |
| End-to-end | Playwright against `pnpm preview` + local Supabase | `apps/app/e2e` | CI nightly + release gate |
| Migration | `supabase db reset` from zero + seed + regression fixtures | CI | every PR touching `supabase/` |
| PDF regression | vitest golden tests: text extraction of generated PDFs, attachment roundtrip, raster hash on fixed inputs | `packages/pdf/tests` | CI |
| PDF regression | cargo: the R4 oracles (O1 draw logs, O2 PDFs, O4 to O10) and the R4 digests of the writer's own bytes; vitest: the Step 6 goldens read back from the app export's file (`tests/helpers/pdfFile.ts`, no pdf-lib), the attachment roundtrip through the module's reader, the N-1 readers | `crates/core/tests/pdf_*.rs`, `apps/app/tests/unit/pdf.*.test.ts`, `packages/core-wasm/tests/pdf-*.test.ts` | CI |
| SQL/TS contract | generated types diff check | CI | every PR touching `supabase/` |
| Rust core and facade | `cargo test` (unit + fixture tests reading the repository JSON in place), `clippy -D warnings`, `cargo deny`; ts-rs types drift check | `crates/*`, `pnpm rust:check` (native, `scripts/rust-check.sh`; same file in the builder image), `pnpm check:all` | before every Rust commit and every merge; CI job `rust` prepared, no runner yet |
| Rust/TS contract | vitest on the built module: byte equality with `JSON.stringify(…, null, 2)` on the real documents, ts-rs types against the Zod output types, the runtime (lazy load, trap → poison → offline re-instantiation, handle rebuild, cap, retry), panic and stack traps on `pkg-proofs`, served size budget and measurements | `packages/core-wasm/tests` (refused, never skipped, when `pkg/` or `pkg-proofs/` is missing or stale) | `pnpm check:all` (after the native `wasm:build`); CI `check` prepared |
@@ -78,8 +78,10 @@ Target after Step 1: roughly 230–250 cases, all current ones untouched.
- `supabase/seed/test/` provides two practitioners, one cabinet, two clients,
three animals, the official sheet, one personal sheet, reports in each
status. RLS tests reuse it.
- Golden PDFs are compared by extracted text and attachment JSON, not by
bytes (pdf-lib dates and ids vary); raster hash only on fixed-size inputs.
- Golden PDFs are compared by the text, images and links read back from the
file and by the attachment JSON; the writer's own bytes are pinned by the
R4 digests at a fixed export time (`goldens/r4/pdf.*.sha256`), rewritten
only for a declared structural change.
## 5. Gates before a merge, and the CI target
+1 -1
View File
@@ -35,7 +35,7 @@ Future units, created only by the step that needs them:
|---|---|---|
| `apps/public-web` | SSR public site (landing, practitioner profiles, search) | plan Step 22 |
| `crates/worker` | Rust job runner (avatars at R1, then emails, expiry, calendar sync, imports) — created at R0, replaces the planned `apps/worker` | R0 (2026-10-12) |
| `packages/*` | shared domain, anatomy, pdf, i18n, supabase, config, ui | see `packages/README.md` |
| `packages/*` | shared domain, anatomy, core-wasm, supabase (i18n, config, ui planned; `pdf` deleted at R4.8) | see `packages/README.md` |
Rust workspace (plan R0, `docs/development/RUST.md`): `Cargo.toml` at the
root with `crates/core` (library), `crates/wasm` (browser facade, built into