From f36dbdd4aeb7ea1c1079e70d0c41a7c04b82619a Mon Sep 17 00:00:00 2001 From: bchanot Date: Mon, 19 Oct 2026 12:17:02 +0200 Subject: [PATCH] =?UTF-8?q?docs(docs):=20RUST.md=20=C2=A710.10=20R4.8=20re?= =?UTF-8?q?cord=20(deleted,=20replaced,=20kept=20with=20reasons,=20census,?= =?UTF-8?q?=20non-regression=20table,=20risks,=20R5=20prerequisites);=20CL?= =?UTF-8?q?AUDE.md,=20README,=20WORKSPACE,=20packages/README,=20TEST=5FSTR?= =?UTF-8?q?ATEGY,=20PDF-FORMAT,=20DEPLOY=20rollback=20R4=20note=20and=20mo?= =?UTF-8?q?dule=20size,=20hardening=20=C2=A75=20pdf-lib=20closed,=20fixtur?= =?UTF-8?q?es=20README,=20CHANGELOG=20R4=20entry;=20TODO=20R4.8=20closed;?= =?UTF-8?q?=20memory=20BDR-064,=20journal?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/memory/decisions.md | 6 ++ .claude/memory/journal.md | 1 + .claude/tasks/TODO.md | 7 +- CHANGELOG.md | 6 ++ CLAUDE.md | 45 ++++---- DEPLOY.md | 16 ++- README.md | 7 +- docs/PDF-FORMAT.md | 47 ++++---- docs/development/PDF_IMPORT_HARDENING.md | 8 +- docs/development/RUST.md | 131 +++++++++++++++++++++++ docs/development/TEST_STRATEGY.md | 8 +- docs/development/WORKSPACE.md | 2 +- packages/README.md | 16 +-- 13 files changed, 238 insertions(+), 62 deletions(-) diff --git a/.claude/memory/decisions.md b/.claude/memory/decisions.md index b97d1f00..e9338697 100644 --- a/.claude/memory/decisions.md +++ b/.claude/memory/decisions.md @@ -481,3 +481,9 @@ Status: active; owner checkpoint R4.6 pending. Context: owner decision 2026-10-19 on the R4.6 report (RUST.md §10.8). Chromium per-export RSS growth on L (≈ 20 MiB, same in TS control, plateau after 8) accepted provisionally: not a leak validation, kept as open reserve; reports must split wasm linear memory / browser global / canvas-blob-URL retention / transient vs persistent; thresholds never changed, no FAIL turned PASS. J1 + J2 on real devices mandatory before any merge; NON TESTÉ kept. R4.7 per plan, no scope widening; S2 export worker not engaged without proof + new decision. Decision (R4.7 design): `pdf::compare` pure module behind feature `cli` (= pdf-read + pdf-write, never in wasm), `Files` trait for fs; binary `src/bin/report_compare.rs` only std::fs file; seed sheet read at run time from `CARGO_MANIFEST_DIR/../../supabase/seed/.../draft` (no drawing); root `report:compare` → `scripts/report-compare.sh` (rust-env + cargo run --release --features cli). A.3 closed lines chosen: `report document invalid: ` (ZodError dumps), `attachment refused: lone UTF-16 surrogate escape at byte N` (surrogates kept/overwritten), `report version null cannot be read` (overflow → null, incl. envelopeVersion overflow where TS printed a rejected-envelope header). Alternatives dropped: `include_str!` of the seed (bakes content, rebuild on seed change); spawning the binary per O10 row in tests (230 spawns, slow): in-process `run` + 8 spawns instead. Status: active; owner checkpoint R4.7 pending. + +## BDR-064 | 2026-10-19 | R4.7 validated; R4.8 deletions: test read-back on own PDF reader, N-1 scanner exported, in-process import helper + +Context: owner validated R4.7 (three A.3 lines accepted, 25 s first compile accepted, no prebuilt binary). R4.8 brief: delete packages/pdf, pdf-lib, TS CLI, redundant tests, generators; keep frozen oracles, goldens, reference PDFs, fixtures; `packages/anatomy` render until R5; audit consumers first; non-regression proofs; checks must fail on missing Rust component; no merge, no R5. +Decision: vitest read-back of the app export without pdf-lib = `tests/helpers/pdfFile.ts`, a reader of the writer's own format (objects, inline dicts, Flate via node:zlib, Kids order, Info, annots, XObjects) + `buildPdfWithAttachment` for the rejection inputs; view goldens from the SVGs captured at the mocked rasteriser (request order = engine order; the bundled partner SVG logo comes first with real branding); `importReportFromPdf` for tests = module `SheetHandle.import_pdf` on a throwaway handle + worker's `pdfImportOf` (`tests/helpers/pdfImport.ts`; `GlueSheetHandle.free()` added to the interface); envelope + N-1 suites read attachments with the frozen O11 scanner, exported through `@faunosteo/core-wasm/n-1` (index re-export only, copies untouched); `pdf.export.app` deleted (re-export from envelope = cargo `export_from_envelope`, no sheet/branding in signature); `@faunosteo/domain` dropped from apps/app (two incidental test uses → core types / N-1 Zod); ledger script reads frozen cases from the committed TSV. Alternatives dropped: keep pdf-lib as test-only devDependency (review R4 row forbids it in apps/app lockfile); move all content-stream checks to cargo (loses app-path coverage of non-fixture reports); `importPdf` on main-thread SheetOps (breaks OD-1). +Status: active; owner checkpoint R4.8 pending. diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index 57853ad8..2e884c4a 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -294,3 +294,4 @@ Owner validated R4.0 plan with conditions (BDR-057, plan §A.6, LRN-059/060). R3 - 2026-10-18 R4.5 done (d420f67..b44dd80): content emitter (60 streams byte-equal, numberToString 703 + V8 pow10 quirk), PNG gate + UPNG decoder (O6 108/108), own object writer (pdf-writer dropped: numberToString everywhere, no dep), serialiser, export + self-check, from_envelope; O2 14 runs equal (streams after B3, planes, links, Info, attachment), qpdf clean, poppler 0 px, O11 + N-1 + TS reader read R4 PDFs, wasm = native digests; Node TS→Rust demo 198→36 ms, L 772→57, CAP 5.7 s→0.22 s; writer linked 531.6 KB gzip (proofs build), shipped 426 KB. Gate PASS 4 LOW → hardenings. Open: one-shot path holds PNGs twice (R4.6 session must stream), L real rasters 5 s native (deflate 6), E3 judged at R4.6. STOP before R4.6. - 2026-10-19 owner validated R4.5; R4.6 go (BDR-062). R4.6 done (9e688b6..): ExportSession (decode on arrival, chunked never-realloc file 1→8 MiB, catalog last, parts → Blob, self-check on attachment chunk), views.rs (compose_highlight requests, raster_size_of O4 57/57), wasm ExportSession + CoreExportSession, app export.ts on session (abort on unmount, User Timing), labels.ts; app suites read content streams (pdfRead.ts), app-path bytes = R4 digests (re-frozen: object order B2, hex upper-case B4), Step 6 goldens via app path, 14 O2 runs through module = digests, request SVG = TS compose text, export Rust → import Rust / TS reader / N-1 ok. E3 Chromium demo 0.67, L 0.74 (Firefox 0.66/0.67, WebKit 0.91/0.95); E4 CAP completes 3 engines (TS stopped by ceiling); peaks lower (L Chromium 805→470 MiB). Linear memory L real 62 MiB flat over 7 exports; Chromium per-export RSS growth ≈ 20 MiB on L same as TS control, plateau after 8. Shipped 519.7 KB gzip (W1). rust-check 0, pnpm check 1,099 tests, ledger 11 pending (R4.7). J2 kit + procedure. STOP before R4.7. - 2026-10-19 owner accepted R4.6 provisionally (Chromium series = open reserve); R4.7 go (BDR-063). R4.7 done: `pdf::compare` (feature `cli`), `report-compare` bin, `scripts/report-compare.sh` behind root `report:compare`, cargo `pdf_compare` 15 tests: O10 230 rows (158 exact, 72 declared A.3 → 62 `report document invalid`, 7 surrogate refusals, 3 `version null`), 11 scenario cases = ledger targets (301 rows, 0 pending), binary spawned 8×. Release binary 7 ms/PDF, 6 MiB RSS; module unchanged 1,535,289 B / 519,795 gzip. RUST.md §10.9, CLAUDE.md, PDF-FORMAT. STOP before R4.8. +- 2026-10-19 owner validated R4.7; R4.8 go (BDR-064). R4.8 done: packages/pdf (70 files, 12,010 lines TS), TS CLI, TS import path, generators, ts-engine comparison deleted; deps pdf-lib / @pdf-lib / resvg / @faunosteo/pdf / tsx / @faunosteo/domain (app) gone, xmldom only under anatomy; test read-back on own `pdfFile.ts` reader, view goldens from captured session SVGs, N-1 scanner exported via core-wasm/n-1, in-process import helper on module reader; `r4.census` suite (8); ledger 301 rows from frozen TSV, 0 pending; app 427 tests, core-wasm 58; docs RUST.md §10.10, CLAUDE.md, README, PDF-FORMAT, DEPLOY rollback R4, CHANGELOG R4 entry, hardening §5 pdf-lib closed. STOP before R4.9. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index c2a411bb..22f352ec 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -255,7 +255,12 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] R4.7b `src/bin/report_compare.rs` (feature `cli`, the only std::fs file): seed sheet from `supabase/seed/anatomical-sheets/equine/draft` (ref, document, map; no asset), arguments relative to `INIT_CWD`, folders expanded, size pre-check, exit code; `scripts/report-compare.sh` (rust-env + cargo run --release --features cli) behind the root `report:compare`; rust-check feature lists widened to `cli` - [x] R4.7c cargo `tests/pdf_compare.rs`: the 230 O10 invocations in-process (158 undeclared rows exact: stdout, stderr, exit; 72 declared rows against the Rust wording per A.3 class), scenario PDF digests = `inputs`, the 11 scenario cases named for the ledger, one spawn of the real binary (usage, INIT_CWD, output = in-process); ledger rows 135–145 targets filled, `scripts/r4-ledger.mjs` regenerated - [x] R4.7d (done 2026-10-19; STOP before R4.8) docs: RUST.md §10.9 (binary, parity 158/72, A.3 lines as printed, timing), CLAUDE.md (command, map), fixtures README if touched, memory journal; rust-check + pnpm check green; report, STOP before R4.8 - - [ ] R4.8 deletions + docs + ledger census + - [ ] R4.8 deletions + docs + ledger census (owner go 2026-10-19, R4.7 validated; stop before R4.9) + - [x] R4.8a test read-back without pdf-lib and without the TS engine: `tests/helpers/pdfFile.ts` (reader of the writer's own files: objects, dicts, Flate streams via zlib, pages in order, Info, annots, XObject sizes), `pdfRead.ts` on it, `goldens.ts` (`readLinks` on it; `snapshotView` from the session's request SVGs captured at the rasteriser; `recordDrawing` dropped), `tests/helpers/pdfImport.ts` (`importReportFromPdf` = the module's `SheetHandle.import_pdf` + the worker's `pdfImportOf`; `importAttachment` = `sheet.ops.importArchive`), `@faunosteo/core-wasm/n-1` exports the frozen O11 scanner (`readAttachmentBytes`); suites rewritten: pdf.goldens (2×1 PNG decode read from the XObject), pdf.roundtrip (foreign PDFs = the r4/read fixtures), archive.envelopes + n-1.reader (N-1 scanner), migration.leaves, goldenReports, helpers/sheet (labels from `@/pdf/labels`), core-wasm pdf-export.session (request SVG vs the frozen R3 compose oracle) + - [x] R4.8b deletions after the consumer audit (grep census per package): `packages/pdf/**` (engine, tests, R3/R4 oracle generators, helpers), `apps/app/src/pdf/import.ts`, `apps/app/scripts/compare-reports.ts` + `compare/*`, `tests/unit/{scripts.compare,pdf.import,pdf.export.app,r4.compare-oracles,r4.tolerance-oracles,r4.writer-oracles}.test.ts`, `tests/helpers/r4-draw.ts` (what survives moves), `packages/core-wasm/tests/pdf-export.ts-engine.test.ts`; deps: `@faunosteo/pdf` (app, core-wasm), `pdf-lib` + `tsx` + app `report:compare` script (app), Dockerfile COPY line; `pnpm install` → lockfile without pdf-lib, @pdf-lib/*, @resvg/resvg-js; `@xmldom/xmldom` only under packages/anatomy (R5); `packages/anatomy` render untouched + - [x] R4.8c ledger: `scripts/r4-ledger.mjs` reads the frozen 301 cases from the committed TSV (no vitest run of deleted suites), re-derives dispositions (deleted suites → `deleted` with their cargo / vitest target, N-1 switch noted, read-back notes), census: every cargo target in `cargo test --list`, every vitest target in the live report, counts = 301; TSV regenerated + - [x] R4.8d non-regression proofs (owner §3): grep census test (no `@faunosteo/pdf`, `pdf-lib`, `@faunosteo/domain` import under apps/app/src, packages/core-wasm/src; the import worker path, the export session path and the root `report:compare` script named by a test), wasm:build, rust-check, pnpm check, pnpm check:all (app image, worker test + image), N-1 reader suites, O2/O3/O7/O8 fixtures still read, 612 leaves, rescue copies and quarantine suites + - [x] R4.8e (done 2026-10-19; STOP before R4.9) docs: RUST.md §10.10 (deleted files and packages, dependencies removed, Rust-only features, TS still legitimate with reasons, tests, risks, R5 prerequisites), CLAUDE.md (stack table, map, PDF conventions, workflow lines), DEPLOY.md (module size, rollback), PDF-FORMAT, PDF_IMPORT_HARDENING §5, TEST_STRATEGY, WORKSPACE, packages/README, fixtures README, CHANGELOG (Unreleased), README; memory (BDR, journal); report, STOP before R4.9 - [ ] R4.9 verifier + security + owner final acceptance → merge on owner signal - [ ] outside R4 (owner 2026-10-17, non-blocking): worker container ignores SIGTERM (Docker kill after 10 s; chore, e.g. `init: true`); README test counter stale (367 vs 1,015) - [ ] R5 curation pipeline + importer + upload tool: seed golden regenerated once (new `documentSha256`, D-R5), `packages/anatomy` + `packages/domain` deleted, docs map; all before Step 13 diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f3cec8d..a1239549 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,12 @@ Versions follow [Semantic Versioning](https://semver.org/). - Improve Safari/iOS PDF download reliability by attaching the download link to the document during activation and delaying object URL revocation by 30 seconds. The file name and content remain unchanged. +### Changed (Rust R4, PDF core, 2026-10-18 to 2026-10-19; on `feature/rust-r4-pdf-core`, not merged) + +- **The PDF archives are read, written and checked by the Rust core** (`crates/core/src/pdf`, `docs/development/RUST.md` §10). Import: the attachment is located by the core's byte scan, inflated under the 16 MiB cap, probed as JSON and read by the R2 layered reader, inside a module worker created for the file and terminated after it (R4.3; the rescue copy follows the same path). Export: an incremental session in the page's engine composes each view, takes the browser's rasters one by one, decodes and writes each image on arrival, and hands the file out in parts (R4.6; no raster and no second copy of the file kept). `pnpm report:compare` runs the core's `report-compare` binary (R4.7). The PDFs are the ones the TypeScript engine wrote: same streams, images, links, metadata and attachment, pixel-identical through poppler on the reference reports; three declared differences only (`Producer`/`Creator` now `Faunosteo`, the attachment's report in contract key order, hostile files refused where the TypeScript repaired them). +- **Removed at R4.8** (2026-10-19): `packages/pdf` (the TypeScript engine and its tests), `pdf-lib`, `@pdf-lib/*` and `@resvg/resvg-js`, the TypeScript CLI `apps/app/scripts/compare-reports.ts`, the application's TypeScript import path and its dependency on `@faunosteo/domain`, the oracle generators. Kept: the frozen oracles, goldens and fixture PDFs, the frozen N-1 readers (`packages/core-wasm/tests/n-1`: the Zod schemas and the byte-scan reader of the R3 build), `packages/anatomy` and `packages/domain` for the curation pipeline until R5. +- Old browsers (Firefox 102 to 112, Safari 16.0 to 16.3) gain the import and the logos; module workers need Firefox 114 (OD-1). Memory: the import worker holds one copy of the file; the export's linear memory stays reserved in the page until reload (OD-2), about 62 MiB for a 44 MiB file; the per-export growth of the browser's own process in Chromium is an open reserve (BDR-063). Module 1,535,289 B raw, 519,795 B gzip (budget 600 KB). + ### Changed (Rust R3, anatomy core, 2026-10-15; validated by the owner's LAN acceptance and merged 2026-10-17) - **The editor canvas draws what the Rust core composes** (`crates/core/src/anatomy`): one SVG reader for every drawing the product shows (bounds checked by a markup scanner before any XML parser, the engine's allowlist matched the same way, a reference graph refusing cycles and expansions over 50,000 elements, a serialiser that re-reads to the same tree), the annotation layer and the view composition, reached through the new `view.canvas` operation and memoised per view. The canvas keeps its in-place selection styling, click and tooltip delegation. The browser-side denylist and the app's dependency on the TypeScript renderer are gone; the PDF engine and the curation pipeline keep the TypeScript renderer until R4 and R5. diff --git a/CLAUDE.md b/CLAUDE.md index cb6bd93b..4eddf01e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -20,8 +20,9 @@ tables `species`, `anatomical_sheets` / `_versions` / `_assets`). Step 5 moved t application onto that model: the editor renders the bundled reference sheet (the curated equine DRAFT, `SHEET_BUNDLE=draft`), writes Report v3 and imports v2 PDFs through a generated migration map. Step 6 made the PDF an autonomous archive: the -`packages/pdf` engine renders from explicit inputs (report, sheet snapshot, branding, -labels, rasterizer) with no network and no DOM, and the attachment is an envelope v1 +PDF engine (TypeScript `packages/pdf` until R4, the Rust core `crates/core/src/pdf` +since) renders from explicit inputs (report, sheet snapshot, branding, labels, +rasterizer) with no network and no DOM, and the attachment is an envelope v1 carrying the report, a lean sheet snapshot, the branding and the labels used, so an old bilan is reproduced identically after any catalog, wording or logo change (`docs/PDF-FORMAT.md`). Step 7 added identity: email/password sign-up with @@ -57,7 +58,7 @@ MVP species: equine only. Open-source MIT. | Styling | Tailwind CSS v4 CSS-first (no postcss config) | 4.x | | State | Zustand | 5 | | Validation | Zod | 3 | -| PDF | pdf-lib | 1.17 | +| PDF | the Rust core's reader and writer (`crates/core/src/pdf`, R4; no PDF library) | — | | SVG raster | native Image/canvas (no dependency) | — | | Animation | motion | 11 | | Tests | vitest + happy-dom + @testing-library/react | 3.x | @@ -191,7 +192,6 @@ faunosteo/ ├─ apps/app/ ← @faunosteo/app (everything below until packages/) ├─ public/ ← favicons, env.js, site header logo only (PDF logos live in src/assets/brand; no anatomy asset) ├─ vite/sheet-bundle.ts ← Vite plugin: `virtual:faunosteo-sheet` (SHEET_BUNDLE=draft|none) -├─ scripts/compare-reports.ts ← the frozen TS archive check CLI (pure part in scripts/compare/report.ts): O10 reference until R4.8, no longer behind `pnpm report:compare` ├─ src/ │ ├─ assets/brand/ ← the four PDF branding logos, bundled by Vite (?inline / ?raw / ?url), never fetched │ ├─ app/ @@ -234,8 +234,8 @@ faunosteo/ │ │ ├─ shell/{AppLayout,SidebarNav,MobileDrawer,UserBadge,SignOutButton}.tsx │ │ └─ account/{SectionCard,SaveRow,AvatarSection,PersonalInfoForm,ProfessionalInfoForm,PreferencesForm,SecurityPanel,RoleBadges}.tsx │ ├─ account/avatarTransport.ts ← magic-byte sniff (never SVG), createImageBitmap → canvas ≤ 1024 px → JPEG transport blob -│ ├─ pdf/ ← the app side of the PDF (export on the core's session since R4.6; the TS reader of packages/pdf until R4.8) -│ │ ├─ export.ts + labels.ts + import.ts ← exportReportToPdf (snapshot + branding + labels → ExportSession, rasters one by one, Blob of the parts, abort on unmount), defaultPdfLabels (25 keys), importReportFromPdf (tests; the app imports through the worker) +│ ├─ pdf/ ← the app side of the PDF (export on the core's session since R4.6; the import runs in the worker, `core/worker`) +│ │ ├─ export.ts + labels.ts ← exportReportToPdf (snapshot + branding + labels → ExportSession, rasters one by one, Blob of the parts, abort on unmount), defaultPdfLabels (25 keys); the tests' in-process import lives in tests/helpers/pdfImport.ts (the module's reader) │ │ └─ branding.ts + browserRasterizer.ts ← bundled src/assets/brand logos (?inline / ?raw) → Branding; Image/canvas Rasterizer adapter │ ├─ lib/{dates,id,assert,supabase}.ts ← supabase.ts: getSupabaseClient() | null │ ├─ styles/{index,tokens}.css ← Tailwind v4 CSS-first @@ -259,8 +259,7 @@ faunosteo/ ├─ packages/domain/ ← @faunosteo/domain (pure, no DOM, zod only): anatomical-sheet (document, integrity rules, sheet-index), report (v3, v2 migration, rebase), archive (sheet snapshot, envelope v1, branding, labels, layered reader) │ ├─ src/anatomical-sheet/{schema,validate,create}.ts ← canonical sheet document + integrity rules │ └─ tests/{fixtures/horse-slice.ts,*.test.ts} -├─ packages/anatomy/ ← @faunosteo/anatomy: legacy importer, curation pipeline, frozen TS annotation-layer renderer + view composer (`./render`: PDF engine until R4, pipeline until R5; the app no longer depends on it) -├─ packages/pdf/ ← @faunosteo/pdf: the frozen TypeScript PDF engine (R3 reference until R4.8; no app consumer since R4.6 but the kept reader) — src/{export,read,text,branding}.ts, attachment/, svg/{parse,sanitize,view}.ts, render/, i18n/; tests reproduce the Step 6 goldens +├─ packages/anatomy/ ← @faunosteo/anatomy: legacy importer, curation pipeline, frozen TS annotation-layer renderer + view composer (`./render`: the pipeline until R5; no app and no PDF consumer since R4.8) │ ├─ src/legacy/{svg,catalog,regions,ids,report,importer}.ts, src/cli/import-equine.ts │ ├─ src/curation/{schema,targets,fields,ops,rename,apply,types,assets,labels,stats,changelog,review-report,overlay,seed,sources,pipeline}.ts ← changeset → draft │ └─ tests/ ← counts preserved, special cases, determinism, curation ops, golden artifacts @@ -334,13 +333,18 @@ PDF export (app side: apps/app/src/pdf; writer: crates/core/src/pdf/write, RUST. → a trap poisons the runtime as any core call; a session of a dropped instance answers core_unavailable; leaving the editor aborts and frees → export_from_envelope (cargo only) re-exports an archive from its own snapshot, branding and labels (no sheet) -PDF import - → readAttachment(bytes) → engine byte scan for the EmbeddedFile stream (TS until R4; no pdf-lib parse on untrusted - bytes, BLK-009): named osteo-report* first, capped native inflation, JSON depth pre-scan - → loadSheetIndex (core loads here) → importAttachment(data, index) → core archive.import: layered reading (report: - v3 as is, v2 migrated, else incompatible_version; envelope metadata tolerant, snapshot/branding/labels strict → - archive kept or snapshotRejected) + rebase on the loaded sheet (unresolved flags, labels kept), one call - → hydrateImport({ report, archive }) → one alert (archive_ok / snapshot_rejected / rebased / migrated) → navigate /editor +PDF import (R4.3, RUST.md §10.5; the core's reader `crates/core/src/pdf/read`, never a PDF library) + → HomeRoute: routing on the first KiB and the size pre-checks → startImport (core/worker/controller) → a module worker + created for the file (import.worker.ts, handler.ts): the compiled module posted by the page, a SheetHandle of the + bundled sheet, the file streamed in chunks into the module's memory (FileBytes), `import_pdf_bytes` + → in the core: byte scan for the EmbeddedFile stream (BLK-009: no object-graph parse): named osteo-report* first, capped + inflation (16 MiB budget shared by the candidates), JSON prescan (depth, value count OD-3), the JavaScript-path + decoding (`json_js`), then archive.import: layered reading (report: v3 as is, v2 migrated, else incompatible_version; + envelope metadata tolerant, snapshot/branding/labels strict → archive kept or snapshotRejected) + rebase on the + loaded sheet (unresolved flags, labels kept), one call + → the settlement mapped by core/worker/outcome.ts (closed kinds; timeout, crash and read error told apart: OD-5), + the worker terminated → hydrateImport({ report, archive }) → one alert (archive_ok / snapshot_rejected / rebased / + migrated) → navigate /editor; a rescue copy (JSON) follows the same worker path (`restore_copy_bytes`) Identity (Step 7, docs/development/IDENTITY.md) → AuthProvider: authStore.start() = client.auth.initialize() (PKCE code in URL) + onAuthStateChange @@ -403,7 +407,7 @@ Database (Supabase) ### Annotation rendering - docs/ANNOTATION-RENDERING.md is the source of truth: the drawing carries no annotation element, the markers are drawn from the document (Rust core for the canvas since R3, `crates/core/src/anatomy`; the frozen TS renderer - `@faunosteo/anatomy/render` for the PDF engine and the pipeline until R4/R5, same output), with the attributes listed there + `@faunosteo/anatomy/render` for the pipeline until R5, same output), with the attributes listed there - Every SVG the product draws goes through the core's one reader (`anatomy::svg::read_svg`): caps before any parse, allowlist, reference graph; a product drawing with anything removed is refused at load, never repaired - Never rely on element id, className or a label for click targeting or behaviour — always `data-anno-id` @@ -414,7 +418,7 @@ Database (Supabase) ### Schemas - The data contracts live in Rust since R2 (`crates/core/src/domain/{sheet,report,archive}`, Zod semantics reproduced: strict where Zod was strict, optional absent, nullable required, Zod schema field order); - `packages/domain` is frozen (no new logic) for the PDF engine, the pipeline and the compare CLI until R4/R5, + `packages/domain` is frozen (no new logic) for the pipeline until R5 (the app no longer depends on it: R4.8), and its Zod copy under `packages/core-wasm/tests/n-1` is the permanent N-1 reader. Never change a contract without bumping the version on a breaking change (additive optional fields keep it); a change touches the Rust struct, the regenerated ts-rs types and the N-1 test @@ -435,7 +439,7 @@ Database (Supabase) - A4 portrait: 595.28 × 841.89 pt - No raw URLs in PDF output — abbreviated citations only - JSON attachment filename must start with osteo-report; content = report archive envelope v1 (report v3 + sheet snapshot + branding + labels); import accepts bare v2 (migrated), bare v3 and envelope v1; a bad snapshot never loses the report -- The writer (`crates/core/src/pdf/write`, since R4.6; the frozen TS engine `packages/pdf` until R4.8) takes no clock and no I/O; rasterisation is the browser's, one raster per core request; every SVG it composes went through the core's one reader. The export is an incremental session (OD-2): the file is written as the rasters arrive, held once, handed out in parts; never accumulate rasters or build the file twice +- The writer (`crates/core/src/pdf/write`, since R4.6; the TypeScript engine deleted at R4.8) takes no clock and no I/O; rasterisation is the browser's, one raster per core request; every SVG it composes went through the core's one reader. The export is an incremental session (OD-2): the file is written as the rasters arrive, held once, handed out in parts; never accumulate rasters or build the file twice - Page layout is frozen by the Step 6 goldens and the R4 oracles (O1 draw logs, O2 PDFs): any visible change needs a user decision and a golden regeneration; the R4 digests (`goldens/r4/pdf.*.sha256`) pin the writer's own bytes and are rewritten only for a declared structural change, with the O2 comparisons re-run - Fonts are StandardFonts (Helvetica/TimesRoman, WinAnsi encoding) — TTF Inter/Fraunces planned but not yet embedded. Any non-Latin-1 glyph in PDF text must use an ASCII surrogate until the TTF migration lands (e.g. `(!)` instead of `⚠`). @@ -562,7 +566,6 @@ RUST.md §9; validated and merged 2026-10-17). Next: R4 (PDF core), R5 (pipeline | tailwindcss 4 | CSS-first utility framework | | zustand 5 | Lightweight global state | | zod 3 | Schema validation + type inference | -| pdf-lib 1.17 | Pure-JS PDF generation + native attachments | | motion 11 | Animations (modals, toasts, observation cards) | | vitest 3 | Test runner | | happy-dom | DOM environment for tests | @@ -585,9 +588,9 @@ RUST.md §9; validated and merged 2026-10-17). Next: R4 (PDF core), R5 (pipeline - Annotation rendering changes require updating docs/ANNOTATION-RENDERING.md first. - Never modify the legacy sources under `supabase/seed/anatomical-sheets/equine/legacy-sources/` — they are the importer's input (the user's historical files). -- Ask before any change to `readReportArchive`, `readArchiveDocument`, `readAttachedDocument` or `rebaseReportOnSheet`, and to their Rust ports (`crates/core/src/domain/{archive/read,report/rebase,report/read,session}.rs`) — they handle user data; archive readers never drop a schema version (envelope v1, Report v2/v3, sheet document v2). +- Ask before any change to the readers of user data: `crates/core/src/domain/{archive/read,report/rebase,report/read,session}.rs` and the PDF reader `crates/core/src/pdf/{read,json_js,import}.rs` (the TypeScript originals `readReportArchive`, `readArchiveDocument`, `readAttachedDocument`, `rebaseReportOnSheet` left with `packages/pdf` at R4.8; the frozen N-1 copies under `packages/core-wasm/tests/n-1` are never edited); archive readers never drop a schema version (envelope v1, Report v2/v3, sheet document v2). - Import never parses untrusted PDF object graphs: the attachment is located by byte scan and inflated under the 16 MiB cap (BLK-009). Opening the import to external users is gated by `docs/development/PDF_IMPORT_HARDENING.md` §5 (BDR-040). -- After PDF roundtrip tests fail: check attachment bytes, then the layered reader (report first, envelope metadata, snapshot/branding/labels), then DecompressionStream inflation. Goldens under `apps/app/tests/fixtures/goldens/step6` are never regenerated without a user decision. +- After PDF roundtrip tests fail: check the attachment bytes (the N-1 scanner reads them in the envelope suites), then the layered reader (report first, envelope metadata, snapshot/branding/labels), then the core's inflation (`pdf/read/inflate.rs`). Goldens under `apps/app/tests/fixtures/goldens/step6` and the R4 digests under `goldens/r4` are never regenerated without a user decision. - Node 24 / pnpm 10.13.1: `corepack enable` or `corepack pnpm`; never a global install. - Rust: native on the Ubuntu development machine (user-level rustup, never a global package, never on the VPS); `pnpm rust:check` and `pnpm wasm:build` run natively, the `:docker` variants reproduce them in the builder image. Before every Rust commit: `pnpm rust:check` green, `pnpm worker:test` green when `crates/worker` changed (Docker only: the sandbox tests and the seccomp derivation never run natively), the ts-rs types under `packages/core-wasm/types` committed, `Cargo.lock` committed. Before a merge: `pnpm check:all`. No CI runs on the forge: a workflow file is never cited as a passed check. The core takes ids, time and bytes as inputs (clippy `disallowed_methods`); the worker child is the only process that decodes untrusted bytes. - Storage or worker change: `pnpm e2e:avatar` (worker container against the local stack, `packages/supabase/tests/e2e`) in addition to `pnpm worker:test`; `docker compose --profile worker` serves the worker from `.env.worker` (never the app `.env`). diff --git a/DEPLOY.md b/DEPLOY.md index 488130ed..6e8ae7d4 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -284,6 +284,16 @@ attachment"; a Step 6 build reads every older PDF (bare v2 and v3). Roll back only with that limit in mind and tell the practitioners which build produced their files. +Rollback note (PDF archives, R4): PDFs written by an R4 build (the core's +writer) are read by the R3 build and by release 0.2.0: the frozen copy of +their reader (`packages/core-wasm/tests/n-1/scanner`, O11) and the frozen +Zod schemas (`n-1/domain`) read every R4 fixture in the test suites +(`n-1.reader`, `n-1.scanner`, `archive.envelopes`); the attachment bytes are +the envelope the TypeScript wrote, in contract key order (D-R11). An R4 +build reads every older PDF (bare v2, bare v3, Step 6 and R3 envelopes). +The TypeScript PDF engine left the repository at R4.8: a rollback is a +previous image, never a rebuild of the old engine. + --- ## Data persistence @@ -668,8 +678,10 @@ only, never `eval` or inline scripts (the proof's inline variant was still blocked). Browser floor for the keyword: Chrome 97, Firefox 102, Safari 16 (the Vite 7 baseline). The `.wasm` file is served as `application/wasm` by `nginx:1.31-alpine` out of the box; since R2 the image precompresses it and -`docker/nginx.conf` serves the `.wasm.gz` with `gzip_static on;` (372 KB -served for 1.18 MB raw, D-R13 budget 600 KB). A proxy in front must pass +`docker/nginx.conf` serves the `.wasm.gz` with `gzip_static on;` (R4.8 +module with the PDF reader, writer and session: 1,535,289 B raw, 519,795 B +gzip -9; R3: 1.18 MB raw, 372 KB served; D-R13 budget 600 KB, report line +550 KB passed at R4.6 and declared). A proxy in front must pass `Content-Encoding` through and keep the `application/wasm` type. Since R4 the PDF import runs in a module worker (`/assets/import.worker-*.js`, diff --git a/README.md b/README.md index 4e519354..8c401205 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ contract. - Tailwind CSS v4 (CSS-first, no postcss config) - Zustand 5 (state) - Zod 3 (schema + runtime validation) -- pdf-lib 1.17 (PDF generation + native attachments) +- PDF reader and writer in the Rust core (`crates/core/src/pdf`, WebAssembly in the app; no pdf-lib since R4.8) - motion 11 (animations) - vitest 3 + happy-dom + @testing-library/react - Supabase (PostgreSQL 17, Auth, RLS) via the pinned CLI; `@supabase/supabase-js` 2; PGlite for Dockerless SQL tests @@ -95,7 +95,7 @@ pnpm typecheck # TypeScript strict, no emit pnpm lint # ESLint 9 flat config, 0 warnings pnpm test:run # vitest run (domain, anatomy, supabase/PGlite, app) pnpm build # tsc + vite build → apps/app/dist/ -pnpm report:compare ... # archive PDF vs its v3 reading, unresolved items flagged +pnpm report:compare ... # archive PDF vs its v3 reading, unresolved items flagged (the core's binary; first run compiles) pnpm rust:check # native: fmt, clippy -D warnings, cargo test, ts-rs export, cargo deny pnpm wasm:build # native: browser module → packages/core-wasm/pkg (the core-wasm proofs run on it) pnpm check:all # everything a branch must pass before a merge, in one command (needs Docker for the image stages) @@ -148,13 +148,12 @@ faunosteo/ pnpm workspace (see docs/development/WORKSPACE.md) ├─ src/state/ Zustand store (Report v3) + persistence adapter + locale store ├─ src/anatomy/ drawing: sanitised asset + annotation layer → inline SVG per view ├─ src/components/ UI primitives, canvas, observation, report -├─ src/pdf/ app side of the PDF: snapshot + branding + browser rasterizer → @faunosteo/pdf engine +├─ src/pdf/ app side of the PDF: snapshot + branding + labels + browser rasterizer → the core's export session ├─ src/lib/ id, dates, assert utilities ├─ src/styles/ Tailwind v4 CSS + design tokens ├─ tests/unit/ vitest tests on the bundled sheet (+ frozen v2 fixtures) ├─ packages/domain/ @faunosteo/domain: canonical sheet document + report document (v3, v2 migration) ├─ packages/anatomy/ @faunosteo/anatomy: legacy importer, curation pipeline, annotation-layer renderer + view composer -├─ packages/pdf/ @faunosteo/pdf: browser-free PDF engine (explicit inputs, archive envelope v1, layered reader) ├─ packages/supabase/ @faunosteo/supabase: typed client, generated types, PGlite SQL test harness ├─ supabase/ CLI config, SQL migrations, local seed (incl. the equine sheet), pgTAP tests ├─ docker/ entrypoint.sh + nginx.conf diff --git a/docs/PDF-FORMAT.md b/docs/PDF-FORMAT.md index dda34d5d..ad395395 100644 --- a/docs/PDF-FORMAT.md +++ b/docs/PDF-FORMAT.md @@ -1,9 +1,11 @@ # PDF format and report archive A Faunosteo PDF is both a document for the reader and an archive of the -bilan. The engine lives in `packages/pdf` (`@faunosteo/pdf`) and renders from -explicit inputs only: no network, no browser DOM, no access to the current -catalog. The attachment carries everything a future reader needs to +bilan. The engine lives in the Rust core (`crates/core/src/pdf`, reached +through the wasm module in the browser and natively by the tools; the +TypeScript `packages/pdf` it reproduced byte for byte was deleted at R4.8, +`docs/development/RUST.md` §10) and renders from explicit inputs only: no +network, no browser DOM, no access to the current catalog. The attachment carries everything a future reader needs to re-render the bilan when the anatomical sheet, the catalog, the labels or the branding have changed since the export. Design record: `docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md`. @@ -19,8 +21,9 @@ branding have changed since the export. Design record: asset), then zones, findings grouped by finding zone, free findings, severity, comment. - Conclusion page: conclusion, recommendations, next appointment. -- Fonts: pdf-lib StandardFonts Helvetica, Helvetica-Bold (body) and - Times-Roman (headings), WinAnsi encoding. Every printed string goes through +- Fonts: the standard Type1 fonts Helvetica, Helvetica-Bold (body) and + Times-Roman (headings), WinAnsi encoding, measured with the AFM widths + pdf-lib 1.17.1 used (`crates/core/src/pdf/write/afm_data.rs`). Every printed string goes through `safeText`: control characters are stripped and any glyph WinAnsi cannot encode becomes `?` (en and em dashes, accented letters, `•`, `·` are fine; `⚠` is not, hence the `(!)` vet-referral marker). @@ -44,7 +47,7 @@ branding have changed since the export. Design record: One native embedded file (`EmbeddedFiles`, `AFRelationship /Data`) named `osteo-report.json`, MIME `application/json`, description -`Faunosteo report archive v1`, Flate-compressed by pdf-lib. Pretty-printed +`Faunosteo report archive v1`, Flate-compressed (zlib level 6). Pretty-printed JSON, camelCase keys (the sheet document inside keeps its own snake_case schema): @@ -227,15 +230,16 @@ The same caps are checked at export, so an export can never write a file its own reader refuses. The branding URL rule exists because a URL is written into every page's -link action: pdf-lib does not escape a literal string, so the engine writes -the URI as a hex string and the schema refuses the characters that could -close a literal. The export re-validates the branding and the labels it +link action: a literal string is not escaped (pdf-lib did not, the core +does not either), so the engine writes the URI as a hex string and the +schema refuses the characters that could close a literal. The export re-validates the branding and the labels it receives (`ArchiveExportError('branding_invalid')` before any page is drawn), even from an archive that passed the reader. The URI itself is the hex form of the URL's ASCII bytes (`<68747470…>`), not UTF-16 with a BOM, so readers that take `/URI` as raw bytes (poppler) follow it. A PNG logo is -checked before pdf-lib's decoder (UPNG) sees it, because that decoder trusts -its input: it sizes its bitmap from whatever IHDR claims, its `inflateRaw` +checked before the decoder sees it (the core's decoder reproduces UPNG's +acceptance, the decoder pdf-lib used, so that an archived logo decodes to +the same planes); the gate exists because that decoder trusted its input: it sizes its bitmap from whatever IHDR claims, its `inflateRaw` never checks the input bound (a truncated or crafted IDAT stream makes it read past the buffer and never return), and it scans text chunks for a NUL byte that may be missing. The gate therefore validates, in order: the @@ -247,10 +251,11 @@ sizes are not computed); an expected raw scanline size, `(ceil(width × bits per pixel / 8) + 1) × height`, of at most 16 MiB, the attachment cap the native inflater enforces (4096² at 8-bit RGBA is 67 MB of scanlines and is refused; a 512 px wide logo is under 1 MB). That cap -bounds what is inflated, not what pdf-lib's decoder then allocates: the -decoder still expands any accepted image to a 32-bit RGBA buffer, so the -4096² pixel cap is what bounds memory (about 430 MB peak measured with four -4096² logos; real logos are 512 px wide). Then the chunk list, which must end on an empty `IEND` +bounds what is inflated, not what the decoder then allocates: the decoder +still expands any accepted image to a 32-bit RGBA buffer, so the 4096² pixel +cap is what bounds memory (about 430 MB peak measured with four 4096² logos +on the TypeScript engine; real logos are 512 px wide; the core frees the +planes as soon as the image object is written). Then the chunk list, which must end on an empty `IEND` that closes the file and may hold only the chunk types the decoder reads by fixed offsets or by declared length (`PLTE`, `tRNS`, `IDAT`, `IEND`, `gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs`, `bKGD`): a second IHDR, an APNG @@ -271,11 +276,13 @@ fallback text and the export reports `logo_failed`. reader in a worker since R4.3) and the `report:compare` tool (the same reader, natively, since R4.7) read the attachment by byte scan, so a crafted object stream is never inflated (tested with a 64 MiB bomb: - milliseconds, no memory growth). pdf-lib stays on the export side, on trusted inputs, and - in tests that inspect exported files. What remains before external users - (`docs/development/PDF_IMPORT_HARDENING.md` §5): a Worker or server-side - budget for any path that still parses PDF objects, the pdf-lib dependency - decision, and read-side caps on report collections. + milliseconds, no memory growth). No PDF library remains in the repository + since R4.8: the core writes the files, the tests read them back with a + reader of the writer's own format (`apps/app/tests/helpers/pdfFile.ts`). + What remains before external users + (`docs/development/PDF_IMPORT_HARDENING.md` §5): the server-side budget + decision for any future path that parses PDF objects, and read-side caps + on report collections. - The scanner reads what is in clear: an attachment whose stream dictionary is malformed beyond `/Length` and `endstream`, or an encrypted file, yields no candidate (`no_attachment`), as the previous reader did in practice. diff --git a/docs/development/PDF_IMPORT_HARDENING.md b/docs/development/PDF_IMPORT_HARDENING.md index 71c858cc..8b3a0f0e 100644 --- a/docs/development/PDF_IMPORT_HARDENING.md +++ b/docs/development/PDF_IMPORT_HARDENING.md @@ -110,9 +110,11 @@ hides its `/Keywords`, so a stripped attachment on such a file reads as brackets deeper than 64 levels (linear byte pre-scan, strings skipped). - Option C or a server-side import (Step 14) with a hard memory and time budget per file, for any path that still parses PDF objects. -- Dependency decision on pdf-lib (unmaintained since 2021): keep it for - generation only, or move to the maintained fork; recorded as a Step 14 - question. +- Dependency decision on pdf-lib (unmaintained since 2021): closed at R4.8 + (2026-10-19): pdf-lib left the repository with the TypeScript engine; the + core's writer (`RUST.md` §10.7, §10.8) generates the PDFs and its reader + reads them; no PDF library remains in the application, its tests or the + tools (`apps/app/tests/unit/r4.census.test.ts` fails on a return). - Report collection caps on read (observations, zones, findings per observation) so a hostile report cannot freeze the editor before the export's own caps refuse it. diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 875badbc..fe7a0d93 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -1960,3 +1960,134 @@ Residual risks and open points (R4.7): - A PDF over 50 MiB is refused by the entry check (`file too large`) before the reader, as before; the reader's own cap would say `not a readable PDF`. + +### 10.10 R4.8: the TypeScript PDF engine deleted (2026-10-19) + +Audit before every deletion (owner's R4.8 brief §2): every importer of +`@faunosteo/pdf`, `pdf-lib`, `@pdf-lib/*`, `@resvg/resvg-js`, `@xmldom/xmldom` +and `@faunosteo/domain` was listed across the application, the glue package, +the tests, the scripts, the pipeline, the workspace files, the Dockerfile, +the CI files and the docs (`grep` census, now the `r4.census` suite). The +application's last consumer of the engine (the TypeScript import path, +tests only since R4.3) and the glue's (the session suite's compose check, +the engine comparison) were moved onto the core or deleted first; the +pipeline's consumers (`packages/anatomy/src/curation/app-review.ts` → +`render/compose.ts`, 22 files → `@faunosteo/domain`) stay until R5. + +Deleted (70 files, 12,010 lines of TypeScript): + +- `packages/pdf/**` (3,122 lines of engine, 8,211 of tests): the export, + the layered reader, the byte-scan attachment reader, the SVG sanitiser and + the view composition, the page renderers, the branding gate, the labels, + the 140 engine cases and the R3 and R4 oracle generators + (`r3.{highlight,sanitiser}-oracles`, `r4.{dates,inflate,png,reader,text}-oracles`) + with their helpers (`resvg`, the reader-input crafters). +- `apps/app/scripts/compare-reports.ts` and `compare/{report,sheet}.ts` + (589 lines, the TypeScript CLI replaced at R4.7), `tests/unit/scripts.compare.test.ts`. +- `apps/app/src/pdf/import.ts` (the TypeScript scanner behind + `importReportFromPdf`, tests only since R4.3) and `tests/unit/pdf.import.test.ts`. +- `tests/unit/pdf.export.app.test.ts` (T2 and scenario 3 through the + engine's `exportFromEnvelope`: the re-export from the envelope alone is + cargo's `export_from_envelope`, whose signature takes no sheet and no + branding, byte-equal to the R4 files on 13 runs). +- The application's generators `r4.{compare,tolerance,writer}-oracles` and + `tests/helpers/r4-draw.ts` (the pdf-lib draw recorder); the glue's + `pdf-export.ts-engine.test.ts` (the Node timings against the engine, + recorded in §10.7). +- Dependencies: `@faunosteo/pdf` (app, core-wasm), `pdf-lib`, `tsx` and + the app's `report:compare` script (app), `@faunosteo/domain` (app: its + two incidental test uses moved to `@faunosteo/core-wasm/types` and the + N-1 Zod copy), the Dockerfile's `COPY packages/pdf/package.json`. + `pnpm-lock.yaml` holds no `pdf-lib`, `@pdf-lib/*` or `@resvg/resvg-js` + any more; `@xmldom/xmldom` only under `packages/anatomy`. + +Replaced in the tests (no case ignored, no golden regenerated): + +| Before | After | +|---|---| +| pdf-lib parsed the app export's file (texts, images, page count, Info, Filespec) | `tests/helpers/pdfFile.ts`, a reader of the writer's own format (objects, dictionaries, Flate streams through `node:zlib`, pages in `/Kids` order, Info, annotations, XObjects); `pdfRead.ts` keeps its content-stream parser on it | +| pdf-lib read the links of the file | the same reader's annotations (`goldens.ts` `readLinks`) | +| pdf-lib built the foreign PDFs of the rejection cases | `buildPdfWithAttachment` (a minimal file with one embedded file in clear) and the frozen hostile fixtures | +| pdf-lib's `embedPng` checked the golden rasters | the XObject sizes read from the app export's file | +| the engine composed the views of the `views.*.json` goldens | the SVGs the core's session asks the rasteriser for, captured at the mocked rasteriser in request order | +| the engine's scanner read the attachment of the envelope and N-1 suites | the frozen O11 scanner, now exported by `@faunosteo/core-wasm/n-1` (`readAttachmentBytes`, `readAttachedDocument`): the R3 build's reader, as a rollback would read | +| `importReportFromPdf` (TypeScript scanner + core) | `tests/helpers/pdfImport.ts`: the module's `SheetHandle.import_pdf` on a handle of the bundled sheet, the envelope mapped by the worker's `pdfImportOf` (the same kinds HomeRoute shows); the main-thread session still never calls the reader (OD-1) | +| the session suite compared each request SVG with the engine's `composeHighlightedViewSvg` | structural facts per request (the view's frame, the highlight, no editor attribute) and the real-raster manifest digests; the text itself is cargo's (`pdf_session` against `compose_highlight`, `anatomy_compose` against the frozen R3 trees) | + +Still TypeScript, with the reason (owner's R4.8 brief §6.4): + +- `packages/anatomy` (the curation pipeline, the legacy importer, the frozen + renderer `render/` and `@xmldom/xmldom`): R5's scope, consumed by + `pnpm import:equine` and the seed golden; no app and no PDF consumer. +- `packages/domain`: the pipeline's schemas until R5; the application and the + glue no longer depend on it. +- `packages/core-wasm/tests/n-1/{domain,scanner}`: the frozen N-1 readers + (the R2 Zod schemas, the R3 byte-scan reader), never edited, permanent. +- `apps/app/src/pdf/{export,labels,branding,browserRasterizer}.ts`: the + application's side of the export (input assembly, the session driver, the + browser's rasteriser), UI glue by the review's boundary. +- `apps/app/src/core/worker/**`: the import worker (protocol, timers, + outcome mapping), browser glue. +- The test readers above (`pdfFile.ts`, `pdfRead.ts`, `pdfImport.ts`): + tooling on the writer's own files, never product logic. +- `scripts/r4-afm-data.mjs`: the generator of `afm_data.rs` from the frozen + O4/O5 oracles (no pdf-lib: reads `afm.json` and `text.json`), kept so the + tables stay reproducible. + +Census and gates: `apps/app/tests/unit/r4.census.test.ts` (8 checks) fails +on a reintroduced import of the engine, pdf-lib, the TS domain or xmldom +under `apps/app/src` or `packages/core-wasm/src`, on a dependency added back, +on a lockfile holding pdf-lib, on an export that does not open the core's +session, on an import that does not go through the worker, on a root +`report:compare` that is not the binary, and on a missing N-1 reader or +oracle folder. The ledger (`scripts/r4-ledger.mjs`) now reads its 301 frozen +cases from the committed TSV (the deleted suites cannot run), re-derives the +dispositions and checks every `cargo` target against `cargo test --list` +and every `vitest (same case…)` target against the live report of the +surviving suites: 150 ported, 29 replaced, 89 kept, 33 deleted, 0 pending. +Test counts after R4.8: apps/app 57 files, 427 cases (460 before: 42 of the +deleted suites, 1 roundtrip case added, 8 census); packages/core-wasm 58 +(60); packages/pdf 146 → 0; cargo unchanged (R4.7: the compare suite +included). `pnpm typecheck`, `pnpm -r lint`, `pnpm check` and +`pnpm rust:check` green; `pnpm check:all` (the app image, the worker test +stage and image) recorded below. + +Non-regression (owner's R4.8 brief §3), each with its proof: + +| Guarantee | Proof | +|---|---| +| the application imports through the core's reader | `r4.census` (HomeRoute → `core/worker`, `handler.ts` → `import_pdf_bytes`, no scanner identifier under `src`); the route and worker suites (`routes.HomeRoute.{import,archive,rescueCopy}`, `core.importWorker`) | +| the export runs on the core's session | `r4.census` (`export.ts` → `openExportSession`); `pdf.goldens`: app-path bytes = the R4 digests at the fixed time | +| `report:compare` is the binary | `r4.census` (root script → `scripts/report-compare.sh` → `--features cli --bin report-compare`); cargo `pdf_compare` spawns it | +| v2 and v3 archives stay compatible | `pdf.roundtrip` (the frozen v2 PDFs migrated, unresolved items flagged), `migration.leaves` (612 leaves exported and read back through the module, 29 unresolved kept), cargo `pdf_import` O7/O8 | +| historical PDFs stay readable | cargo `pdf_import` and `pdf_serialize::restoration_round_trip_loses_nothing` on the Step 6 and R3 fixtures; `pdf.goldens` on `report-v3.bare.pdf`; the N-1 suites on the R4 files | +| the validated renders stay conformant | `pdf.goldens` (Step 6 text, images, links read from the file; the R4 digests), cargo `pdf_layout` (O1), `pdf_serialize` (O2, poppler 0 pixel recorded at R4.5) | +| rescue copies and data-loss protections | `routes.HomeRoute.rescueCopy` (13), `routes.coreStopped` (15), `state.coreFailures` (5), the quarantine suites, unchanged | +| no TypeScript engine active in parallel | `r4.census`; the only TypeScript readers left are the frozen N-1 copies, read by tests only | + +Residual risks and open points (R4.8): + +- The test reader `pdfFile.ts` reads the writer's own format only (one + dictionary per line, direct `/Length`, classic xref): a change of the + writer's layout changes it too; it is tooling, pinned by the suites that + use it. +- `tests/helpers/pdfImport.ts` runs the module's reader on the test + process's main instance (a throwaway handle, freed): the application never + does (OD-1); the worker path keeps its own suites. +- A stale `packages/pdf/node_modules` folder (git-ignored symlinks of the + previous install) may linger on a development machine after the pull: + `rm -rf packages/pdf` by hand; the census checks the package files, not + the folder. +- The open points of §10.8 stand: the Chromium per-export growth (BDR-063, + an open reserve), J1 and J2 on real devices before any merge, the WebKit + reserves BDR-058/059, the hostile-file protections (`PDF_IMPORT_HARDENING` + §5 gate open), the module at 519,795 B gzip of a 600 KB budget. R4.8 + measured nothing new on memory or time and claims nothing. + +R5 prerequisites (the pipeline port): `packages/anatomy` (curation pipeline, +importer, `render/`, the review drawings) and `packages/domain` are the last +TypeScript business packages; their oracles (the seed golden, the 612-leaf +semantics, the determinism and count suites, the R3 layer oracle) are in +place; `documentSha256` will change once (D-R5) and every older archive will +rebase on import; `@xmldom/xmldom`, `tsx` and `prettier` on generated files +leave with them. diff --git a/docs/development/TEST_STRATEGY.md b/docs/development/TEST_STRATEGY.md index 9e04fd60..374ebac7 100644 --- a/docs/development/TEST_STRATEGY.md +++ b/docs/development/TEST_STRATEGY.md @@ -15,7 +15,7 @@ Status: Step 0 deliverable, 2026-10-08. Current state: 32 files under | RLS | pgTAP, per table, per role | `supabase/tests/rls` | CI, release gate | | End-to-end | Playwright against `pnpm preview` + local Supabase | `apps/app/e2e` | CI nightly + release gate | | Migration | `supabase db reset` from zero + seed + regression fixtures | CI | every PR touching `supabase/` | -| PDF regression | vitest golden tests: text extraction of generated PDFs, attachment roundtrip, raster hash on fixed inputs | `packages/pdf/tests` | CI | +| PDF regression | cargo: the R4 oracles (O1 draw logs, O2 PDFs, O4 to O10) and the R4 digests of the writer's own bytes; vitest: the Step 6 goldens read back from the app export's file (`tests/helpers/pdfFile.ts`, no pdf-lib), the attachment roundtrip through the module's reader, the N-1 readers | `crates/core/tests/pdf_*.rs`, `apps/app/tests/unit/pdf.*.test.ts`, `packages/core-wasm/tests/pdf-*.test.ts` | CI | | SQL/TS contract | generated types diff check | CI | every PR touching `supabase/` | | Rust core and facade | `cargo test` (unit + fixture tests reading the repository JSON in place), `clippy -D warnings`, `cargo deny`; ts-rs types drift check | `crates/*`, `pnpm rust:check` (native, `scripts/rust-check.sh`; same file in the builder image), `pnpm check:all` | before every Rust commit and every merge; CI job `rust` prepared, no runner yet | | Rust/TS contract | vitest on the built module: byte equality with `JSON.stringify(…, null, 2)` on the real documents, ts-rs types against the Zod output types, the runtime (lazy load, trap → poison → offline re-instantiation, handle rebuild, cap, retry), panic and stack traps on `pkg-proofs`, served size budget and measurements | `packages/core-wasm/tests` (refused, never skipped, when `pkg/` or `pkg-proofs/` is missing or stale) | `pnpm check:all` (after the native `wasm:build`); CI `check` prepared | @@ -78,8 +78,10 @@ Target after Step 1: roughly 230–250 cases, all current ones untouched. - `supabase/seed/test/` provides two practitioners, one cabinet, two clients, three animals, the official sheet, one personal sheet, reports in each status. RLS tests reuse it. -- Golden PDFs are compared by extracted text and attachment JSON, not by - bytes (pdf-lib dates and ids vary); raster hash only on fixed-size inputs. +- Golden PDFs are compared by the text, images and links read back from the + file and by the attachment JSON; the writer's own bytes are pinned by the + R4 digests at a fixed export time (`goldens/r4/pdf.*.sha256`), rewritten + only for a declared structural change. ## 5. Gates before a merge, and the CI target diff --git a/docs/development/WORKSPACE.md b/docs/development/WORKSPACE.md index a9cb7d81..06fa5d4e 100644 --- a/docs/development/WORKSPACE.md +++ b/docs/development/WORKSPACE.md @@ -35,7 +35,7 @@ Future units, created only by the step that needs them: |---|---|---| | `apps/public-web` | SSR public site (landing, practitioner profiles, search) | plan Step 22 | | `crates/worker` | Rust job runner (avatars at R1, then emails, expiry, calendar sync, imports) — created at R0, replaces the planned `apps/worker` | R0 (2026-10-12) | -| `packages/*` | shared domain, anatomy, pdf, i18n, supabase, config, ui | see `packages/README.md` | +| `packages/*` | shared domain, anatomy, core-wasm, supabase (i18n, config, ui planned; `pdf` deleted at R4.8) | see `packages/README.md` | Rust workspace (plan R0, `docs/development/RUST.md`): `Cargo.toml` at the root with `crates/core` (library), `crates/wasm` (browser facade, built into diff --git a/packages/README.md b/packages/README.md index d72c7f40..d7951fd3 100644 --- a/packages/README.md +++ b/packages/README.md @@ -10,19 +10,21 @@ Planned packages and the step that creates each one: |---|---|---| | `@faunosteo/domain` | Zod schemas: anatomical sheet document + integrity rules (**created**, Step 4); report document, entitlement capabilities later | Step 4 (sheet model) | | `@faunosteo/anatomy` | legacy importer (**created**, Step 4b), curation pipeline: changeset → working draft + report migration map (**created**, Steps 4c/5), pure annotation-layer renderer `@faunosteo/anatomy/render` (**created**, Step 5; since R3 used by the PDF engine and the pipeline only, the app canvas draws the Rust core's composition) | Step 4b (importer) | -| `@faunosteo/pdf` | browser-free PDF engine (**created**, Step 6): report archive export from explicit inputs (report, sheet snapshot, branding, labels, rasterizer), layered archive reading, SVG sanitiser, page renderers | Step 6 (PDF archives) | +| `@faunosteo/pdf` | browser-free PDF engine (**created** Step 6, **deleted** R4.8, 2026-10-19): its reader, writer and compare tool live in the Rust core (`crates/core/src/pdf`); its byte-scan reader stays as the frozen N-1 copy under `core-wasm/tests/n-1/scanner` | Step 6; gone at R4.8 | | `@faunosteo/i18n` | FR/EN dictionaries (no React) + `useT` | first second consumer (public-web or worker) | | `@faunosteo/supabase` | browser client factory, generated database types, PGlite harness running the pgTAP suite, sheet publish gate | **created**, Step 3 | | `@faunosteo/core-wasm` | the Rust core compiled to WebAssembly: `CoreRuntime` (lazy load, poison on a trap, offline re-instantiation, sheet handles per instance), Node source, `pkg/` shipped and `pkg-proofs/` (built by `pnpm wasm:build`, git-ignored, BUILD_STAMP), TypeScript types of the contracts generated by ts-rs (committed), the frozen N-1 Zod reader (`./n-1`), the trap, size and measurement tests | **created**, R0; runtime and app consumers R2 | Since R2 the Rust core (`crates/core`) holds the business logic `domain` held for the application (`docs/development/RUST.md` §8); `domain` stays frozen -for the PDF engine, the pipeline and the compare CLI. Since R3 the core also -holds the SVG reader, the annotation layer and the view composition the editor -canvas draws (`RUST.md` §9); `anatomy/render` stays frozen for the engine and the -pipeline. `pdf` follows at R4 (`docs/development/RUST_ARCHITECTURE_REVIEW.md` Part -VI); each TypeScript package is deleted in the phase that switches its last -consumer (`domain` and `anatomy` at R5, `pdf` at R4). +for the pipeline only (the application no longer depends on it: R4.8). Since +R3 the core also holds the SVG reader, the annotation layer and the view +composition the editor canvas draws (`RUST.md` §9); `anatomy/render` stays +frozen for the pipeline. Since R4 (`RUST.md` §10) the core reads, writes and +compares the PDF archives; `pdf` was deleted at R4.8 +(`docs/development/RUST_ARCHITECTURE_REVIEW.md` Part VI); each TypeScript +package is deleted in the phase that switches its last consumer (`domain` and +`anatomy` at R5). | `@faunosteo/config` | runtime config resolution shared by app, public-web, worker | with public-web | | `@faunosteo/ui` | UI primitives and design tokens | only when public-web reuses them |