CLAUDE.md, WORKSPACE, packages/README, README, sheet model, SUPABASE, TEST_STRATEGY, PDF-FORMAT and ANNOTATION-RENDERING follow the deletion; RUST.md 11.23 records the census, counts, the historical TS replay procedure and hashes; CHANGELOG, TODO, R5.7 plan and contract.
330 lines
24 KiB
Markdown
330 lines
24 KiB
Markdown
# PDF format and report archive
|
||
|
||
A Faunosteo PDF is both a document for the reader and an archive of the
|
||
bilan. The engine lives in the Rust core (`crates/core/src/pdf`, reached
|
||
through the wasm module in the browser and natively by the tools; the
|
||
TypeScript `packages/pdf` it reproduced byte for byte was deleted at R4.8,
|
||
`docs/development/RUST.md` §10) and renders from explicit inputs only: no
|
||
network, no browser DOM, no access to the current catalog. The attachment carries everything a future reader needs to
|
||
re-render the bilan when the anatomical sheet, the catalog, the labels or the
|
||
branding have changed since the export. Design record:
|
||
`docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md`.
|
||
|
||
## Pages
|
||
|
||
- A4 portrait, 595.28 × 841.89 pt, 36 pt side margins.
|
||
- One page per observation, then one conclusion page. Every page carries the
|
||
header (logo, practitioner, animal, owner, date) and the footer (site logo,
|
||
"powered by" credit, partner credit, page `n / N`).
|
||
- Observation page: eyebrow + title, the views of the observation drawn with
|
||
its zones highlighted (main view first, head view as a crop of the same
|
||
asset), then zones, findings grouped by finding zone, free findings,
|
||
severity, comment.
|
||
- Conclusion page: conclusion, recommendations, next appointment.
|
||
- Fonts: the standard Type1 fonts Helvetica, Helvetica-Bold (body) and
|
||
Times-Roman (headings), WinAnsi encoding, measured with the AFM widths
|
||
pdf-lib 1.17.1 used (`crates/core/src/pdf/write/afm_data.rs`). Every printed string goes through
|
||
`safeText`: control characters are stripped and any glyph WinAnsi cannot
|
||
encode becomes `?` (en and em dashes, accented letters, `•`, `·` are fine;
|
||
`⚠` is not, hence the `(!)` vet-referral marker).
|
||
- Drawings: the sheet SVG asset is read by the core's one SVG reader (`anatomy::svg`, since R3/R4; xmldom left at R5.7), sanitised (allowlist
|
||
below), framed on the view box, the marker layer is generated from the
|
||
document and the observation's zones are filled `#b85a3f` at 0.85 opacity
|
||
with a blur halo on the main view; every other marker is dropped. The SVG
|
||
is rasterised at 1200 px wide (aspect clamped to [0.2, 2.0]) by the
|
||
rasterizer the caller provides: `Image` + canvas in the browser, resvg in
|
||
Node tests. A failed raster leaves the page without that image and reports
|
||
a warning; the export never fails for a drawing.
|
||
- Document information: title, author, subject, dates = `exportedAt`, and
|
||
`Keywords: faunosteo-archive/1` so a PDF whose attachment was stripped by a
|
||
viewer is still recognised.
|
||
- Saved without object streams (`useObjectStreams: false`) since BLK-009: the
|
||
Filespec, its file name and the Info dictionary stay in clear, where the
|
||
byte-scan reader below can see them. Files produced earlier pack them into
|
||
an `/ObjStm`; their attachment stream is still in clear and still imports.
|
||
|
||
## Attachment: the report archive envelope (v1)
|
||
|
||
One native embedded file (`EmbeddedFiles`, `AFRelationship /Data`) named
|
||
`osteo-report.json`, MIME `application/json`, description
|
||
`Faunosteo report archive v1`, Flate-compressed (zlib level 6). Pretty-printed
|
||
JSON, camelCase keys (the sheet document inside keeps its own snake_case
|
||
schema):
|
||
|
||
```json
|
||
{
|
||
"format": "faunosteo-report-archive",
|
||
"envelopeVersion": 1,
|
||
"exportedAt": "2026-10-11T09:00:00.000Z",
|
||
"locale": "fr",
|
||
"report": { "version": 3, "...": "Report v3 exactly as exported" },
|
||
"sheetSnapshot": {
|
||
"status": "draft",
|
||
"ref": {
|
||
"sheetId": "…",
|
||
"versionId": null,
|
||
"documentSha256": "…",
|
||
"schemaVersion": 2,
|
||
"speciesId": "equine"
|
||
},
|
||
"document": {
|
||
"schema_version": 2,
|
||
"...": "lean, valid anatomical sheet document"
|
||
},
|
||
"assets": [{ "svgAssetId": "…", "sha256": "…", "svg": "<svg …>" }]
|
||
},
|
||
"branding": {
|
||
"siteUrl": "https://…",
|
||
"headerLogo": { "kind": "png", "base64": "…" },
|
||
"footerLogo": { "kind": "png", "base64": "…" },
|
||
"poweredBy": {
|
||
"logo": { "kind": "png", "base64": "…" },
|
||
"url": "https://…",
|
||
"fallbackText": "…"
|
||
},
|
||
"partner": {
|
||
"logo": { "kind": "svg", "svg": "<svg …>" },
|
||
"url": "https://…",
|
||
"fallbackText": "…"
|
||
}
|
||
},
|
||
"labels": {
|
||
"fr": { "pdf.section.zones": "ZONES", "...": "25 keys" },
|
||
"en": { "...": "" }
|
||
}
|
||
}
|
||
```
|
||
|
||
### What the snapshot holds
|
||
|
||
The snapshot is the sheet the bilan was built on, reduced to what the bilan
|
||
needs, as a **valid** anatomical sheet document (schema v2, same integrity
|
||
rules as the catalog):
|
||
|
||
| Part | Content |
|
||
| ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| `ref` | sheet id, version id, sha256 of the full catalog document (provenance pointer), schema version, species |
|
||
| `document.views`, `annotations`, `categories`, `vertebral_config` | all of them: geometry, colours, names, anatomical relations (`finding_zone_id`, `vertebral_region_code`, `opens_view_id`, laterality) |
|
||
| `document.finding_zones` | every zone with its name; for the zones the report's findings use (resolved or `unresolved`), the categories pruned to the referenced paths with the name and vet flag of every node on the path; other zones keep `categories: []` |
|
||
| `document.references` | emptied (bibliography) |
|
||
| `document.name`, `description`, `scope_disclaimer`, `attribution` | kept |
|
||
| `assets` | the SVG text of every asset a view reaches, with the catalog's content hash when known |
|
||
|
||
`branding` is the exact set of images, URLs and credit texts used at export;
|
||
the application builds it from logos bundled at build time
|
||
(`apps/app/src/assets/brand`), so the export makes no network request.
|
||
`labels` is the engine dictionary in both languages. A re-export of the
|
||
archive uses these three blocks and nothing from the current application, so
|
||
a new logo, a reworded heading or a curated sheet never changes how an old
|
||
bilan is reproduced.
|
||
|
||
Invariant: `report.sheet.documentSha256 === sheetSnapshot.ref.documentSha256`
|
||
(and the sheet ids match). The export refuses to write a file that breaks it.
|
||
|
||
### Sizes
|
||
|
||
Measured on the demo report (6 observations, both views, equine draft):
|
||
|
||
| Branding | Stored in the PDF (Flate) | Raw JSON |
|
||
| ----------------------------------------- | ------------------------- | -------- |
|
||
| no logos | 55 KB | 189 KB |
|
||
| full branding (3 PNG logos + partner SVG) | 136 KB | 306 KB |
|
||
|
||
The drawing is the largest part (47 KB stored); the archived PNG logos add
|
||
about 80 KB stored. The 250 KB target applies to the stored size.
|
||
|
||
## Reading an archive
|
||
|
||
`readReportArchive(pdfBytes, migrationMap)` reads in layers so the report is
|
||
never lost because of its wrapper:
|
||
|
||
1. The file must weigh at most 50 MiB and carry `%PDF-` within its first
|
||
KiB (`invalid_pdf`). Nothing else of the PDF syntax is required.
|
||
2. The attachment is located by a byte scan, never by parsing the PDF object
|
||
graph (BLK-009): every `/Type /EmbeddedFile` stream dictionary in clear
|
||
(a stream object can never sit inside an object stream) is a candidate,
|
||
with its direct `/Length` or, failing that, the distance to `endstream`;
|
||
a `/Filespec` dictionary in clear whose `/EF /F` points at the stream
|
||
gives it a name. Candidates named `osteo-report*` come first, then file
|
||
order; Flate data is fed to the platform inflater in 16 KiB slices, so
|
||
memory stays bounded in browsers too, under a 16 MiB cap (cancelled
|
||
beyond it); at most 16 MiB of candidate content is retained and parsed in
|
||
all (a cancelled inflation is not charged, so up to 32 oversized streams
|
||
may each cost one capped inflation of CPU, bounded, no memory retained),
|
||
a candidate not opening on `{` or nesting brackets deeper than 64 levels
|
||
is skipped unparsed, and the first one whose text is a JSON object with a
|
||
numeric `envelopeVersion` or `version` wins. Object streams,
|
||
cross-reference streams and page contents are never inflated.
|
||
No candidate: `attachment_stripped` when a `/Keywords` string in clear
|
||
contains `faunosteo-archive`, else `no_attachment`. A stamped file saved
|
||
with object streams (before BLK-009) hides its stamp and reads as
|
||
`no_attachment`.
|
||
3. An attachment whose JSON escapes a lone UTF-16 surrogate is refused
|
||
whole (`lone_surrogate`, "malformed text, import refused, file not
|
||
modified"): the core cannot hold it without altering it and never
|
||
rewrites it (L1, `docs/development/RUST.md` §8.7).
|
||
The report is located (`data.report` inside an envelope, `data` for a
|
||
bare attachment) and read with the report reader: v3 as is, v2 migrated
|
||
through the sheet's `report-migration-map.json` (items the map cannot
|
||
place are kept with their labels and flagged `unresolved: true`), any
|
||
other version → `incompatible_version` with the version found.
|
||
4. Envelope metadata is read tolerantly (unknown keys ignored within v1). An
|
||
unknown `envelopeVersion` keeps the report and drops the snapshot
|
||
(`snapshotRejected: 'unsupported_envelope'`). The snapshot, branding and
|
||
labels are validated strictly; any failure drops the snapshot with a
|
||
closed reason (`schema`, `caps`, `integrity`, `sheet_mismatch`) and keeps
|
||
the report.
|
||
|
||
| Archive | Produced by | Read as |
|
||
| -------------- | ------------------------- | ---------------------------------------------------------- |
|
||
| bare Report v2 | application before Step 5 | `origin 'v2'`, migrated, unresolved counts |
|
||
| bare Report v3 | Step 5 | `origin 'bare_v3'`, no archive |
|
||
| envelope v1 | Step 6 | `origin 'envelope_v1'`, archive kept or `snapshotRejected` |
|
||
| envelope v≥2 | future | report kept when its version is known, snapshot dropped |
|
||
|
||
Readers never drop a schema version: sheet document schema v2 stays readable
|
||
as long as envelope v1 archives exist, as Report v2 does today. A rollback to
|
||
a pre-Step-6 application reads an envelope as "no attachment".
|
||
|
||
### In the application
|
||
|
||
Two operations are distinct:
|
||
|
||
- **Re-read or re-export an archive**: `exportFromEnvelope(envelope, { locale, rasterizer, exportedAt })`
|
||
uses the archived snapshot, branding and labels. The imported archive is
|
||
kept in the session (`osteo:archive`), immutable.
|
||
- **Resume a bilan for editing**: the editor works on the loaded sheet. When
|
||
the archive's sheet differs from the loaded one, the report is **rebased**
|
||
on a working copy: zones and findings the loaded sheet cannot place are
|
||
flagged `unresolved: true`, their labels are filled from the archived
|
||
snapshot, and `report.sheet` becomes the loaded sheet. The import message
|
||
states the number of items not found; a degraded import is never shown as
|
||
a perfect restoration.
|
||
|
||
## Limits and sanitising
|
||
|
||
| Item | Cap |
|
||
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||
| PDF file | 50 MiB |
|
||
| attachment, raw and inflated | 16 MiB |
|
||
| snapshot assets | ≤ 4, ≤ 2 MiB each, ≤ 6 MiB total |
|
||
| snapshot document | ≤ 64 views, ≤ 1,024 annotations, ≤ 8,192 finding nodes, depth ≤ 8, names ≤ 512 chars |
|
||
| branding | PNG base64 ≤ 512 KiB, SVG ≤ 256 KiB, http(s) URLs ≤ 2,048 chars |
|
||
| branding URL | `http(s)://` then printable ASCII only, without `"` `'` `(` `)` `<` `>` `\`, no space or control character |
|
||
| branding PNG | valid signature, one IHDR (13 bytes) first with a colour type and bit depth the spec allows, compression and filter method 0, no interlace; each side 1 to 8,192 px, ≤ 16,777,216 px (4096²), expected raw scanline size ≤ 16 MiB; then only `PLTE` `tRNS` `IDAT` `gAMA` `cHRM` `sRGB` `sBIT` `pHYs` `bKGD`, no chunk past the buffer, an empty `IEND` as the last bytes; the IDAT stream inflated natively and yielding exactly the raw scanline size |
|
||
| report at export | ≤ 500 observations, ≤ 8 views per observation |
|
||
| SVG sanitiser | ≤ 20,000 elements, nesting ≤ 64, ≤ 256 `<use>`, blur `stdDeviation` ≤ 100 |
|
||
|
||
Every SVG that reaches the engine (sheet asset or branding) is untrusted: a
|
||
text containing `<!DOCTYPE` or `<!ENTITY` is refused; the parsed tree is
|
||
sanitised by allowlist (shapes, groups, gradients, the filter primitives the
|
||
drawings use, `clipPath`, `mask`, `switch`, `use`, `title`, `desc`; geometry,
|
||
presentation, gradient and filter attributes; `href` and `url()` only to a
|
||
`#fragment`; no `style` element, no `script`, `image`, `a`, `foreignObject`,
|
||
`text`, no event handler). An attribute value with a backslash is removed (a
|
||
CSS escape such as `u\72l(` would hide a reference), and a `style` or
|
||
presentation attribute may call no function other than `url(#…)`: `rgb(`,
|
||
`image-set(`, `expression(` and the like are removed with the attribute. The
|
||
reference drawings pass unchanged (tested).
|
||
The same caps are checked at export, so an export can never write a file its
|
||
own reader refuses.
|
||
|
||
The branding URL rule exists because a URL is written into every page's
|
||
link action: a literal string is not escaped (pdf-lib did not, the core
|
||
does not either), so the engine writes the URI as a hex string and the
|
||
schema refuses the characters that could close a literal. The export re-validates the branding and the labels it
|
||
receives (`ArchiveExportError('branding_invalid')` before any page is
|
||
drawn), even from an archive that passed the reader. The URI itself is the
|
||
hex form of the URL's ASCII bytes (`<68747470…>`), not UTF-16 with a BOM,
|
||
so readers that take `/URI` as raw bytes (poppler) follow it. A PNG logo is
|
||
checked before the decoder sees it (the core's decoder reproduces UPNG's
|
||
acceptance, the decoder pdf-lib used, so that an archived logo decodes to
|
||
the same planes); the gate exists because that decoder trusted its input: it sizes its bitmap from whatever IHDR claims, its `inflateRaw`
|
||
never checks the input bound (a truncated or crafted IDAT stream makes it
|
||
read past the buffer and never return), and it scans text chunks for a NUL
|
||
byte that may be missing. The gate therefore validates, in order: the
|
||
signature and a first 13-byte IHDR; a width and height of at least 1 under
|
||
the pixel budget; a colour type and bit depth the PNG spec allows (0: 1 to
|
||
16 bits, 2 and 4 and 6: 8 or 16, 3: 1 to 8), compression and filter method
|
||
0 and no interlace (the archived logos are never interlaced, so Adam7 pass
|
||
sizes are not computed); an expected raw scanline size,
|
||
`(ceil(width × bits per pixel / 8) + 1) × height`, of at most 16 MiB, the
|
||
attachment cap the native inflater enforces (4096² at 8-bit RGBA is 67 MB
|
||
of scanlines and is refused; a 512 px wide logo is under 1 MB). That cap
|
||
bounds what is inflated, not what the decoder then allocates: the decoder
|
||
still expands any accepted image to a 32-bit RGBA buffer, so the 4096² pixel
|
||
cap is what bounds memory (about 430 MB peak measured with four 4096² logos
|
||
on the TypeScript engine; real logos are 512 px wide; the core frees the
|
||
planes as soon as the image object is written). Then the chunk list, which must end on an empty `IEND`
|
||
that closes the file and may hold only the chunk types the decoder reads by
|
||
fixed offsets or by declared length (`PLTE`, `tRNS`, `IDAT`, `IEND`,
|
||
`gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs`, `bKGD`): a second IHDR, an APNG
|
||
chunk, a text or profile chunk (`tEXt`, `iTXt`, `zTXt`, `iCCP`), an unknown
|
||
type, a chunk length past the end of the buffer, a missing IEND or bytes
|
||
after it refuse the slot. Last, the IDAT payloads are concatenated and
|
||
inflated with the platform's `DecompressionStream`, bounded to the expected
|
||
raw size: a truncated, corrupt or oversized stream, or one that does not
|
||
yield exactly that size, refuses the slot. The allowlist alone would not
|
||
make the decoder safe; the native inflation is what keeps it from reading
|
||
an unbounded stream. The archive accepts only PNGs ending on IEND, even
|
||
though the decoder reads a PNG without one. A refused slot alone prints its
|
||
fallback text and the export reports `logo_failed`.
|
||
|
||
### Known limits
|
||
|
||
- pdf-lib is out of the import path (BLK-009): the browser (the core's
|
||
reader in a worker since R4.3) and the `report:compare` tool (the same
|
||
reader, natively, since R4.7) read the attachment by byte scan, so a
|
||
crafted object stream is never inflated (tested with a 64 MiB bomb:
|
||
milliseconds, no memory growth). No PDF library remains in the repository
|
||
since R4.8: the core writes the files, the tests read them back with a
|
||
reader of the writer's own format (`apps/app/tests/helpers/pdfFile.ts`).
|
||
What remains before external users
|
||
(`docs/development/PDF_IMPORT_HARDENING.md` §5): the server-side budget
|
||
decision for any future path that parses PDF objects, and read-side caps
|
||
on report collections.
|
||
- The scanner reads what is in clear: an attachment whose stream dictionary
|
||
is malformed beyond `/Length` and `endstream`, or an encrypted file, yields
|
||
no candidate (`no_attachment`), as the previous reader did in practice.
|
||
- `<use>` expansion is bounded by the rasterizer, not by the sanitiser,
|
||
which only counts the `<use>` elements (≤ 256). resvg (tests) refuses an
|
||
oversized tree; the production path is the browser rasterizer
|
||
(`apps/app/src/pdf/browserRasterizer.ts`, `Image` + canvas), whose
|
||
limits on `<use>` expansion are not measured.
|
||
- Report v3 collections are not capped on read; the export refuses above
|
||
500 observations and 8 views per observation.
|
||
|
||
## Checking an archive against its migration
|
||
|
||
`pnpm report:compare <file.pdf | folder>...` reads the attachment of one or
|
||
more PDFs and prints it next to its v3 reading on the bundled reference
|
||
sheet (the core's `report-compare` binary since R4.7, `RUST.md` §10.9: the
|
||
R4 reader and the R2 layered reader, the same lines as the TypeScript tool
|
||
it replaces, three failure lines reworded as declared there). It writes no
|
||
output file:
|
||
|
||
- an envelope prints a header line `archive v1 (snapshot kept)` or
|
||
`archive v1 (snapshot rejected: <reason>)`;
|
||
- a v2 archive PDF: every legacy zone (`data-anno-id`, side, vertebrae) and
|
||
every legacy finding path is shown with its canonical target and the
|
||
current label of that target (`same` when the path did not change);
|
||
- a v3 report: every zone and finding is checked against the sheet, and the
|
||
header says whether the report was written on the bundled sheet document;
|
||
- anything the sheet cannot place is printed as `!! UNRESOLVED` with the
|
||
reason: unknown legacy zone id, finding removed from the sheet by the
|
||
curation, finding that never belonged to the legacy catalog, or a v3
|
||
finding flagged `unresolved` in the report itself.
|
||
|
||
Options: `--summary` (one line per file, counts and unresolved counts),
|
||
`--full` (add the practitioner, animal and owner names and the real free
|
||
texts; by default only identifiers, catalog labels, snapshots of unresolved
|
||
items, counts and placeholders such as `comment (present)` are printed),
|
||
`--locale fr|en` (labels), `--diff a.pdf b.pdf` (compare the v3 content of
|
||
two PDFs: an archive and the PDF exported after importing it should print
|
||
`identical v3 content`; by default the texts are compared as placeholders,
|
||
so an edited text shows only when it appears or disappears; `--full`
|
||
compares the real texts and prints the differing lines). Every untrusted
|
||
string is printed with control characters escaped as `\uXXXX`. The tool
|
||
exits 2 when a file could not be read (no attachment, unknown version,
|
||
oversized) and prints the reason.
|