Files
faunosteo/docs/PDF-FORMAT.md
bchanot 4e0a6c04d6 docs(docs): R5.7 census, ledger, reproducibility, RUST.md 11.23
CLAUDE.md, WORKSPACE, packages/README, README, sheet model, SUPABASE, TEST_STRATEGY,
PDF-FORMAT and ANNOTATION-RENDERING follow the deletion; RUST.md 11.23 records the census,
counts, the historical TS replay procedure and hashes; CHANGELOG, TODO, R5.7 plan and
contract.
2026-10-25 04:00:15 +01:00

330 lines
24 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# PDF format and report archive
A Faunosteo PDF is both a document for the reader and an archive of the
bilan. The engine lives in the Rust core (`crates/core/src/pdf`, reached
through the wasm module in the browser and natively by the tools; the
TypeScript `packages/pdf` it reproduced byte for byte was deleted at R4.8,
`docs/development/RUST.md` §10) and renders from explicit inputs only: no
network, no browser DOM, no access to the current catalog. The attachment carries everything a future reader needs to
re-render the bilan when the anatomical sheet, the catalog, the labels or the
branding have changed since the export. Design record:
`docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md`.
## Pages
- A4 portrait, 595.28 × 841.89 pt, 36 pt side margins.
- One page per observation, then one conclusion page. Every page carries the
header (logo, practitioner, animal, owner, date) and the footer (site logo,
"powered by" credit, partner credit, page `n / N`).
- Observation page: eyebrow + title, the views of the observation drawn with
its zones highlighted (main view first, head view as a crop of the same
asset), then zones, findings grouped by finding zone, free findings,
severity, comment.
- Conclusion page: conclusion, recommendations, next appointment.
- Fonts: the standard Type1 fonts Helvetica, Helvetica-Bold (body) and
Times-Roman (headings), WinAnsi encoding, measured with the AFM widths
pdf-lib 1.17.1 used (`crates/core/src/pdf/write/afm_data.rs`). Every printed string goes through
`safeText`: control characters are stripped and any glyph WinAnsi cannot
encode becomes `?` (en and em dashes, accented letters, `•`, `·` are fine;
`⚠` is not, hence the `(!)` vet-referral marker).
- Drawings: the sheet SVG asset is read by the core's one SVG reader (`anatomy::svg`, since R3/R4; xmldom left at R5.7), sanitised (allowlist
below), framed on the view box, the marker layer is generated from the
document and the observation's zones are filled `#b85a3f` at 0.85 opacity
with a blur halo on the main view; every other marker is dropped. The SVG
is rasterised at 1200 px wide (aspect clamped to [0.2, 2.0]) by the
rasterizer the caller provides: `Image` + canvas in the browser, resvg in
Node tests. A failed raster leaves the page without that image and reports
a warning; the export never fails for a drawing.
- Document information: title, author, subject, dates = `exportedAt`, and
`Keywords: faunosteo-archive/1` so a PDF whose attachment was stripped by a
viewer is still recognised.
- Saved without object streams (`useObjectStreams: false`) since BLK-009: the
Filespec, its file name and the Info dictionary stay in clear, where the
byte-scan reader below can see them. Files produced earlier pack them into
an `/ObjStm`; their attachment stream is still in clear and still imports.
## Attachment: the report archive envelope (v1)
One native embedded file (`EmbeddedFiles`, `AFRelationship /Data`) named
`osteo-report.json`, MIME `application/json`, description
`Faunosteo report archive v1`, Flate-compressed (zlib level 6). Pretty-printed
JSON, camelCase keys (the sheet document inside keeps its own snake_case
schema):
```json
{
"format": "faunosteo-report-archive",
"envelopeVersion": 1,
"exportedAt": "2026-10-11T09:00:00.000Z",
"locale": "fr",
"report": { "version": 3, "...": "Report v3 exactly as exported" },
"sheetSnapshot": {
"status": "draft",
"ref": {
"sheetId": "…",
"versionId": null,
"documentSha256": "…",
"schemaVersion": 2,
"speciesId": "equine"
},
"document": {
"schema_version": 2,
"...": "lean, valid anatomical sheet document"
},
"assets": [{ "svgAssetId": "…", "sha256": "…", "svg": "<svg …>" }]
},
"branding": {
"siteUrl": "https://…",
"headerLogo": { "kind": "png", "base64": "…" },
"footerLogo": { "kind": "png", "base64": "…" },
"poweredBy": {
"logo": { "kind": "png", "base64": "…" },
"url": "https://…",
"fallbackText": "…"
},
"partner": {
"logo": { "kind": "svg", "svg": "<svg …>" },
"url": "https://…",
"fallbackText": "…"
}
},
"labels": {
"fr": { "pdf.section.zones": "ZONES", "...": "25 keys" },
"en": { "...": "" }
}
}
```
### What the snapshot holds
The snapshot is the sheet the bilan was built on, reduced to what the bilan
needs, as a **valid** anatomical sheet document (schema v2, same integrity
rules as the catalog):
| Part | Content |
| ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ref` | sheet id, version id, sha256 of the full catalog document (provenance pointer), schema version, species |
| `document.views`, `annotations`, `categories`, `vertebral_config` | all of them: geometry, colours, names, anatomical relations (`finding_zone_id`, `vertebral_region_code`, `opens_view_id`, laterality) |
| `document.finding_zones` | every zone with its name; for the zones the report's findings use (resolved or `unresolved`), the categories pruned to the referenced paths with the name and vet flag of every node on the path; other zones keep `categories: []` |
| `document.references` | emptied (bibliography) |
| `document.name`, `description`, `scope_disclaimer`, `attribution` | kept |
| `assets` | the SVG text of every asset a view reaches, with the catalog's content hash when known |
`branding` is the exact set of images, URLs and credit texts used at export;
the application builds it from logos bundled at build time
(`apps/app/src/assets/brand`), so the export makes no network request.
`labels` is the engine dictionary in both languages. A re-export of the
archive uses these three blocks and nothing from the current application, so
a new logo, a reworded heading or a curated sheet never changes how an old
bilan is reproduced.
Invariant: `report.sheet.documentSha256 === sheetSnapshot.ref.documentSha256`
(and the sheet ids match). The export refuses to write a file that breaks it.
### Sizes
Measured on the demo report (6 observations, both views, equine draft):
| Branding | Stored in the PDF (Flate) | Raw JSON |
| ----------------------------------------- | ------------------------- | -------- |
| no logos | 55 KB | 189 KB |
| full branding (3 PNG logos + partner SVG) | 136 KB | 306 KB |
The drawing is the largest part (47 KB stored); the archived PNG logos add
about 80 KB stored. The 250 KB target applies to the stored size.
## Reading an archive
`readReportArchive(pdfBytes, migrationMap)` reads in layers so the report is
never lost because of its wrapper:
1. The file must weigh at most 50 MiB and carry `%PDF-` within its first
KiB (`invalid_pdf`). Nothing else of the PDF syntax is required.
2. The attachment is located by a byte scan, never by parsing the PDF object
graph (BLK-009): every `/Type /EmbeddedFile` stream dictionary in clear
(a stream object can never sit inside an object stream) is a candidate,
with its direct `/Length` or, failing that, the distance to `endstream`;
a `/Filespec` dictionary in clear whose `/EF /F` points at the stream
gives it a name. Candidates named `osteo-report*` come first, then file
order; Flate data is fed to the platform inflater in 16 KiB slices, so
memory stays bounded in browsers too, under a 16 MiB cap (cancelled
beyond it); at most 16 MiB of candidate content is retained and parsed in
all (a cancelled inflation is not charged, so up to 32 oversized streams
may each cost one capped inflation of CPU, bounded, no memory retained),
a candidate not opening on `{` or nesting brackets deeper than 64 levels
is skipped unparsed, and the first one whose text is a JSON object with a
numeric `envelopeVersion` or `version` wins. Object streams,
cross-reference streams and page contents are never inflated.
No candidate: `attachment_stripped` when a `/Keywords` string in clear
contains `faunosteo-archive`, else `no_attachment`. A stamped file saved
with object streams (before BLK-009) hides its stamp and reads as
`no_attachment`.
3. An attachment whose JSON escapes a lone UTF-16 surrogate is refused
whole (`lone_surrogate`, "malformed text, import refused, file not
modified"): the core cannot hold it without altering it and never
rewrites it (L1, `docs/development/RUST.md` §8.7).
The report is located (`data.report` inside an envelope, `data` for a
bare attachment) and read with the report reader: v3 as is, v2 migrated
through the sheet's `report-migration-map.json` (items the map cannot
place are kept with their labels and flagged `unresolved: true`), any
other version → `incompatible_version` with the version found.
4. Envelope metadata is read tolerantly (unknown keys ignored within v1). An
unknown `envelopeVersion` keeps the report and drops the snapshot
(`snapshotRejected: 'unsupported_envelope'`). The snapshot, branding and
labels are validated strictly; any failure drops the snapshot with a
closed reason (`schema`, `caps`, `integrity`, `sheet_mismatch`) and keeps
the report.
| Archive | Produced by | Read as |
| -------------- | ------------------------- | ---------------------------------------------------------- |
| bare Report v2 | application before Step 5 | `origin 'v2'`, migrated, unresolved counts |
| bare Report v3 | Step 5 | `origin 'bare_v3'`, no archive |
| envelope v1 | Step 6 | `origin 'envelope_v1'`, archive kept or `snapshotRejected` |
| envelope v≥2 | future | report kept when its version is known, snapshot dropped |
Readers never drop a schema version: sheet document schema v2 stays readable
as long as envelope v1 archives exist, as Report v2 does today. A rollback to
a pre-Step-6 application reads an envelope as "no attachment".
### In the application
Two operations are distinct:
- **Re-read or re-export an archive**: `exportFromEnvelope(envelope, { locale, rasterizer, exportedAt })`
uses the archived snapshot, branding and labels. The imported archive is
kept in the session (`osteo:archive`), immutable.
- **Resume a bilan for editing**: the editor works on the loaded sheet. When
the archive's sheet differs from the loaded one, the report is **rebased**
on a working copy: zones and findings the loaded sheet cannot place are
flagged `unresolved: true`, their labels are filled from the archived
snapshot, and `report.sheet` becomes the loaded sheet. The import message
states the number of items not found; a degraded import is never shown as
a perfect restoration.
## Limits and sanitising
| Item | Cap |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| PDF file | 50 MiB |
| attachment, raw and inflated | 16 MiB |
| snapshot assets | ≤ 4, ≤ 2 MiB each, ≤ 6 MiB total |
| snapshot document | ≤ 64 views, ≤ 1,024 annotations, ≤ 8,192 finding nodes, depth ≤ 8, names ≤ 512 chars |
| branding | PNG base64 ≤ 512 KiB, SVG ≤ 256 KiB, http(s) URLs ≤ 2,048 chars |
| branding URL | `http(s)://` then printable ASCII only, without `"` `'` `(` `)` `<` `>` `\`, no space or control character |
| branding PNG | valid signature, one IHDR (13 bytes) first with a colour type and bit depth the spec allows, compression and filter method 0, no interlace; each side 1 to 8,192 px, ≤ 16,777,216 px (4096²), expected raw scanline size ≤ 16 MiB; then only `PLTE` `tRNS` `IDAT` `gAMA` `cHRM` `sRGB` `sBIT` `pHYs` `bKGD`, no chunk past the buffer, an empty `IEND` as the last bytes; the IDAT stream inflated natively and yielding exactly the raw scanline size |
| report at export | ≤ 500 observations, ≤ 8 views per observation |
| SVG sanitiser | ≤ 20,000 elements, nesting ≤ 64, ≤ 256 `<use>`, blur `stdDeviation` ≤ 100 |
Every SVG that reaches the engine (sheet asset or branding) is untrusted: a
text containing `<!DOCTYPE` or `<!ENTITY` is refused; the parsed tree is
sanitised by allowlist (shapes, groups, gradients, the filter primitives the
drawings use, `clipPath`, `mask`, `switch`, `use`, `title`, `desc`; geometry,
presentation, gradient and filter attributes; `href` and `url()` only to a
`#fragment`; no `style` element, no `script`, `image`, `a`, `foreignObject`,
`text`, no event handler). An attribute value with a backslash is removed (a
CSS escape such as `u\72l(` would hide a reference), and a `style` or
presentation attribute may call no function other than `url(#…)`: `rgb(`,
`image-set(`, `expression(` and the like are removed with the attribute. The
reference drawings pass unchanged (tested).
The same caps are checked at export, so an export can never write a file its
own reader refuses.
The branding URL rule exists because a URL is written into every page's
link action: a literal string is not escaped (pdf-lib did not, the core
does not either), so the engine writes the URI as a hex string and the
schema refuses the characters that could close a literal. The export re-validates the branding and the labels it
receives (`ArchiveExportError('branding_invalid')` before any page is
drawn), even from an archive that passed the reader. The URI itself is the
hex form of the URL's ASCII bytes (`<68747470…>`), not UTF-16 with a BOM,
so readers that take `/URI` as raw bytes (poppler) follow it. A PNG logo is
checked before the decoder sees it (the core's decoder reproduces UPNG's
acceptance, the decoder pdf-lib used, so that an archived logo decodes to
the same planes); the gate exists because that decoder trusted its input: it sizes its bitmap from whatever IHDR claims, its `inflateRaw`
never checks the input bound (a truncated or crafted IDAT stream makes it
read past the buffer and never return), and it scans text chunks for a NUL
byte that may be missing. The gate therefore validates, in order: the
signature and a first 13-byte IHDR; a width and height of at least 1 under
the pixel budget; a colour type and bit depth the PNG spec allows (0: 1 to
16 bits, 2 and 4 and 6: 8 or 16, 3: 1 to 8), compression and filter method
0 and no interlace (the archived logos are never interlaced, so Adam7 pass
sizes are not computed); an expected raw scanline size,
`(ceil(width × bits per pixel / 8) + 1) × height`, of at most 16 MiB, the
attachment cap the native inflater enforces (4096² at 8-bit RGBA is 67 MB
of scanlines and is refused; a 512 px wide logo is under 1 MB). That cap
bounds what is inflated, not what the decoder then allocates: the decoder
still expands any accepted image to a 32-bit RGBA buffer, so the 4096² pixel
cap is what bounds memory (about 430 MB peak measured with four 4096² logos
on the TypeScript engine; real logos are 512 px wide; the core frees the
planes as soon as the image object is written). Then the chunk list, which must end on an empty `IEND`
that closes the file and may hold only the chunk types the decoder reads by
fixed offsets or by declared length (`PLTE`, `tRNS`, `IDAT`, `IEND`,
`gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs`, `bKGD`): a second IHDR, an APNG
chunk, a text or profile chunk (`tEXt`, `iTXt`, `zTXt`, `iCCP`), an unknown
type, a chunk length past the end of the buffer, a missing IEND or bytes
after it refuse the slot. Last, the IDAT payloads are concatenated and
inflated with the platform's `DecompressionStream`, bounded to the expected
raw size: a truncated, corrupt or oversized stream, or one that does not
yield exactly that size, refuses the slot. The allowlist alone would not
make the decoder safe; the native inflation is what keeps it from reading
an unbounded stream. The archive accepts only PNGs ending on IEND, even
though the decoder reads a PNG without one. A refused slot alone prints its
fallback text and the export reports `logo_failed`.
### Known limits
- pdf-lib is out of the import path (BLK-009): the browser (the core's
reader in a worker since R4.3) and the `report:compare` tool (the same
reader, natively, since R4.7) read the attachment by byte scan, so a
crafted object stream is never inflated (tested with a 64 MiB bomb:
milliseconds, no memory growth). No PDF library remains in the repository
since R4.8: the core writes the files, the tests read them back with a
reader of the writer's own format (`apps/app/tests/helpers/pdfFile.ts`).
What remains before external users
(`docs/development/PDF_IMPORT_HARDENING.md` §5): the server-side budget
decision for any future path that parses PDF objects, and read-side caps
on report collections.
- The scanner reads what is in clear: an attachment whose stream dictionary
is malformed beyond `/Length` and `endstream`, or an encrypted file, yields
no candidate (`no_attachment`), as the previous reader did in practice.
- `<use>` expansion is bounded by the rasterizer, not by the sanitiser,
which only counts the `<use>` elements (≤ 256). resvg (tests) refuses an
oversized tree; the production path is the browser rasterizer
(`apps/app/src/pdf/browserRasterizer.ts`, `Image` + canvas), whose
limits on `<use>` expansion are not measured.
- Report v3 collections are not capped on read; the export refuses above
500 observations and 8 views per observation.
## Checking an archive against its migration
`pnpm report:compare <file.pdf | folder>...` reads the attachment of one or
more PDFs and prints it next to its v3 reading on the bundled reference
sheet (the core's `report-compare` binary since R4.7, `RUST.md` §10.9: the
R4 reader and the R2 layered reader, the same lines as the TypeScript tool
it replaces, three failure lines reworded as declared there). It writes no
output file:
- an envelope prints a header line `archive v1 (snapshot kept)` or
`archive v1 (snapshot rejected: <reason>)`;
- a v2 archive PDF: every legacy zone (`data-anno-id`, side, vertebrae) and
every legacy finding path is shown with its canonical target and the
current label of that target (`same` when the path did not change);
- a v3 report: every zone and finding is checked against the sheet, and the
header says whether the report was written on the bundled sheet document;
- anything the sheet cannot place is printed as `!! UNRESOLVED` with the
reason: unknown legacy zone id, finding removed from the sheet by the
curation, finding that never belonged to the legacy catalog, or a v3
finding flagged `unresolved` in the report itself.
Options: `--summary` (one line per file, counts and unresolved counts),
`--full` (add the practitioner, animal and owner names and the real free
texts; by default only identifiers, catalog labels, snapshots of unresolved
items, counts and placeholders such as `comment (present)` are printed),
`--locale fr|en` (labels), `--diff a.pdf b.pdf` (compare the v3 content of
two PDFs: an archive and the PDF exported after importing it should print
`identical v3 content`; by default the texts are compared as placeholders,
so an edited text shows only when it appears or disappears; `--full`
compares the real texts and prints the differing lines). Every untrusted
string is printed with control characters escaped as `\uXXXX`. The tool
exits 2 when a file could not be read (no attachment, unknown version,
oversized) and prints the reason.