fix(pdf): bounded attachment extraction by byte scan — no pdf-lib parse on import, export without object streams (BLK-009)

This commit is contained in:
bchanot committed 2026-10-11 16:42:45 +02:00
1 parent 62b8a43e09
commit 60ee71208f
14 files changed
+1054 -198

No files matched your search

+86 -45
View File
@@ -35,6 +35,10 @@ branding have changed since the export. Design record:
- Document information: title, author, subject, dates = `exportedAt`, and
`Keywords: faunosteo-archive/1` so a PDF whose attachment was stripped by a
viewer is still recognised.
- Saved without object streams (`useObjectStreams: false`) since BLK-009: the
Filespec, its file name and the Info dictionary stay in clear, where the
byte-scan reader below can see them. Files produced earlier pack them into
an `/ObjStm`; their attachment stream is still in clear and still imports.
## Attachment: the report archive envelope (v1)
@@ -53,18 +57,38 @@ schema):
"report": { "version": 3, "...": "Report v3 exactly as exported" },
"sheetSnapshot": {
"status": "draft",
"ref": { "sheetId": "…", "versionId": null, "documentSha256": "…", "schemaVersion": 2, "speciesId": "equine" },
"document": { "schema_version": 2, "...": "lean, valid anatomical sheet document" },
"assets": [ { "svgAssetId": "…", "sha256": "…", "svg": "<svg …>" } ]
"ref": {
"sheetId": "…",
"versionId": null,
"documentSha256": "…",
"schemaVersion": 2,
"speciesId": "equine"
},
"document": {
"schema_version": 2,
"...": "lean, valid anatomical sheet document"
},
"assets": [{ "svgAssetId": "…", "sha256": "…", "svg": "<svg …>" }]
},
"branding": {
"siteUrl": "https://…",
"headerLogo": { "kind": "png", "base64": "…" },
"footerLogo": { "kind": "png", "base64": "…" },
"poweredBy": { "logo": { "kind": "png", "base64": "…" }, "url": "https://…", "fallbackText": "…" },
"partner": { "logo": { "kind": "svg", "svg": "<svg …>" }, "url": "https://…", "fallbackText": "…" }
"poweredBy": {
"logo": { "kind": "png", "base64": "…" },
"url": "https://…",
"fallbackText": "…"
},
"partner": {
"logo": { "kind": "svg", "svg": "<svg …>" },
"url": "https://…",
"fallbackText": "…"
}
},
"labels": { "fr": { "pdf.section.zones": "ZONES", "...": "25 keys" }, "en": { "...": "" } }
"labels": {
"fr": { "pdf.section.zones": "ZONES", "...": "25 keys" },
"en": { "...": "" }
}
}
```
@@ -74,14 +98,14 @@ The snapshot is the sheet the bilan was built on, reduced to what the bilan
needs, as a **valid** anatomical sheet document (schema v2, same integrity
rules as the catalog):
| Part | Content |
|---|---|
| `ref` | sheet id, version id, sha256 of the full catalog document (provenance pointer), schema version, species |
| `document.views`, `annotations`, `categories`, `vertebral_config` | all of them: geometry, colours, names, anatomical relations (`finding_zone_id`, `vertebral_region_code`, `opens_view_id`, laterality) |
| `document.finding_zones` | every zone with its name; for the zones the report's findings use (resolved or `unresolved`), the categories pruned to the referenced paths with the name and vet flag of every node on the path; other zones keep `categories: []` |
| `document.references` | emptied (bibliography) |
| `document.name`, `description`, `scope_disclaimer`, `attribution` | kept |
| `assets` | the SVG text of every asset a view reaches, with the catalog's content hash when known |
| Part | Content |
| ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ref` | sheet id, version id, sha256 of the full catalog document (provenance pointer), schema version, species |
| `document.views`, `annotations`, `categories`, `vertebral_config` | all of them: geometry, colours, names, anatomical relations (`finding_zone_id`, `vertebral_region_code`, `opens_view_id`, laterality) |
| `document.finding_zones` | every zone with its name; for the zones the report's findings use (resolved or `unresolved`), the categories pruned to the referenced paths with the name and vet flag of every node on the path; other zones keep `categories: []` |
| `document.references` | emptied (bibliography) |
| `document.name`, `description`, `scope_disclaimer`, `attribution` | kept |
| `assets` | the SVG text of every asset a view reaches, with the catalog's content hash when known |
`branding` is the exact set of images, URLs and credit texts used at export;
the application builds it from logos bundled at build time
@@ -98,10 +122,10 @@ Invariant: `report.sheet.documentSha256 === sheetSnapshot.ref.documentSha256`
Measured on the demo report (6 observations, both views, equine draft):
| Branding | Stored in the PDF (Flate) | Raw JSON |
|---|---|---|
| no logos | 55 KB | 189 KB |
| full branding (3 PNG logos + partner SVG) | 136 KB | 306 KB |
| Branding | Stored in the PDF (Flate) | Raw JSON |
| ----------------------------------------- | ------------------------- | -------- |
| no logos | 55 KB | 189 KB |
| full branding (3 PNG logos + partner SVG) | 136 KB | 306 KB |
The drawing is the largest part (47 KB stored); the archived PNG logos add
about 80 KB stored. The 250 KB target applies to the stored size.
@@ -111,11 +135,22 @@ about 80 KB stored. The 250 KB target applies to the stored size.
`readReportArchive(pdfBytes, migrationMap)` reads in layers so the report is
never lost because of its wrapper:
1. The PDF must load (`invalid_pdf`, also above 50 MiB).
2. The attachment named `osteo-report*` is taken first, else the first
embedded file; Flate streams are inflated under a 16 MiB cap. No
attachment: `attachment_stripped` when the PDF carries the Faunosteo
keyword, else `no_attachment`.
1. The file must weigh at most 50 MiB and carry `%PDF-` within its first
KiB (`invalid_pdf`). Nothing else of the PDF syntax is required.
2. The attachment is located by a byte scan, never by parsing the PDF object
graph (BLK-009): every `/Type /EmbeddedFile` stream dictionary in clear
(a stream object can never sit inside an object stream) is a candidate,
with its direct `/Length` or, failing that, the distance to `endstream`;
a `/Filespec` dictionary in clear whose `/EF /F` points at the stream
gives it a name. Candidates named `osteo-report*` come first, then file
order; Flate data is inflated by the platform inflater under a 16 MiB cap
(cancelled beyond it), and the first candidate whose text is a JSON
object with a numeric `envelopeVersion` or `version` wins. Object
streams, cross-reference streams and page contents are never inflated.
No candidate: `attachment_stripped` when a `/Keywords` string in clear
contains `faunosteo-archive`, else `no_attachment`. A stamped file saved
with object streams (before BLK-009) hides its stamp and reads as
`no_attachment`.
3. The report is located (`data.report` inside an envelope, `data` for a
bare attachment) and read with the report reader: v3 as is, v2 migrated
through the sheet's `report-migration-map.json` (items the map cannot
@@ -128,12 +163,12 @@ never lost because of its wrapper:
closed reason (`schema`, `caps`, `integrity`, `sheet_mismatch`) and keeps
the report.
| Archive | Produced by | Read as |
|---|---|---|
| bare Report v2 | application before Step 5 | `origin 'v2'`, migrated, unresolved counts |
| bare Report v3 | Step 5 | `origin 'bare_v3'`, no archive |
| envelope v1 | Step 6 | `origin 'envelope_v1'`, archive kept or `snapshotRejected` |
| envelope v≥2 | future | report kept when its version is known, snapshot dropped |
| Archive | Produced by | Read as |
| -------------- | ------------------------- | ---------------------------------------------------------- |
| bare Report v2 | application before Step 5 | `origin 'v2'`, migrated, unresolved counts |
| bare Report v3 | Step 5 | `origin 'bare_v3'`, no archive |
| envelope v1 | Step 6 | `origin 'envelope_v1'`, archive kept or `snapshotRejected` |
| envelope v≥2 | future | report kept when its version is known, snapshot dropped |
Readers never drop a schema version: sheet document schema v2 stays readable
as long as envelope v1 archives exist, as Report v2 does today. A rollback to
@@ -156,17 +191,17 @@ Two operations are distinct:
## Limits and sanitising
| Item | Cap |
|---|---|
| PDF file | 50 MiB |
| attachment, raw and inflated | 16 MiB |
| snapshot assets | ≤ 4, ≤ 2 MiB each, ≤ 6 MiB total |
| snapshot document | ≤ 64 views, ≤ 1,024 annotations, ≤ 8,192 finding nodes, depth ≤ 8, names ≤ 512 chars |
| branding | PNG base64 ≤ 512 KiB, SVG ≤ 256 KiB, http(s) URLs ≤ 2,048 chars |
| branding URL | `http(s)://` then printable ASCII only, without `"` `'` `(` `)` `<` `>` `\`, no space or control character |
| branding PNG | valid signature, one IHDR (13 bytes) first with a colour type and bit depth the spec allows, compression and filter method 0, no interlace; each side 1 to 8,192 px, ≤ 16,777,216 px (4096²), expected raw scanline size ≤ 16 MiB; then only `PLTE` `tRNS` `IDAT` `gAMA` `cHRM` `sRGB` `sBIT` `pHYs` `bKGD`, no chunk past the buffer, an empty `IEND` as the last bytes; the IDAT stream inflated natively and yielding exactly the raw scanline size |
| report at export | ≤ 500 observations, ≤ 8 views per observation |
| SVG sanitiser | ≤ 20,000 elements, nesting ≤ 64, ≤ 256 `<use>`, blur `stdDeviation` ≤ 100 |
| Item | Cap |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| PDF file | 50 MiB |
| attachment, raw and inflated | 16 MiB |
| snapshot assets | ≤ 4, ≤ 2 MiB each, ≤ 6 MiB total |
| snapshot document | ≤ 64 views, ≤ 1,024 annotations, ≤ 8,192 finding nodes, depth ≤ 8, names ≤ 512 chars |
| branding | PNG base64 ≤ 512 KiB, SVG ≤ 256 KiB, http(s) URLs ≤ 2,048 chars |
| branding URL | `http(s)://` then printable ASCII only, without `"` `'` `(` `)` `<` `>` `\`, no space or control character |
| branding PNG | valid signature, one IHDR (13 bytes) first with a colour type and bit depth the spec allows, compression and filter method 0, no interlace; each side 1 to 8,192 px, ≤ 16,777,216 px (4096²), expected raw scanline size ≤ 16 MiB; then only `PLTE` `tRNS` `IDAT` `gAMA` `cHRM` `sRGB` `sBIT` `pHYs` `bKGD`, no chunk past the buffer, an empty `IEND` as the last bytes; the IDAT stream inflated natively and yielding exactly the raw scanline size |
| report at export | ≤ 500 observations, ≤ 8 views per observation |
| SVG sanitiser | ≤ 20,000 elements, nesting ≤ 64, ≤ 256 `<use>`, blur `stdDeviation` ≤ 100 |
Every SVG that reaches the engine (sheet asset or branding) is untrusted: a
text containing `<!DOCTYPE` or `<!ENTITY` is refused; the parsed tree is
@@ -223,11 +258,17 @@ fallback text and the export reports `logo_failed`.
### Known limits
- pdf-lib inflates PDF object streams without a cap when loading: a crafted
PDF of a few hundred KB can expand to hundreds of MB in the tab before the
attachment is even looked at. The 50 MiB file cap bounds the input, not
the expansion. Mitigation planned: parse in a Worker, or locate the
attachment without a full load.
- pdf-lib is out of the import path (BLK-009): the browser and the
`report:compare` CLI read the attachment by byte scan, so a crafted object
stream is never inflated (tested with a 64 MiB bomb: milliseconds, no
memory growth). pdf-lib stays on the export side, on trusted inputs, and
in tests that inspect exported files. What remains before external users
(`docs/development/PDF_IMPORT_HARDENING.md` §5): a Worker or server-side
budget for any path that still parses PDF objects, the pdf-lib dependency
decision, and read-side caps on report collections.
- The scanner reads what is in clear: an attachment whose stream dictionary
is malformed beyond `/Length` and `endstream`, or an encrypted file, yields
no candidate (`no_attachment`), as the previous reader did in practice.
- `<use>` expansion is bounded by the rasterizer, not by the sanitiser,
which only counts the `<use>` elements (≤ 256). resvg (tests) refuses an
oversized tree; the production path is the browser rasterizer