126 lines
11 KiB
Markdown
126 lines
11 KiB
Markdown
# PDF import hardening — BLK-009 analysis
|
|
|
|
Status 2026-10-11: analysis requested by the user after Step 6. §4 landed the
|
|
same day on `feature/step-06-pdf-archives` (options A + B, see §4 for the
|
|
commit); §5 lists what must land before the import is opened to external
|
|
users.
|
|
|
|
## 1. The problem
|
|
|
|
`readReportArchive` loads the whole PDF with pdf-lib (`PDFDocument.load`)
|
|
before looking for the attachment. pdf-lib's parser inflates every object
|
|
stream (`/ObjStm`) and cross-reference stream eagerly with its own, uncapped
|
|
inflater. Measured by the security audit: a 408 KB PDF expanded to 400 MB in
|
|
1.4 s (468 MB RSS); a 131 KB PDF to 128 MB. The ratio is about 1000:1, so the
|
|
50 MiB file cap still allows tens of gigabytes of expansion. Our 16 MiB caps
|
|
protect the attachment stream only, which the reader inflates itself.
|
|
|
|
Effect: a crafted PDF freezes the practitioner's tab or kills it (out of
|
|
memory). No code execution, no data access. A file size limit cannot help: the
|
|
amplification lives inside the compression.
|
|
|
|
Why `/ObjStm` cannot simply be refused: pdf-lib's `save()` writes object
|
|
streams by default (`useObjectStreams: true`), so every Faunosteo PDF
|
|
produced so far carries them, and refusing them would refuse the archive.
|
|
|
|
## 2. What the reader actually needs from the file
|
|
|
|
One embedded file stream, found through its `/Filespec`, and the document
|
|
keyword stamp. A PDF stream object can never live inside an object stream:
|
|
the `EmbeddedFile` dictionary and its data are always in clear text in the
|
|
file (`<< /Type /EmbeddedFile /Filter /FlateDecode /Length N … >> stream …
|
|
endstream`). Only the small dictionary objects (Filespec, Info, catalog) may
|
|
be packed into object streams.
|
|
|
|
So the attachment can be located by scanning the bytes, without parsing the
|
|
PDF object graph at all.
|
|
|
|
## 3. Options
|
|
|
|
| Option | Protection | Cost | Verdict |
|
|
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
|
|
| A. Byte-scan reader: find `/Type /EmbeddedFile` streams, read `/Length` (direct integer, as pdf-lib writes it; fall back to `endstream` search), inflate with the existing capped `inflateFlate`, parse JSON; no `PDFDocument.load` on import | removes pdf-lib from the import path entirely: the only inflation is our capped one | one module (~120 lines) + tests on every fixture (v2, bare v3, envelope, object streams on/off, several streams, foreign PDF) | **recommended, can land now** |
|
|
| B. Export with `useObjectStreams: false` | the Filespec name, the Info keywords and the Filespec→EF link become visible to a byte scan, so the filename-first rule and the `attachment_stripped` detection keep working without the object graph; files a few KB larger | one save option; older PDFs keep their object streams, which A tolerates | **recommended with A** |
|
|
| C. Load in a Web Worker with a timeout and a memory budget | contains the damage (the tab survives, the worker dies) | worker bundling, message protocol, no memory cap API in browsers (only termination on timeout) | defence in depth, later |
|
|
| D. Pre-scan declared `/Length` of `/ObjStm` streams and refuse above a sum | partial: declared lengths are the compressed sizes, the expansion is unknown | small | insufficient alone |
|
|
| E. Replace pdf-lib's parser (fork `@cantoo/pdf-lib`, or `pdf.js` for reading) | maintained parser; pdf.js has its own limits | dependency change, larger bundle | evaluate for Step 14 (server-side import) |
|
|
|
|
With A, the import path reads bytes, finds at most a few streams, inflates one
|
|
under 16 MiB, parses JSON under the existing schema caps. pdf-lib stays on
|
|
the export side (trusted inputs) and in the `report:compare` CLI (operator
|
|
tool, Node, acceptable risk but same fix applies).
|
|
|
|
Edge cases A must handle: several `EmbeddedFile` streams (pick the one whose
|
|
Filespec name starts with `osteo-report` when visible, else the first whose
|
|
inflated content parses as a report or an envelope); `/Length` as an indirect
|
|
reference (pdf-lib writes direct lengths; other producers may not: fall back
|
|
to scanning for `endstream`); `\r\n` versus `\n` after `stream`; encrypted
|
|
PDFs (today loaded with `ignoreEncryption`: the attachment stream of an
|
|
encrypted file is encrypted too and would fail to inflate → `no_attachment`,
|
|
same outcome as today in practice); the 50 MiB file cap stays as the scan
|
|
bound.
|
|
|
|
## 4. Done 2026-10-11 (feature/step-06-pdf-archives, commit: see commit 60ee712)
|
|
|
|
1. Option A in `packages/pdf/src/attachment/scan.ts`
|
|
(`findEmbeddedFiles`, `readAttachmentBytes`, `hasArchiveStamp`), used by
|
|
`readReportArchive`; `readAttachedDocument(bytes)` takes the file bytes
|
|
and the `report:compare` CLI runs on the scanner too. No
|
|
`PDFDocument.load` remains under `packages/pdf/src` or `apps/app/scripts`.
|
|
2. Option B on export (`pdfDoc.save({ useObjectStreams: false })`): no visible
|
|
change; goldens unchanged (they record drawn text, images, links); the
|
|
envelope roundtrip tests prove the reader finds the attachment in both
|
|
layouts.
|
|
3. Tests (`packages/pdf/tests/scan.test.ts`): the three frozen fixtures (object
|
|
streams) and a fresh export; two embedded files with a decoy first;
|
|
indirect and wrong `/Length`; `\r\n` after `stream`; `/Type/EmbeddedFile`
|
|
without a space; hex and literal names and keywords; garbage inputs and
|
|
the 32-candidate cap; a 64 MiB object-stream bomb next to the real
|
|
attachment (report in milliseconds, RSS flat) and an `EmbeddedFile` bomb
|
|
(inflation cancelled at 16 MiB, `no_attachment`).
|
|
4. Docs: `docs/PDF-FORMAT.md` "Reading an archive" and "Known limits".
|
|
|
|
Known trade-off: a stamped PDF saved before this change (object streams)
|
|
hides its `/Keywords`, so a stripped attachment on such a file reads as
|
|
`no_attachment` instead of `attachment_stripped`.
|
|
|
|
## 5. Prerequisites before external users (Step 7+ sync, Step 14 worker)
|
|
|
|
- Option A merged and the CLI on the scanner; no `PDFDocument.load` on any
|
|
untrusted input in the browser.
|
|
- Done at R4 (2026-10-18, `docs/development/RUST.md` §10.3 to §10.5): the
|
|
reader runs in Rust inside a disposable worker; the inflate bound and the
|
|
hostile classes were measured in Chromium 156, Firefox 157 and WebKit 27.2
|
|
(headless): a 16 MiB + 1 bomb costs 63 MiB of linear memory and is
|
|
refused, 32 bombs 68, a 16 MiB token-dense document 16 MiB under the
|
|
400,000-value cap (784 MiB with it off), the 1 GiB module ceiling traps;
|
|
the application build answers the same outcome and message as the
|
|
TypeScript reader on 666 of 678 (file, engine) pairs, the other 12 being
|
|
the four declared hostile files. Device run (J1, `R4_J1.md`): the owner
|
|
reports the functional checks executed on 2026-10-18 with no failure;
|
|
the device list, versions, scenario coverage, timings and memory
|
|
observations are recorded in `R4_J1_RESULTS.md` as they are confirmed.
|
|
The gate below stays open: imports are not offered to external users.
|
|
- Done: the candidate CPU/byte budget on the scanner: one 16 MiB budget of
|
|
retained and parsed content shared by every candidate (a cancelled
|
|
inflation is not charged, so up to 32 oversized streams may each cost one
|
|
capped inflation of CPU, bounded, no memory retained), a monotonic
|
|
`endstream` search over candidates sorted by data offset and bounded to
|
|
the cap, and no JSON parse for a candidate not opening on `{` or nesting
|
|
brackets deeper than 64 levels (linear byte pre-scan, strings skipped).
|
|
- Option C or a server-side import (Step 14) with a hard memory and time
|
|
budget per file, for any path that still parses PDF objects.
|
|
- Dependency decision on pdf-lib (unmaintained since 2021): keep it for
|
|
generation only, or move to the maintained fork; recorded as a Step 14
|
|
question.
|
|
- Report collection caps on read (observations, zones, findings per
|
|
observation) so a hostile report cannot freeze the editor before the
|
|
export's own caps refuse it.
|
|
- The remaining low findings of the Step 6 audits: `<use>` nesting budget in
|
|
the sanitiser (today bounded by the rasterizer on the PDF path; since R3 the
|
|
Rust reader refuses reference cycles and expansions over 50,000 elements,
|
|
`docs/development/RUST.md` §9.2, and the engine gets it when R4 switches it to
|
|
that reader: still open for the PDF path), the decoder memory
|
|
envelope of PNG logos (bounded by the 4096² cap, about 430 MB worst case
|
|
for four logos).
|