docs: PDF import hardening analysis (BLK-009) — byte-scan reader and object-stream-free export now, worker and parser replacement before external users

This commit is contained in:
bchanot committed 2026-10-11 16:12:22 +02:00
1 parent 293a6e0b36
commit 5ed4a4b620
3 files changed
+96 -1

No files matched your search

+1 -1
View File
@@ -80,4 +80,4 @@ Status: open (no real data yet; treat before Step 7+ persistence of user reports
## BLK-009 | 2026-10-11 | pdf-lib inflates PDF object streams without cap at `PDFDocument.load` (import path)
Security audit Step 6 (MEDIUM, pre-existing since Step 1 import): `PDFDocument.load` decodes every `/ObjStm` and xref stream eagerly with its own uncapped FlateStream; measured 408 KB PDF → 400 MB in 1.4 s, 468 MB RSS; `MAX_PDF_BYTES` 50 MiB allows ~50 GB. Attachment cap (16 MiB) covers the attachment only. Impact: tab freeze/OOM for the practitioner on a crafted PDF, no code execution. Cannot refuse `/ObjStm` (pdf-lib `save()` emits object streams in our own PDFs). Options: parse in a Web Worker (bounded blast radius), or locate the attachment by scanning bytes without a full load, or a capped custom decoder. pdf-lib 1.17.1 unmaintained since 2021 (LOW in the same audit).
Status: open (documented in docs/PDF-FORMAT.md "Known limits"); candidate for Step 14 worker or a hardening task before external users.
Status: open; analysis + plan in docs/development/PDF_IMPORT_HARDENING.md (2026-10-11): byte-scan reader (no pdf-lib load on import) + `useObjectStreams: false` at export can land now; worker/server import + pdf-lib replacement are prerequisites before external users.
+1
View File
@@ -105,6 +105,7 @@ For production deployment (VPS provisioning, firewall, TLS, nginx reverse proxy,
- [docs/development/STEP5_MIGRATION_ANALYSIS.md](docs/development/STEP5_MIGRATION_ANALYSIS.md) — how the application moved onto the canonical sheet (Report v3)
- [docs/PDF-FORMAT.md](docs/PDF-FORMAT.md) — PDF pages, report archive envelope (snapshot, branding, labels), reading and limits
- [docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md](docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md) — why and how the PDF became an autonomous archive (Step 6)
- [docs/development/PDF_IMPORT_HARDENING.md](docs/development/PDF_IMPORT_HARDENING.md) — PDF import against decompression bombs: what lands now, what is required before external users
- [docs/development/SUPABASE.md](docs/development/SUPABASE.md) — local Supabase stack, migrations, RLS tests, generated types
- [DEPLOY.md](DEPLOY.md) — production VPS deploy guide (Docker + nginx + TLS + hardening)
- [CHANGELOG.md](CHANGELOG.md) — versioned release notes
+94
View File
@@ -0,0 +1,94 @@
# PDF import hardening — BLK-009 analysis
Status 2026-10-11: analysis requested by the user after Step 6. Nothing here is
implemented; §4 says what can land now and §5 what must land before the
import is opened to external users.
## 1. The problem
`readReportArchive` loads the whole PDF with pdf-lib (`PDFDocument.load`)
before looking for the attachment. pdf-lib's parser inflates every object
stream (`/ObjStm`) and cross-reference stream eagerly with its own, uncapped
inflater. Measured by the security audit: a 408 KB PDF expanded to 400 MB in
1.4 s (468 MB RSS); a 131 KB PDF to 128 MB. The ratio is about 1000:1, so the
50 MiB file cap still allows tens of gigabytes of expansion. Our 16 MiB caps
protect the attachment stream only, which the reader inflates itself.
Effect: a crafted PDF freezes the practitioner's tab or kills it (out of
memory). No code execution, no data access. A file size limit cannot help: the
amplification lives inside the compression.
Why `/ObjStm` cannot simply be refused: pdf-lib's `save()` writes object
streams by default (`useObjectStreams: true`), so every Faunosteo PDF
produced so far carries them, and refusing them would refuse the archive.
## 2. What the reader actually needs from the file
One embedded file stream, found through its `/Filespec`, and the document
keyword stamp. A PDF stream object can never live inside an object stream:
the `EmbeddedFile` dictionary and its data are always in clear text in the
file (`<< /Type /EmbeddedFile /Filter /FlateDecode /Length N … >> stream …
endstream`). Only the small dictionary objects (Filespec, Info, catalog) may
be packed into object streams.
So the attachment can be located by scanning the bytes, without parsing the
PDF object graph at all.
## 3. Options
| Option | Protection | Cost | Verdict |
|---|---|---|---|
| A. Byte-scan reader: find `/Type /EmbeddedFile` streams, read `/Length` (direct integer, as pdf-lib writes it; fall back to `endstream` search), inflate with the existing capped `inflateFlate`, parse JSON; no `PDFDocument.load` on import | removes pdf-lib from the import path entirely: the only inflation is our capped one | one module (~120 lines) + tests on every fixture (v2, bare v3, envelope, object streams on/off, several streams, foreign PDF) | **recommended, can land now** |
| B. Export with `useObjectStreams: false` | the Filespec name, the Info keywords and the Filespec→EF link become visible to a byte scan, so the filename-first rule and the `attachment_stripped` detection keep working without the object graph; files a few KB larger | one save option; older PDFs keep their object streams, which A tolerates | **recommended with A** |
| C. Load in a Web Worker with a timeout and a memory budget | contains the damage (the tab survives, the worker dies) | worker bundling, message protocol, no memory cap API in browsers (only termination on timeout) | defence in depth, later |
| D. Pre-scan declared `/Length` of `/ObjStm` streams and refuse above a sum | partial: declared lengths are the compressed sizes, the expansion is unknown | small | insufficient alone |
| E. Replace pdf-lib's parser (fork `@cantoo/pdf-lib`, or `pdf.js` for reading) | maintained parser; pdf.js has its own limits | dependency change, larger bundle | evaluate for Step 14 (server-side import) |
With A, the import path reads bytes, finds at most a few streams, inflates one
under 16 MiB, parses JSON under the existing schema caps. pdf-lib stays on
the export side (trusted inputs) and in the `report:compare` CLI (operator
tool, Node, acceptable risk but same fix applies).
Edge cases A must handle: several `EmbeddedFile` streams (pick the one whose
Filespec name starts with `osteo-report` when visible, else the first whose
inflated content parses as a report or an envelope); `/Length` as an indirect
reference (pdf-lib writes direct lengths; other producers may not: fall back
to scanning for `endstream`); `\r\n` versus `\n` after `stream`; encrypted
PDFs (today loaded with `ignoreEncryption`: the attachment stream of an
encrypted file is encrypted too and would fail to inflate → `no_attachment`,
same outcome as today in practice); the 50 MiB file cap stays as the scan
bound.
## 4. What can be done now (before merge of Step 6, or as the next chore)
1. Option A in `packages/pdf/src/attachment/scan.ts`, used by
`readReportArchive`; `readAttachedDocument(pdfDoc)` kept for the CLI until
it moves to the scanner too.
2. Option B on export (`pdfDoc.save({ useObjectStreams: false })`): no visible
change; goldens unaffected (they record drawn text, images, links); the
envelope roundtrip tests prove the reader still finds the attachment in
both layouts.
3. Tests: every existing import fixture through the scanner; a crafted
object-stream bomb (the audit's recipe: a few hundred KB inflating to
hundreds of MB) imports in milliseconds without inflating it; a PDF with
two embedded files; a PDF whose `/Length` is indirect.
4. Docs: `docs/PDF-FORMAT.md` "Reading an archive" and "Known limits".
Estimated size: one executor phase, about 300 lines including tests.
## 5. Prerequisites before external users (Step 7+ sync, Step 14 worker)
- Option A merged and the CLI on the scanner; no `PDFDocument.load` on any
untrusted input in the browser.
- Option C or a server-side import (Step 14) with a hard memory and time
budget per file, for any path that still parses PDF objects.
- Dependency decision on pdf-lib (unmaintained since 2021): keep it for
generation only, or move to the maintained fork; recorded as a Step 14
question.
- Report collection caps on read (observations, zones, findings per
observation) so a hostile report cannot freeze the editor before the
export's own caps refuse it.
- The remaining low findings of the Step 6 audits: `<use>` nesting budget in
the sanitiser (today bounded by the rasterizer), the decoder memory
envelope of PNG logos (bounded by the 4096² cap, about 430 MB worst case
for four logos).