From 5ed4a4b6207d368256edfac5ff1ff0bada5bac60 Mon Sep 17 00:00:00 2001 From: bchanot Date: Sun, 11 Oct 2026 16:12:22 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20PDF=20import=20hardening=20analysis=20(?= =?UTF-8?q?BLK-009)=20=E2=80=94=20byte-scan=20reader=20and=20object-stream?= =?UTF-8?q?-free=20export=20now,=20worker=20and=20parser=20replacement=20b?= =?UTF-8?q?efore=20external=20users?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/memory/blockers.md | 2 +- README.md | 1 + docs/development/PDF_IMPORT_HARDENING.md | 94 ++++++++++++++++++++++++ 3 files changed, 96 insertions(+), 1 deletion(-) create mode 100644 docs/development/PDF_IMPORT_HARDENING.md diff --git a/.claude/memory/blockers.md b/.claude/memory/blockers.md index 03be7215..8e6a6fd5 100644 --- a/.claude/memory/blockers.md +++ b/.claude/memory/blockers.md @@ -80,4 +80,4 @@ Status: open (no real data yet; treat before Step 7+ persistence of user reports ## BLK-009 | 2026-10-11 | pdf-lib inflates PDF object streams without cap at `PDFDocument.load` (import path) Security audit Step 6 (MEDIUM, pre-existing since Step 1 import): `PDFDocument.load` decodes every `/ObjStm` and xref stream eagerly with its own uncapped FlateStream; measured 408 KB PDF → 400 MB in 1.4 s, 468 MB RSS; `MAX_PDF_BYTES` 50 MiB allows ~50 GB. Attachment cap (16 MiB) covers the attachment only. Impact: tab freeze/OOM for the practitioner on a crafted PDF, no code execution. Cannot refuse `/ObjStm` (pdf-lib `save()` emits object streams in our own PDFs). Options: parse in a Web Worker (bounded blast radius), or locate the attachment by scanning bytes without a full load, or a capped custom decoder. pdf-lib 1.17.1 unmaintained since 2021 (LOW in the same audit). -Status: open (documented in docs/PDF-FORMAT.md "Known limits"); candidate for Step 14 worker or a hardening task before external users. +Status: open; analysis + plan in docs/development/PDF_IMPORT_HARDENING.md (2026-10-11): byte-scan reader (no pdf-lib load on import) + `useObjectStreams: false` at export can land now; worker/server import + pdf-lib replacement are prerequisites before external users. diff --git a/README.md b/README.md index 355efc19..7b2caccc 100644 --- a/README.md +++ b/README.md @@ -105,6 +105,7 @@ For production deployment (VPS provisioning, firewall, TLS, nginx reverse proxy, - [docs/development/STEP5_MIGRATION_ANALYSIS.md](docs/development/STEP5_MIGRATION_ANALYSIS.md) — how the application moved onto the canonical sheet (Report v3) - [docs/PDF-FORMAT.md](docs/PDF-FORMAT.md) — PDF pages, report archive envelope (snapshot, branding, labels), reading and limits - [docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md](docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md) — why and how the PDF became an autonomous archive (Step 6) +- [docs/development/PDF_IMPORT_HARDENING.md](docs/development/PDF_IMPORT_HARDENING.md) — PDF import against decompression bombs: what lands now, what is required before external users - [docs/development/SUPABASE.md](docs/development/SUPABASE.md) — local Supabase stack, migrations, RLS tests, generated types - [DEPLOY.md](DEPLOY.md) — production VPS deploy guide (Docker + nginx + TLS + hardening) - [CHANGELOG.md](CHANGELOG.md) — versioned release notes diff --git a/docs/development/PDF_IMPORT_HARDENING.md b/docs/development/PDF_IMPORT_HARDENING.md new file mode 100644 index 00000000..37403042 --- /dev/null +++ b/docs/development/PDF_IMPORT_HARDENING.md @@ -0,0 +1,94 @@ +# PDF import hardening — BLK-009 analysis + +Status 2026-10-11: analysis requested by the user after Step 6. Nothing here is +implemented; §4 says what can land now and §5 what must land before the +import is opened to external users. + +## 1. The problem + +`readReportArchive` loads the whole PDF with pdf-lib (`PDFDocument.load`) +before looking for the attachment. pdf-lib's parser inflates every object +stream (`/ObjStm`) and cross-reference stream eagerly with its own, uncapped +inflater. Measured by the security audit: a 408 KB PDF expanded to 400 MB in +1.4 s (468 MB RSS); a 131 KB PDF to 128 MB. The ratio is about 1000:1, so the +50 MiB file cap still allows tens of gigabytes of expansion. Our 16 MiB caps +protect the attachment stream only, which the reader inflates itself. + +Effect: a crafted PDF freezes the practitioner's tab or kills it (out of +memory). No code execution, no data access. A file size limit cannot help: the +amplification lives inside the compression. + +Why `/ObjStm` cannot simply be refused: pdf-lib's `save()` writes object +streams by default (`useObjectStreams: true`), so every Faunosteo PDF +produced so far carries them, and refusing them would refuse the archive. + +## 2. What the reader actually needs from the file + +One embedded file stream, found through its `/Filespec`, and the document +keyword stamp. A PDF stream object can never live inside an object stream: +the `EmbeddedFile` dictionary and its data are always in clear text in the +file (`<< /Type /EmbeddedFile /Filter /FlateDecode /Length N … >> stream … +endstream`). Only the small dictionary objects (Filespec, Info, catalog) may +be packed into object streams. + +So the attachment can be located by scanning the bytes, without parsing the +PDF object graph at all. + +## 3. Options + +| Option | Protection | Cost | Verdict | +|---|---|---|---| +| A. Byte-scan reader: find `/Type /EmbeddedFile` streams, read `/Length` (direct integer, as pdf-lib writes it; fall back to `endstream` search), inflate with the existing capped `inflateFlate`, parse JSON; no `PDFDocument.load` on import | removes pdf-lib from the import path entirely: the only inflation is our capped one | one module (~120 lines) + tests on every fixture (v2, bare v3, envelope, object streams on/off, several streams, foreign PDF) | **recommended, can land now** | +| B. Export with `useObjectStreams: false` | the Filespec name, the Info keywords and the Filespec→EF link become visible to a byte scan, so the filename-first rule and the `attachment_stripped` detection keep working without the object graph; files a few KB larger | one save option; older PDFs keep their object streams, which A tolerates | **recommended with A** | +| C. Load in a Web Worker with a timeout and a memory budget | contains the damage (the tab survives, the worker dies) | worker bundling, message protocol, no memory cap API in browsers (only termination on timeout) | defence in depth, later | +| D. Pre-scan declared `/Length` of `/ObjStm` streams and refuse above a sum | partial: declared lengths are the compressed sizes, the expansion is unknown | small | insufficient alone | +| E. Replace pdf-lib's parser (fork `@cantoo/pdf-lib`, or `pdf.js` for reading) | maintained parser; pdf.js has its own limits | dependency change, larger bundle | evaluate for Step 14 (server-side import) | + +With A, the import path reads bytes, finds at most a few streams, inflates one +under 16 MiB, parses JSON under the existing schema caps. pdf-lib stays on +the export side (trusted inputs) and in the `report:compare` CLI (operator +tool, Node, acceptable risk but same fix applies). + +Edge cases A must handle: several `EmbeddedFile` streams (pick the one whose +Filespec name starts with `osteo-report` when visible, else the first whose +inflated content parses as a report or an envelope); `/Length` as an indirect +reference (pdf-lib writes direct lengths; other producers may not: fall back +to scanning for `endstream`); `\r\n` versus `\n` after `stream`; encrypted +PDFs (today loaded with `ignoreEncryption`: the attachment stream of an +encrypted file is encrypted too and would fail to inflate → `no_attachment`, +same outcome as today in practice); the 50 MiB file cap stays as the scan +bound. + +## 4. What can be done now (before merge of Step 6, or as the next chore) + +1. Option A in `packages/pdf/src/attachment/scan.ts`, used by + `readReportArchive`; `readAttachedDocument(pdfDoc)` kept for the CLI until + it moves to the scanner too. +2. Option B on export (`pdfDoc.save({ useObjectStreams: false })`): no visible + change; goldens unaffected (they record drawn text, images, links); the + envelope roundtrip tests prove the reader still finds the attachment in + both layouts. +3. Tests: every existing import fixture through the scanner; a crafted + object-stream bomb (the audit's recipe: a few hundred KB inflating to + hundreds of MB) imports in milliseconds without inflating it; a PDF with + two embedded files; a PDF whose `/Length` is indirect. +4. Docs: `docs/PDF-FORMAT.md` "Reading an archive" and "Known limits". + +Estimated size: one executor phase, about 300 lines including tests. + +## 5. Prerequisites before external users (Step 7+ sync, Step 14 worker) + +- Option A merged and the CLI on the scanner; no `PDFDocument.load` on any + untrusted input in the browser. +- Option C or a server-side import (Step 14) with a hard memory and time + budget per file, for any path that still parses PDF objects. +- Dependency decision on pdf-lib (unmaintained since 2021): keep it for + generation only, or move to the maintained fork; recorded as a Step 14 + question. +- Report collection caps on read (observations, zones, findings per + observation) so a hostile report cannot freeze the editor before the + export's own caps refuse it. +- The remaining low findings of the Step 6 audits: `` nesting budget in + the sanitiser (today bounded by the rasterizer), the decoder memory + envelope of PNG logos (bounded by the 4096² cap, about 430 MB worst case + for four logos).