fix(pdf): bounded attachment extraction by byte scan — no pdf-lib parse on import, export without object streams (BLK-009)
This commit is contained in:
1 parent
62b8a43e09
commit
60ee71208f
14 files changed
+1054
-198
No files matched your search
@@ -1,8 +1,9 @@
|
||||
# PDF import hardening — BLK-009 analysis
|
||||
|
||||
Status 2026-10-11: analysis requested by the user after Step 6. Nothing here is
|
||||
implemented; §4 says what can land now and §5 what must land before the
|
||||
import is opened to external users.
|
||||
Status 2026-10-11: analysis requested by the user after Step 6. §4 landed the
|
||||
same day on `feature/step-06-pdf-archives` (options A + B, see §4 for the
|
||||
commit); §5 lists what must land before the import is opened to external
|
||||
users.
|
||||
|
||||
## 1. The problem
|
||||
|
||||
@@ -36,13 +37,13 @@ PDF object graph at all.
|
||||
|
||||
## 3. Options
|
||||
|
||||
| Option | Protection | Cost | Verdict |
|
||||
|---|---|---|---|
|
||||
| A. Byte-scan reader: find `/Type /EmbeddedFile` streams, read `/Length` (direct integer, as pdf-lib writes it; fall back to `endstream` search), inflate with the existing capped `inflateFlate`, parse JSON; no `PDFDocument.load` on import | removes pdf-lib from the import path entirely: the only inflation is our capped one | one module (~120 lines) + tests on every fixture (v2, bare v3, envelope, object streams on/off, several streams, foreign PDF) | **recommended, can land now** |
|
||||
| B. Export with `useObjectStreams: false` | the Filespec name, the Info keywords and the Filespec→EF link become visible to a byte scan, so the filename-first rule and the `attachment_stripped` detection keep working without the object graph; files a few KB larger | one save option; older PDFs keep their object streams, which A tolerates | **recommended with A** |
|
||||
| C. Load in a Web Worker with a timeout and a memory budget | contains the damage (the tab survives, the worker dies) | worker bundling, message protocol, no memory cap API in browsers (only termination on timeout) | defence in depth, later |
|
||||
| D. Pre-scan declared `/Length` of `/ObjStm` streams and refuse above a sum | partial: declared lengths are the compressed sizes, the expansion is unknown | small | insufficient alone |
|
||||
| E. Replace pdf-lib's parser (fork `@cantoo/pdf-lib`, or `pdf.js` for reading) | maintained parser; pdf.js has its own limits | dependency change, larger bundle | evaluate for Step 14 (server-side import) |
|
||||
| Option | Protection | Cost | Verdict |
|
||||
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |
|
||||
| A. Byte-scan reader: find `/Type /EmbeddedFile` streams, read `/Length` (direct integer, as pdf-lib writes it; fall back to `endstream` search), inflate with the existing capped `inflateFlate`, parse JSON; no `PDFDocument.load` on import | removes pdf-lib from the import path entirely: the only inflation is our capped one | one module (~120 lines) + tests on every fixture (v2, bare v3, envelope, object streams on/off, several streams, foreign PDF) | **recommended, can land now** |
|
||||
| B. Export with `useObjectStreams: false` | the Filespec name, the Info keywords and the Filespec→EF link become visible to a byte scan, so the filename-first rule and the `attachment_stripped` detection keep working without the object graph; files a few KB larger | one save option; older PDFs keep their object streams, which A tolerates | **recommended with A** |
|
||||
| C. Load in a Web Worker with a timeout and a memory budget | contains the damage (the tab survives, the worker dies) | worker bundling, message protocol, no memory cap API in browsers (only termination on timeout) | defence in depth, later |
|
||||
| D. Pre-scan declared `/Length` of `/ObjStm` streams and refuse above a sum | partial: declared lengths are the compressed sizes, the expansion is unknown | small | insufficient alone |
|
||||
| E. Replace pdf-lib's parser (fork `@cantoo/pdf-lib`, or `pdf.js` for reading) | maintained parser; pdf.js has its own limits | dependency change, larger bundle | evaluate for Step 14 (server-side import) |
|
||||
|
||||
With A, the import path reads bytes, finds at most a few streams, inflates one
|
||||
under 16 MiB, parses JSON under the existing schema caps. pdf-lib stays on
|
||||
@@ -59,22 +60,29 @@ encrypted file is encrypted too and would fail to inflate → `no_attachment`,
|
||||
same outcome as today in practice); the 50 MiB file cap stays as the scan
|
||||
bound.
|
||||
|
||||
## 4. What can be done now (before merge of Step 6, or as the next chore)
|
||||
## 4. Done 2026-10-11 (feature/step-06-pdf-archives, commit: see `git log -- packages/pdf/src/attachment/scan.ts`)
|
||||
|
||||
1. Option A in `packages/pdf/src/attachment/scan.ts`, used by
|
||||
`readReportArchive`; `readAttachedDocument(pdfDoc)` kept for the CLI until
|
||||
it moves to the scanner too.
|
||||
1. Option A in `packages/pdf/src/attachment/scan.ts`
|
||||
(`findEmbeddedFiles`, `readAttachmentBytes`, `hasArchiveStamp`), used by
|
||||
`readReportArchive`; `readAttachedDocument(bytes)` takes the file bytes
|
||||
and the `report:compare` CLI runs on the scanner too. No
|
||||
`PDFDocument.load` remains under `packages/pdf/src` or `apps/app/scripts`.
|
||||
2. Option B on export (`pdfDoc.save({ useObjectStreams: false })`): no visible
|
||||
change; goldens unaffected (they record drawn text, images, links); the
|
||||
envelope roundtrip tests prove the reader still finds the attachment in
|
||||
both layouts.
|
||||
3. Tests: every existing import fixture through the scanner; a crafted
|
||||
object-stream bomb (the audit's recipe: a few hundred KB inflating to
|
||||
hundreds of MB) imports in milliseconds without inflating it; a PDF with
|
||||
two embedded files; a PDF whose `/Length` is indirect.
|
||||
change; goldens unchanged (they record drawn text, images, links); the
|
||||
envelope roundtrip tests prove the reader finds the attachment in both
|
||||
layouts.
|
||||
3. Tests (`packages/pdf/tests/scan.test.ts`): the three frozen fixtures (object
|
||||
streams) and a fresh export; two embedded files with a decoy first;
|
||||
indirect and wrong `/Length`; `\r\n` after `stream`; `/Type/EmbeddedFile`
|
||||
without a space; hex and literal names and keywords; garbage inputs and
|
||||
the 32-candidate cap; a 64 MiB object-stream bomb next to the real
|
||||
attachment (report in milliseconds, RSS flat) and an `EmbeddedFile` bomb
|
||||
(inflation cancelled at 16 MiB, `no_attachment`).
|
||||
4. Docs: `docs/PDF-FORMAT.md` "Reading an archive" and "Known limits".
|
||||
|
||||
Estimated size: one executor phase, about 300 lines including tests.
|
||||
Known trade-off: a stamped PDF saved before this change (object streams)
|
||||
hides its `/Keywords`, so a stripped attachment on such a file reads as
|
||||
`no_attachment` instead of `attachment_stripped`.
|
||||
|
||||
## 5. Prerequisites before external users (Step 7+ sync, Step 14 worker)
|
||||
|
||||
|
||||
Reference in new issue
Block a user