docs(pdf): PNG budget wording (pixel cap bounds decoder memory); chore(memory): EVAL-013, journal, TODO 6.6 closed and 6.8 awaiting user

This commit is contained in:
bchanot committed 2026-10-11 15:06:12 +02:00
1 parent cb95627a11
commit 80e95ca61d
4 files changed
+21 -4

No files matched your search

+9
View File
@@ -97,3 +97,12 @@ Result: verifier pass 1 ECARTS(2) = floor only (EVAL-011 prose "eslint-disable"
Anomalies: verifier stage runs after executor edits only (concurrent reads otherwise); gates.sh needs `EVIDENCE: PENDING` placeholder lines on new criteria. Residuals noted, not fixed: ZodError message echoes closed-vocabulary received values (never names), `compareReport` computes v3 lines twice, template-literal lines >80 chars, locals >5 in two functions. Anomalies: verifier stage runs after executor edits only (concurrent reads otherwise); gates.sh needs `EVIDENCE: PENDING` placeholder lines on new criteria. Residuals noted, not fixed: ZodError message echoes closed-vocabulary received values (never names), `compareReport` computes v3 lines twice, template-literal lines >80 chars, locals >5 in two functions.
Action: await user manual validation of Step 5; residuals in TODO follow-up. Action: await user manual validation of Step 5; residuals in TODO follow-up.
---
## EVAL-013 | 2026-10-11 | Step 6 autonomous PDF archives — 5 phases by executors, 3 verifier passes, 4 security audits
Method: analysis (3 analyzers + measurements) challenged by 3 plan-challengers (correctness/robustness/simplicity, 48 findings folded in) → user validation with D7/D8/D13 adjustments → Part II contract (exact types) → one executor per phase (0 freeze goldens, 1 domain, 2 anatomy compose, 3 engine, 4 app) with closed contracts, orchestrator commits → whole-step verifier + security auditor → fix loops.
Result: Phase 0 froze 39 goldens (text/images/links per locale × logo mode, view facts) + 2 route characterisation tests with the OLD code; engine reproduced 36 goldens with 0 mismatch (views: data-* 2→0 and hidden-marker titles declared relaxed). Verifier pass 1 ECARTS(31) = all oracle/scope defects of MY contract (grep on parametrised table, xmldom filter, narrow FILE SCOPE, 3 deleted tests with moved coverage) → contract amended → pass 2 CONFORME 13/13. Security pass 1 BLOCK(1) HIGH real + PoC: archived branding URL written unescaped into `/URI` (PDF object injection via `exportFromEnvelope`), +3 MEDIUM/4 LOW → executor fixed (hex URI, strict URL schema, export re-validation, PNG IHDR budget, CSS-escape rules, NaN guard, file.size) → pass 2 PASS but 2 MEDIUM incl. a regression the fix introduced (UTF-16 BOM URI dead in poppler) + PNG double-IHDR bypass → executor → pass 3 PASS + 1 MEDIUM (UPNG nextZero spin on tEXt) → executor (chunk allowlist + IEND) → pass 4 PASS + 3 MEDIUM (UPNG `inflateRaw` spins on truncated IDAT with allowlisted chunks only; IHDR fields unvalidated; zero dimension) → executor: native bounded `inflateFlate` of the IDAT against the exact raw size, IHDR fields, sides ≥ 1, no interlace, raw ≤ 16 MiB → pass 5 focused. Lesson: a third-party decoder on untrusted bytes needs a native oracle in front, not a chunk allowlist. Final: 656 tests (194/21/130/88/223), typecheck, lint, build, docker ×4 green; goldens byte-identical since 8b86b92.
Anomalies: contract oracles written fast = 2 verifier passes lost (lesson: run every CHECK once before dispatching the verifier); security fix introduced a functional regression invisible to tests because the test reader decoded the BOM (lesson: assert raw bytes, cross-check with a real reader like poppler); executors denied `rm`/`git rm` (orchestrator deletes); one executor used `git stash` despite "no git state" (now forbidden explicitly in briefs); demo envelope with archived branding = 136 KB stored / 306 KB raw (raw > 250 KB, stored < 250 KB: reported, user said data first).
Action: await user manual validation + UX decision on re-export of the kept archive; BLK-009 (pdf-lib ObjStm inflation) before external users; react-router advisories (pre-existing) to bump in a chore.
+4
View File
@@ -199,3 +199,7 @@ Step 4 validated + finished into develop. User decisions on the 7 open points
## 2026-10-10 (session — Step 5 closed, Step 6 analysis) ## 2026-10-10 (session — Step 5 closed, Step 6 analysis)
Step 5 validated + closed by user; LRN-045 (permanent will-change blurs text) written on develop. Step 6 analysis on chore/step-06-pdf-archive-analysis: 3 analyzers (PDF pipeline, data model, app/tests) → facts: export reads live SheetIndex (labels, views, asset, all annotations of view), 4 fetched logos + 3 hard-coded URLs, DOM in drawing/highlight/raster, attachment = bare v3 Flate'd by pdf-lib, getAttachments dead on pdf-lib 1.17.1, no report↔sheet check anywhere. Measured: lean snapshot envelope 172 KB raw / 51 KB stored (3 obs), 228 / 54 worst; SVG = 90% of stored. Plan: snapshot = lean AnatomicalSheetDocument + assets inline (reuses index/renderer/integrity rules), packages/pdf engine (xmldom compose promoted from app-review.ts, Rasterizer + Branding inputs, no DOM lib), envelope v1 layered reading (report never lost), rebase on import/restore keeps report.sheet == snapshot.ref, export-side caps. 3 plan-challengers GO WITH FIXES (16+19+13 findings folded in). docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md; TODO 6.0–6.7. Awaiting user decisions D1/D4/D6/D7/D8/D10/D12/D13; no implementation. Step 5 validated + closed by user; LRN-045 (permanent will-change blurs text) written on develop. Step 6 analysis on chore/step-06-pdf-archive-analysis: 3 analyzers (PDF pipeline, data model, app/tests) → facts: export reads live SheetIndex (labels, views, asset, all annotations of view), 4 fetched logos + 3 hard-coded URLs, DOM in drawing/highlight/raster, attachment = bare v3 Flate'd by pdf-lib, getAttachments dead on pdf-lib 1.17.1, no report↔sheet check anywhere. Measured: lean snapshot envelope 172 KB raw / 51 KB stored (3 obs), 228 / 54 worst; SVG = 90% of stored. Plan: snapshot = lean AnatomicalSheetDocument + assets inline (reuses index/renderer/integrity rules), packages/pdf engine (xmldom compose promoted from app-review.ts, Rasterizer + Branding inputs, no DOM lib), envelope v1 layered reading (report never lost), rebase on import/restore keeps report.sheet == snapshot.ref, export-side caps. 3 plan-challengers GO WITH FIXES (16+19+13 findings folded in). docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md; TODO 6.0–6.7. Awaiting user decisions D1/D4/D6/D7/D8/D10/D12/D13; no implementation.
## 2026-10-11 (Step 6 implementation)
User validated the Step 6 analysis with D7 (branding archived), D8 (imported snapshot kept, explicit rebase, no new UI without UX gate), D13 (explicit import messages). Analysis chore branch finished; feature/step-06-pdf-archives. Part II contract (exact types) + BDR-039. Phases by executors: 0 freeze (39 goldens, demo fixture, bare v3 PDF, route characterisation), 1 domain (archive/{snapshot,envelope,branding,labels}, sheet-index moved, rebase, bundle asset hashes), 2 anatomy (composeViewSvg + highlightLayer promoted from app-review), 3 engine packages/pdf (xmldom composition, allowlist sanitiser, safeText, Rasterizer, envelope v1, layered reader, resvg test), 4 app (export/import on the engine, archive in session `osteo:archive`, rebase + 4 messages, legacy modules removed). Verifier ECARTS(31, all my contract's oracles/scope) → CONFORME. Security BLOCK(1) HIGH (URI injection via archived branding) → 4 fix loops (hex URI → ASCII hex after poppler regression, URL schema, export re-validation, PNG IHDR budget → chunk allowlist + IEND → native IDAT inflation against exact raw size + IHDR fields, CSS escapes, NaN, file.size) → PASS. Docs: PDF-FORMAT.md real, CLAUDE/README/WORKSPACE/DEPLOY/CHANGELOG. 656 tests, build + docker green. Sizes: demo envelope 55 KB stored no logos / 136 KB with archived branding (306 KB raw). LRN-046, BLK-009, EVAL-013. Awaiting user validation + UX decision for re-export of the kept archive; Step 7 not started.
+2 -1
View File
@@ -107,7 +107,8 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
- [x] 6.3 Phase 2 anatomy (6940926: composeViewSvg + highlightLayer, 130 anatomy tests): render/compose.ts promoted from curation/app-review.ts, highlight on the layer tree, review drawing golden unchanged - [x] 6.3 Phase 2 anatomy (6940926: composeViewSvg + highlightLayer, 130 anatomy tests): render/compose.ts promoted from curation/app-review.ts, highlight on the layer tree, review drawing golden unchanged
- [x] 6.4 Phase 3 packages/pdf (c179560: 53 engine tests, 36 goldens exact, resvg raster, demo envelope 55 KB / 136 KB stored): scaffold (no DOM lib, restricted globals, vitest node, Dockerfile, CI, README), svg/sanitize allowlist + horse.svg zero-removal, render moved, Branding/Rasterizer/exportedAt/safeText/dict, exportReportArchive (T3 both logo modes), readReportArchive (filename-first, layered, attachment_stripped; T4-T6, T8), T1/T7/T9/T10 resvg/T11, xmldom bundle size ≤ 50 KB gz - [x] 6.4 Phase 3 packages/pdf (c179560: 53 engine tests, 36 goldens exact, resvg raster, demo envelope 55 KB / 136 KB stored): scaffold (no DOM lib, restricted globals, vitest node, Dockerfile, CI, README), svg/sanitize allowlist + horse.svg zero-removal, render moved, Branding/Rasterizer/exportedAt/safeText/dict, exportReportArchive (T3 both logo modes), readReportArchive (filename-first, layered, attachment_stripped; T4-T6, T8), T1/T7/T9/T10 resvg/T11, xmldom bundle size ≤ 50 KB gz
- [x] 6.5 Phase 4 app (9bd5173: 222 app tests, archive kept in session, rebase + alerts, legacy PDF modules removed, xmldom ≈ 31 KB gz in the main chunk): browserRasterizer, branding.ts, export/import on the engine, rebase on import + draft restore, 2 new messages, old render/attachment/highlight code + pdf.* keys removed, report:compare on engine API, T2/T14/T15, suite migrated 1:1 - [x] 6.5 Phase 4 app (9bd5173: 222 app tests, archive kept in session, rebase + alerts, legacy PDF modules removed, xmldom ≈ 31 KB gz in the main chunk): browserRasterizer, branding.ts, export/import on the engine, rebase on import + draft restore, 2 new messages, old render/attachment/highlight code + pdf.* keys removed, report:compare on engine API, T2/T14/T15, suite migrated 1:1
- [ ] 6.6 Phase 5 docs + gates (docs synced: PDF-FORMAT, CLAUDE, README, WORKSPACE, DEPLOY, CHANGELOG; root check 580 tests + build + docker build green; verifier + security gates running): PDF-FORMAT.md real, CLAUDE.md map, WORKSPACE.md rule, DEPLOY.md rollback note, CHANGELOG, BDR-014 note; pnpm check, docker build, verifier + security gates, EVAL; manual validation list - [x] 6.6 Phase 5 docs + gates (docs synced; verifier ECARTS(31 oracle/scope) → CONFORME 13/13; security BLOCK(1) → 4 fix loops → PASS; 656 tests, build, docker ×4 green)
- [ ] 6.8 AWAITING USER: manual validation (protocol in the end-of-step report), UX decision for "re-export the kept archive" (store + engine ready, no button), wording of the 4 import messages; then finish into develop and Milestone A release 0.2.0: PDF-FORMAT.md real, CLAUDE.md map, WORKSPACE.md rule, DEPLOY.md rollback note, CHANGELOG, BDR-014 note; pnpm check, docker build, verifier + security gates, EVAL; manual validation list
- [ ] 6.7 follow-ups recorded for Step 13: catalog asset cap aligned with the archive cap (2 MiB), allowlist sanitiser at upload, app canvas sanitiser (EVAL-011) - [ ] 6.7 follow-ups recorded for Step 13: catalog asset cap aligned with the archive cap (2 MiB), allowlist sanitiser at upload, app canvas sanitiser (EVAL-011)
- [ ] 5 follow-up: allowlist SVG sanitiser before Storage-served assets (Step 13); orphan findings UX (D5) at the report UX step; report:compare residuals (ZodError reason genericised, v3 lines computed once, long template lines); 16 MiB attachment cap without direct test - [ ] 5 follow-up: allowlist SVG sanitiser before Storage-served assets (Step 13); orphan findings UX (D5) at the report UX step; report:compare residuals (ZodError reason genericised, v3 lines computed once, long template lines); 16 MiB attachment cap without direct test
+6 -3
View File
@@ -199,9 +199,12 @@ the pixel budget; a colour type and bit depth the PNG spec allows (0: 1 to
0 and no interlace (the archived logos are never interlaced, so Adam7 pass 0 and no interlace (the archived logos are never interlaced, so Adam7 pass
sizes are not computed); an expected raw scanline size, sizes are not computed); an expected raw scanline size,
`(ceil(width × bits per pixel / 8) + 1) × height`, of at most 16 MiB, the `(ceil(width × bits per pixel / 8) + 1) × height`, of at most 16 MiB, the
attachment cap the native inflater enforces, which is the effective budget attachment cap the native inflater enforces (4096² at 8-bit RGBA is 67 MB
(4096² at 8-bit RGBA is 67 MB of scanlines and is refused; a 512 px wide of scanlines and is refused; a 512 px wide logo is under 1 MB). That cap
logo is under 1 MB); then the chunk list, which must end on an empty `IEND` bounds what is inflated, not what pdf-lib's decoder then allocates: the
decoder still expands any accepted image to a 32-bit RGBA buffer, so the
4096² pixel cap is what bounds memory (about 430 MB peak measured with four
4096² logos; real logos are 512 px wide). Then the chunk list, which must end on an empty `IEND`
that closes the file and may hold only the chunk types the decoder reads by that closes the file and may hold only the chunk types the decoder reads by
fixed offsets or by declared length (`PLTE`, `tRNS`, `IDAT`, `IEND`, fixed offsets or by declared length (`PLTE`, `tRNS`, `IDAT`, `IEND`,
`gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs`, `bKGD`): a second IHDR, an APNG `gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs`, `bKGD`): a second IHDR, an APNG