diff --git a/.claude/memory/evals.md b/.claude/memory/evals.md index 8e80a667..9701adcb 100644 --- a/.claude/memory/evals.md +++ b/.claude/memory/evals.md @@ -97,3 +97,12 @@ Result: verifier pass 1 ECARTS(2) = floor only (EVAL-011 prose "eslint-disable" Anomalies: verifier stage runs after executor edits only (concurrent reads otherwise); gates.sh needs `EVIDENCE: PENDING` placeholder lines on new criteria. Residuals noted, not fixed: ZodError message echoes closed-vocabulary received values (never names), `compareReport` computes v3 lines twice, template-literal lines >80 chars, locals >5 in two functions. Action: await user manual validation of Step 5; residuals in TODO follow-up. + +--- + +## EVAL-013 | 2026-10-11 | Step 6 autonomous PDF archives — 5 phases by executors, 3 verifier passes, 4 security audits + +Method: analysis (3 analyzers + measurements) challenged by 3 plan-challengers (correctness/robustness/simplicity, 48 findings folded in) → user validation with D7/D8/D13 adjustments → Part II contract (exact types) → one executor per phase (0 freeze goldens, 1 domain, 2 anatomy compose, 3 engine, 4 app) with closed contracts, orchestrator commits → whole-step verifier + security auditor → fix loops. +Result: Phase 0 froze 39 goldens (text/images/links per locale × logo mode, view facts) + 2 route characterisation tests with the OLD code; engine reproduced 36 goldens with 0 mismatch (views: data-* 2→0 and hidden-marker titles declared relaxed). Verifier pass 1 ECARTS(31) = all oracle/scope defects of MY contract (grep on parametrised table, xmldom filter, narrow FILE SCOPE, 3 deleted tests with moved coverage) → contract amended → pass 2 CONFORME 13/13. Security pass 1 BLOCK(1) HIGH real + PoC: archived branding URL written unescaped into `/URI` (PDF object injection via `exportFromEnvelope`), +3 MEDIUM/4 LOW → executor fixed (hex URI, strict URL schema, export re-validation, PNG IHDR budget, CSS-escape rules, NaN guard, file.size) → pass 2 PASS but 2 MEDIUM incl. a regression the fix introduced (UTF-16 BOM URI dead in poppler) + PNG double-IHDR bypass → executor → pass 3 PASS + 1 MEDIUM (UPNG nextZero spin on tEXt) → executor (chunk allowlist + IEND) → pass 4 PASS + 3 MEDIUM (UPNG `inflateRaw` spins on truncated IDAT with allowlisted chunks only; IHDR fields unvalidated; zero dimension) → executor: native bounded `inflateFlate` of the IDAT against the exact raw size, IHDR fields, sides ≥ 1, no interlace, raw ≤ 16 MiB → pass 5 focused. Lesson: a third-party decoder on untrusted bytes needs a native oracle in front, not a chunk allowlist. Final: 656 tests (194/21/130/88/223), typecheck, lint, build, docker ×4 green; goldens byte-identical since 8b86b92. +Anomalies: contract oracles written fast = 2 verifier passes lost (lesson: run every CHECK once before dispatching the verifier); security fix introduced a functional regression invisible to tests because the test reader decoded the BOM (lesson: assert raw bytes, cross-check with a real reader like poppler); executors denied `rm`/`git rm` (orchestrator deletes); one executor used `git stash` despite "no git state" (now forbidden explicitly in briefs); demo envelope with archived branding = 136 KB stored / 306 KB raw (raw > 250 KB, stored < 250 KB: reported, user said data first). +Action: await user manual validation + UX decision on re-export of the kept archive; BLK-009 (pdf-lib ObjStm inflation) before external users; react-router advisories (pre-existing) to bump in a chore. diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index 13d40d7b..c7ce085b 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -199,3 +199,7 @@ Step 4 validated + finished into develop. User decisions on the 7 open points ## 2026-10-10 (session — Step 5 closed, Step 6 analysis) Step 5 validated + closed by user; LRN-045 (permanent will-change blurs text) written on develop. Step 6 analysis on chore/step-06-pdf-archive-analysis: 3 analyzers (PDF pipeline, data model, app/tests) → facts: export reads live SheetIndex (labels, views, asset, all annotations of view), 4 fetched logos + 3 hard-coded URLs, DOM in drawing/highlight/raster, attachment = bare v3 Flate'd by pdf-lib, getAttachments dead on pdf-lib 1.17.1, no report↔sheet check anywhere. Measured: lean snapshot envelope 172 KB raw / 51 KB stored (3 obs), 228 / 54 worst; SVG = 90% of stored. Plan: snapshot = lean AnatomicalSheetDocument + assets inline (reuses index/renderer/integrity rules), packages/pdf engine (xmldom compose promoted from app-review.ts, Rasterizer + Branding inputs, no DOM lib), envelope v1 layered reading (report never lost), rebase on import/restore keeps report.sheet == snapshot.ref, export-side caps. 3 plan-challengers GO WITH FIXES (16+19+13 findings folded in). docs/development/STEP6_PDF_ARCHIVE_ANALYSIS.md; TODO 6.0–6.7. Awaiting user decisions D1/D4/D6/D7/D8/D10/D12/D13; no implementation. + +## 2026-10-11 (Step 6 implementation) + +User validated the Step 6 analysis with D7 (branding archived), D8 (imported snapshot kept, explicit rebase, no new UI without UX gate), D13 (explicit import messages). Analysis chore branch finished; feature/step-06-pdf-archives. Part II contract (exact types) + BDR-039. Phases by executors: 0 freeze (39 goldens, demo fixture, bare v3 PDF, route characterisation), 1 domain (archive/{snapshot,envelope,branding,labels}, sheet-index moved, rebase, bundle asset hashes), 2 anatomy (composeViewSvg + highlightLayer promoted from app-review), 3 engine packages/pdf (xmldom composition, allowlist sanitiser, safeText, Rasterizer, envelope v1, layered reader, resvg test), 4 app (export/import on the engine, archive in session `osteo:archive`, rebase + 4 messages, legacy modules removed). Verifier ECARTS(31, all my contract's oracles/scope) → CONFORME. Security BLOCK(1) HIGH (URI injection via archived branding) → 4 fix loops (hex URI → ASCII hex after poppler regression, URL schema, export re-validation, PNG IHDR budget → chunk allowlist + IEND → native IDAT inflation against exact raw size + IHDR fields, CSS escapes, NaN, file.size) → PASS. Docs: PDF-FORMAT.md real, CLAUDE/README/WORKSPACE/DEPLOY/CHANGELOG. 656 tests, build + docker green. Sizes: demo envelope 55 KB stored no logos / 136 KB with archived branding (306 KB raw). LRN-046, BLK-009, EVAL-013. Awaiting user validation + UX decision for re-export of the kept archive; Step 7 not started. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index a812b2b9..d68065cd 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -107,7 +107,8 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] 6.3 Phase 2 anatomy (6940926: composeViewSvg + highlightLayer, 130 anatomy tests): render/compose.ts promoted from curation/app-review.ts, highlight on the layer tree, review drawing golden unchanged - [x] 6.4 Phase 3 packages/pdf (c179560: 53 engine tests, 36 goldens exact, resvg raster, demo envelope 55 KB / 136 KB stored): scaffold (no DOM lib, restricted globals, vitest node, Dockerfile, CI, README), svg/sanitize allowlist + horse.svg zero-removal, render moved, Branding/Rasterizer/exportedAt/safeText/dict, exportReportArchive (T3 both logo modes), readReportArchive (filename-first, layered, attachment_stripped; T4-T6, T8), T1/T7/T9/T10 resvg/T11, xmldom bundle size ≤ 50 KB gz - [x] 6.5 Phase 4 app (9bd5173: 222 app tests, archive kept in session, rebase + alerts, legacy PDF modules removed, xmldom ≈ 31 KB gz in the main chunk): browserRasterizer, branding.ts, export/import on the engine, rebase on import + draft restore, 2 new messages, old render/attachment/highlight code + pdf.* keys removed, report:compare on engine API, T2/T14/T15, suite migrated 1:1 - - [ ] 6.6 Phase 5 docs + gates (docs synced: PDF-FORMAT, CLAUDE, README, WORKSPACE, DEPLOY, CHANGELOG; root check 580 tests + build + docker build green; verifier + security gates running): PDF-FORMAT.md real, CLAUDE.md map, WORKSPACE.md rule, DEPLOY.md rollback note, CHANGELOG, BDR-014 note; pnpm check, docker build, verifier + security gates, EVAL; manual validation list + - [x] 6.6 Phase 5 docs + gates (docs synced; verifier ECARTS(31 oracle/scope) → CONFORME 13/13; security BLOCK(1) → 4 fix loops → PASS; 656 tests, build, docker ×4 green) + - [ ] 6.8 AWAITING USER: manual validation (protocol in the end-of-step report), UX decision for "re-export the kept archive" (store + engine ready, no button), wording of the 4 import messages; then finish into develop and Milestone A release 0.2.0: PDF-FORMAT.md real, CLAUDE.md map, WORKSPACE.md rule, DEPLOY.md rollback note, CHANGELOG, BDR-014 note; pnpm check, docker build, verifier + security gates, EVAL; manual validation list - [ ] 6.7 follow-ups recorded for Step 13: catalog asset cap aligned with the archive cap (2 MiB), allowlist sanitiser at upload, app canvas sanitiser (EVAL-011) - [ ] 5 follow-up: allowlist SVG sanitiser before Storage-served assets (Step 13); orphan findings UX (D5) at the report UX step; report:compare residuals (ZodError reason genericised, v3 lines computed once, long template lines); 16 MiB attachment cap without direct test diff --git a/docs/PDF-FORMAT.md b/docs/PDF-FORMAT.md index 9e029430..436edbd7 100644 --- a/docs/PDF-FORMAT.md +++ b/docs/PDF-FORMAT.md @@ -199,9 +199,12 @@ the pixel budget; a colour type and bit depth the PNG spec allows (0: 1 to 0 and no interlace (the archived logos are never interlaced, so Adam7 pass sizes are not computed); an expected raw scanline size, `(ceil(width × bits per pixel / 8) + 1) × height`, of at most 16 MiB, the -attachment cap the native inflater enforces, which is the effective budget -(4096² at 8-bit RGBA is 67 MB of scanlines and is refused; a 512 px wide -logo is under 1 MB); then the chunk list, which must end on an empty `IEND` +attachment cap the native inflater enforces (4096² at 8-bit RGBA is 67 MB +of scanlines and is refused; a 512 px wide logo is under 1 MB). That cap +bounds what is inflated, not what pdf-lib's decoder then allocates: the +decoder still expands any accepted image to a 32-bit RGBA buffer, so the +4096² pixel cap is what bounds memory (about 430 MB peak measured with four +4096² logos; real logos are 512 px wide). Then the chunk list, which must end on an empty `IEND` that closes the file and may hold only the chunk types the decoder reads by fixed offsets or by declared length (`PLTE`, `tRNS`, `IDAT`, `IEND`, `gAMA`, `cHRM`, `sRGB`, `sBIT`, `pHYs`, `bKGD`): a second IHDR, an APNG