feat(pdf): archive check tool pnpm report:compare (attachment vs its v3 reading, unresolved items flagged, --diff/--summary/--identity); readAttachedDocument split out of extractReportJson; stale scaffold:species script removed; docs

This commit is contained in:
bchanot committed 2026-10-10 15:44:20 +02:00
1 parent 1e0491f3bc
commit dc78a00835
13 files changed
+753 -37

No files matched your search

+4 -1
View File
@@ -95,7 +95,10 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
- [x] 5.5 PDF shim + attachment v3 + v2 import (gate D3)
- [x] 5.6 delete legacy (registry, species/*, zoneMatch, spineZones, svgConvention, D4 tools), SVG convention doc replaced, CLAUDE.md map, CHANGELOG, plan note, memory
- [x] 5.7 visual before/after (cmp renders), root check + docker + real stack green; verifier ×3 (gaps closed), security PASS (LOW fixed); EVAL-011. Manual FR/EN flow = user validation
- [ ] 5 follow-up: allowlist SVG sanitiser before Storage-served assets (Step 13); orphan findings UX (D5) at the report UX step; per-leaf adapter test over the 612 legacy leaves (map-level coverage today)
- [ ] 5.8 per-leaf migration (user ask 2026-10-10, during manual validation): every legacy leaf (612 = 442 identical, 141 moved, 29 removed) as a v2 finding → migrated → resolved to a draft leaf (anatomy, map file) and through sheetIndex.nodeAt + PDF roundtrip (app); the 27 legacy annotations as zones on their views
- [ ] 5.9 archive check tool `pnpm report:compare <pdf|dir>...`: v2 attachment vs migrated v3 side by side, unresolved flagged (removed vs unknown), identity opt-in, --diff of two PDFs, --summary; attachment.ts split read/parse (additive, D3); docs PDF-FORMAT/CLAUDE/README
- [ ] 5.10 final controls on the last lot: stale root script scaffold:species removed; fresh verifier + security on the whole diff (criteria 16-18); EVAL-011 addendum
- [ ] 5 follow-up: allowlist SVG sanitiser before Storage-served assets (Step 13); orphan findings UX (D5) at the report UX step
---
+1 -1
View File
@@ -11,7 +11,7 @@ Versions follow [Semantic Versioning](https://semver.org/).
- **Supabase foundation** — local stack (`supabase/config.toml`), migrations (schemas `private` / `audit`, `profiles`, platform roles with audited RPCs), pgTAP suite run on PGlite (`pnpm test:run`) and on the real stack (`pnpm db:test`), `@faunosteo/supabase` typed client + generated types. No UI change.
- **Anatomical sheet model** — `@faunosteo/domain`: canonical sheet document (views tree, annotations with stable ids and explicit laterality, ordered vertebral regions, typed finding categories, explicit annotation → finding zone links, FR/EN labels) with integrity validation; tables `species`, `anatomical_sheets`, `anatomical_sheet_versions` (published rows immutable, one draft per sheet, deletion preconditions), `anatomical_sheet_assets`; lifecycle RPCs (create, draft, publish, deprecate, delete, fork with provenance, archive, assets) audited and under RLS. Spec: `docs/development/ANATOMICAL_SHEET_MODEL.md`.
- **Application on the canonical sheet (Step 5)** — the editor renders the bundled reference sheet (curated equine draft, `SHEET_BUNDLE=draft`; a default build carries no sheet): markers drawn from the document with the legacy look (`@faunosteo/anatomy/render`), head view as a crop of the same drawing, zone interaction service, pickers on the sheet's regions (usual count, extension to the documented range) and finding zones. Report document **v3** (`packages/domain/report`): zones by annotation and view ids, findings by zone and path, severity codes, exact sheet reference; v2 PDFs and drafts migrate through the generated `report-migration-map.json`, unresolved items kept with their labels. Removed: species registry, label matchers, SVG convention parser, `/editors`, `/svg-editor`, `pathology-editor.html`, `scaffold:species`; the legacy sources now live under `supabase/seed/anatomical-sheets/equine/legacy-sources/`.
- **Application on the canonical sheet (Step 5)** — the editor renders the bundled reference sheet (curated equine draft, `SHEET_BUNDLE=draft`; a default build carries no sheet): markers drawn from the document with the legacy look (`@faunosteo/anatomy/render`), head view as a crop of the same drawing, zone interaction service, pickers on the sheet's regions (usual count, extension to the documented range) and finding zones. Report document **v3** (`packages/domain/report`): zones by annotation and view ids, findings by zone and path, severity codes, exact sheet reference; v2 PDFs and drafts migrate through the generated `report-migration-map.json`, unresolved items kept with their labels. Removed: species registry, label matchers, SVG convention parser, `/editors`, `/svg-editor`, `pathology-editor.html`, `scaffold:species`; the legacy sources now live under `supabase/seed/anatomical-sheets/equine/legacy-sources/`. Archive check tool `pnpm report:compare` (attachment vs its v3 reading, unresolved items flagged); the 612 legacy catalog leaves are tested through the migration in both packages.
- **Equine sheet curation (Step 4c)** — document schema v2 (references registry, `review_status` on findings and annotations, configurable vertebral `count_range`; migration `20261010100000`, pgTAP 008); `@faunosteo/anatomy` curation pipeline: authored changeset (`curation/changes.json`, levels A/B/C, before/after records) applied to the legacy baseline → working draft with `CHANGELOG.md`, `REVIEW-REPORT.md` (10 sections for the osteopath) and before/after review drawings; one cleaned SVG asset, head view as a crop. Content: research-backed audit of the horse sheet (44 changes, 48 verifiable references): sphenoid zone restructured, NAV terminology, FR/EN translations, vertebral ranges, ten SVG points relocated, carpus zone added, veterinary red flags, legal disclaimer anchored on R. 243-6 / R. 243-8, osteopathic-model status on unvalidated concepts, 11 questions for the professional. Still a private DRAFT, not published. Docker resources prefixed `faunosteo_` (Compose project and container).
- **Equine import (Step 4b)** — `@faunosteo/anatomy` deterministic importer: `Horse.svg` + `Horse_head.svg` + `pathologies.json` → canonical document, legacy id map, French import report, idempotent seed creating the official sheet `faunosteo-cheval` as a private DRAFT (never published by code). Storage bucket `anatomical-sheet-assets` (content-addressed SVGs, read through the sheet, operator upload script).
+5 -2
View File
@@ -88,6 +88,7 @@ pnpm db:types:check # CI diff
pnpm build:preview # SHEET_BUNDLE=draft build (manual validation, preview deployments)
pnpm preview:draft # same, then serves it
pnpm import:equine # regenerate legacy-import/ and draft/ (document, migration map, review drawings)
pnpm report:compare <pdf|dir>... # archive check: attachment vs its v3 reading, unresolved items flagged (docs/PDF-FORMAT.md)
```
## Docker commands
@@ -125,7 +126,8 @@ pnpm test # vitest watch — dev loop
```
Test files live in `apps/app/tests/unit/*.test.ts(x)`; golden fixtures in
`apps/app/tests/fixtures/`. Setup: `apps/app/tests/setup.ts` imports
`apps/app/tests/fixtures/`. The 612 legacy catalog leaves run through the v2 → v3 migration
in both `packages/anatomy` (against the draft document) and the app (sheet index + PDF roundtrip). Setup: `apps/app/tests/setup.ts` imports
@testing-library/jest-dom. Suites run on the bundled reference sheet (`tests/helpers/sheet.ts`); frozen v2 fixtures under `tests/fixtures/`.
SQL: `supabase/tests/*.sql` (pgTAP) run by `packages/supabase/tests/db/pgtap.test.ts`
on PGlite (part of `pnpm test:run`) and by `pnpm db:test` on the real stack.
@@ -148,6 +150,7 @@ faunosteo/
├─ apps/app/ ← @faunosteo/app (everything below until packages/)
├─ public/ ← logos only (no anatomy asset: the sheet is bundled)
├─ vite/sheet-bundle.ts ← Vite plugin: `virtual:faunosteo-sheet` (SHEET_BUNDLE=draft|none)
├─ scripts/compare-reports.ts ← `pnpm report:compare`: archive PDF vs v3 reading (pure part in scripts/compare/report.ts)
├─ src/
│ ├─ app/
│ │ ├─ App.tsx ← root router
@@ -414,7 +417,7 @@ Database (Supabase)
- Annotation rendering changes require updating docs/ANNOTATION-RENDERING.md first.
- Never modify the legacy sources under `supabase/seed/anatomical-sheets/equine/legacy-sources/` — they are the
importer's input (the user's historical files).
- Ask before any change to importReportFromPdf or extractReportJson — these handle user data.
- Ask before any change to importReportFromPdf, extractReportJson or readAttachedDocument — these handle user data.
- After PDF roundtrip tests fail: check attachment bytes, then Zod validation (v3, or v2 through the migration map), then DecompressionStream inflation.
- Node 24 / pnpm 10.13.1: `corepack enable` or `corepack pnpm`; never a global install.
- SQL change: `pnpm --filter @faunosteo/supabase test:run` (PGlite) before commit; `pnpm db:reset && pnpm db:test`
+2 -1
View File
@@ -78,8 +78,9 @@ docker compose up --build
```bash
pnpm typecheck # TypeScript strict, no emit
pnpm lint # ESLint 9 flat config, 0 warnings
pnpm test:run # vitest run (app 278 + supabase 13, incl. 79 pgTAP checks on PGlite)
pnpm test:run # vitest run (domain, anatomy, supabase/PGlite, app)
pnpm build # tsc + vite build → apps/app/dist/
pnpm report:compare <pdf|dir>... # archive PDF vs its v3 reading, unresolved items flagged
```
---
+3 -1
View File
@@ -11,7 +11,8 @@
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "vitest",
"test:run": "vitest run",
"lint": "eslint . --max-warnings 0"
"lint": "eslint . --max-warnings 0",
"report:compare": "tsx scripts/compare-reports.ts"
},
"dependencies": {
"@faunosteo/anatomy": "workspace:*",
@@ -39,6 +40,7 @@
"eslint-plugin-react-hooks": "^5.1.0",
"globals": "^15.14.0",
"happy-dom": "^20.10.6",
"tsx": "^4.23.15",
"typescript": "^5.7.2",
"typescript-eslint": "^8.19.0",
"vite": "^7.3.6",
+135
View File
@@ -0,0 +1,135 @@
/**
* Archive check: reads the report attached to one or more Faunosteo PDFs and
* prints it next to its v3 reading on the bundled reference sheet.
*
* pnpm report:compare <file.pdf | folder>... [--summary] [--identity]
* [--locale fr|en] [--diff a.pdf b.pdf]
*
* A v2 archive PDF is shown legacy id → canonical id, line by line; a v3 PDF
* is checked against the sheet. Items the sheet cannot place are marked
* `!! UNRESOLVED` with the reason. `--diff` compares the v3 content of two
* PDFs (an archive and its re-export after import, typically). Names are
* printed only with `--identity`. Reads only; never writes a file.
*/
import { readFileSync, readdirSync, statSync } from 'node:fs'
import { basename, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { PDFDocument } from 'pdf-lib'
import {
readReportDocument,
REPORT_VERSION,
UnsupportedReportVersionError,
} from '@faunosteo/domain'
import { readAttachedDocument } from '../src/pdf/attachment'
import { MAX_PDF_BYTES } from '../src/pdf/import'
import { parseBundledSheet } from '../src/sheets/provider'
import { buildSheetIndex, type SheetIndex } from '../src/services/sheetIndex'
import {
compareReport,
diffLines,
parseArgs,
renderComparison,
renderSummary,
type CliArgs,
type Comparison,
} from './compare/report'
const SEED_DIR = resolve(
fileURLToPath(import.meta.url),
'../../../../supabase/seed/anatomical-sheets/equine/draft',
)
/** pnpm runs the script from apps/app; paths are typed from the caller's folder. */
const CWD = process.env.INIT_CWD ?? process.cwd()
function loadSheet(): SheetIndex {
const json = (name: string) =>
JSON.parse(readFileSync(join(SEED_DIR, name), 'utf8')) as never
// The comparison needs the document and the map, not the drawings.
return buildSheetIndex(
parseBundledSheet({
status: 'draft',
ref: json('sheet-ref.json'),
document: json('document.json'),
migrationMap: json('report-migration-map.json'),
assets: {},
}),
)
}
function expandFiles(args: string[]): string[] {
const out: string[] = []
for (const arg of args) {
const path = resolve(CWD, arg)
if (statSync(path).isDirectory()) {
const pdfs = readdirSync(path)
.filter((f) => f.toLowerCase().endsWith('.pdf'))
.sort()
out.push(...pdfs.map((f) => join(path, f)))
} else out.push(path)
}
return out
}
async function compareFile(
path: string,
sheet: SheetIndex,
args: CliArgs,
): Promise<Comparison> {
const bytes = new Uint8Array(readFileSync(path))
if (bytes.byteLength > MAX_PDF_BYTES) throw new Error('file too large')
const pdf = await PDFDocument.load(bytes, { ignoreEncryption: true })
const raw = await readAttachedDocument(pdf)
const version = (raw as { version: number }).version
try {
const outcome = readReportDocument(raw, sheet.sheet.migrationMap)
return compareReport(raw, outcome, version !== REPORT_VERSION, sheet, args)
} catch (err) {
if (err instanceof UnsupportedReportVersionError) {
throw new Error(`report version ${version} cannot be read`)
}
throw err
}
}
async function main(argv: string[]): Promise<number> {
const args = parseArgs(argv)
const files = expandFiles(args.files)
if (files.length === 0 || (args.diff && files.length !== 2)) {
console.error(
'usage: pnpm report:compare <file.pdf | folder>... [--summary] [--identity] [--locale fr|en] [--diff a.pdf b.pdf]',
)
return 1
}
const sheet = loadSheet()
const results: Comparison[] = []
let failures = 0
for (const file of files) {
const name = basename(file)
try {
const c = await compareFile(file, sheet, args)
results.push(c)
if (!args.diff) {
const lines = args.summary
? [renderSummary(name, c)]
: [...renderComparison(name, c), '']
console.log(lines.join('\n'))
}
} catch (err) {
failures += 1
console.log(`!! ${name}: ${err instanceof Error ? err.message : err}`)
}
}
if (args.diff && results.length === 2) {
const diff = diffLines(results[0]!.normalized, results[1]!.normalized)
console.log(diff.length ? diff.join('\n') : 'identical v3 content')
}
return failures ? 2 : 0
}
main(process.argv.slice(2)).then(
(code) => process.exit(code),
(err) => {
console.error(err instanceof Error ? err.message : err)
process.exit(1)
},
)
+330
View File
@@ -0,0 +1,330 @@
import {
ReportV2Schema,
findingPathKey,
type Finding,
type FindingV2,
type MigrationOutcome,
type ObservationZoneV2,
type Report,
type ReportZone,
} from '@faunosteo/domain'
import type { Locale } from '../../src/i18n/types'
import { pickLabel, type SheetIndex } from '../../src/services/sheetIndex'
/**
* Pure comparison of a report document found in a PDF with its v3 reading:
* a v2 archive is shown legacy id → canonical id line by line, a v3 report
* is checked against the bundled sheet. Every item the sheet cannot place is
* marked `!! UNRESOLVED` with the reason, nothing is summarised away.
*/
export const UNRESOLVED = '!! UNRESOLVED'
export interface CompareOptions {
locale: Locale
/** Print practitioner, animal and owner names (off by default). */
identity: boolean
}
export interface Comparison {
/** `v2` when the attachment was migrated, `v3` when read as is. */
source: 'v2' | 'v3'
header: string[]
observations: string[][]
footer: string[]
/** Lines in the v3 vocabulary only, for `--diff` between two PDFs. */
normalized: string[]
counts: Counts
}
export interface Counts {
observations: number
zones: number
unresolvedZones: number
findings: number
unresolvedFindings: number
}
const q = (text: string) => `"${text}"`
function zoneRef(z: { annotationId: string; viewId: string; side?: string }) {
return `${z.annotationId}@${z.viewId}${z.side ? ' ' + z.side : ''}`
}
function findingRef(f: { findingZoneId: string; path: readonly string[] }) {
return findingPathKey(f.findingZoneId, f.path)
}
function vertebraeOf(z: { vertebrae?: { region: string; index: number }[] }) {
return z.vertebrae?.length
? ` [${z.vertebrae.map((v) => `${v.region}${v.index}`).join(', ')}]`
: ''
}
/** A v3 zone against the sheet: current label, or why it cannot be shown. */
function describeZone(
z: ReportZone,
sheet: SheetIndex,
locale: Locale,
): string {
const ref = `${zoneRef(z)}${vertebraeOf(z)}`
const anno = sheet.annotation(z.annotationId)
if (z.unresolved) {
return `${ref} ${q(z.labelFrSnapshot ?? '')} ${UNRESOLVED} legacy zone kept from the archive`
}
if (!anno || anno.view_id !== z.viewId) {
return `${ref} ${q(z.labelFrSnapshot ?? '')} !! not in the sheet (unflagged)`
}
return `${ref} ${q(pickLabel(sheet.name(anno), locale))}`
}
/** A v3 finding against the sheet: current label, or why it cannot be shown. */
function describeFinding(
f: Finding,
sheet: SheetIndex,
locale: Locale,
): string {
const snapshot = q(locale === 'en' ? f.labelEnSnapshot : f.labelFrSnapshot)
const vet = f.requiresVetReferral ? ' (!)' : ''
const side = f.side ? ` ${f.side}` : ''
const ref = `${findingRef(f)}${side}${vet}`
if (f.unresolved) {
return `${ref} ${snapshot} ${UNRESOLVED} kept with its snapshot`
}
const node = sheet.nodeAt(f.findingZoneId, f.path)
if (!node) return `${ref} ${snapshot} !! not in the sheet (unflagged)`
if (node.children.length > 0) {
return `${ref} ${snapshot} !! points at a category, not a leaf`
}
return `${ref} ${q(pickLabel(node.name, locale))}`
}
/** Legacy zone → migrated zone. */
function describeZoneMigration(
legacy: ObservationZoneV2,
out: ReportZone,
sheet: SheetIndex,
locale: Locale,
): string {
const side = legacy.side ? ` ${legacy.side}` : ''
const from = `${legacy.boneId}${legacy.detailSourceId ? '@' + legacy.detailSourceId : ''}${side}${vertebraeOf(legacy)}`
if (out.unresolved) {
return `${from} ${UNRESOLVED} unknown legacy id, kept as ${zoneRef(out)}`
}
const anno = sheet.annotation(out.annotationId)
const label = anno ? q(pickLabel(sheet.name(anno), locale)) : '!! missing'
return `${from} ${q(out.labelFrSnapshot ?? '')} → ${zoneRef(out)} ${label}`
}
/** Legacy finding → migrated finding, with the fate the map decided. */
function describeFindingMigration(
legacy: FindingV2,
out: Finding,
sheet: SheetIndex,
locale: Locale,
): string {
const map = sheet.sheet.migrationMap.finding_paths
const vet = legacy.requiresVetReferral ? ' (!)' : ''
const key = `${findingPathKey(legacy.zoneId, legacy.path)}${vet}`
const snapshot = q(
locale === 'en' ? legacy.labelEnSnapshot : legacy.labelFrSnapshot,
)
if (out.unresolved) {
const why = Object.hasOwn(map, findingPathKey(legacy.zoneId, legacy.path))
? 'removed from the sheet, kept with its label'
: 'never in the legacy catalog, kept with its label'
return `${key} ${snapshot} ${UNRESOLVED} ${why}`
}
const node = sheet.nodeAt(out.findingZoneId, out.path)
const label = node ? q(pickLabel(node.name, locale)) : '!! missing'
const target =
findingRef(out) === findingPathKey(legacy.zoneId, legacy.path)
? 'same'
: findingRef(out)
return `${key} ${snapshot} → ${target} ${label}`
}
function observationLines(
obs: Report['observations'][number],
sheet: SheetIndex,
opts: CompareOptions,
): string[] {
const lines = [
`[${obs.id}] severity ${obs.severity ?? 'none'}`,
...obs.zones.map((z) => `zone ${describeZone(z, sheet, opts.locale)}`),
...obs.findings.map(
(f) => `finding ${describeFinding(f, sheet, opts.locale)}`,
),
...obs.freeTextFindings.map((t) => `text ${q(t)}`),
]
if (obs.comment) lines.push(`comment ${q(obs.comment)}`)
return lines
}
function migratedObservationLines(
raw: unknown,
report: Report,
sheet: SheetIndex,
opts: CompareOptions,
): string[][] {
const v2 = ReportV2Schema.parse(raw)
return v2.observations.map((legacy, i) => {
const out = report.observations[i]!
const severity = legacy.severity
? `${legacy.severity} → ${out.severity}`
: 'none'
const lines = [`[${legacy.id}] severity ${severity}`]
legacy.zones.forEach((z, j) => {
const line = describeZoneMigration(z, out.zones[j]!, sheet, opts.locale)
lines.push(`zone ${line}`)
})
legacy.findings.forEach((f, j) => {
const line = describeFindingMigration(
f,
out.findings[j]!,
sheet,
opts.locale,
)
lines.push(`finding ${line}`)
})
lines.push(...legacy.freeTextFindings.map((t) => `text ${q(t)}`))
if (legacy.comment) lines.push(`comment ${q(legacy.comment)}`)
return lines
})
}
function countsOf(report: Report): Counts {
const zones = report.observations.flatMap((o) => o.zones)
const findings = report.observations.flatMap((o) => o.findings)
return {
observations: report.observations.length,
zones: zones.length,
unresolvedZones: zones.filter((z) => z.unresolved).length,
findings: findings.length,
unresolvedFindings: findings.filter((f) => f.unresolved).length,
}
}
function headerLines(
source: 'v2' | 'v3',
report: Report,
sheet: SheetIndex,
opts: CompareOptions,
): string[] {
const ref = sheet.sheet.ref
const sameSheet =
report.sheet.sheetId === ref.sheetId &&
report.sheet.documentSha256 === ref.documentSha256
const sheetLine = sameSheet
? `sheet ${ref.sheetId} (${sheet.sheet.status}, matches the bundled sheet)`
: `sheet ${report.sheet.sheetId} !! written on another sheet document (sha ${report.sheet.documentSha256.slice(0, 12)}…)`
const c = countsOf(report)
const lines = [
source === 'v2' ? 'v2 → v3 (migrated)' : 'v3 (read as is)',
sheetLine,
`id ${report.id} · date ${report.date} · ${c.observations} observations · ${c.zones} zones (${c.unresolvedZones} unresolved) · ${c.findings} findings (${c.unresolvedFindings} unresolved)`,
]
if (opts.identity) {
lines.push(
`practitioner ${q(report.practitioner.name)} · animal ${q(report.animal.name)} · owner ${q(report.owner.name)}`,
)
}
return lines
}
function footerLines(report: Report): string[] {
const lines: string[] = []
if (report.conclusion) lines.push(`conclusion ${q(report.conclusion)}`)
if (report.recommendations) {
lines.push(`recommendations ${q(report.recommendations)}`)
}
if (report.nextAppointment) {
lines.push(`next appointment ${JSON.stringify(report.nextAppointment)}`)
}
return lines
}
/**
* `raw` is the attachment as found in the PDF, `outcome` its v3 reading
* (`readReportDocument`); `migrated` tells which vocabulary the lines use.
*/
export function compareReport(
raw: unknown,
outcome: MigrationOutcome,
migrated: boolean,
sheet: SheetIndex,
opts: CompareOptions,
): Comparison {
const { report } = outcome
const source = migrated ? 'v2' : 'v3'
const v3Lines = report.observations.map((o) =>
observationLines(o, sheet, opts),
)
return {
source,
header: headerLines(source, report, sheet, opts),
observations: migrated
? migratedObservationLines(raw, report, sheet, opts)
: v3Lines,
footer: footerLines(report),
normalized: [...v3Lines.flat(), ...footerLines(report)],
counts: countsOf(report),
}
}
export function renderComparison(name: string, c: Comparison): string[] {
return [
`== ${name}`,
...c.header.map((l) => ` ${l}`),
...c.observations.flatMap((lines) =>
lines.map((l, i) => (i === 0 ? ` ${l}` : ` ${l}`)),
),
...c.footer.map((l) => ` ${l}`),
]
}
export function renderSummary(name: string, c: Comparison): string {
const n = c.counts
return `${name} · ${c.source === 'v2' ? 'v2→v3' : 'v3'} · ${n.observations} obs · zones ${n.zones}/${n.unresolvedZones} unresolved · findings ${n.findings}/${n.unresolvedFindings} unresolved`
}
/** Lines of A absent from B and lines of B absent from A, in order. */
export function diffLines(a: string[], b: string[]): string[] {
const inB = new Set(b)
const inA = new Set(a)
return [
...a.filter((l) => !inB.has(l)).map((l) => `- ${l}`),
...b.filter((l) => !inA.has(l)).map((l) => `+ ${l}`),
]
}
export interface CliArgs extends CompareOptions {
files: string[]
summary: boolean
diff: boolean
}
export function parseArgs(argv: string[]): CliArgs {
const args: CliArgs = {
files: [],
locale: 'fr',
identity: false,
summary: false,
diff: false,
}
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i]!
if (arg === '--identity') args.identity = true
else if (arg === '--summary') args.summary = true
else if (arg === '--diff') args.diff = true
else if (arg === '--locale') {
const value = argv[++i]
if (value !== 'fr' && value !== 'en') {
throw new Error('--locale takes fr or en')
}
args.locale = value
} else if (arg.startsWith('--')) throw new Error(`unknown option ${arg}`)
else args.files.push(arg)
}
return args
}
+41 -29
View File
@@ -43,10 +43,45 @@ export interface ExtractedReport extends MigrationOutcome {
migrated: boolean
}
/**
* The raw report document carried by the PDF, of whatever version, before
* any migration: what the archive check tool shows next to the v3 result.
* Throws when the PDF carries no Faunosteo attachment or an oversized one.
*/
export async function readAttachedDocument(
pdfDoc: PDFDocument,
): Promise<unknown> {
const bytes = await attachmentBytes(pdfDoc)
if (bytes.byteLength > MAX_ATTACHMENT_BYTES) {
throw new Error('attachment too large')
}
const data: unknown = JSON.parse(new TextDecoder().decode(bytes))
// No numeric version at all: not a Faunosteo attachment (legacy message), not an old one.
if (versionOf(data) === null) {
throw new Error('attachment carries no report version')
}
return data
}
export async function extractReportJson(
pdfDoc: PDFDocument,
map: ReportMigrationMap,
): Promise<ExtractedReport> {
const data = await readAttachedDocument(pdfDoc)
try {
return {
...readReportDocument(data, map),
migrated: versionOf(data) !== REPORT_VERSION,
}
} catch (err) {
if (err instanceof UnsupportedReportVersionError) {
throw new IncompatibleReportVersionError(err.foundVersion)
}
throw err
}
}
async function attachmentBytes(pdfDoc: PDFDocument): Promise<Uint8Array> {
const maybeApi = pdfDoc as unknown as {
getAttachments?: () => Array<{ name: string; data: Uint8Array }>
}
@@ -57,44 +92,21 @@ export async function extractReportJson(
list.find(
(a) => a.name.startsWith('osteo-report') && a.name.endsWith('.json'),
) ?? list[0]
return parseAttachment(hit.data, map)
return hit.data
}
}
const fallback = await traverseForJsonAttachment(pdfDoc)
if (!fallback) throw new Error('No osteo-report attachment found')
return parseAttachment(fallback, map)
return fallback
}
/** Hard ceiling on an attachment (inflated or not): a report is a few hundred KB at most. */
const MAX_ATTACHMENT_BYTES = 16 * 1024 * 1024
function parseAttachment(
bytes: Uint8Array,
map: ReportMigrationMap,
): ExtractedReport {
if (bytes.byteLength > MAX_ATTACHMENT_BYTES) {
throw new Error('attachment too large')
}
const data: unknown = JSON.parse(new TextDecoder().decode(bytes))
const version =
data !== null &&
typeof data === 'object' &&
typeof (data as { version?: unknown }).version === 'number'
? (data as { version: number }).version
: null
// No numeric version at all: not a Faunosteo attachment (legacy message), not an old one.
if (version === null) throw new Error('attachment carries no report version')
try {
return {
...readReportDocument(data, map),
migrated: version !== REPORT_VERSION,
}
} catch (err) {
if (err instanceof UnsupportedReportVersionError) {
throw new IncompatibleReportVersionError(err.foundVersion)
}
throw err
}
function versionOf(data: unknown): number | null {
if (data === null || typeof data !== 'object') return null
const v = (data as { version?: unknown }).version
return typeof v === 'number' ? v : null
}
interface RawObj {
+203
View File
@@ -0,0 +1,203 @@
/**
* The archive check tool (scripts/compare-reports.ts) on the frozen v2 PDF,
* on a synthetic v2 report with removed and unknown items, and on a v3 report.
*/
import { describe, it, expect, beforeAll } from 'vitest'
import { PDFDocument } from 'pdf-lib'
import { readFileSync } from 'node:fs'
import { fileURLToPath } from 'node:url'
import { dirname, resolve } from 'node:path'
import { readReportDocument, type ReportV2 } from '@faunosteo/domain'
import { readAttachedDocument } from '@/pdf/attachment'
import type { SheetIndex } from '@/services/sheetIndex'
import {
compareReport,
diffLines,
parseArgs,
renderComparison,
renderSummary,
UNRESOLVED,
} from '../../scripts/compare/report'
import { loadTestSheet, minimalReport } from '../helpers/sheet'
const HERE = dirname(fileURLToPath(import.meta.url))
const OPTS = { locale: 'fr' as const, identity: false }
let sheet: SheetIndex
function minimalV2(): ReportV2 {
return JSON.parse(
readFileSync(resolve(HERE, '../fixtures/report-v2.minimal.json'), 'utf8'),
) as ReportV2
}
function compareRaw(raw: unknown, identity = false) {
const outcome = readReportDocument(raw, sheet.sheet.migrationMap)
const migrated = (raw as { version: number }).version !== 3
return compareReport(raw, outcome, migrated, sheet, { ...OPTS, identity })
}
beforeAll(async () => {
sheet = await loadTestSheet()
})
describe('compareReport on the frozen v2 archive PDF', () => {
it('shows every legacy zone and finding next to its canonical target', async () => {
const bytes = readFileSync(resolve(HERE, '../fixtures/report-v2.full.pdf'))
const pdf = await PDFDocument.load(new Uint8Array(bytes))
const raw = await readAttachedDocument(pdf)
const c = compareRaw(raw)
const lines = renderComparison('full.pdf', c)
expect(c.source).toBe('v2')
expect(lines[0]).toBe('== full.pdf')
expect(lines).toContain(' v2 → v3 (migrated)')
expect(lines.join('\n')).toContain(
'point-mpbwtst0 left "Hanche (coxo-fémorale)" → hip@main left "Hanche (coxo-fémorale)"',
)
expect(lines.join('\n')).toContain(
'cervical/cervical-observations/stumbling-proprioceptive (!) "Trébuchements / déficit proprioceptif rapporté" → cervical/cervical-consultation-reasons/stumbling-proprioceptive',
)
expect(lines.join('\n')).toContain('severity modérée → moderate')
expect(lines.some((l) => l.includes(UNRESOLVED))).toBe(false)
expect(c.counts).toEqual({
observations: 3,
zones: 5,
unresolvedZones: 0,
findings: 4,
unresolvedFindings: 0,
})
expect(renderSummary('full.pdf', c)).toBe(
'full.pdf · v2→v3 · 3 obs · zones 5/0 unresolved · findings 4/0 unresolved',
)
expect(lines.join('\n')).not.toContain('practitioner')
expect(renderComparison('x', compareRaw(raw, true)).join('\n')).toContain(
'practitioner "Dr. Claire Dupont"',
)
})
})
describe('compareReport on unresolved items', () => {
it('names a removed finding, a finding unknown to the catalog and an unknown zone', () => {
const v2: ReportV2 = {
...minimalV2(),
observations: [
{
id: 'o',
zones: [{ boneId: 'point-gone' }],
findings: [
{
zoneId: 'ssb',
path: ['ssb-dysfunctions', 'sutures', 'sphenozygomatic'],
labelFrSnapshot: 'Suture sphéno-zygomatique',
labelEnSnapshot: 'Sphenozygomatic suture',
requiresVetReferral: true,
},
{
zoneId: 'nowhere',
path: ['x'],
labelFrSnapshot: 'X',
labelEnSnapshot: 'X',
},
],
freeTextFindings: [],
comment: '',
},
],
}
const text = renderComparison('odd.pdf', compareRaw(v2)).join('\n')
expect(text).toContain(
`point-gone ${UNRESOLVED} unknown legacy id, kept as point-gone@main`,
)
expect(text).toContain(
`ssb/ssb-dysfunctions/sutures/sphenozygomatic (!) "Suture sphéno-zygomatique" ${UNRESOLVED} removed from the sheet, kept with its label`,
)
expect(text).toContain(
`nowhere/x "X" ${UNRESOLVED} never in the legacy catalog, kept with its label`,
)
expect(text).toContain('1 zones (1 unresolved) · 2 findings (2 unresolved)')
})
it('reads a v3 report as is and flags what the sheet cannot place', () => {
const report = minimalReport(sheet, {
observations: [
{
id: 'o3',
zones: [{ annotationId: 'hip', viewId: 'main', side: 'left' }],
findings: [
{
findingZoneId: 'hip',
path: ['hip-dysfunctions', 'articular', 'restriction-flexion'],
labelFrSnapshot: 'Restriction en flexion',
labelEnSnapshot: 'Flexion restriction',
},
{
findingZoneId: 'atm',
path: ['atm-observations', 'headshaking'],
labelFrSnapshot: 'Headshaking (secouement de tête)',
labelEnSnapshot: 'Headshaking',
unresolved: true,
},
{
findingZoneId: 'hip',
path: ['hip-dysfunctions'],
labelFrSnapshot: 'Catégorie',
labelEnSnapshot: 'Category',
},
],
freeTextFindings: [],
comment: '',
},
],
})
const c = compareRaw(report)
const text = renderComparison('v3.pdf', c).join('\n')
expect(c.source).toBe('v3')
expect(text).toContain('v3 (read as is)')
expect(text).toContain('hip@main left "Hanche (coxo-fémorale)"')
expect(text).toContain(
'hip/hip-dysfunctions/articular/restriction-flexion "Restriction en flexion"',
)
expect(text).toContain(
`atm/atm-observations/headshaking "Headshaking (secouement de tête)" ${UNRESOLVED} kept with its snapshot`,
)
expect(text).toContain('!! points at a category, not a leaf')
expect(c.counts.unresolvedFindings).toBe(1)
})
it('warns when a v3 report was written on another sheet document', () => {
const report = minimalReport(sheet, {
sheet: { ...sheet.sheet.ref, documentSha256: 'b'.repeat(64) },
})
const text = renderComparison('other.pdf', compareRaw(report)).join('\n')
expect(text).toContain(
'!! written on another sheet document (sha bbbbbbbbbbbb…)',
)
})
})
describe('diff and arguments', () => {
it('diffs the v3 content of a migrated v2 report and its v3 re-reading', () => {
const v2 = minimalV2()
const migrated = readReportDocument(v2, sheet.sheet.migrationMap).report
const a = compareRaw(v2)
const b = compareRaw(migrated)
expect(diffLines(a.normalized, b.normalized)).toEqual([])
const edited = { ...migrated, conclusion: 'changed' }
expect(diffLines(a.normalized, compareRaw(edited).normalized)).toEqual([
'+ conclusion "changed"',
])
})
it('parses files and options, refuses unknown ones', () => {
expect(parseArgs(['a.pdf', '--summary', 'dir', '--locale', 'en'])).toEqual({
files: ['a.pdf', 'dir'],
locale: 'en',
identity: false,
summary: true,
diff: false,
})
expect(parseArgs(['--identity', '--diff', 'a', 'b']).identity).toBe(true)
expect(() => parseArgs(['--locale', 'de'])).toThrow('fr or en')
expect(() => parseArgs(['--json'])).toThrow('unknown option')
})
})
+1 -1
View File
@@ -23,6 +23,6 @@
"outDir": "dist",
"composite": false
},
"include": ["src", "tests"],
"include": ["src", "tests", "scripts"],
"exclude": ["node_modules", "dist"]
}
+24
View File
@@ -3,6 +3,7 @@
Placeholder — to be written when implementing src/pdf/.
Will cover:
- A4 portrait dimensions (595.28 × 841.89 pt)
- Page layout: header / body / footer zones
- Observation page structure
@@ -27,3 +28,26 @@ Import (`importReportFromPdf(bytes, migrationMap)`) accepts:
or finding the map cannot place is kept with its snapshots and flagged
`unresolved: true`; the outcome reports `migrated` and the unresolved counts;
- any other version: `ImportError('incompatible_version', foundVersion)`.
## Checking an archive against its migration
`pnpm report:compare <file.pdf | folder>...` reads the attachment of one or
more PDFs and prints it next to its v3 reading on the bundled reference
sheet, without writing anything:
- a v2 archive PDF: every legacy zone (`data-anno-id`, side, vertebrae) and
every legacy finding path is shown with its canonical target and the
current label of that target (`same` when the path did not change);
- a v3 PDF: every zone and finding is checked against the sheet, and the
header says whether the report was written on the bundled sheet document;
- anything the sheet cannot place is printed as `!! UNRESOLVED` with the
reason: unknown legacy zone id, finding removed from the sheet by the
curation, finding that never belonged to the legacy catalog, or a v3
finding flagged `unresolved` in the report itself.
Options: `--summary` (one line per file, counts and unresolved counts),
`--identity` (print the practitioner, animal and owner names, off by
default), `--locale fr|en` (labels), `--diff a.pdf b.pdf` (compare the v3
content of two PDFs: an archive and the PDF exported after importing it
should print `identical v3 content`). The tool exits 2 when a file could not
be read (no attachment, unknown version, oversized) and prints the reason.
+1 -1
View File
@@ -19,7 +19,7 @@
"lint": "pnpm -r lint",
"format": "prettier --write .",
"check": "pnpm typecheck && pnpm test:run && pnpm lint && pnpm build",
"scaffold:species": "pnpm --filter @faunosteo/app scaffold:species",
"report:compare": "pnpm --filter @faunosteo/app report:compare",
"import:equine": "pnpm --filter @faunosteo/anatomy import:equine",
"db:start": "supabase start",
"db:stop": "supabase stop",
+3
View File
@@ -90,6 +90,9 @@ importers:
happy-dom:
specifier: ^20.10.6
version: 20.10.6
tsx:
specifier: ^4.23.15
version: 4.23.15
typescript:
specifier: ^5.7.2
version: 5.9.3