diff --git a/docs/development/R4_J1.md b/docs/development/R4_J1.md new file mode 100644 index 00000000..651b8669 --- /dev/null +++ b/docs/development/R4_J1.md @@ -0,0 +1,74 @@ +# R4 J1: device acceptance of the Rust import + +Plan `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md` §A.4 (J1), owner +conditions BDR-058. The import of the application runs in a disposable +worker on the Rust core since R4.3 (`docs/development/RUST.md` §10.5); the +export is still the TypeScript engine (R4.6). Headless runs in Chromium, +Firefox and WebKit are recorded in RUST.md §10.4 and §10.5; nothing below is +validated until it has run on the owner's devices. + +## 1. Serve the build on the LAN (owner's machine) + +Two surfaces, both on the DEV origin (`scripts/dev-env.sh`, +`FAUNOSTEO_DEV_HOST`), never on every interface: + +1. The development server, as for the R3 acceptance: `pnpm dev` (local + stack) or `pnpm dev:offline` (local tool), `http://:5173`. + No CSP header, HMR on. +2. The production build behind the DEPLOY CSP (what the proofs ran): + `pnpm build:preview` then + `node scripts/browser/serve.mjs --host "$FAUNOSTEO_DEV_HOST" --port 4173`, + `http://:4173`. Static files, the nginx SPA fallback, + the DEPLOY `Content-Security-Policy` with `'wasm-unsafe-eval'` on every + response. `--csp without` serves the same build without the keyword (the + import must then show the engine message). Stop it with Ctrl-C; it binds + the one address given. Run both surfaces for the memory items (2 is the + one that matters for the CSP and the worker chunk's caching). + +## 2. The kit + +`node scripts/r4-j1-kit.mjs [dir]` writes a folder (about 13 MB, synthetic +data only) with `ATTENDU.md` listing the expected message of every file: + +- `1-reference/`: Step 5 v2 and v3 PDFs, the R3 exports (FR / EN, with and + without logos, the real-raster export, the store-path export), the Step 6 + envelopes. Every one opens the editor. +- `2-hostile/`: the 51 PDFs of the O8 corpus with their message (the four + declared R4 differences applied: `values-1m`, `surrogate-content-overwritten`, + `surrogate-version-overwritten`, `two-files-decoy-first`). +- `3-copies/`: rescue copies (valid, BOM, no extension, invalid UTF-8, + invalid JSON, version 1, oversized 18 MB). + +Copy the folder to the devices (AirDrop, a share, a USB key); on iPhone pick +from Files. + +## 3. Checks per device (computer, iPhone, low-end Android: the OD-6 list) + +Record the device, the browser and its version, and the surface (1 or 2). + +| # | Check | Expected | +|---|---|---| +| J1.1 | `1-reference/`: pick each file from Home | the editor opens; the R3 and Step 6 envelopes alert « Restauration complète. Archive Faunosteo importée… »; v2 files alert the migration count; no engine message | +| J1.2 | `2-hostile/`: pick each file | the message of `ATTENDU.md`, the file untouched, Home stays usable (button enabled again) | +| J1.3 | `3-copies/`: pick each file | per `ATTENDU.md`; a refused copy leaves the tab's draft in place | +| J1.4 | Engine unavailable: surface 2 with `--csp without`, pick `r3-demo.fr.logos.pdf` | « Le moteur de lecture n'a pas pu être chargé. Réessayez l'import. »; a second pick shows it again; nothing else changes | +| J1.5 | Engine recovered: back to surface 2 with the keyword, same pick | the editor opens | +| J1.6 | Interrupted import: pick `2-hostile/values-1m.pdf` and tap « Nouveau bilan » at once, or leave Home within the first second | no alert, no editor from the import, the new page intact | +| J1.7 | Offline import: open the editor once (engine loaded), enable airplane mode, back to Home, pick a reference PDF | the editor opens (the worker chunk and the module come from the page and the cache) | +| J1.8 | Large legitimate export: from a computer, open `1-reference/r3-demo.fr.logos.pdf`, then on the phone import the L export (owner: the 60-observation report, export it from the editor on the computer first, about 42 MB) | the editor opens within the task timer (20 s); note the time to the editor; the tab survives | +| J1.9 | Memory on the phone: 6 imports of the L export in a row (Home → pick → editor → Home) | no reload of the tab by the system, no Safari memory warning; note any slowdown | +| J1.10 | Repetition: 20 picks of `2-hostile/values-1m.pdf` on the phone | 20 times the message, the tab alive | +| J1.11 | Rescue copy under the cap on the phone: `3-copies/copie-valide.json`, then `copie-trop-grande.json` | the editor; the refusal | +| J1.12 | Timing: note the time from the pick to the editor for `r3-demo.fr.logos.pdf` and for the L export per device | reported in RUST.md §10.5; the 20 s timer is confirmed or moved by the §B.3 rule (≥ 10 × the slowest legitimate import) | + +Browsers: Safari on iPhone (the WebKit path that headless WPE does not +stand for), Chrome and Firefox on the computer, the Android browser. A +device or browser not run is reported as not run. + +## 4. What to report + +The table above with the result per device, the timings of J1.8 and J1.12, +any message that differs from `ATTENDU.md`, any system reload or memory +warning, and the device list actually run. The results go to RUST.md §10.5 +and `docs/development/PDF_IMPORT_HARDENING.md` §5 (the gate stays open: +imports are not offered to external users). diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 50e9ad8b..69d8dc7f 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -1328,21 +1328,66 @@ Suites: handler and client in-process (`core.importWorker`, `core.importClient`), the six existing Home / core-failure suites migrated to the real worker logic (53 cases, titles kept), 458 app tests. -Memory (R4.3a, measurement page, three engines, summed / content RSS -growth MiB, TS · worker read · page read where measured): +Memory, large files (summed / content RSS growth MiB; TS reader · Rust, +worker read, R4.3a page · Rust, page read, R4.3d application build): | File | Chromium | Firefox | WebKit | |---|---|---|---| -| 50 MiB file, attachment at the end | 56/55 · 65/66 · page read pending | 79/59 · 67/65 · pending | 76/60 · 272/251 · 106 (probe) | -| 16 MiB rescue copy | – · 33/33 | – · 29/30 | – · 133/124 | -| 16 MiB dense numbers | – · 33/33 | – · 32/30 | – · 108/98 | -| CAP envelope | – · 16/17 | – · 19/16 | – · 41/37 | +| 50 MiB file (`size-exactly-50mib`) | 56/55 · 65/66 · 75/73 | 79/59 · 67/65 · 103/78 | 76/60 · 272/251 · 145/126 | +| 32 MiB (`32-raw-1mib-brackets`) | 46/40 · 48/48 · 75/70 | 53/52 · 49/47 · 88/62 | 50/39 · 185/170 · 124/112 | +| 20 MiB (`32-open-streams`) | 27/26 · 38/36 · 61/55 | 26/25 · 35/34 · 63/44 | 35/28 · 127/116 · 96/88 | +| 10 MiB (`one-10mib`, opens) | 58/53 · 26/26 · 35/33 | 61/38 · 26/25 · 58/36 | 79/73 · 81/74 · 76/72 | +| 16 MiB rescue copy (page) | – · 33/33 · – | – · 29/30 · – | – · 133/124 · – | +| CAP envelope (page) | – · 16/17 · – | – · 19/16 · – | – · 41/37 · – | -The streaming sink alone removed the file's JavaScript copy in Chromium and -Firefox (within 9 MiB of the TypeScript, or below it); the rescue copy drops -from 52 to 20 MiB of linear memory and reads faster (221 → 128 ms in -Chromium). The page-side read of the application build is measured in the -R4.3d run below. +The streaming sink removed the file's JavaScript copy; the page-side read +brought WebKit from 272 to 145 MiB on a 50 MiB file (the probe without the +application around it gave 106; the application build adds about 40, three +samples 141 to 158). What remains in WebKit is its own cost of the module's +linear memory (the sink alone, no file read, measured at 81 MiB for 55 MiB +of linear memory) plus the worker; it cannot be removed without scanning +the file without holding it, which the reader's contract (§B.1.2, one 16 MiB +shared budget over candidates found anywhere in the file) does not allow +cheaply. Note the 50 MiB cap is older than R4 (`MAX_PDF_BYTES`): an export of +about 140 observations with rasters reaches it; L (121 views, 42 MB) imports. + +Application build, the same 226 files and engines as the TypeScript baseline +(R4.3d, `scripts/browser/results/2026-10-18-r4-app-import.json`, dist of +e659924 built with `SHEET_BUNDLE=draft`; the hardenings of 8ddf185 are not in +that build, they change no outcome): outcome and message identical on 666 of +678 pairs, the 12 others the four declared files; 0 page errors; W5: 0 +Chromium long task over 50 ms (the TypeScript showed 91 and 173 ms); small +files cost 25 to 35 ms more (the worker start and its sheet handle per +import); the `tolerance` group costs half the TypeScript's memory. Duration +medians, TS → Rust (ms): Chromium base 53 → 80, r3 63 → 100, step6 61 → 90, +tolerance 57 → 82, read 66 → 75; Firefox 39 → 71, 73 → 107, 69 → 104, +53 → 79, 78 → 81; WebKit 52 → 80, 62 → 89, 66 → 91, 59 → 78, 75 → 77. + +CSP (`r4-app-csp.mjs`): with the keyword the reference PDF opens the editor +in the three engines (the known footer image violation only); without it, +« Le moteur de lecture n'a pas pu être chargé. Réessayez l'import. » in the +three engines, the button enabled again, no page error, every worker closed; +Chromium and Firefox report the worker's `wasm-eval` refusal through +`report-uri`, WebKit reports nothing for a worker. + +Leak series on the application (R4.3d, content RSS, init = first import, +then six imports of the same document with `page.goto('/')` and the replace +dialog between them, 2 s idle each, 5 s at the end; init delta / growth over +the last five / the same after the final idle, three repetitions): Chromium +(forced GC) L 49 / 7.6, 9.7, 8.0 / 1.6, 3.6, 0.5; dense 10.6 / 2.1 ×3 / −0.6; +copy 42 / 6.8, 11.0, 4.8 / −3.5, 1.1, −4.4, per-import deltas at or under 2 MiB +after init. Firefox (idle only) L 72 / 42, 35, 44 / −5, −7, −13; dense 36 / +8.8, 9.1, 10.7 / −6; copy 60 / −2.6, 17.3, −5.0 / −21: a GC sawtooth reclaimed +by the final reading, as its navigation-only control. WebKit (idle only) L +102 to 110 / 53, 67, 53 / 44, 65, 46; dense 44 / 38, 37, 34 / 40; copy 88 / +69, 35, 44 / 61, 35, 41: above the 10 MiB tolerance in every series, but the +navigation-only control (same readings, no pick) grows 27 MiB too (about +5 MiB per same-tab navigation, the page cache), and a 12-import series on one +page levels off at 365 to 382 MiB; a surplus over the control remains (about +2 MiB per import for the refused document, 6 for one that opens the editor) +that cannot be called reclaimable without a forced GC. Twenty picks of the +dense document on one WebKit Home page: 20 workers created and closed, 0 +page error, content 330 → 379 MiB, peak 405. Leak series (R4.3a measurement page, content RSS, init = first import): Chromium (GC forced) init 9 to 11 MiB, growth over the next five within @@ -1355,3 +1400,26 @@ steps (dense +17 to +20 once; copy +12 to +15 then +17 to +19) and drop 3 to off (−2.3 over its last five). Twenty consecutive dense imports in WebKit: all refused, no worker `error` event (the one seen in an R4.2e smoke run did not reproduce). + +Residual risks and open points (R4.3, for the J1 checkpoint): + +- WebKit, 50 MiB file: 145 MiB of content growth against 76 for the + TypeScript (BDR-058 gate "resolved or decided with numbers"): the owner + decides between accepting it (an absolute peak of about 150 MiB, below the + worker-read 324), lowering the PDF cap, or a streaming scanner later. +- WebKit leak criterion: not demonstrable headless without a forced GC + (above); the one-page series levels off; the device run at J1 gives the + real signal (Safari's memory warning or a reload). +- A hidden tab pauses the task timer by design (§B.1.5 item 3): a hostile + file keeps the worker's CPU and up to about 51 MiB busy in the background + until the tab is shown again; the core's own bounds (R4.2) are the only + limit then. +- The page reads the first KiB of the file to route it (unchanged since + R3); nothing else of the file is parsed on the page. +- The `IMPORT_TIMEOUT_MS` 20 s and the 10 s start timer are hypotheses to + confirm on the owner's devices (plan §B.3 rule). +- The import is 25 to 35 ms slower on small files (worker start and sheet + handle per import); posting the handle's work is not worth a shared + worker, which would keep hostile memory alive between imports. +- The TypeScript reader stays in `packages/pdf` until R4.8 (parity and the + full switch); `apps/app/src/pdf/import.ts` is now used by tests only. diff --git a/scripts/browser/serve.mjs b/scripts/browser/serve.mjs index 159fef22..a23e4ea3 100644 --- a/scripts/browser/serve.mjs +++ b/scripts/browser/serve.mjs @@ -187,6 +187,7 @@ export function startServer({ csp = 'with', mounts = {}, reportViolations = false, + host = HOST, } = {}) { const enforced = cspVariant(productionCsp(), csp) const policy = reportViolations @@ -198,8 +199,8 @@ export function startServer({ ) return new Promise((done, fail) => { server.once('error', fail) - server.listen(port, HOST, () => { - const url = `http://${HOST}:${server.address().port}` + server.listen(port, host, () => { + const url = `http://${host}:${server.address().port}` done({ url, csp: policy, reports, close: () => server.close() }) }) }) @@ -211,6 +212,9 @@ async function main() { port: { type: 'string', default: '4173' }, csp: { type: 'string', default: 'with' }, r4: { type: 'boolean', default: false }, + // The owner's device acceptance (J1): the production build behind the DEPLOY CSP on + // the DEV origin (scripts/dev-env.sh), never on every interface. + host: { type: 'string', default: HOST }, }, }) if (!['with', 'without'].includes(values.csp)) { @@ -220,6 +224,7 @@ async function main() { port: Number(values.port), csp: values.csp, mounts: values.r4 ? R4_MOUNTS : {}, + host: values.host, }) console.log(`serving apps/app/dist on ${server.url}`) if (values.r4) console.log(`R4 import page on ${server.url}/r4/`) diff --git a/scripts/r4-j1-kit.mjs b/scripts/r4-j1-kit.mjs new file mode 100644 index 00000000..5bf4d136 --- /dev/null +++ b/scripts/r4-j1-kit.mjs @@ -0,0 +1,132 @@ +// R4 J1 device acceptance kit (plan 2026-10-17 §A.4, BDR-058): gathers in one +// folder the files the owner picks from a phone or a computer, each with the +// message expected (docs/development/R4_J1.md). Reference PDFs (Step 5 v2 and +// v3, Step 6 envelopes, R3 exports), a hostile folder with the expected +// message per file (from the frozen O8 oracle, the declared R4 differences +// applied), rescue copies (valid, BOM, invalid UTF-8, oversized, refused) and +// the large synthetic L export built like the application exports it is NOT +// here (it needs the browser: export the L report from the editor, see the +// doc). Synthetic data only. Usage: node scripts/r4-j1-kit.mjs [out-dir] +// (default: a `faunosteo-j1-kit` folder next to the repository). +import { mkdirSync, readdirSync, readFileSync, writeFileSync } from 'node:fs' +import { join, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' + +const ROOT = resolve(fileURLToPath(new URL('..', import.meta.url))) +const FIXTURES = join(ROOT, 'apps/app/tests/fixtures') +const OUT = resolve(process.argv[2] ?? join(ROOT, '..', 'faunosteo-j1-kit')) + +// The FR messages of apps/app/src/i18n/fr.ts, by kind (kept in step by hand). +const MESSAGES = { + editor: 'ouvre le bilan dans l’éditeur', + invalid_pdf: "Ce fichier n'est pas un PDF lisible.", + no_attachment: + "Ce PDF n'a pas été généré par Faunosteo (pas de données détectées).", + attachment_stripped: + 'Ce PDF Faunosteo a perdu ses données embarquées (pièce jointe supprimée…). Utilisez le fichier d’origine.', + incompatible_version: + 'Ce PDF est au format v{version}, que cette version de l’application ne sait pas lire…', + lone_surrogate: + 'Ce PDF Faunosteo contient un texte mal encodé que cette version ne peut pas relire sans l’altérer. Import refusé…', + invalid_copy: + 'Ce fichier n’est pas une copie de secours Faunosteo lisible par cette version. Import refusé…', +} + +/** The R4 outcome of an O8 case (RUST.md §10.3 declared differences applied). */ +const DECLARED = { + 'values-1m': 'attachment_stripped', + 'surrogate-content-overwritten': 'lone_surrogate', + 'surrogate-version-overwritten': 'lone_surrogate', + 'two-files-decoy-first': 'lone_surrogate', +} + +function copy(from, toDir, name = from.split('/').pop()) { + mkdirSync(toDir, { recursive: true }) + writeFileSync(join(toDir, name), readFileSync(from)) + return name +} + +const lines = ['# Kit J1 (R4) — fichiers et message attendu', ''] + +// 1. Reference PDFs: every one opens the editor. +lines.push('## 1-reference : chaque fichier ouvre l’éditeur', '') +const reference = join(OUT, '1-reference') +for (const f of [ + 'report-v2.full.pdf', + 'report-v2.minimal.pdf', + 'report-v3.bare.pdf', +]) { + lines.push(`- ${copy(join(FIXTURES, f), reference)} (Step 5)`) +} +for (const f of readdirSync(join(FIXTURES, 'pdf/r3')).filter((n) => + n.endsWith('.pdf'), +)) { + lines.push( + `- ${copy(join(FIXTURES, 'pdf/r3', f), reference, `r3-${f}`)} (R3)`, + ) +} +lines.push( + `- ${copy(join(FIXTURES, 'pdf/r3/real/v2full-obs1.fr.logos.pdf'), reference, 'r3-real-raster.pdf')} (R3, rasters réels)`, +) +for (const f of readdirSync(join(FIXTURES, 'pdf/step6')).filter((n) => + n.endsWith('.pdf'), +)) { + lines.push( + `- ${copy(join(FIXTURES, 'pdf/step6', f), reference, `step6-${f}`)} (Step 6)`, + ) +} + +// 2. Hostile folder from the O8 oracle. +lines.push('', '## 2-hostile : message attendu par fichier (corpus O8)', '') +const oracle = JSON.parse( + readFileSync( + join(ROOT, 'crates/core/tests/oracles/r4/tolerance.json'), + 'utf8', + ), +) +const hostile = join(OUT, '2-hostile') +for (const c of oracle.cases) { + const kind = DECLARED[c.id] ?? (c.outcome.ok ? 'editor' : c.outcome.kind) + const name = copy(join(ROOT, c.pdf), hostile) + const expected = MESSAGES[kind].replace( + '{version}', + String(c.outcome.foundVersion ?? ''), + ) + lines.push(`- ${name} : ${expected}`) +} + +// 3. Rescue copies. +lines.push('', '## 3-copies : copies de secours', '') +const copies = join(OUT, '3-copies') +mkdirSync(copies, { recursive: true }) +const demo = readFileSync(join(FIXTURES, 'report-v3.demo.json'), 'utf8') +const write = (name, data, note) => { + writeFileSync(join(copies, name), data) + lines.push(`- ${name} : ${note}`) +} +write('copie-valide.json', demo, MESSAGES.editor) +write('copie-bom.json', `${demo}`, `${MESSAGES.editor} (BOM accepté)`) +write( + 'copie-sans-extension', + demo, + `${MESSAGES.editor} (routée par le contenu)`, +) +write( + 'copie-utf8-invalide.json', + Buffer.concat([ + Buffer.from('{"version": 3, "x": "'), + Buffer.from([0xff]), + Buffer.from('"}'), + ]), + MESSAGES.invalid_copy, +) +write('copie-json-invalide.json', '{"version": 3,', MESSAGES.invalid_copy) +write('copie-version-1.json', '{"version": 1}', MESSAGES.invalid_copy) +write( + 'copie-trop-grande.json', + `{"version":3,"a":[${'1,'.repeat((9 * 1024 * 1024) / 2)}1]}`, + `${MESSAGES.invalid_copy} (18 Mo, au-dessus de 16 Mio)`, +) + +writeFileSync(join(OUT, 'ATTENDU.md'), `${lines.join('\n')}\n`) +console.log(`kit written to ${OUT} (${lines.length} lines in ATTENDU.md)`)