test(tests): R4.2e browser measurements of the Rust import (module worker page, same perimeter as the TS baseline)

r4-import.mjs runs the 226 baseline files and the hostile classes through a disposable
module worker calling SheetHandle.import_pdf in Chromium 156, Firefox 157 and WebKit 27.2
under the RSS watchdog (not a guaranteed limit); compare-imports.mjs prints TS vs Rust;
serve.mjs gains opt-in mounts (/r4/, /pkg/). Results 2026-10-18-r4-import.json: 666/678
agreements, the 12 others are the 4 declared files; W4 met (max 68 MiB linear); the
1 GiB ceiling traps in every engine.
This commit is contained in:
bchanot committed 2026-10-18 16:58:42 +02:00
1 parent cad485b13c
commit c84186b032
14 files changed
+3135 -17

No files matched your search

+101 -5
View File
@@ -1,10 +1,11 @@
# Browser harness (R4.1e)
# Browser harness (R4.1e, R4.2e)
Measures the application in real browser engines (Chromium, Firefox, WebKit)
through Playwright: the TypeScript baselines of R4 (export and import time
and memory, plan `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md` O12,
§B.3, §B.4, §B.8) and the browser columns of the O9 inflate oracle. Later
slices reuse the server for the CSP and worker proofs.
§B.3, §B.4, §B.8) and the browser columns of the O9 inflate oracle. Since
R4.2e it also measures the Rust import in a disposable module worker on the
baseline's perimeter, its hostile classes and its memory protections.
> **Watchdog caveat.** Every browser runs under a 4 GiB RSS watchdog that is
> **not a guaranteed memory limit**. It samples every 50 ms, so a faster
@@ -23,6 +24,10 @@ slices reuse the server for the CSP and worker proofs.
SHEET_BUNDLE=draft corepack pnpm --filter @faunosteo/app build
```
`r4-import.mjs` does not load the application: it needs the core module
instead, `packages/core-wasm/pkg` (and `pkg-proofs` for the memory proofs),
built by `corepack pnpm wasm:build`. It never builds either.
- Playwright outside the repository (no project dependency): set
`PLAYWRIGHT_NODE_MODULES` to a `node_modules` that holds `playwright-core`,
with its browsers installed in `~/.cache/ms-playwright`. On the
@@ -47,11 +52,17 @@ All from the repository root.
| `node scripts/browser/import-baseline.mjs` | import baseline only (`results/<date>-import.json`) |
| `node scripts/browser/inflate-verdicts.mjs` | O9 browser columns → `crates/core/tests/oracles/r4/inflate.browsers.json` |
| `node scripts/browser/summarize.mjs <results.json>` | prints the markdown tables of a results file |
| `node scripts/browser/serve.mjs --r4` | the same server, plus the R4 import page at `/r4/` and the module builds |
| `node scripts/browser/r4-import.mjs` | Rust import in a module worker, `results/<date>-r4-import.json`, tables on stdout |
| `node scripts/browser/compare-imports.mjs <ts.json> <r4.json>` | TS baseline vs Rust import: durations, RSS growth, per-file outcome agreement |
Options of the baseline scripts: `--engines chromium,firefox,webkit` (default
all three), `--out <file>`, `--reports demo,L,CAP` (export runs to keep),
`--groups base,r3,step6,tolerance,read` (import groups to keep). Progress goes
to stderr, the tables to stdout.
to stderr, the tables to stdout. `r4-import.mjs` takes `--engines`, `--out`,
`--groups`, plus `--sections import,hostile,repeat,proofs`, `--classes <id,…>`
(hostile classes to keep) and `--scratch <dir>` (where the hostile files and
the module copies live during the run, default the OS temporary directory).
## The server (`serve.mjs`)
@@ -67,6 +78,10 @@ to stderr, the tables to stdout.
- `/env.js` is the text `docker/entrypoint.sh` writes when no `APP_*`
variable is set: the local tool, no Supabase (the build's stub `env.js`
is not served).
- `--r4` (or `startServer({ mounts })`) adds `/r4/` (`r4-import-page/`),
`/pkg/` and `/pkg-proofs/` (`packages/core-wasm`). A mounted path is a
file or a 404, never the SPA fallback, and never a file outside its
directory. Without the option the server answers exactly as before.
## What each script measures
@@ -146,15 +161,96 @@ null past the row's limit or on any error), and writes
and each engine's (`null` or `{ length, sha256 }`), plus the rows where an
engine disagrees with Node.
### Rust import (`r4-import.mjs`)
The page `/r4/` (`r4-import-page/`) reproduces the import worker plan
§B.1.5 gives the application at R4.3. The main thread compiles the module
once at page load (`WebAssembly.compileStreaming`) and routes the picked
file as HomeRoute does (first KiB: `%PDF-`, a JSON object, a `.json` name).
Each import then gets its own module worker: the compiled `Module` posted by
structured clone (an engine that refuses the clone is recorded and the
worker fetches the URL instead), the `File` and the sheet bundle text posted
on `ready`, the file read in the worker, the size re-checked (50 MiB PDF,
16 MiB copy), one `SheetHandle`, `import_pdf` or the strict `TextDecoder`
then `draft.restore`, one result string parsed on the main thread,
`terminate()` in every case. Timers: 10 s until `ready`, 120 s for the task.
The script copies `pkg/` and `pkg-proofs/` at start and serves the copies,
so a rebuild during the run cannot mix two modules; the results name the
module by size, sha256, memory maximum and build stamp. One browser per
engine, a fresh context per run, the file set on the page's input
(`setInputFiles`, so the worker reads a real `File`). Sections:
- `import`: the files and groups of `import-baseline.mjs`, same listing and
`.gen.json` expansion. `outcome` is `ok`, `refused` (the envelope's
`kind`), `failed` (a worker failure: phase and kind), `timeout`,
`stopped_by_ceiling` or `error`. Durations: `durationMs` (Node clock, from
setting the file to the result), `pageMs` (call to result), `startMs`
(worker creation to `ready`), `taskMs` and its phases (read, sheet,
run, worker clock), `parseMs`, `compileMs` (page load). Linear memory of
the worker's instance at `ready`, before and after the run. RSS summed and
content before, peak and right after, as the baseline. Chromium: long
tasks of the main thread.
- `hostile`: the classes of `packages/core-wasm/tests/pdf-import.budget.test.ts`
built in Node with its builders: the L and CAP envelopes (the demo archive
read by the shipped module, the report swapped in, a fresh
`archive.snapshot`), 16 MiB dense numbers, dense strings, depth 64, the
dense rescue copy, the 16 MiB inflation, the 16 MiB + 1 bomb, 32 bombs,
the 50 MiB file.
- `repeat`: the L envelope and the 16 MiB dense document imported three
times in one page, the settled level after each: a one-time residue stops
growing, a leak keeps growing.
- `proofs`, on `pkg-proofs` (worker-side fetch and compile, so the worker's
CSP is exercised): `allocate_for_test` 256 MiB then 1100 MiB (the 1 GiB
linear ceiling must trap), and `import_pdf_unbounded_for_test` on the
16 MiB dense document (the cost the OD-3 value cap prevents).
Settled level: forced GC (CDP `HeapProfiler.collectGarbage`) and 1 s in
Chromium, before and after every run; the plan §B.3 check (content process
within 10 MiB of its pre-import level once the worker is terminated) is
recorded per run, never a failure. Firefox and WebKit get 5 s idle instead,
outside the `import` section; idle is not a GC, so their deltas are
indicative only. Every run records the CSP violations seen by the page and
by the worker (`securitypolicyviolation`), page errors and console errors.
### Comparison (`compare-imports.mjs`)
Pairs the runs of a TS baseline file and an R4 file by engine and file.
Per engine and group: files, agreement, median durations (Node clock on
both sides, plus the Rust page clock), max summed and content RSS growth
(peak minus before, the baseline's formula). Outcome agreement: TS `editor`
↔ Rust `ok`, a TS inline error ↔ a Rust refusal or worker failure. Every
disagreement is listed with both outcomes; the declared ones (plan §A.3:
`values-1m` for OD-3, the three overwritten-duplicate lone surrogates for
OD-4) are tagged, each must appear in every engine, and any other one is
printed as `NOT DECLARED`. A finer list compares the message each side
would show (the TS text mapped to its `home.import.*` key, the Rust kind
through the plan §B.1.8 table). A pair present on one side only is printed
as a perimeter difference. The French texts come from
`apps/app/src/i18n/fr.ts`, imported through Node 24's type stripping (Node
prints a `MODULE_TYPELESS_PACKAGE_JSON` warning on stderr, harmless).
The perimeter is the same files, groups, engines, RSS metrics and watchdog;
the page is not. The TS baseline ran the application (Home, core loaded
during the import, hydrate, navigation to `/editor`); the Rust runs use the
measurement page, which compiles the module at page load and stops at the
parsed result. The R4.3 application will be measured by `import-baseline.mjs`
itself.
## Layout
```
scripts/browser/
├─ serve.mjs, ts-baseline.mjs, export-baseline.mjs, import-baseline.mjs
├─ inflate-verdicts.mjs, summarize.mjs
├─ r4-import.mjs, compare-imports.mjs
├─ r4-import-page/ index.html, main.js (compile, routing, one worker per
│ import), worker.js (the import worker), proofs-worker.js
├─ lib/ playwright (load, launch under the watchdog, close), watchdog + its
│ sampler thread, proc (/proc reads), page-probes (init scripts),
│ harness (CLI, results file), summary (tables), app-build, host,
│ inputs, stats, async, paths
│ inputs, stats, async, paths; R4.2e: import-inputs (the baseline's
│ file set), core-module (bundle, module facts and copies, Node
│ instance), hostile-classes (budget-suite builders), r4-summary
└─ results/ raw results files
```