Files
faunosteo/scripts/browser/README.md
T

414 lines
25 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Browser harness (R4.1e, R4.2e, R4.3a, R4.3d)
Measures the application in real browser engines (Chromium, Firefox, WebKit)
through Playwright: the TypeScript baselines of R4 (export and import time
and memory, plan `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md` O12,
§B.3, §B.4, §B.8) and the browser columns of the O9 inflate oracle. Since
R4.2e it also measures the Rust import in a disposable module worker on the
baseline's perimeter, its hostile classes and its memory protections; since
R4.3a, the worker's two read paths on the large files and the leak series;
since R4.3d, the application itself on the Rust path (its import worker):
the import baseline against the TypeScript one, the CSP proof, the leak
series and the WebKit repetition through the real Home page.
> **Watchdog caveat.** Every browser runs under a 4 GiB RSS watchdog that is
> **not a guaranteed memory limit**. It samples every 50 ms, so a faster
> allocation can overshoot the ceiling before the kill; summed VmRSS counts
> shared pages once per process; a process outside the browser's tree is not
> seen. A run it stops is recorded as `stopped_by_ceiling`, which is a
> result, not a harness failure. The same sampling misses a peak shorter
> than 50 ms (an import that reads, peaks and ends within one interval).
## Prerequisites
- Linux (the watchdog reads `/proc`), Node 24.
- A production build that embeds the reference sheet. The scripts check it
and refuse to run without it; they never build by themselves:
```bash
SHEET_BUNDLE=draft corepack pnpm --filter @faunosteo/app build
```
`r4-import.mjs` does not load the application: it needs the core module
instead, `packages/core-wasm/pkg` (and `pkg-proofs` for the memory proofs),
built by `corepack pnpm wasm:build`. It never builds either.
- Playwright outside the repository (no project dependency): set
`PLAYWRIGHT_NODE_MODULES` to a `node_modules` that holds `playwright-core`,
with its browsers installed in `~/.cache/ms-playwright`. On the
development machine:
```bash
export PLAYWRIGHT_NODE_MODULES=$HOME/.local/lib/node_modules/gsd-pi/node_modules
```
(playwright-core 1.64.0: Chromium headless shell 156, Firefox 157,
WebKit 27.2, all headless.)
## Commands
All from the repository root.
| Command | What it does |
| -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `node scripts/browser/serve.mjs [--port 4173] [--csp with\|without]` | serves `apps/app/dist` on `127.0.0.1` until Ctrl-C |
| `node scripts/browser/ts-baseline.mjs` | export then import baseline, one file `results/<date>-ts-baseline.json`, tables on stdout |
| `node scripts/browser/export-baseline.mjs` | export baseline only (`results/<date>-export.json`) |
| `node scripts/browser/import-baseline.mjs` | import baseline only (`results/<date>-import.json`) |
| `node scripts/browser/inflate-verdicts.mjs` | O9 browser columns → `crates/core/tests/oracles/r4/inflate.browsers.json` |
| `node scripts/browser/summarize.mjs <results.json>` | prints the markdown tables of a results file |
| `node scripts/browser/serve.mjs --r4` | the same server, plus the R4 import page at `/r4/` and the module builds |
| `node scripts/browser/r4-import.mjs` | Rust import in a module worker, `results/<date>-r4-import.json`, tables on stdout |
| `node scripts/browser/compare-imports.mjs <ts.json> <r4.json>` | TS baseline vs Rust import: durations, RSS growth, per-file outcome agreement |
| `node scripts/browser/r4-large-files.mjs` | files of 10 MiB and more, both read paths, `results/<date>-r4-large-files.json`, table |
| `node scripts/browser/r4-leak-series.mjs` | leak series per engine and document, WebKit repetition, `results/<date>-r4-leak-series.json` |
| `node scripts/browser/import-baseline.mjs --out <file>` | R4.3d: the same baseline on the Rust application build (`results/<date>-r4-app-import.json`) |
| `node scripts/browser/compare-app-imports.mjs <ts.json> <rust.json>` | two application import baselines: groups, large files, outcome and message per file |
| `node scripts/browser/r4-app-csp.mjs` | the import under both CSP variants and `report-uri`, `results/<date>-r4-csp.json` |
| `node scripts/browser/r4-app-leak-series.mjs` | leak series and WebKit repetition on Home, `results/<date>-r4-app-leak-series.json` |
Options of the baseline scripts: `--engines chromium,firefox,webkit` (default
all three), `--out <file>`, `--reports demo,L,CAP` (export runs to keep),
`--groups base,r3,step6,tolerance,read` (import groups to keep). Progress goes
to stderr, the tables to stdout. `r4-import.mjs` takes `--engines`, `--out`,
`--groups`, plus `--sections import,hostile,repeat,proofs`, `--classes <id,…>`
(hostile classes to keep), `--read stream|arrayBuffer` (the worker's read
path, default `stream`) and `--scratch <dir>` (where the hostile files and
the module copies live during the run, default the OS temporary directory).
`r4-large-files.mjs` takes `--engines`, `--paths stream,arrayBuffer`, `--ts
<ts-baseline.json>` (default the newest `results/*-ts-baseline.json`),
`--scratch`, `--out`. `r4-leak-series.mjs` takes `--engines`, `--sections
leak,webkit-repeat`, `--reps 3`, `--imports 6`, `--webkit-imports 20`,
`--read`, `--scratch`, `--out`. `r4-app-csp.mjs` takes `--engines`, `--out`;
`r4-app-leak-series.mjs` takes `--engines`, `--sections leak,webkit-repeat`
(add `control` for the navigation-only control),
`--reps 3`, `--imports 6`, `--webkit-imports 20`, `--scratch`, `--out`. The
results files are written before the tables are printed. Exit codes:
`compare-app-imports.mjs` and `r4-app-csp.mjs` exit 1 when a difference is
not declared or a run is not as expected.
## The server (`serve.mjs`)
- `apps/app/dist` with the SPA fallback of `docker/nginx.conf`: an existing
file is served, a missing `/assets/` file is a 404, any other path gets
`index.html`. Binds `127.0.0.1` only; the LAN DEV origin is unaffected.
- `.wasm` as `application/wasm` (uncompressed; production serves the
`.wasm.gz` sibling with `gzip_static`).
- The `Content-Security-Policy` of DEPLOY.md (the nginx `add_header` line,
read at start) on every response, with `'wasm-unsafe-eval'` added to
`script-src` as DEPLOY.md requires for R2+ images; `--csp without` serves
the header without the keyword.
- `/env.js` is the text `docker/entrypoint.sh` writes when no `APP_*`
variable is set: the local tool, no Supabase (the build's stub `env.js`
is not served).
- `--r4` (or `startServer({ mounts })`) adds `/r4/` (`r4-import-page/`),
`/pkg/` and `/pkg-proofs/` (`packages/core-wasm`). A mounted path is a
file or a 404, never the SPA fallback, and never a file outside its
directory. Without the option the server answers exactly as before.
- `startServer({ reportViolations: true })` (R4.3d, `r4-app-csp.mjs`; no CLI
option) appends `report-uri /__csp-report` to the policy, its directives
unchanged, and keeps every report the browser POSTs there (64 KiB cap per
report) in `server.reports`.
## What each script measures
Every run records the engine version; every results file records the host
(CPU, RAM, MemAvailable at start and end), the build (mtime, chunk sizes,
HEAD), the served CSP and the watchdog facts with its caveat. `headCommit`
is HEAD when the run starts, which is not always the commit the build was
made from; since R4.3d the build facts carry each chunk's sha256 (sheet,
entry, wasm, import worker), which identify the build itself.
Memory comes from the watchdog's samples of the browser's process tree:
summed VmRSS of all its processes, the content processes alone (Chromium
renderers, Firefox Web Content, WebKit `WPEWebProcess`), and the largest
single process.
### Export (`export-baseline.mjs`)
Reports: demo (`report-v3.demo.json`, 3 runs), L and CAP
(`tests/fixtures/r4/reports/`, 3 runs and 1 run), locale fr, each run in a
fresh browser and context. Before navigation the tab gets `osteo:draft`
exactly as `persistence.ts` writes it (`JSON.stringify(report)`),
`osteo:disclaimer-shown` (EditorRoute's first-run flag) and `osteo:locale`
`fr`. The harness opens `/editor`, waits for the canvas and clicks
« Exporter en PDF ».
- `exportMs`: page clock, from the click to `URL.createObjectURL` of the PDF
Blob (download.ts); `clickToEndMs`: Node clock, click to the download
event.
- `pdfBytes`, the suggested file name.
- `rasters`: every PNG Blob `HTMLCanvasElement.prototype.toBlob` produced
(wrapped, calls through): count, sizes; the first is the partner SVG logo,
the others are the views; `dataUrlFallbacks` counts `toDataURL` calls.
- RSS before (settled), peak (click to the end of the export), after
(settled). Settled = after a forced GC (CDP `HeapProfiler.collectGarbage`)
and 1 s in Chromium, after 5 s idle in Firefox and WebKit. The PDF Blob is
still referenced by its object URL for 30 s after the download
(download.ts), so "after" includes it.
- Chromium JS heap (CDP `Performance.getMetrics`), before and after.
- Every dialog (accepted), page errors, console errors, CSP violations.
- `outcome`: `downloaded`, `stopped_by_ceiling`, `timeout` (5 min demo,
30 min L, 45 min CAP), `error`. A run that did not download keeps its
peak, `stoppedAtBytes` (summed RSS when the watchdog fired) and
`rastersBeforeEnd` (PNG blobs encoded so far, read every 2 s).
### Import (`import-baseline.mjs`)
One browser per engine (relaunched after a stop), a fresh context per file.
The harness opens `/`, waits for « Choisir un fichier » to be enabled and
sets the file on the import input. The run ends at the first of: navigation
to `/editor`, an alert, the inline import error (`timeout` after 120 s).
Files: `report-v2.full.pdf`, `report-v2.minimal.pdf`, `report-v3.bare.pdf`,
every PDF under `pdf/r3/` (recursively), `pdf/step6/`, `pdf/r4/tolerance/`,
and the hostile inputs of `pdf/r4/read/` as `crates/core/tests/oracles/r4/
read.json` lists them (inputs listed there but absent are reported as
missing). A `<slug>.gen.json` description is expanded (checked against its
length and digest) to a temporary `<slug>.pdf`, removed after its run.
- `durationMs`: Node clock, from setting the file to the first end;
`pageMs`: page clock, from the input's `change` event to the first alert
or navigation.
- `outcome` (`editor`, `error`, `dialog`, `none`, `timeout`,
`stopped_by_ceiling`), the alert texts, the inline error text, the final
path.
- RSS (summed and content) before the file is set and peak until the first
end.
- Chromium only: main-thread long tasks during the import
(`PerformanceObserver('longtask')`): count, longest, total. Firefox and
WebKit do not expose the entry type (`null`).
### O9 inflate verdicts (`inflate-verdicts.mjs`)
Reads `crates/core/tests/oracles/r4/inflate.json` and its inputs under
`apps/app/tests/fixtures/pdf/r4/inflate/` (each checked against the recorded
length and sha256), runs in each engine, on the application's Home page, a
copy of `inflateFlate` from the R3 engine (now the frozen N-1 copy `packages/core-wasm/tests/n-1/scanner/attachment/inflate.ts`)
(`DecompressionStream('deflate')`, 16 KiB awaited slices, concurrent drain,
null past the row's limit or on any error), and writes
`inflate.browsers.json`: per row, Node's verdict copied from `inflate.json`
and each engine's (`null` or `{ length, sha256 }`), plus the rows where an
engine disagrees with Node.
### Rust import (`r4-import.mjs`)
The page `/r4/` (`r4-import-page/`) reproduces the import worker plan
§B.1.5 gives the application at R4.3. The main thread compiles the module
once at page load (`WebAssembly.compileStreaming`) and routes the picked
file as HomeRoute does (first KiB: `%PDF-`, a JSON object, a `.json` name).
Each import then gets its own module worker: the compiled `Module` posted by
structured clone (an engine that refuses the clone is recorded and the
worker fetches the URL instead), the `File` and the sheet bundle text posted
on `ready`, the size re-checked before any read (a PDF over 50 MiB answers
`invalid_pdf`, a copy over 16 MiB `invalid`), the file read in the worker,
one `SheetHandle`, one result string parsed on the main thread,
`terminate()` in every case. Timers: 10 s until `ready`, 120 s for the task.
The worker reads the file one of two ways (`readPath`, per call or for the
page with `/r4/?read=arrayBuffer`):
- `stream` (default, R4.3a): `file.stream()` pushed chunk by chunk into a
`FileBytes(file.size)` sink in linear memory, then `import_pdf_bytes` or
`restore_copy_bytes`, which consume the sink (the core does the copy's
strict UTF-8 decode, BOM and value budget). A BYOB reader reads into one
reused 1 MiB buffer, so no read leaves a dead chunk for the GC; a stream
that refuses BYOB is read with its default reader, each chunk dropped once
pushed; an engine without `stream` reads the file whole and pushes it
once. A read that ends short of `file.size`, or a push past it, is a
`read` failure, never a PDF verdict.
- `arrayBuffer` (R4.2e): one ArrayBuffer of the file, `import_pdf` (the glue
copies it into linear memory) or the strict `TextDecoder` then
`draft.restore`, both alive during the call.
The result's `diag` names the path asked and the one that ran (`readVia`:
`stream-byob`, `stream-default`, `arrayBuffer-fallback`, `arrayBuffer`),
the chunk count and the largest chunk. Every `error` and `messageerror`
event of a worker is recorded with its text (`workerEvents`); the first one
settles the run as a `failed` outcome, none is suppressed.
The script copies `pkg/` and `pkg-proofs/` at start and serves the copies,
so a rebuild during the run cannot mix two modules; the results name the
module by size, sha256, memory maximum and build stamp. One browser per
engine, a fresh context per run, the file set on the page's input
(`setInputFiles`, so the worker reads a real `File`). Sections:
- `import`: the files and groups of `import-baseline.mjs`, same listing and
`.gen.json` expansion. `outcome` is `ok`, `refused` (the envelope's
`kind`), `failed` (a worker failure: phase and kind), `timeout`,
`stopped_by_ceiling` or `error`. Durations: `durationMs` (Node clock, from
setting the file to the result), `pageMs` (call to result), `startMs`
(worker creation to `ready`), `taskMs` and its phases (read, sheet,
run, worker clock), `parseMs`, `compileMs` (page load). Linear memory of
the worker's instance at `ready`, before and after the run. RSS summed and
content before, peak and right after, as the baseline. Chromium: long
tasks of the main thread.
- `hostile`: the classes of `packages/core-wasm/tests/pdf-import.budget.test.ts`
built in Node with its builders: the L and CAP envelopes (the demo archive
read by the shipped module, the report swapped in, a fresh
`archive.snapshot`), 16 MiB dense numbers, dense strings, depth 64, the
dense rescue copy, the 16 MiB inflation, the 16 MiB + 1 bomb, 32 bombs,
the 50 MiB file.
- `repeat`: the L envelope and the 16 MiB dense document imported three
times in one page, the settled level after each: a one-time residue stops
growing, a leak keeps growing.
- `proofs`, on `pkg-proofs` (worker-side fetch and compile, so the worker's
CSP is exercised): `allocate_for_test` 256 MiB then 1100 MiB (the 1 GiB
linear ceiling must trap), and `import_pdf_unbounded_for_test` on the
16 MiB dense document (the cost the OD-3 value cap prevents).
Settled level: forced GC (CDP `HeapProfiler.collectGarbage`) and 1 s in
Chromium, before and after every run; the plan §B.3 check (content process
within 10 MiB of its pre-import level once the worker is terminated) is
recorded per run, never a failure. Firefox and WebKit get 5 s idle instead,
outside the `import` section; idle is not a GC, so their deltas are
indicative only. Every run records the CSP violations seen by the page and
by the worker (`securitypolicyviolation`), page errors and console errors.
### Large files (`r4-large-files.mjs`)
The inputs of the import baseline's `read` group of 10 MiB and more (sizes
from the files or their `.gen.json` descriptions), plus the hostile classes
`file-50mib`, `CAP-envelope`, `dense-numbers-16mib` and
`dense-numbers-copy-16mib`, each imported once per engine and per read path
(`stream`, `arrayBuffer`), the two paths of a file back to back and their
order alternated from one file to the next. Same row as `r4-import.mjs`: page
and Node-clock durations, summed and content RSS before, peak and right
after, linear memory, chunks; a fresh context per run, the worker
terminated. The settled level before and after each run is a forced GC + 1 s
in Chromium (the after-GC delta) and 5 s idle in Firefox and WebKit. The
table puts beside both paths the TS baseline's growth for the same file
(peak minus the level before the file is set, Node-clock duration), copied
into the results file (`tsBaseline`).
### Leak series (`r4-leak-series.mjs`)
Per engine, per document (the L envelope, the 16 MiB dense numbers refused
by the value budget, the 16 MiB rescue copy), three repetitions, each in ONE
page: the file set once, the start level, one import (the init cost, counted
apart), six more imports, the worker terminated after each, 2 s idle, a
reading after each import, then 5 s idle and a final reading. Chromium forces
a GC (CDP) and waits 1 s before every reading; Firefox and WebKit are read
after idle only, which is not a GC, so their levels are indicative. The row
keeps the content and summed levels, the init delta (L1 − L0), the six
per-import deltas, the recurring growth over the last five imports
(L7 − L2) and the same at the final reading, against the 10 MiB tolerance of
BDR-058, and the series peak. `webkit-repeat`: WebKit, one page, 20
consecutive imports of the 16 MiB dense document, a worker per import; the
outcomes, every worker `error` / `messageerror` event with its text, the
content level after each import and the series peak.
### Comparison (`compare-imports.mjs`)
Pairs the runs of a TS baseline file and an R4 file by engine and file.
Per engine and group: files, agreement, median durations (Node clock on
both sides, plus the Rust page clock), max summed and content RSS growth
(peak minus before, the baseline's formula). Outcome agreement: TS `editor`
↔ Rust `ok`, a TS inline error ↔ a Rust refusal or worker failure. Every
disagreement is listed with both outcomes; the declared ones (plan §A.3:
`values-1m` for OD-3, the three overwritten-duplicate lone surrogates for
OD-4) are tagged, each must appear in every engine, and any other one is
printed as `NOT DECLARED`. A finer list compares the message each side
would show (the TS text mapped to its `home.import.*` key, the Rust kind
through the plan §B.1.8 table). A pair present on one side only is printed
as a perimeter difference. The French texts come from
`apps/app/src/i18n/fr.ts`, imported through Node 24's type stripping (Node
prints a `MODULE_TYPELESS_PACKAGE_JSON` warning on stderr, harmless).
The perimeter is the same files, groups, engines, RSS metrics and watchdog;
the page is not. The TS baseline ran the application (Home, core loaded
during the import, hydrate, navigation to `/editor`); the Rust runs use the
measurement page, which compiles the module at page load and stops at the
parsed result. The R4.3 application is measured by `import-baseline.mjs`
itself and compared by `compare-app-imports.mjs` (below).
### Application comparison (`compare-app-imports.mjs`, R4.3d)
Two results files of `import-baseline.mjs`: the TypeScript application
(R4.1) and the Rust one (the import worker), so the page, files, groups,
engines, RSS metrics and watchdog are the same and only the build differs.
Per engine and group: files, same outcome, same outcome and message, median
Node-clock durations, max summed and content RSS growth, the longest
Chromium main-thread long task on each side (W5: none over 50 ms; the
`longtask` entry type only reports tasks over 50 ms) and the Rust runs that
had one. The message is what the practitioner sees: the inline error text,
or the alert lines after an import that opened the editor, compared as
exact text. The declared differences (plan §A.3) are expected with their
own message in every engine: `values-1m` → `error_attachment_stripped`
(OD-3), `surrogate-content-overwritten`, `surrogate-version-overwritten`,
`two-files-decoy-first` → `error_lone_surrogate` (OD-4), the TS side opening
the editor. Any other outcome or message difference is printed as `NOT
DECLARED`; it, a declared file not as declared, or a pair present on one
side only sets exit code 1. A last table puts the `read` group's files of
10 MiB and more side by side, per engine.
### CSP proof (`r4-app-csp.mjs`, R4.3d)
One reference PDF (`pdf/r3/demo.fr.logos.pdf`) picked on Home, a fresh
browser per engine and policy, four passes: the served policy `with`
(DEPLOY.md, `'wasm-unsafe-eval'`) must open the editor with no violation
other than the footer's external image; `without` (the keyword removed) must
show the inline line « Le moteur de lecture n'a pas pu être chargé.
Réessayez l'import. », the pick button enabled again, no page error, and a
second pick of the same file must show the line again (counted by a
`MutationObserver` on the error line, `lib/page-probes.mjs`
`countErrorLines`). Then both policies again with `report-uri` appended
(the enforced directives unchanged). Observed: the document's
`securitypolicyviolation` events, every console message of the page and its
workers (`ConsoleMessage.worker()`), the workers created and closed, and in
the reporting passes the reports the browser POSTs (document or worker
path, directive, blocked URI). A worker's own violation event fires in its
scope, out of the harness's reach: the reports are the channel that names
it, where the engine sends them.
### Application leak series (`r4-app-leak-series.mjs`, R4.3d)
The documents: the L envelope and the 16 MiB dense numbers built by
`lib/hostile-classes.mjs` with `packages/core-wasm/pkg` (its wasm digest is
recorded beside the build's), and a rescue copy, `report-v3.demo.json`
copied as `copie-de-secours-demo.json`; all in a scratch directory removed
at the end. Per engine, document and repetition, a fresh browser and ONE
tab: Home, the start level (forced GC + 1 s in Chromium, 5 s idle
elsewhere), then one import (the init cost, counted apart) and six more,
each by `page.goto('/')` and the pick button; when the session in progress
brings up « Remplacer le bilan en cours ? », « Remplacer » opens the picker
(Playwright's `filechooser`), which gets the file. The end of an import is
the first of: `/editor`, an alert, the inline error line (120 s timeout);
then 2 s idle and a reading (Chromium: forced GC + 1 s first; Firefox and
WebKit: idle only, indicative), 5 s idle and a final reading after the
last. A reading after an import that opened the editor includes the editor
page and its own core instance. The row keeps the levels (content and
summed), the init delta, the six per-import deltas, the growth over the
last five (L7 − L2) and at the final reading against the 10 MiB tolerance
of BDR-058, the series peak, each import's outcome, message and durations,
the workers created and closed, and the console errors (page or worker).
`webkit-repeat`: WebKit, one Home page without navigation, 20 consecutive
picks of the 16 MiB dense document; the inline lines with their counts,
page errors, console errors (worker ones and the footer image apart),
workers created and closed, the content level after each import and the
series peak. `control` (opt-in, `--sections control`): the series without
the pick, the same tab, `page.goto('/')` steps, idle and readings, so the
navigations' own cost can be told from the imports' (rows in `control`).
## Layout
```
scripts/browser/
├─ serve.mjs, ts-baseline.mjs, export-baseline.mjs, import-baseline.mjs
├─ inflate-verdicts.mjs, summarize.mjs
├─ r4-import.mjs, compare-imports.mjs, r4-large-files.mjs, r4-leak-series.mjs
├─ compare-app-imports.mjs, r4-app-csp.mjs, r4-app-leak-series.mjs (R4.3d)
├─ r4-import-page/ index.html, main.js (compile, routing, one worker per
│ import, worker events), worker.js (the import worker, both read
│ paths), proofs-worker.js
├─ lib/ playwright (load, launch under the watchdog, close), watchdog + its
│ sampler thread, proc (/proc reads), page-probes (init scripts),
│ harness (CLI, results file), summary (tables), app-build, host,
│ inputs, stats, async, paths; R4.2e: import-inputs (the baseline's
│ file set), core-module (bundle, module facts and copies, Node
│ instance), hostile-classes (budget-suite builders), r4-summary;
│ R4.3a: r4-runs (page, import race, settled levels, run row, engine
│ loop, prepared module run), r4-memory-summary (large-file and leak
│ tables)
└─ results/ raw results files
```