test(tests): R4.1e browser harness and TS baselines (Chromium, Firefox, WebKit)
Playwright harness under scripts/browser: 127.0.0.1 server with the DEPLOY CSP, export and import baselines of the TypeScript engine under a 4 GiB RSS watchdog (not a guaranteed memory limit), O9 browser columns in inflate.browsers.json (26 rows agree with Node in all three engines). Results of 2026-10-18 committed; the O9 Node test no longer expects browser columns in inflate.json.
This commit is contained in:
1 parent
4bdfda59ba
commit
4439fd2420
23 files changed
+20903
-18
No files matched your search
@@ -0,0 +1,160 @@
|
||||
# Browser harness (R4.1e)
|
||||
|
||||
Measures the application in real browser engines (Chromium, Firefox, WebKit)
|
||||
through Playwright: the TypeScript baselines of R4 (export and import time
|
||||
and memory, plan `.claude/tasks/plans/2026-10-17-rust-r4-pdf-core.md` O12,
|
||||
§B.3, §B.4, §B.8) and the browser columns of the O9 inflate oracle. Later
|
||||
slices reuse the server for the CSP and worker proofs.
|
||||
|
||||
> **Watchdog caveat.** Every browser runs under a 4 GiB RSS watchdog that is
|
||||
> **not a guaranteed memory limit**. It samples every 50 ms, so a faster
|
||||
> allocation can overshoot the ceiling before the kill; summed VmRSS counts
|
||||
> shared pages once per process; a process outside the browser's tree is not
|
||||
> seen. A run it stops is recorded as `stopped_by_ceiling`, which is a
|
||||
> result, not a harness failure.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Linux (the watchdog reads `/proc`), Node 24.
|
||||
- A production build that embeds the reference sheet. The scripts check it
|
||||
and refuse to run without it; they never build by themselves:
|
||||
|
||||
```bash
|
||||
SHEET_BUNDLE=draft corepack pnpm --filter @faunosteo/app build
|
||||
```
|
||||
|
||||
- Playwright outside the repository (no project dependency): set
|
||||
`PLAYWRIGHT_NODE_MODULES` to a `node_modules` that holds `playwright-core`,
|
||||
with its browsers installed in `~/.cache/ms-playwright`. On the
|
||||
development machine:
|
||||
|
||||
```bash
|
||||
export PLAYWRIGHT_NODE_MODULES=/home/bchanot/.local/lib/node_modules/gsd-pi/node_modules
|
||||
```
|
||||
|
||||
(playwright-core 1.64.0: Chromium headless shell 156, Firefox 157,
|
||||
WebKit 27.2, all headless.)
|
||||
|
||||
## Commands
|
||||
|
||||
All from the repository root.
|
||||
|
||||
| Command | What it does |
|
||||
| -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `node scripts/browser/serve.mjs [--port 4173] [--csp with\|without]` | serves `apps/app/dist` on `127.0.0.1` until Ctrl-C |
|
||||
| `node scripts/browser/ts-baseline.mjs` | export then import baseline, one file `results/<date>-ts-baseline.json`, tables on stdout |
|
||||
| `node scripts/browser/export-baseline.mjs` | export baseline only (`results/<date>-export.json`) |
|
||||
| `node scripts/browser/import-baseline.mjs` | import baseline only (`results/<date>-import.json`) |
|
||||
| `node scripts/browser/inflate-verdicts.mjs` | O9 browser columns → `crates/core/tests/oracles/r4/inflate.browsers.json` |
|
||||
| `node scripts/browser/summarize.mjs <results.json>` | prints the markdown tables of a results file |
|
||||
|
||||
Options of the baseline scripts: `--engines chromium,firefox,webkit` (default
|
||||
all three), `--out <file>`, `--reports demo,L,CAP` (export runs to keep),
|
||||
`--groups base,r3,step6,tolerance,read` (import groups to keep). Progress goes
|
||||
to stderr, the tables to stdout.
|
||||
|
||||
## The server (`serve.mjs`)
|
||||
|
||||
- `apps/app/dist` with the SPA fallback of `docker/nginx.conf`: an existing
|
||||
file is served, a missing `/assets/` file is a 404, any other path gets
|
||||
`index.html`. Binds `127.0.0.1` only; the LAN DEV origin is unaffected.
|
||||
- `.wasm` as `application/wasm` (uncompressed; production serves the
|
||||
`.wasm.gz` sibling with `gzip_static`).
|
||||
- The `Content-Security-Policy` of DEPLOY.md (the nginx `add_header` line,
|
||||
read at start) on every response, with `'wasm-unsafe-eval'` added to
|
||||
`script-src` as DEPLOY.md requires for R2+ images; `--csp without` serves
|
||||
the header without the keyword.
|
||||
- `/env.js` is the text `docker/entrypoint.sh` writes when no `APP_*`
|
||||
variable is set: the local tool, no Supabase (the build's stub `env.js`
|
||||
is not served).
|
||||
|
||||
## What each script measures
|
||||
|
||||
Every run records the engine version; every results file records the host
|
||||
(CPU, RAM, MemAvailable at start and end), the build (mtime, chunk sizes,
|
||||
HEAD), the served CSP and the watchdog facts with its caveat.
|
||||
|
||||
Memory comes from the watchdog's samples of the browser's process tree:
|
||||
summed VmRSS of all its processes, the content processes alone (Chromium
|
||||
renderers, Firefox Web Content, WebKit `WPEWebProcess`), and the largest
|
||||
single process.
|
||||
|
||||
### Export (`export-baseline.mjs`)
|
||||
|
||||
Reports: demo (`report-v3.demo.json`, 3 runs), L and CAP
|
||||
(`tests/fixtures/r4/reports/`, 3 runs and 1 run), locale fr, each run in a
|
||||
fresh browser and context. Before navigation the tab gets `osteo:draft`
|
||||
exactly as `persistence.ts` writes it (`JSON.stringify(report)`),
|
||||
`osteo:disclaimer-shown` (EditorRoute's first-run flag) and `osteo:locale`
|
||||
`fr`. The harness opens `/editor`, waits for the canvas and clicks
|
||||
« Exporter en PDF ».
|
||||
|
||||
- `exportMs`: page clock, from the click to `URL.createObjectURL` of the PDF
|
||||
Blob (download.ts); `clickToEndMs`: Node clock, click to the download
|
||||
event.
|
||||
- `pdfBytes`, the suggested file name.
|
||||
- `rasters`: every PNG Blob `HTMLCanvasElement.prototype.toBlob` produced
|
||||
(wrapped, calls through): count, sizes; the first is the partner SVG logo,
|
||||
the others are the views; `dataUrlFallbacks` counts `toDataURL` calls.
|
||||
- RSS before (settled), peak (click to the end of the export), after
|
||||
(settled). Settled = after a forced GC (CDP `HeapProfiler.collectGarbage`)
|
||||
and 1 s in Chromium, after 5 s idle in Firefox and WebKit. The PDF Blob is
|
||||
still referenced by its object URL for 30 s after the download
|
||||
(download.ts), so "after" includes it.
|
||||
- Chromium JS heap (CDP `Performance.getMetrics`), before and after.
|
||||
- Every dialog (accepted), page errors, console errors, CSP violations.
|
||||
- `outcome`: `downloaded`, `stopped_by_ceiling`, `timeout` (5 min demo,
|
||||
30 min L, 45 min CAP), `error`. A run that did not download keeps its
|
||||
peak, `stoppedAtBytes` (summed RSS when the watchdog fired) and
|
||||
`rastersBeforeEnd` (PNG blobs encoded so far, read every 2 s).
|
||||
|
||||
### Import (`import-baseline.mjs`)
|
||||
|
||||
One browser per engine (relaunched after a stop), a fresh context per file.
|
||||
The harness opens `/`, waits for « Choisir un fichier » to be enabled and
|
||||
sets the file on the import input. The run ends at the first of: navigation
|
||||
to `/editor`, an alert, the inline import error (`timeout` after 120 s).
|
||||
|
||||
Files: `report-v2.full.pdf`, `report-v2.minimal.pdf`, `report-v3.bare.pdf`,
|
||||
every PDF under `pdf/r3/` (recursively), `pdf/step6/`, `pdf/r4/tolerance/`,
|
||||
and the hostile inputs of `pdf/r4/read/` as `crates/core/tests/oracles/r4/
|
||||
read.json` lists them (inputs listed there but absent are reported as
|
||||
missing). A `<slug>.gen.json` description is expanded (checked against its
|
||||
length and digest) to a temporary `<slug>.pdf`, removed after its run.
|
||||
|
||||
- `durationMs`: Node clock, from setting the file to the first end;
|
||||
`pageMs`: page clock, from the input's `change` event to the first alert
|
||||
or navigation.
|
||||
- `outcome` (`editor`, `error`, `dialog`, `none`, `timeout`,
|
||||
`stopped_by_ceiling`), the alert texts, the inline error text, the final
|
||||
path.
|
||||
- RSS (summed and content) before the file is set and peak until the first
|
||||
end.
|
||||
- Chromium only: main-thread long tasks during the import
|
||||
(`PerformanceObserver('longtask')`): count, longest, total. Firefox and
|
||||
WebKit do not expose the entry type (`null`).
|
||||
|
||||
### O9 inflate verdicts (`inflate-verdicts.mjs`)
|
||||
|
||||
Reads `crates/core/tests/oracles/r4/inflate.json` and its inputs under
|
||||
`apps/app/tests/fixtures/pdf/r4/inflate/` (each checked against the recorded
|
||||
length and sha256), runs in each engine, on the application's Home page, a
|
||||
copy of `inflateFlate` from `packages/pdf/src/attachment/inflate.ts`
|
||||
(`DecompressionStream('deflate')`, 16 KiB awaited slices, concurrent drain,
|
||||
null past the row's limit or on any error), and writes
|
||||
`inflate.browsers.json`: per row, Node's verdict copied from `inflate.json`
|
||||
and each engine's (`null` or `{ length, sha256 }`), plus the rows where an
|
||||
engine disagrees with Node.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
scripts/browser/
|
||||
├─ serve.mjs, ts-baseline.mjs, export-baseline.mjs, import-baseline.mjs
|
||||
├─ inflate-verdicts.mjs, summarize.mjs
|
||||
├─ lib/ playwright (load, launch under the watchdog, close), watchdog + its
|
||||
│ sampler thread, proc (/proc reads), page-probes (init scripts),
|
||||
│ harness (CLI, results file), summary (tables), app-build, host,
|
||||
│ inputs, stats, async, paths
|
||||
└─ results/ raw results files
|
||||
```
|
||||
Reference in new issue
Block a user