Files
faunosteo/docs/development/BROWSER_TESTS.md
T
bchanot 431999e45c docs(docs): V1-2.5 checkpoint corrections in browser tests, identity, contract
Eight phone skips by design, the unreadable-permissions and live-region checks, species order and Paris expiry restated in contract 7.2 and 4, PermissionsUnavailable in the folder map.
2026-11-03 00:14:19 +01:00

173 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Browser tests (Playwright and axe)
Status: V1-1.3, 2026-10-25 (decision D14, `V1-1_OWNERS_ANIMALS.md` §12.5).
Chromium only until V1-5, except the canvas click proof (V1-2.2), which also
runs on Firefox and WebKit, desktop and the WebKit phone, when the pinned
engines are installed. The suites live in `apps/app/e2e`.
## 1. Suites
| Command | Backend | Dev server | Content |
|---|---|---|---|
| `pnpm e2e:ui` | none | Vite on `127.0.0.1:5191` | `e2e/ui/dialog.spec.ts`: the D13 contract of the shared Dialog on a test page; `e2e/ui/local-tool.spec.ts`: the disclaimer, edit and replace dialogs on the real local-tool routes |
| `pnpm e2e:shell` | the isolated stack of `pnpm db:sandbox` | Vite on `127.0.0.1:5192` | `e2e/shell/shell.spec.ts`: sign-in, dashboard, the account photo confirmation, the phone drawer; `e2e/shell/clients.spec.ts`: the client journeys (V1-1.5); `e2e/shell/animals.spec.ts`: the animal and relation journeys (V1-1.6, contract §11.8); `e2e/shell/responsive.spec.ts`: no sideways scroll on nine shell pages, the archive confirmations and the drawer at 320, 360, 375, 390 and 430 px (phone project) and 1280 px, long values, the dashboard e-mail shown whole (RESP-01); `e2e/shell/admin.spec.ts` (V1-2.5): A0 the owner's bootstrap (two TOTP factors from Sécurité, the step-up at `/app/admin` and inside a factor action, no second factor from `aal1`, refresh keeping `aal1` and `aal2`, rights granted and revoked with their reason), A1 a sheet created by an editor on a new inactive species, S1 a species created inactive then activated by the owner, the creation page when the own-grants read fails (« Impossible de vérifier vos autorisations » and « Réessayer », never the form; Retry brings the form back), access per right (a reviewer-only account refused on the creation page), axe and no sideways scroll on every admin page; `e2e/shell/catalogue.spec.ts` (V1-2.5): P1 a reader searches, opens a sheet, picks a zone, reads a retired version, drafts and archived sheets never listed, the same refusal for an existing and an unknown sheet without access, 320, 375 and 430 px with the whole drawing in the box at opening; at 320 and 375 px the zone panel's live region present before a pick and naming the picked zone, the « Références » card starting at or below the drawing's bottom |
Each suite runs on two Chromium projects: `chromium-desktop` (1280 × 800) and
`chromium-375` (375 × 667, mobile viewport, touch). A test tagged `@phone`
runs on the phone project only (the drawer has no desktop counterpart). Both
languages run: the local tool and the shell through the browser locale and
the account's locale, the test page through `?lang=en`.
The test page `e2e/harness/dialog.html` is served by the dev server only
(`/e2e/harness/dialog.html`); the build has one entry, `index.html`, so it
never ships. It holds the D13 cases (basic, long content, refused close,
vanishing opener) and no product journey.
The shell suite never reaches DEV. Its dev server is built with the Supabase
URL `http://sandbox.faunosteo.invalid` (a name that resolves nowhere,
RFC 6761); `e2e/support/sandbox.ts` replays `/auth/v1`, `/rest/v1` and
`/storage/v1` on the sandbox's loopback ports, the way Kong maps them on the
CLI stack. Each test creates its own confirmed account through the sandbox's
GoTrue admin API (random address under `sandbox.local`, random password, a
fictitious display name, the profile locale of the test), and seeds the clean
avatar the worker would have produced (the `assets` row as postgres, the
object with the service key). Rows around the subject of a test (clients,
animals, relations) are written as postgres by `e2e/support/records.ts`,
every value quoted as an SQL literal. The V1-2.5 suites set up what lies
before the page the same way: platform roles, catalogue grants and other
accounts' verified TOTP factors by `e2e/support/admin.ts` (roles through the
operator path, grants as postgres, a factor enrolled and verified on the real
GoTrue, its secret kept in the test's memory); official sheets by
`e2e/support/catalogue.ts` as SQL fixtures (published, retired, archived and
draft-only versions carrying the curated equine draft under names of their
own, the drawing in the private bucket under its content key), each published
or retired version's fingerprint computed in Node by the shipped core module
(`packages/core-wasm/pkg`, `preview.state`), the one the page runs. A0, A1, S1
and the creation page's unreadable-permissions test are desktop journeys, skipped on the phone
project (8 skips by design); the admin screens at 375 px have their own test.
`pnpm db:sandbox down` removes all of it.
`pnpm db:sandbox e2e-shell <args>` passes its arguments to Playwright: one
spec file, `--project`, `--grep`. Every shell suite removes its routes after
each test without waiting for them (`dropLateReplays`), so an answer still in
flight when the last check passed is dropped with the context instead of
failing the test (the V1-1.6 verifier saw the client journey fail that way once
the client page reloads its animals after a restore).
## 2. Prerequisites
- The repository's Node and pnpm, `pnpm install`.
- The Chromium build of the pinned Playwright (1.64.1, Chromium 1248):
`pnpm --filter @faunosteo/app exec playwright install chromium`. It
downloads into `~/.cache/ms-playwright` and needs no `sudo`. Never pass
`--with-deps`: it installs system packages with `sudo`, an owner action. On
the Ubuntu development machine the headless shell runs with the libraries
already present.
- `pnpm wasm:build`: the local tool runs the core, and the UI suite stops on a
missing or stale module (`packages/core-wasm/tests/global-setup.ts`).
- For `pnpm e2e:shell`: Docker (rootless works) and the sandbox up,
`pnpm db:sandbox up fresh` (or `up upgrade <version>`).
- Ports 5191 and 5192 free: each server uses `--strictPort` and is never
reused, so the owner's `pnpm dev` on 5173 is never picked up by mistake.
## 3. Guarantees
- Request guard (`e2e/support/browser.ts`, automatic in every test): an
allowlist of origins per suite (the dev server, plus the fake Supabase
origin for the shell). Any other request, DEV's API on the LAN address
included, is aborted and fails the test with its URL.
- No fixture value in a URL: the local tool's names and the shell account's
email, password and display name are checked against every request URL;
a match fails the test and the report masks the value.
- axe (`@axe-core/playwright` 4.14.0, rules tagged WCAG 2.0 to 2.2 A and AA
plus best practices): zero serious or critical violation on each scanned
state, colours measured once animations have settled. Three pre-existing
findings, outside the V1-1.3 code, are listed with their reason in
`KNOWN_VIOLATIONS` and in TODO RR-7: the language switcher's names (« FR »
shown, « Passer en français » announced), the top bar account link (display
name shown, « Mon compte » announced), the dashboard's `dl` holding a status
row without `dt`/`dd`. Each entry names the axe rule, the selector, the
opening markup of the element and the route where it lives; a node is let
through only when all four match, so a new screen or a new element never
inherits an exception (owner brief 2026-10-26).
- Reports: `apps/app/e2e/test-results/` and `apps/app/e2e/playwright-report/`,
git-ignored. The UI suite keeps a trace and a screenshot of a failed test
(no backend, no secret). The shell suite keeps the terminal output and a
screenshot of a failed test only: a trace would hold the session tokens, and
an HTML report the typed values, the fictitious accounts' passwords included
(found in the V1-1.5 report check, 2026-10-26; the HTML reporter was removed
from the shell config). Screenshots and the failure's page snapshot show
fictitious names and addresses only.
## 4. Reading a failure
The terminal shows each step and the failing assertion. The HTML report of
the UI suite: `pnpm --filter @faunosteo/app exec playwright show-report
e2e/playwright-report/ui` (the shell suite has none); a UI trace: `pnpm --filter @faunosteo/app exec playwright
show-trace <path printed in the failure>`. Setup failures are named: a missing
Chromium (Playwright prints the install command), a stale core module
(`run pnpm wasm:build`), the shell suite outside its wrapper
(`SANDBOX_ANON_KEY missing: run pnpm e2e:shell`), the sandbox down
(`db-sandbox: not running`).
## 5. In `pnpm check:all` and a future CI
`pnpm check:all` runs `pnpm e2e:ui` when the pinned Chromium is installed;
otherwise it prints `SKIPPED` with the reason and the install command, and
goes on. `pnpm e2e:shell` needs the sandbox and stays a separate run, like
`pnpm e2e:avatar` (= `pnpm db:sandbox avatar-e2e`). The last lines of a green run state each browser suite:
`PASSED`, `SKIPPED` (environment missing, with the reason) or `NOT RUN`
(`e2e:shell`). A suite that runs and fails stops `check:all` with
`== FAILED: <stage>` and a non-zero exit, never a warning. Both suites keep
their own commands and run alone. A CI runner installs the same Chromium
(`playwright install chromium` in its image) and runs `pnpm e2e:ui`; the shell
suite needs Docker in the runner for the sandbox.
## 6. Limits
- Chromium only, but `e2e/ui/canvas-clicks.spec.ts` (V1-2.2: projects
`chromium-320` (320 × 568, touch, the smallest preview the markers' visible
size is measured on), `firefox-desktop`, `webkit-desktop`, `webkit-375`, the
last three added when `playwright install firefox webkit` has put the pinned
engines in the cache; a missing one is named in the run's output and its
projects do not run; harness `e2e/harness/canvas.{html,tsx}`, dev server
only). WebKit snaps a real tap or click to whole CSS pixels (measured
2026-11-01: a tap at x 100.1 lands right of an edge at 100.4, a click at
100.9 left of it; Chromium keeps the fraction on a click, rounds a tap), so a
real press keeps 1.5 px of margin from any edge it tests; below that margin
the spec asks the browser's own hit test (`elementFromPoint`) at the exact
point. `e2e/ui/interior-clicks.spec.ts` runs on the same six projects
(V1-2.2, the interior rule): the core's probe cases
(`e2e/fixtures/interior-probe.json`, generated and drift-checked by
`crates/core/tests/interior_probe.rs`, `MAKE_INTERIOR_PROBE=1` rewrites it) in
the phone preview's 286 × 310 box (harness `e2e/harness/interior.{html,ts}`,
dev server only), then the equine zones in the canvas harness. Chromium's touch
adjustment gives a point the taps up to about 10 px around its disc
(measured 2026-11-01, `chromium-320` and `chromium-375`; WebKit 375 does
not), so the real press on an equine zone goes where its fill lies farthest
from every disc. Firefox, WebKit
and the 412 × 915 viewport join every suite at V1-5
(contract §12.5). WebKit's system libraries may need `sudo` (owner action).
Desktop WebKit is not iOS Safari: the owner's manual check on a real iPhone
and on a computer over the LAN stays a validation criterion of every mobile
step.
- Tab past a modal dialog's last control reaches the browser's own interface
for one step, then the first control (the platform's behaviour, D13); the
page behind is never reached. The tests assert exactly that walk.
- Safari does not focus a button on a pointer click, so a dialog opened by a
click has no opener to return to there; a keyboard opening always has one.
- The vitest suites (happy-dom) cover the React wiring of the dialogs only:
happy-dom has no top layer, no inertness and no dialog focusing steps.
- axe's target-size rule measures a control partly under the sticky top bar
as cut. Two phone states of the animal suite are reached at the bottom of a
long page (the client page after linking an animal, a refused duplicate);
they are scanned from the top of the page (`expectAccessibleFromTop`), where
nothing sits under the bar. The controls are 44 px high either way.
## 7. Adding a test
Import `test` and `expect` from `e2e/support/browser.ts` (the request guard
comes with them), read every visible string from `@/i18n/{fr,en}`, use
fictitious values only, and pass the values a test creates to
`requests.forbid(...)`. A new backend origin goes into the suite's
`allowedOrigins`, never into a wildcard.