138 lines
8.4 KiB
Markdown
138 lines
8.4 KiB
Markdown
# Browser tests (Playwright and axe)
|
||
|
||
Status: V1-1.3, 2026-10-25 (decision D14, `V1-1_OWNERS_ANIMALS.md` §12.5).
|
||
Chromium only until V1-5. The suites live in `apps/app/e2e`.
|
||
|
||
## 1. Suites
|
||
|
||
| Command | Backend | Dev server | Content |
|
||
|---|---|---|---|
|
||
| `pnpm e2e:ui` | none | Vite on `127.0.0.1:5191` | `e2e/ui/dialog.spec.ts`: the D13 contract of the shared Dialog on a test page; `e2e/ui/local-tool.spec.ts`: the disclaimer, edit and replace dialogs on the real local-tool routes |
|
||
| `pnpm e2e:shell` | the isolated stack of `pnpm db:sandbox` | Vite on `127.0.0.1:5192` | `e2e/shell/shell.spec.ts`: sign-in, dashboard, the account photo confirmation, the phone drawer; `e2e/shell/clients.spec.ts`: the client journeys (V1-1.5); `e2e/shell/animals.spec.ts`: the animal and relation journeys (V1-1.6, contract §11.8); `e2e/shell/responsive.spec.ts`: no sideways scroll on nine shell pages, the archive confirmations and the drawer at 320, 360, 375, 390 and 430 px (phone project) and 1280 px, long values, the dashboard e-mail shown whole (RESP-01) |
|
||
|
||
Each suite runs on two Chromium projects: `chromium-desktop` (1280 × 800) and
|
||
`chromium-375` (375 × 667, mobile viewport, touch). A test tagged `@phone`
|
||
runs on the phone project only (the drawer has no desktop counterpart). Both
|
||
languages run: the local tool and the shell through the browser locale and
|
||
the account's locale, the test page through `?lang=en`.
|
||
|
||
The test page `e2e/harness/dialog.html` is served by the dev server only
|
||
(`/e2e/harness/dialog.html`); the build has one entry, `index.html`, so it
|
||
never ships. It holds the D13 cases (basic, long content, refused close,
|
||
vanishing opener) and no product journey.
|
||
|
||
The shell suite never reaches DEV. Its dev server is built with the Supabase
|
||
URL `http://sandbox.faunosteo.invalid` (a name that resolves nowhere,
|
||
RFC 6761); `e2e/support/sandbox.ts` replays `/auth/v1`, `/rest/v1` and
|
||
`/storage/v1` on the sandbox's loopback ports, the way Kong maps them on the
|
||
CLI stack. Each test creates its own confirmed account through the sandbox's
|
||
GoTrue admin API (random address under `sandbox.local`, random password, a
|
||
fictitious display name, the profile locale of the test), and seeds the clean
|
||
avatar the worker would have produced (the `assets` row as postgres, the
|
||
object with the service key). Rows around the subject of a test (clients,
|
||
animals, relations) are written as postgres by `e2e/support/records.ts`,
|
||
every value quoted as an SQL literal. `pnpm db:sandbox down` removes all of it.
|
||
`pnpm db:sandbox e2e-shell <args>` passes its arguments to Playwright: one
|
||
spec file, `--project`, `--grep`. Every shell suite removes its routes after
|
||
each test without waiting for them (`dropLateReplays`), so an answer still in
|
||
flight when the last check passed is dropped with the context instead of
|
||
failing the test (the V1-1.6 verifier saw the client journey fail that way once
|
||
the client page reloads its animals after a restore).
|
||
|
||
## 2. Prerequisites
|
||
|
||
- The repository's Node and pnpm, `pnpm install`.
|
||
- The Chromium build of the pinned Playwright (1.64.1, Chromium 1248):
|
||
`pnpm --filter @faunosteo/app exec playwright install chromium`. It
|
||
downloads into `~/.cache/ms-playwright` and needs no `sudo`. Never pass
|
||
`--with-deps`: it installs system packages with `sudo`, an owner action. On
|
||
the Ubuntu development machine the headless shell runs with the libraries
|
||
already present.
|
||
- `pnpm wasm:build`: the local tool runs the core, and the UI suite stops on a
|
||
missing or stale module (`packages/core-wasm/tests/global-setup.ts`).
|
||
- For `pnpm e2e:shell`: Docker (rootless works) and the sandbox up,
|
||
`pnpm db:sandbox up fresh` (or `up upgrade <version>`).
|
||
- Ports 5191 and 5192 free: each server uses `--strictPort` and is never
|
||
reused, so the owner's `pnpm dev` on 5173 is never picked up by mistake.
|
||
|
||
## 3. Guarantees
|
||
|
||
- Request guard (`e2e/support/browser.ts`, automatic in every test): an
|
||
allowlist of origins per suite (the dev server, plus the fake Supabase
|
||
origin for the shell). Any other request, DEV's API on the LAN address
|
||
included, is aborted and fails the test with its URL.
|
||
- No fixture value in a URL: the local tool's names and the shell account's
|
||
email, password and display name are checked against every request URL;
|
||
a match fails the test and the report masks the value.
|
||
- axe (`@axe-core/playwright` 4.14.0, rules tagged WCAG 2.0 to 2.2 A and AA
|
||
plus best practices): zero serious or critical violation on each scanned
|
||
state, colours measured once animations have settled. Three pre-existing
|
||
findings, outside the V1-1.3 code, are listed with their reason in
|
||
`KNOWN_VIOLATIONS` and in TODO RR-7: the language switcher's names (« FR »
|
||
shown, « Passer en français » announced), the top bar account link (display
|
||
name shown, « Mon compte » announced), the dashboard's `dl` holding a status
|
||
row without `dt`/`dd`. Each entry names the axe rule, the selector, the
|
||
opening markup of the element and the route where it lives; a node is let
|
||
through only when all four match, so a new screen or a new element never
|
||
inherits an exception (owner brief 2026-10-26).
|
||
- Reports: `apps/app/e2e/test-results/` and `apps/app/e2e/playwright-report/`,
|
||
git-ignored. The UI suite keeps a trace and a screenshot of a failed test
|
||
(no backend, no secret). The shell suite keeps the terminal output and a
|
||
screenshot of a failed test only: a trace would hold the session tokens, and
|
||
an HTML report the typed values, the fictitious accounts' passwords included
|
||
(found in the V1-1.5 report check, 2026-10-26; the HTML reporter was removed
|
||
from the shell config). Screenshots and the failure's page snapshot show
|
||
fictitious names and addresses only.
|
||
|
||
## 4. Reading a failure
|
||
|
||
The terminal shows each step and the failing assertion. The HTML report of
|
||
the UI suite: `pnpm --filter @faunosteo/app exec playwright show-report
|
||
e2e/playwright-report/ui` (the shell suite has none); a UI trace: `pnpm --filter @faunosteo/app exec playwright
|
||
show-trace <path printed in the failure>`. Setup failures are named: a missing
|
||
Chromium (Playwright prints the install command), a stale core module
|
||
(`run pnpm wasm:build`), the shell suite outside its wrapper
|
||
(`SANDBOX_ANON_KEY missing: run pnpm e2e:shell`), the sandbox down
|
||
(`db-sandbox: not running`).
|
||
|
||
## 5. In `pnpm check:all` and a future CI
|
||
|
||
`pnpm check:all` runs `pnpm e2e:ui` when the pinned Chromium is installed;
|
||
otherwise it prints `SKIPPED` with the reason and the install command, and
|
||
goes on. `pnpm e2e:shell` needs the sandbox and stays a separate run, like
|
||
`pnpm e2e:avatar`. The last lines of a green run state each browser suite:
|
||
`PASSED`, `SKIPPED` (environment missing, with the reason) or `NOT RUN`
|
||
(`e2e:shell`). A suite that runs and fails stops `check:all` with
|
||
`== FAILED: <stage>` and a non-zero exit, never a warning. Both suites keep
|
||
their own commands and run alone. A CI runner installs the same Chromium
|
||
(`playwright install chromium` in its image) and runs `pnpm e2e:ui`; the shell
|
||
suite needs Docker in the runner for the sandbox.
|
||
|
||
## 6. Limits
|
||
|
||
- Chromium only. Firefox, WebKit and the 412 × 915 viewport join at V1-5
|
||
(contract §12.5). WebKit's system libraries may need `sudo` (owner action).
|
||
Desktop WebKit is not iOS Safari: the owner's manual check on a real iPhone
|
||
and on a computer over the LAN stays a validation criterion of every mobile
|
||
step.
|
||
- Tab past a modal dialog's last control reaches the browser's own interface
|
||
for one step, then the first control (the platform's behaviour, D13); the
|
||
page behind is never reached. The tests assert exactly that walk.
|
||
- Safari does not focus a button on a pointer click, so a dialog opened by a
|
||
click has no opener to return to there; a keyboard opening always has one.
|
||
- The vitest suites (happy-dom) cover the React wiring of the dialogs only:
|
||
happy-dom has no top layer, no inertness and no dialog focusing steps.
|
||
- axe's target-size rule measures a control partly under the sticky top bar
|
||
as cut. Two phone states of the animal suite are reached at the bottom of a
|
||
long page (the client page after linking an animal, a refused duplicate);
|
||
they are scanned from the top of the page (`expectAccessibleFromTop`), where
|
||
nothing sits under the bar. The controls are 44 px high either way.
|
||
|
||
## 7. Adding a test
|
||
|
||
Import `test` and `expect` from `e2e/support/browser.ts` (the request guard
|
||
comes with them), read every visible string from `@/i18n/{fr,en}`, use
|
||
fictitious values only, and pass the values a test creates to
|
||
`requests.forbid(...)`. A new backend origin goes into the suite's
|
||
`allowedOrigins`, never into a wildcard.
|