From 1f9fa8b8354aaf82f3bba49293e84a42b1a11439 Mon Sep 17 00:00:00 2001 From: bchanot Date: Wed, 28 Oct 2026 20:51:17 +0100 Subject: [PATCH] chore(memory): V1-2.1 raw old-schema repro log, evidence, LRN, TODO --- .../2026-10-28-v1-2-1-corrections-evidence.md | 41 ++++--- .../2026-10-28-v1-2-1-repro-old-schema.out | 105 ++++++++++++++++++ .claude/memory/journal.md | 1 + .claude/memory/learnings.md | 3 + .claude/tasks/TODO.md | 2 +- 5 files changed, 136 insertions(+), 16 deletions(-) create mode 100644 .claude/audits/2026-10-28-v1-2-1-repro-old-schema.out diff --git a/.claude/audits/2026-10-28-v1-2-1-corrections-evidence.md b/.claude/audits/2026-10-28-v1-2-1-corrections-evidence.md index 5ecbc33b..5beb4e89 100644 --- a/.claude/audits/2026-10-28-v1-2-1-corrections-evidence.md +++ b/.claude/audits/2026-10-28-v1-2-1-corrections-evidence.md @@ -28,6 +28,10 @@ F17 service_role inserted an owner row directly into profile_roles F16 sha256 of an asset used by a published version changed ``` +Re-run on 2026-10-28 for the final decisions pass, raw psql output committed as +`2026-10-28-v1-2-1-repro-old-schema.out` (sandbox `up fresh` from a copy holding only the 12 +migrations before M4; the same NOTICE lines, `repro exit 0`, the transaction rolled back). + F18 (last-owner race) was reproduced with two sessions; its test is `packages/supabase/tests/sandbox/sheets.sandbox.ts`. @@ -57,17 +61,17 @@ files of this pass: ## 4. Real-stack results on the corrected HEAD Final decisions pass (owner brief « V1-2.1 : décisions finales du propriétaire »), code at -`5620399` (M6 option A, text size measure with the bounded number walk, pgTAP 015/017/020; -only `.claude/` differs at the final HEAD). Earlier figures (1 333 pgTAP) were the `2af61e7` -state; an intermediate pass at `7eca441` gave 1 352 before the review findings below. +`2379d28` (M6 option A, text size measure with the bounded, linear number walk, pgTAP +015/017/020; only `.claude/` differs at the final HEAD). Earlier figures (1 333 pgTAP) were the `2af61e7` +state; intermediate passes at `7eca441` (1 352) and `5620399` (1 807) preceded the review findings below. | Run | Fresh stack | Upgraded stack (M3 + Git fixture, then M4+M5) | |---|---|---| -| pgTAP (`test-db`, 18 files) | 1 807/1 807 | 1 807/1 807 | -| sandbox suite, run 1 | 86/86 | 86/86 | -| sandbox suite, run 2 (same stack) | 86/86 | 86/86 | +| pgTAP (`test-db`, 18 files) | 1 808/1 808 | 1 808/1 808 | +| sandbox suite, run 1 | 87/87 | 87/87 | +| sandbox suite, run 2 (same stack) | 87/87 | 87/87 | | of which two-session concurrency (`sheets-concurrency`) | 16/16 each run | 16/16 each run | -| pgTAP again after both suite runs | 1 807/1 807 | 1 807/1 807 | +| pgTAP again after both suite runs | 1 808/1 808 | 1 808/1 808 | | `types --check` | types match | types match | PGlite: 128/128 (packages/supabase: the 18 pgTAP files and the closed-code census). @@ -82,7 +86,8 @@ New in this pass: as text and over it as binary jsonb saved whole; creation over the cap refused; numbers that JavaScript prints with an exponent stored and measured in plain form; a document under the cap in compact JavaScript form but over it as measured refused; 9 000 numbers of 131 072 digits - refused with the closed code; the table CHECK definition. Every refusal answers + refused with the closed code; 600 000 small numbers within the cap saved (the number walk is + linear); the table CHECK definition. Every refusal answers `document.too_large` (23514), never a raw error. - 017 `publish.species.other` refused alone. - 020 (449): the AR-1 B matrix of §15 in full (6 caller classes × aal1/aal2 × grant/revoke × 4 @@ -91,7 +96,13 @@ New in this pass: `revoke_catalog_right`, `lookup_catalog_account`, `list_catalog_grants`) refused to every non-owner class at aal2 (`rights.owner`) and at aal1 (`rights.mfa`). -Number bomb on the real image (`runs2/fresh/bomb.log`, sandbox): the shipped function answers +Number walk, linear (security gate on `7c4bf12`: `jsonb_path_query` row by row is quadratic on +PostgreSQL 17, 0.15 / 0.53 / 2.17 s for 50k / 100k / 200k numbers on the real image; replaced by +one `jsonb_path_query_array` call: 65 / 128 / 262 / 414 ms for 100k / 200k / 400k / 600k). Over +HTTP on both stacks, each run: 600 000 numbers (1 801 760 bytes of text) saved in 677-695 ms; +1 000 000 numbers answered `document.too_large` in 1 019-1 078 ms (budget 2 000 ms). + +Number bomb on the real image (sandbox, re-run on `2379d28`): the shipped function answers `document.too_large`; the same payload under the previous body (`octet_length(p_doc::text)` alone, rolled back) fails with `out of memory` after a 1 GiB buffer. @@ -101,14 +112,14 @@ Synthetic document: 64 views, 1 024 annotations, 8 192 finding nodes (depth 8), groups, 4 drawings of 2 000 000 bytes. The labels are calibrated on the contract's text measure alone: text 2 076 707 bytes (cap 2 097 152), binary jsonb 2 346 648 bytes, over the cap and accepted, as the measure requires. HTTP wall-clock through PostgREST on the development machine, -code `5620399`, milliseconds (min / median / max): +code `2379d28`, milliseconds (min / median / max): | Stack, run | save (7) | submit (5) | approve (3) | publish (3) | |---|---|---|---|---| -| fresh, 1 | 135 / 140 / 149 | 5 / 6 / 7 | 195 / 285 / 290 | 208 / 297 / 315 | -| fresh, 2 | 130 / 139 / 145 | 4 / 5 / 6 | 200 / 286 / 301 | 201 / 285 / 304 | -| upgraded, 1 | 130 / 141 / 156 | 5 / 5 / 6 | 196 / 286 / 296 | 306 / 308 / 318 | -| upgraded, 2 | 132 / 136 / 146 | 4 / 5 / 6 | 210 / 284 / 293 | 204 / 291 / 298 | +| fresh, 1 | 127 / 137 / 155 | 4 / 5 / 6 | 190 / 285 / 292 | 195 / 288 / 303 | +| fresh, 2 | 130 / 142 / 155 | 4 / 7 / 13 | 196 / 284 / 297 | 205 / 302 / 306 | +| upgraded, 1 | 135 / 139 / 148 | 5 / 6 / 6 | 192 / 282 / 294 | 196 / 292 / 296 | +| upgraded, 2 | 127 / 137 / 151 | 5 / 5 / 7 | 199 / 286 / 295 | 220 / 292 / 299 | The core's at-caps fixture is V1-2.2 work; the contract fixes the budgets at the V1-2.3 checkpoint and measures verification (120 s) there. @@ -141,7 +152,7 @@ RPC), then M4+M5 by `supabase migration up`. bytes (sha256 equal to the key); the same editor at aal1, a catalogue reader and an anonymous caller are refused (the historical sheet has no published version). -The rehearsal is reproducible from the repository, re-run on `7eca441` and `5620399` (final decisions pass) +The rehearsal is reproducible from the repository, re-run on `7eca441`, `5620399` and `2379d28` (final decisions pass) with the same results, the only diff lines being the bucket limit and the service_role privileges: `bash .claude/audits/2026-10-28-v1-2-1-m7/m7-rehearsal.sh [work dir]` (sandbox only; it applies M4+M5 with `supabase migration up --db-url` on the sandbox port, a step `scripts/db-sandbox.sh` diff --git a/.claude/audits/2026-10-28-v1-2-1-repro-old-schema.out b/.claude/audits/2026-10-28-v1-2-1-repro-old-schema.out new file mode 100644 index 00000000..201a8052 --- /dev/null +++ b/.claude/audits/2026-10-28-v1-2-1-repro-old-schema.out @@ -0,0 +1,105 @@ + assign_platform_role +-------------------------------------- + 634abb88-1ccb-4bb5-9424-a384b5c31e74 +(1 row) + + assign_platform_role +-------------------------------------- + d4de6c72-8e49-40a9-a012-4dfbab54e8b3 +(1 row) + + as_user +--------- + +(1 row) + +NOTICE: F1 observed: plain account created a personal PUBLIC sheet for species other: creator=t official=f visibility=public species=other + as_user +--------- + +(1 row) + +NOTICE: F5 observed: an account with an UNCONFIRMED email created a sheet + as_user +--------- + +(1 row) + +NOTICE: F6 observed: set_sheet_visibility changes visibility with no gate + as_user +--------- + +(1 row) + +NOTICE: F15 observed: another plain account reads the public personal sheet: 1 row(s) + as_user +--------- + +(1 row) + +NOTICE: F9 observed: editor rewrote draft content by direct UPDATE (no RPC): change_note=direct +NOTICE: F9 observed: sheet renamed by direct UPDATE: Renamed directly + publish_sheet_version +-------------------------------------- + a45915a7-1677-4560-abe9-72d7d0c018e8 +(1 row) + +NOTICE: F4 observed: version published with a drawing never confirmed: uploaded_at null = t + archive_anatomical_sheet +-------------------------------------- + f2411a8d-0f70-4bac-8e4c-cc7084977f42 +(1 row) + +NOTICE: F2 observed: the draft of an ARCHIVED sheet was published +NOTICE: F2 observed: create_sheet_draft opened a draft on an ARCHIVED sheet + as_user +--------- + +(1 row) + +NOTICE: F8 observed: after archiving, another account reads 0 published version(s) of the sheet + as_user +--------- + +(1 row) + + restore_anatomical_sheet +-------------------------------------- + f2411a8d-0f70-4bac-8e4c-cc7084977f42 +(1 row) + + deprecate_sheet_version +-------------------------------------- + a45915a7-1677-4560-abe9-72d7d0c018e8 +(1 row) + +NOTICE: F13 observed: a DEPRECATED (once published) version was deleted physically: rows left 0 + as_user +--------- + +(1 row) + +NOTICE: F3 observed: source drawing confirmed, copied rows confirmed: false + as_user +--------- + +(1 row) + +NOTICE: F14 observed: a platform admin at aal1 (claims carry no aal) published an official sheet + as_user +--------- + +(1 row) + +NOTICE: F19 observed: grant_platform_role(owner) accepted at aal1 (claims: {"sub": "a1000000-0000-0000-0000-000000000001", "role": "authenticated"}) +NOTICE: F19 observed: the new owner holds 0 verified factor(s); active owner rows now 1 + set_config +------------------------- + {"role":"service_role"} +(1 row) + +NOTICE: F7 observed: upload confirmed with no object in Storage: objects=0 uploaded_at set=t +NOTICE: F12 observed: service_role published a draft by direct UPDATE: published +NOTICE: F17 observed: service_role inserted an owner row directly into profile_roles +NOTICE: F16 observed: sha256 of an asset used by a published version changed: 00000000 +repro exit 0 diff --git a/.claude/memory/journal.md b/.claude/memory/journal.md index f07fa284..8cf518bb 100644 --- a/.claude/memory/journal.md +++ b/.claude/memory/journal.md @@ -479,3 +479,4 @@ R5.4: `draft/{labels,changelog,review_report,drawing,overlay,app_review,svg_comp - V1-2.1 corrections after audit (3914eaa CHANGES REQUIRED): M1 approve needs active sheet, M2 base only from published round version, LOW fixes (rights.mfa first, closed codes, budgets, view.asset.unknown at submit, upload policy = registrant, append-only acks, deprecate/working-copy locks). Found+fixed: published round withdrawable, approver-author not refused at publish. Found: jsonb pg_column_size > text size (CHECK binds before contract measure) → closed code; owner decision. pgTAP 1325 fresh+upgraded, sandbox 86/86 x2 both stacks, caps timing max 374 ms, M7 rehearsal object+metadata identical, check:all green. Commits d05f75d..4a82941 pushed. M6 + M7 criterion + size measure → owner. Fresh verifier + security gates dispatched. - V1-2.1 final owner decisions (2af61e7): M6 A grant.self_review dropped, size = octet_length(content::text) (table CHECK replaced in M5), M7 Git fixture wording in contract + evidence. pgTAP 1352 fresh+upgraded, sandbox 86/86 x2 both stacks (concurrency 16/16), caps text 2 076 707 / jsonb 2 346 648 accepted, PGlite 127. DEV metadata identical. BDR-085. - V1-2.1 review pass on 1281667: verifier ECARTS(6) (missing proofs: AR-1 B matrix, non-owner rights RPCs, publish.species.other; LOW docs), security PASS + MEDIUM number-exponent OOM (9000 x 1e131071 = 1 GiB text, raw OOM). Fixed 3eba6d8: bounded number walk before text measure; pgTAP 020 (449). Re-run 5620399: pgTAP 1807 x2 both stacks, suite 86/86 x2 both. Correction: earlier 'DEV metadata identical' = DEV container state only (ids, start times), no DEV data read. +- V1-2.1 second reviews on 7c4bf12: security BLOCK HIGH = my number walk quadratic (jsonb_path_query per row, PG17 list_delete_first; 200k numbers 2.2 s native, ~26 s near cap). Fixed 538d103: jsonb_path_query_array + jsonb_array_elements (linear, 600k 0.4 s). Verifier ECARTS(1) LOW: red-on-old raw log now committed (.out). Rerun 2379d28: pgTAP 1808, suite 87/87 x2 both stacks. diff --git a/.claude/memory/learnings.md b/.claude/memory/learnings.md index 8768c2d7..ee674458 100644 --- a/.claude/memory/learnings.md +++ b/.claude/memory/learnings.md @@ -660,3 +660,6 @@ Adapting pgTAP across PGlite and the real Supabase image: the fixtures must resp - Real-stack-only divergences: `storage.objects` has a storage-API trigger forbidding direct DELETE/UPDATE (PGlite allows it) — assert "no write policy" via `pg_policies` census, not a runtime throws_ok. PostgREST maps a `raise ... errcode='42501'` to HTTP 403 with code `42501` (not P0001). An EXECUTE-revoked function raises "permission denied for function" before its own body — test that, not the body's custom error. - A narrow operator read RPC returning JSON `null` for "no row" is cleaner than raising (PostgREST won't surface a raise as a 404 the client recognises); the tool treats `null` as absent. - Real GoTrue proves what fabricated rows can't: `pnpm db:sandbox test` enrol+verify TOTP → aal2 token, then the role RPC works at aal2 and is refused (mfa_required, 403) at aal1; F18 cross-revoke under real concurrency leaves one owner. + +## LRN-096 — jsonb_path_query row iteration is quadratic; numeric text has no exponent (2026-10-28) +PostgreSQL 17 `jsonb_path_query` (SRF) builds its whole match list on the first call and pops rows with `list_delete_first` -> O(n^2) when iterated row by row (FOR loop, count(*)): 200k matches 2.2 s, 400k ~27 s. Early `exit` saves nothing. Use one `jsonb_path_query_array(...)` then `jsonb_array_elements` (linear: 600k 0.4 s). Also: `numeric` prints no exponent, so `octet_length(jsonb::text)` of `1e131071` x 9000 (90 kB payload) = 1.2 GB -> raw OOM; sum printed numbers first. Apply to any SQL walk over user jsonb (status walks `sheet_all_reviewed`, `sheet_has_to_confirm` use the same SRF pattern, bounded today by core caps). diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 4a51f9c9..c4fb38d4 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -561,7 +561,7 @@ Order (BDR-071): P0 docs + decisions → V1-1 clients and animals → V1-2 sheet - [x] H2 M7 Git fixture accepted: contract §12.3/§15/§16.2 + evidence say Git fixture with sha256 = DEV row, bytes/metadata/relations before/after, no DEV Storage read, live DEV object integrity not shown - [x] H3 size measure = contract `octet_length(content::text)` (UTF-8 bytes of the jsonb text form): drop `pg_column_size` from `check_document_structure`, M5 replaces the base table CHECK by the text measure; contract defines the form for SQL, TS, Rust; pgTAP: cap − 1, cap, cap + 1, multibyte, dense (jsonb > cap, text ≤ cap) accepted; refusals `document.too_large` - [ ] H4 gates: pgTAP fresh + upgraded, PGlite + unit, sandbox suite ×2, concurrency, fresh + M3→M5 (M7 rehearsal), types, check:all; fresh verifier + security on final HEAD; DEV metadata before/after — sandbox done on 7eca441 (pgTAP 1352 ×2 per stack, suite 86/86 ×2 per stack, types match, M7 diff = bucket + service_role only, DEV containers untouched: same ids, start times, restart counts; nothing on DEV read); check:all + reviews on final HEAD pending - - [x] H6 review findings on 1281667 (verifier ECARTS(6), security PASS + 1 MEDIUM): number-exponent amplification → raw OOM (bounded number-text pre-sum, closed code, pgTAP); AR-1 B §15 matrix (015); rights RPCs refused for non-owners (015); `publish.species.other` alone (017); docs §1, §9.5, §12.2 vectors, evidence §3; then gates again + fresh reviews — done 3eba6d8/5620399: pgTAP 1807 ×2 per stack, suite 86/86 ×2 per stack, PGlite 128, bomb proof on real image; final check:all + fresh reviews pending + - [x] H6 review findings on 1281667 (verifier ECARTS(6), security PASS + 1 MEDIUM): number-exponent amplification → raw OOM (bounded number-text pre-sum, closed code, pgTAP); AR-1 B §15 matrix (015); rights RPCs refused for non-owners (015); `publish.species.other` alone (017); docs §1, §9.5, §12.2 vectors, evidence §3; then gates again + fresh reviews — done 3eba6d8/5620399: pgTAP 1807 ×2 per stack, suite 86/86 ×2 per stack, PGlite 128, bomb proof on real image; second reviews on 7c4bf12: verifier ECARTS(1) LOW (raw red-on-old log → committed .out), security BLOCK(1) HIGH quadratic number walk → linear 538d103, perf fixture 2379d28; rerun 2379d28: pgTAP 1808 ×2 per stack, suite 87/87 ×2 per stack; final check:all + fresh reviews pending - [ ] H5 memory (BDR M6/size, journal), commit + push branch, final report - [ ] V1-2.2 core verdict, preview, change list, edit primitives, SVG cleaning · V1-2.3 drawings and worker verification (M6) · V1-2.4 wireframes (editor, review, admin entry) · V1-2.5 catalogue, admin space, rights, species, MFA · V1-2.6 editor level 1 · V1-2.7 preview, review, publication UI · V1-2.8 equine working copy and review preparation · V1-2.9 FR/EN acceptance and closure (owner checkpoints V1-2.3, V1-2.4, V1-2.6, V1-2.7, V1-2.8, V1-2.9 + each DEV operation of contract §16.11)