chore(memory): V1-2.1 raw old-schema repro log, evidence, LRN, TODO

This commit is contained in:
bchanot committed 2026-10-28 20:51:17 +01:00
1 parent 2379d28d9b
commit 1f9fa8b835
5 files changed
+136 -16

No files matched your search

@@ -28,6 +28,10 @@ F17 service_role inserted an owner row directly into profile_roles
F16 sha256 of an asset used by a published version changed F16 sha256 of an asset used by a published version changed
``` ```
Re-run on 2026-10-28 for the final decisions pass, raw psql output committed as
`2026-10-28-v1-2-1-repro-old-schema.out` (sandbox `up fresh` from a copy holding only the 12
migrations before M4; the same NOTICE lines, `repro exit 0`, the transaction rolled back).
F18 (last-owner race) was reproduced with two sessions; its test is F18 (last-owner race) was reproduced with two sessions; its test is
`packages/supabase/tests/sandbox/sheets.sandbox.ts`. `packages/supabase/tests/sandbox/sheets.sandbox.ts`.
@@ -57,17 +61,17 @@ files of this pass:
## 4. Real-stack results on the corrected HEAD ## 4. Real-stack results on the corrected HEAD
Final decisions pass (owner brief « V1-2.1 : décisions finales du propriétaire »), code at Final decisions pass (owner brief « V1-2.1 : décisions finales du propriétaire »), code at
`5620399` (M6 option A, text size measure with the bounded number walk, pgTAP 015/017/020; `2379d28` (M6 option A, text size measure with the bounded, linear number walk, pgTAP
only `.claude/` differs at the final HEAD). Earlier figures (1 333 pgTAP) were the `2af61e7` 015/017/020; only `.claude/` differs at the final HEAD). Earlier figures (1 333 pgTAP) were the `2af61e7`
state; an intermediate pass at `7eca441` gave 1 352 before the review findings below. state; intermediate passes at `7eca441` (1 352) and `5620399` (1 807) preceded the review findings below.
| Run | Fresh stack | Upgraded stack (M3 + Git fixture, then M4+M5) | | Run | Fresh stack | Upgraded stack (M3 + Git fixture, then M4+M5) |
|---|---|---| |---|---|---|
| pgTAP (`test-db`, 18 files) | 1 807/1 807 | 1 807/1 807 | | pgTAP (`test-db`, 18 files) | 1 808/1 808 | 1 808/1 808 |
| sandbox suite, run 1 | 86/86 | 86/86 | | sandbox suite, run 1 | 87/87 | 87/87 |
| sandbox suite, run 2 (same stack) | 86/86 | 86/86 | | sandbox suite, run 2 (same stack) | 87/87 | 87/87 |
| of which two-session concurrency (`sheets-concurrency`) | 16/16 each run | 16/16 each run | | of which two-session concurrency (`sheets-concurrency`) | 16/16 each run | 16/16 each run |
| pgTAP again after both suite runs | 1 807/1 807 | 1 807/1 807 | | pgTAP again after both suite runs | 1 808/1 808 | 1 808/1 808 |
| `types --check` | types match | types match | | `types --check` | types match | types match |
PGlite: 128/128 (packages/supabase: the 18 pgTAP files and the closed-code census). PGlite: 128/128 (packages/supabase: the 18 pgTAP files and the closed-code census).
@@ -82,7 +86,8 @@ New in this pass:
as text and over it as binary jsonb saved whole; creation over the cap refused; numbers that as text and over it as binary jsonb saved whole; creation over the cap refused; numbers that
JavaScript prints with an exponent stored and measured in plain form; a document under the cap JavaScript prints with an exponent stored and measured in plain form; a document under the cap
in compact JavaScript form but over it as measured refused; 9 000 numbers of 131 072 digits in compact JavaScript form but over it as measured refused; 9 000 numbers of 131 072 digits
refused with the closed code; the table CHECK definition. Every refusal answers refused with the closed code; 600 000 small numbers within the cap saved (the number walk is
linear); the table CHECK definition. Every refusal answers
`document.too_large` (23514), never a raw error. `document.too_large` (23514), never a raw error.
- 017 `publish.species.other` refused alone. - 017 `publish.species.other` refused alone.
- 020 (449): the AR-1 B matrix of §15 in full (6 caller classes × aal1/aal2 × grant/revoke × 4 - 020 (449): the AR-1 B matrix of §15 in full (6 caller classes × aal1/aal2 × grant/revoke × 4
@@ -91,7 +96,13 @@ New in this pass:
`revoke_catalog_right`, `lookup_catalog_account`, `list_catalog_grants`) refused to every `revoke_catalog_right`, `lookup_catalog_account`, `list_catalog_grants`) refused to every
non-owner class at aal2 (`rights.owner`) and at aal1 (`rights.mfa`). non-owner class at aal2 (`rights.owner`) and at aal1 (`rights.mfa`).
Number bomb on the real image (`runs2/fresh/bomb.log`, sandbox): the shipped function answers Number walk, linear (security gate on `7c4bf12`: `jsonb_path_query` row by row is quadratic on
PostgreSQL 17, 0.15 / 0.53 / 2.17 s for 50k / 100k / 200k numbers on the real image; replaced by
one `jsonb_path_query_array` call: 65 / 128 / 262 / 414 ms for 100k / 200k / 400k / 600k). Over
HTTP on both stacks, each run: 600 000 numbers (1 801 760 bytes of text) saved in 677-695 ms;
1 000 000 numbers answered `document.too_large` in 1 019-1 078 ms (budget 2 000 ms).
Number bomb on the real image (sandbox, re-run on `2379d28`): the shipped function answers
`document.too_large`; the same payload under the previous body (`octet_length(p_doc::text)` `document.too_large`; the same payload under the previous body (`octet_length(p_doc::text)`
alone, rolled back) fails with `out of memory` after a 1 GiB buffer. alone, rolled back) fails with `out of memory` after a 1 GiB buffer.
@@ -101,14 +112,14 @@ Synthetic document: 64 views, 1 024 annotations, 8 192 finding nodes (depth 8),
groups, 4 drawings of 2 000 000 bytes. The labels are calibrated on the contract's text measure groups, 4 drawings of 2 000 000 bytes. The labels are calibrated on the contract's text measure
alone: text 2 076 707 bytes (cap 2 097 152), binary jsonb 2 346 648 bytes, over the cap and alone: text 2 076 707 bytes (cap 2 097 152), binary jsonb 2 346 648 bytes, over the cap and
accepted, as the measure requires. HTTP wall-clock through PostgREST on the development machine, accepted, as the measure requires. HTTP wall-clock through PostgREST on the development machine,
code `5620399`, milliseconds (min / median / max): code `2379d28`, milliseconds (min / median / max):
| Stack, run | save (7) | submit (5) | approve (3) | publish (3) | | Stack, run | save (7) | submit (5) | approve (3) | publish (3) |
|---|---|---|---|---| |---|---|---|---|---|
| fresh, 1 | 135 / 140 / 149 | 5 / 6 / 7 | 195 / 285 / 290 | 208 / 297 / 315 | | fresh, 1 | 127 / 137 / 155 | 4 / 5 / 6 | 190 / 285 / 292 | 195 / 288 / 303 |
| fresh, 2 | 130 / 139 / 145 | 4 / 5 / 6 | 200 / 286 / 301 | 201 / 285 / 304 | | fresh, 2 | 130 / 142 / 155 | 4 / 7 / 13 | 196 / 284 / 297 | 205 / 302 / 306 |
| upgraded, 1 | 130 / 141 / 156 | 5 / 5 / 6 | 196 / 286 / 296 | 306 / 308 / 318 | | upgraded, 1 | 135 / 139 / 148 | 5 / 6 / 6 | 192 / 282 / 294 | 196 / 292 / 296 |
| upgraded, 2 | 132 / 136 / 146 | 4 / 5 / 6 | 210 / 284 / 293 | 204 / 291 / 298 | | upgraded, 2 | 127 / 137 / 151 | 5 / 5 / 7 | 199 / 286 / 295 | 220 / 292 / 299 |
The core's at-caps fixture is V1-2.2 work; the contract fixes the budgets at the V1-2.3 The core's at-caps fixture is V1-2.2 work; the contract fixes the budgets at the V1-2.3
checkpoint and measures verification (120 s) there. checkpoint and measures verification (120 s) there.
@@ -141,7 +152,7 @@ RPC), then M4+M5 by `supabase migration up`.
bytes (sha256 equal to the key); the same editor at aal1, a catalogue reader and an anonymous bytes (sha256 equal to the key); the same editor at aal1, a catalogue reader and an anonymous
caller are refused (the historical sheet has no published version). caller are refused (the historical sheet has no published version).
The rehearsal is reproducible from the repository, re-run on `7eca441` and `5620399` (final decisions pass) The rehearsal is reproducible from the repository, re-run on `7eca441`, `5620399` and `2379d28` (final decisions pass)
with the same results, the only diff lines being the bucket limit and the service_role with the same results, the only diff lines being the bucket limit and the service_role
privileges: `bash .claude/audits/2026-10-28-v1-2-1-m7/m7-rehearsal.sh [work dir]` (sandbox only; it applies privileges: `bash .claude/audits/2026-10-28-v1-2-1-m7/m7-rehearsal.sh [work dir]` (sandbox only; it applies
M4+M5 with `supabase migration up --db-url` on the sandbox port, a step `scripts/db-sandbox.sh` M4+M5 with `supabase migration up --db-url` on the sandbox port, a step `scripts/db-sandbox.sh`
@@ -0,0 +1,105 @@
assign_platform_role
--------------------------------------
634abb88-1ccb-4bb5-9424-a384b5c31e74
(1 row)
assign_platform_role
--------------------------------------
d4de6c72-8e49-40a9-a012-4dfbab54e8b3
(1 row)
as_user
---------
(1 row)
NOTICE: F1 observed: plain account created a personal PUBLIC sheet for species other: creator=t official=f visibility=public species=other
as_user
---------
(1 row)
NOTICE: F5 observed: an account with an UNCONFIRMED email created a sheet
as_user
---------
(1 row)
NOTICE: F6 observed: set_sheet_visibility changes visibility with no gate
as_user
---------
(1 row)
NOTICE: F15 observed: another plain account reads the public personal sheet: 1 row(s)
as_user
---------
(1 row)
NOTICE: F9 observed: editor rewrote draft content by direct UPDATE (no RPC): change_note=direct
NOTICE: F9 observed: sheet renamed by direct UPDATE: Renamed directly
publish_sheet_version
--------------------------------------
a45915a7-1677-4560-abe9-72d7d0c018e8
(1 row)
NOTICE: F4 observed: version published with a drawing never confirmed: uploaded_at null = t
archive_anatomical_sheet
--------------------------------------
f2411a8d-0f70-4bac-8e4c-cc7084977f42
(1 row)
NOTICE: F2 observed: the draft of an ARCHIVED sheet was published
NOTICE: F2 observed: create_sheet_draft opened a draft on an ARCHIVED sheet
as_user
---------
(1 row)
NOTICE: F8 observed: after archiving, another account reads 0 published version(s) of the sheet
as_user
---------
(1 row)
restore_anatomical_sheet
--------------------------------------
f2411a8d-0f70-4bac-8e4c-cc7084977f42
(1 row)
deprecate_sheet_version
--------------------------------------
a45915a7-1677-4560-abe9-72d7d0c018e8
(1 row)
NOTICE: F13 observed: a DEPRECATED (once published) version was deleted physically: rows left 0
as_user
---------
(1 row)
NOTICE: F3 observed: source drawing confirmed, copied rows confirmed: false
as_user
---------
(1 row)
NOTICE: F14 observed: a platform admin at aal1 (claims carry no aal) published an official sheet
as_user
---------
(1 row)
NOTICE: F19 observed: grant_platform_role(owner) accepted at aal1 (claims: {"sub": "a1000000-0000-0000-0000-000000000001", "role": "authenticated"})
NOTICE: F19 observed: the new owner holds 0 verified factor(s); active owner rows now 1
set_config
-------------------------
{"role":"service_role"}
(1 row)
NOTICE: F7 observed: upload confirmed with no object in Storage: objects=0 uploaded_at set=t
NOTICE: F12 observed: service_role published a draft by direct UPDATE: published
NOTICE: F17 observed: service_role inserted an owner row directly into profile_roles
NOTICE: F16 observed: sha256 of an asset used by a published version changed: 00000000
repro exit 0
+1
View File
@@ -479,3 +479,4 @@ R5.4: `draft/{labels,changelog,review_report,drawing,overlay,app_review,svg_comp
- V1-2.1 corrections after audit (3914eaa CHANGES REQUIRED): M1 approve needs active sheet, M2 base only from published round version, LOW fixes (rights.mfa first, closed codes, budgets, view.asset.unknown at submit, upload policy = registrant, append-only acks, deprecate/working-copy locks). Found+fixed: published round withdrawable, approver-author not refused at publish. Found: jsonb pg_column_size > text size (CHECK binds before contract measure) → closed code; owner decision. pgTAP 1325 fresh+upgraded, sandbox 86/86 x2 both stacks, caps timing max 374 ms, M7 rehearsal object+metadata identical, check:all green. Commits d05f75d..4a82941 pushed. M6 + M7 criterion + size measure → owner. Fresh verifier + security gates dispatched. - V1-2.1 corrections after audit (3914eaa CHANGES REQUIRED): M1 approve needs active sheet, M2 base only from published round version, LOW fixes (rights.mfa first, closed codes, budgets, view.asset.unknown at submit, upload policy = registrant, append-only acks, deprecate/working-copy locks). Found+fixed: published round withdrawable, approver-author not refused at publish. Found: jsonb pg_column_size > text size (CHECK binds before contract measure) → closed code; owner decision. pgTAP 1325 fresh+upgraded, sandbox 86/86 x2 both stacks, caps timing max 374 ms, M7 rehearsal object+metadata identical, check:all green. Commits d05f75d..4a82941 pushed. M6 + M7 criterion + size measure → owner. Fresh verifier + security gates dispatched.
- V1-2.1 final owner decisions (2af61e7): M6 A grant.self_review dropped, size = octet_length(content::text) (table CHECK replaced in M5), M7 Git fixture wording in contract + evidence. pgTAP 1352 fresh+upgraded, sandbox 86/86 x2 both stacks (concurrency 16/16), caps text 2 076 707 / jsonb 2 346 648 accepted, PGlite 127. DEV metadata identical. BDR-085. - V1-2.1 final owner decisions (2af61e7): M6 A grant.self_review dropped, size = octet_length(content::text) (table CHECK replaced in M5), M7 Git fixture wording in contract + evidence. pgTAP 1352 fresh+upgraded, sandbox 86/86 x2 both stacks (concurrency 16/16), caps text 2 076 707 / jsonb 2 346 648 accepted, PGlite 127. DEV metadata identical. BDR-085.
- V1-2.1 review pass on 1281667: verifier ECARTS(6) (missing proofs: AR-1 B matrix, non-owner rights RPCs, publish.species.other; LOW docs), security PASS + MEDIUM number-exponent OOM (9000 x 1e131071 = 1 GiB text, raw OOM). Fixed 3eba6d8: bounded number walk before text measure; pgTAP 020 (449). Re-run 5620399: pgTAP 1807 x2 both stacks, suite 86/86 x2 both. Correction: earlier 'DEV metadata identical' = DEV container state only (ids, start times), no DEV data read. - V1-2.1 review pass on 1281667: verifier ECARTS(6) (missing proofs: AR-1 B matrix, non-owner rights RPCs, publish.species.other; LOW docs), security PASS + MEDIUM number-exponent OOM (9000 x 1e131071 = 1 GiB text, raw OOM). Fixed 3eba6d8: bounded number walk before text measure; pgTAP 020 (449). Re-run 5620399: pgTAP 1807 x2 both stacks, suite 86/86 x2 both. Correction: earlier 'DEV metadata identical' = DEV container state only (ids, start times), no DEV data read.
- V1-2.1 second reviews on 7c4bf12: security BLOCK HIGH = my number walk quadratic (jsonb_path_query per row, PG17 list_delete_first; 200k numbers 2.2 s native, ~26 s near cap). Fixed 538d103: jsonb_path_query_array + jsonb_array_elements (linear, 600k 0.4 s). Verifier ECARTS(1) LOW: red-on-old raw log now committed (.out). Rerun 2379d28: pgTAP 1808, suite 87/87 x2 both stacks.
+3
View File
@@ -660,3 +660,6 @@ Adapting pgTAP across PGlite and the real Supabase image: the fixtures must resp
- Real-stack-only divergences: `storage.objects` has a storage-API trigger forbidding direct DELETE/UPDATE (PGlite allows it) — assert "no write policy" via `pg_policies` census, not a runtime throws_ok. PostgREST maps a `raise ... errcode='42501'` to HTTP 403 with code `42501` (not P0001). An EXECUTE-revoked function raises "permission denied for function" before its own body — test that, not the body's custom error. - Real-stack-only divergences: `storage.objects` has a storage-API trigger forbidding direct DELETE/UPDATE (PGlite allows it) — assert "no write policy" via `pg_policies` census, not a runtime throws_ok. PostgREST maps a `raise ... errcode='42501'` to HTTP 403 with code `42501` (not P0001). An EXECUTE-revoked function raises "permission denied for function" before its own body — test that, not the body's custom error.
- A narrow operator read RPC returning JSON `null` for "no row" is cleaner than raising (PostgREST won't surface a raise as a 404 the client recognises); the tool treats `null` as absent. - A narrow operator read RPC returning JSON `null` for "no row" is cleaner than raising (PostgREST won't surface a raise as a 404 the client recognises); the tool treats `null` as absent.
- Real GoTrue proves what fabricated rows can't: `pnpm db:sandbox test` enrol+verify TOTP → aal2 token, then the role RPC works at aal2 and is refused (mfa_required, 403) at aal1; F18 cross-revoke under real concurrency leaves one owner. - Real GoTrue proves what fabricated rows can't: `pnpm db:sandbox test` enrol+verify TOTP → aal2 token, then the role RPC works at aal2 and is refused (mfa_required, 403) at aal1; F18 cross-revoke under real concurrency leaves one owner.
## LRN-096 — jsonb_path_query row iteration is quadratic; numeric text has no exponent (2026-10-28)
PostgreSQL 17 `jsonb_path_query` (SRF) builds its whole match list on the first call and pops rows with `list_delete_first` -> O(n^2) when iterated row by row (FOR loop, count(*)): 200k matches 2.2 s, 400k ~27 s. Early `exit` saves nothing. Use one `jsonb_path_query_array(...)` then `jsonb_array_elements` (linear: 600k 0.4 s). Also: `numeric` prints no exponent, so `octet_length(jsonb::text)` of `1e131071` x 9000 (90 kB payload) = 1.2 GB -> raw OOM; sum printed numbers first. Apply to any SQL walk over user jsonb (status walks `sheet_all_reviewed`, `sheet_has_to_confirm` use the same SRF pattern, bounded today by core caps).
+1 -1
View File
@@ -561,7 +561,7 @@ Order (BDR-071): P0 docs + decisions → V1-1 clients and animals → V1-2 sheet
- [x] H2 M7 Git fixture accepted: contract §12.3/§15/§16.2 + evidence say Git fixture with sha256 = DEV row, bytes/metadata/relations before/after, no DEV Storage read, live DEV object integrity not shown - [x] H2 M7 Git fixture accepted: contract §12.3/§15/§16.2 + evidence say Git fixture with sha256 = DEV row, bytes/metadata/relations before/after, no DEV Storage read, live DEV object integrity not shown
- [x] H3 size measure = contract `octet_length(content::text)` (UTF-8 bytes of the jsonb text form): drop `pg_column_size` from `check_document_structure`, M5 replaces the base table CHECK by the text measure; contract defines the form for SQL, TS, Rust; pgTAP: cap − 1, cap, cap + 1, multibyte, dense (jsonb > cap, text ≤ cap) accepted; refusals `document.too_large` - [x] H3 size measure = contract `octet_length(content::text)` (UTF-8 bytes of the jsonb text form): drop `pg_column_size` from `check_document_structure`, M5 replaces the base table CHECK by the text measure; contract defines the form for SQL, TS, Rust; pgTAP: cap − 1, cap, cap + 1, multibyte, dense (jsonb > cap, text ≤ cap) accepted; refusals `document.too_large`
- [ ] H4 gates: pgTAP fresh + upgraded, PGlite + unit, sandbox suite ×2, concurrency, fresh + M3→M5 (M7 rehearsal), types, check:all; fresh verifier + security on final HEAD; DEV metadata before/after — sandbox done on 7eca441 (pgTAP 1352 ×2 per stack, suite 86/86 ×2 per stack, types match, M7 diff = bucket + service_role only, DEV containers untouched: same ids, start times, restart counts; nothing on DEV read); check:all + reviews on final HEAD pending - [ ] H4 gates: pgTAP fresh + upgraded, PGlite + unit, sandbox suite ×2, concurrency, fresh + M3→M5 (M7 rehearsal), types, check:all; fresh verifier + security on final HEAD; DEV metadata before/after — sandbox done on 7eca441 (pgTAP 1352 ×2 per stack, suite 86/86 ×2 per stack, types match, M7 diff = bucket + service_role only, DEV containers untouched: same ids, start times, restart counts; nothing on DEV read); check:all + reviews on final HEAD pending
- [x] H6 review findings on 1281667 (verifier ECARTS(6), security PASS + 1 MEDIUM): number-exponent amplification → raw OOM (bounded number-text pre-sum, closed code, pgTAP); AR-1 B §15 matrix (015); rights RPCs refused for non-owners (015); `publish.species.other` alone (017); docs §1, §9.5, §12.2 vectors, evidence §3; then gates again + fresh reviews — done 3eba6d8/5620399: pgTAP 1807 ×2 per stack, suite 86/86 ×2 per stack, PGlite 128, bomb proof on real image; final check:all + fresh reviews pending - [x] H6 review findings on 1281667 (verifier ECARTS(6), security PASS + 1 MEDIUM): number-exponent amplification → raw OOM (bounded number-text pre-sum, closed code, pgTAP); AR-1 B §15 matrix (015); rights RPCs refused for non-owners (015); `publish.species.other` alone (017); docs §1, §9.5, §12.2 vectors, evidence §3; then gates again + fresh reviews — done 3eba6d8/5620399: pgTAP 1807 ×2 per stack, suite 86/86 ×2 per stack, PGlite 128, bomb proof on real image; second reviews on 7c4bf12: verifier ECARTS(1) LOW (raw red-on-old log → committed .out), security BLOCK(1) HIGH quadratic number walk → linear 538d103, perf fixture 2379d28; rerun 2379d28: pgTAP 1808 ×2 per stack, suite 87/87 ×2 per stack; final check:all + fresh reviews pending
- [ ] H5 memory (BDR M6/size, journal), commit + push branch, final report - [ ] H5 memory (BDR M6/size, journal), commit + push branch, final report
- [ ] V1-2.2 core verdict, preview, change list, edit primitives, SVG cleaning · V1-2.3 drawings and worker verification (M6) · V1-2.4 wireframes (editor, review, admin entry) · V1-2.5 catalogue, admin space, rights, species, MFA · V1-2.6 editor level 1 · V1-2.7 preview, review, publication UI · V1-2.8 equine working copy and review preparation · V1-2.9 FR/EN acceptance and closure (owner checkpoints V1-2.3, V1-2.4, V1-2.6, V1-2.7, V1-2.8, V1-2.9 + each DEV operation of contract §16.11) - [ ] V1-2.2 core verdict, preview, change list, edit primitives, SVG cleaning · V1-2.3 drawings and worker verification (M6) · V1-2.4 wireframes (editor, review, admin entry) · V1-2.5 catalogue, admin space, rights, species, MFA · V1-2.6 editor level 1 · V1-2.7 preview, review, publication UI · V1-2.8 equine working copy and review preparation · V1-2.9 FR/EN acceptance and closure (owner checkpoints V1-2.3, V1-2.4, V1-2.6, V1-2.7, V1-2.8, V1-2.9 + each DEV operation of contract §16.11)