diff --git a/.claude/memory/evals.md b/.claude/memory/evals.md index 4efa97cc..5b0bbe70 100644 --- a/.claude/memory/evals.md +++ b/.claude/memory/evals.md @@ -205,3 +205,9 @@ Method: check:all green 21:16:39 (app 434, core-wasm 58, domain 262, anatomy 141 Anomalies: LOW report.rs `Map` index panics on hand-built ImportResult → fixed (get chain) after verdict, importer suite + --check rerun green; LOW quadratic vertebrae scan, symlink-following `listed()` in --check, lint allows (accepted, operator inputs). check:all ran on pre-commit tree (code identical to 03bdb5e). Verifier floor note: plan B.3.5 "(proposal, not implemented)" heading from R5.1, intended. Action: checkpoint R5.2 delivered, STOP before R5.3. + +## EVAL-028 | 2026-10-23 | R5.3 gates: fresh verifier ×4 + security; successive passes found real defects + +Method: check:all after each fix round; security-auditor (semgrep 725 rules, checklist) → PASS, 6 LOW (3 fixed: Map index, destination claimed + canonicalised, INIT_CWD absolute only); fresh verifier each round, no history. Pass 1 ECARTS(1) (floor on a plan heading) + 2 unreachable divergences aligned; pass 2 null-on-optional (LRN-072); pass 3 (first attempt died on API credit error, rerun on Opus) repeated keys refused by serde derive, NUL touch-mark silent drop (LRN-073), `--out symlink/` trailing slash followed the link; pass 4 C1-C12 met, C13 docs gaps (lossy UTF-8 read, 2 contrived splits undeclared, stale counts). +Anomalies: green byte parity on real data hid every finding: all were hostile-input or file-protection defects. A verdict reported after post-pass edits was not re-verified → owner required a fresh pass on the final state. +Action: refusal tests now compare reasons by stage; Object.prototype characterised + tested; migration-map debt characterised; final fresh verifier + security on the final R5.3 state before R5.4. Next ports: budget ≥3 verifier passes, probe hostile inputs (null, duplicates, reserved names, symlinks, encodings) in the oracle from the start. diff --git a/.claude/memory/learnings.md b/.claude/memory/learnings.md index ebca527d..28de6583 100644 --- a/.claude/memory/learnings.md +++ b/.claude/memory/learnings.md @@ -529,3 +529,13 @@ Apply: shortest-float port → take digits from `zmij`/ryu-class crate, never `{ R5.2 reproduced Prettier 3.8.3 `parser: json` over `JSON.stringify(v, null, 2)` and markdown table alignment in ~350 lines Rust, byte-exact on 968 synthetic probes + 7 seed files + 3 reports. Rules that mattered: objects always expanded (stringify newline after `{`), array breaks when >1 element and all elements arrays >1, or child breaks, or flat form + trailing comma > 80 cols; broken all-number arrays use `fill`, others one per line; `printNumber` drops `e+`; width = length if all bytes 0x20..0x7F else per code point (controls/combining/VS16 = 0, East Asian wide/fullwidth = 2) after RGI emoji replaced by spaces. Emoji table not reproduced → refuse (`UnsupportedWidth`), declared. Probe generator in TS (vitest, `MAKE_R5_ORACLES=1`) sweeping depth × key width × count × digits around the 80-col boundary found both bugs (`e+21` form, width fast path) before any real file did. Apply: any port that must match a formatter's bytes: sweep the boundary with synthetic probes written by the formatter itself; transcribe tables from the installed package (same table lives in `index.mjs` and plugin), never from memory; refuse unreproduced classes loudly instead of approximating. + +## LRN-072 | 2026-10-23 | serde `Option` reads JSON `null` as absent; Zod `.optional()` refuses it + +Found by R5.3 verifier pass 2: changeset `scope` / `position` / `field` declared `Option` + `#[serde(default)]` → `"scope": null` parsed as absent, Zod refused (`expected 'self' | 'subtree', received null`). Committed data had none → every parity test green, only a hostile input shows it. Core already had `js::present` (R2) for exactly this; port forgot it. Complement of LRN-048 (serializer side: `.optional()` absent vs nullable `null`). +Apply: porting a Zod schema to serde: `.optional()` → `default, deserialize_with = "js::present"`; `.nullable()` → `js::nullable`; add one `null` refusal case per optional key to the differential oracle. + +## LRN-073 | 2026-10-23 | In-band markers replacing object identity must be reserved keys refused at input + +R5.3 applier replaced TS object identity (touched items) with a transient key `"\0touched"`, stripped after each change. Verifier pass 3: an authored value holding that key was taken for a mark and the cleanup deleted it silently, where TS refused it at the strict gate → silent author-data loss. Fix: refuse any NUL-prefixed key in the changeset at parse (`reserved key`), test it. +Apply: any in-band sentinel (key, id prefix, magic value) gets a reserved namespace, a refusal at the input boundary and a test; never rely on "no author would write that". Prefer out-of-band state (side table by path) when the cost is similar. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 525bf0fb..8b506a0c 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -315,6 +315,8 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [x] R5.3c `import-equine`: both folders (`/legacy-import`, `/draft`), `--check` compares the produced files and lists the R5.4 files as pending (not counted), `--out` refuses a non-empty folder, a symlink, a file, an empty path; `scripts/import-equine.sh` + root script `r5:import-equine` - [x] R5.3d proofs (draft 5 files byte-exact, 12/12 `--check`, 46 changes + 825 records + stats equal, 612 leaves 442/141/29, 58 synthetic cases: 9 byte-exact + 49 refused alike + 5 references files, rust:check green): `tests/curation_draft.rs` (draft files byte-exact vs committed, `pipeline.json` draft section: 46 changes, 825 records, stats, migration stats, assets, issues; determinism; 612-leaf walk through the core's migration on the produced map); TS generator `r5.curation-synthetic-oracles.test.ts` → `oracles/r5/curation-synthetic.json` (mini sheet: every op kind, positions, subtree scopes, review rule B/C, renames with cascades, re-homing, refusals) run by the Rust suite; `pnpm rust:check`, `pnpm check:all` - [x] R5.3e (security PASS 6 LOW / 3 fixed; verifier 4 passes, last C1-C12 met + C13 docs fixed after; check:all green 00:10:03) gates: security auditor (diff), independent verifier on the contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`; docs RUST.md §11.12-11.13, CHANGELOG, CLAUDE.md commands; memory → checkpoint report, STOP before R5.4 + - [x] R5.3f owner 2026-10-23: Object.prototype member characterised (safe, accepted, test), migration-map debt kept + characterisation test, refusal reasons compared by stage; final fresh verifier + security on the R5.3 state + - [ ] debt (owner 2026-10-23, after R5): migration map replays cascade records by their op (`applyRecord` / `apply_record`): a view, zone, region, category or cited-reference rename corrupts the map; fix both sides as a declared change before any changeset uses such a rename (RUST.md §11.12) - [ ] R5.4 review overlays + app drawings on the R3 layer/compose (parsed trees), CHANGELOG + REVIEW-REPORT - [ ] R5.5 `upload-sheet-assets` bin + runbook (local stack run) - [ ] R5.6 D-R5 switch (convention, aliases, membership test in gated readers with per-file go, seed regenerated once by the Rust tool, `.prettierignore`, CHANGELOG) diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 4a5f3314..cb92ae51 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -2901,7 +2901,7 @@ re-cleaned; the review drawings and the two Markdown reports are R5.4. Contract | `pipeline.rs` | `buildEquineDraft` (CHG-000 from the frozen asset's sha256, byte size and uuid, the head view box parsed from the legacy SVG; the reserved id refused), `renderDraftArtifacts` for the R5.3 files (`document.json`, `assets.json`, `sheet-ref.json` with `document_sha256` = sha256 of the `document.json` bytes, `report-migration-map.json`, `equine_draft.sql`), `parseViewBox` | | `bin/import_equine.rs` | both folders; `--check` compares the produced files with the committed seed and lists the six R5.4 files of the draft as pending (`shortcut:` comment, removed at R5.4); `--out ` writes `/legacy-import` and `/draft` into a destination that is absent or an empty folder, refuses an empty path, a symbolic link, a path that is not a folder, a folder holding any file; an absent destination is created before any write (nothing can appear between the check and the writes) and the files go to its canonical path, which the success line prints (a symbolic link among the parent folders is visible there); a relative path is typed from `INIT_CWD` when absolute, else the current folder; nothing is ever deleted | -Parity, every expectation written by the TypeScript (`tests/curation_draft.rs`, 8 tests): +Parity, every expectation written by the TypeScript (`tests/curation_draft.rs`, 10 tests): - the five R5.3 text files of the committed `draft/` byte for byte, the frozen `horse.svg` described by its sha256, size and asset id; `pnpm r5:import-equine --check`: 12 files, 0 @@ -2916,15 +2916,17 @@ Parity, every expectation written by the TypeScript (`tests/curation_draft.rs`, - the 612 legacy leaves through the core's `read_report_document` on the produced map: 442 identical, 141 moved, 29 removed, every resolved leaf a leaf of the produced document, the 29 flagged paths exactly those the map sends to null, the FR snapshots kept; -- 58 synthetic changesets on the mini sheet (`oracles/r5/curation-synthetic.json`, - generator `packages/anatomy/tests/r5.curation-synthetic-oracles.test.ts`): 9 accepted +- 64 synthetic changesets on the mini sheet (`oracles/r5/curation-synthetic.json`, + generator `packages/anatomy/tests/r5.curation-synthetic-oracles.test.ts`): 11 accepted cases (every op kind, numeric and anchored positions, subtree scopes with skipped equal values, the review rule on levels A, B and C, the five rename cascades and a reference - rename, re-homing, two chained changes) byte for byte on the formatted document, the - records, the stats, the migration map and the seed SQL; 48 refusals (19 schema, `null` - on `scope`, `position` and `field` included, 29 applier, integrity and shape gates - included) refused alike; 5 references files refused - alike; one declared difference below. + rename, re-homing, two chained changes, the two migration-map debt cases below) byte for + byte on the formatted document, the records, the stats, the migration map and the seed + SQL; 51 refusals (19 schema, `null` on `scope`, `position` and `field` included, 32 + applier, integrity and shape gates included) refused alike, with the same reason: the + same message for an op or change refusal (change id and op index included), the same + stage for a schema or gate refusal; 5 references files refused alike; two declared + differences below (one refusal, one acceptance), each held by its own test. Differences declared (none reachable by the committed files, since the folders are reproduced byte for byte): @@ -2936,17 +2938,25 @@ reproduced byte for byte): | a lone surrogate escape (`"\ud800"`) in any string of the two files | `JSON.parse` kept it | refused (the core's L1 rule: no lone surrogate is ever read) | | nesting deeper than 128 levels (an `add` of a 70-level finding chain) | accepted | refused at parse (serde_json's depth limit, kept as a resource bound) | | a number beyond the double range (`1e400`) in a value a later op overwrites or removes | `JSON.parse` read Infinity, the value was discarded, the change accepted (a surviving value is refused by both at the gate) | refused at parse (serde_json's range check) | -| a `set` field path through an inherited `Object.prototype` member (`name.constructor`) whose parent a later op replaces | `structuredClone` of the function read there threw, change refused | the member is a plain missing key: accepted (a surviving key is refused by both at the gate) | +| a `set` field path ending in an inherited `Object.prototype` member (`name.constructor`, `name.toString`, `name.valueOf`…) | `readField` returned the inherited function and the record's `structuredClone` threw (`… could not be cloned`): change refused at the op | the member is an ordinary missing key: written, `before` recorded as null. Kept alone, the shape gate refuses it (every object of the sheet contract is strict). Replaced by a later op of the run (a `set name` on the same item), the change is accepted: the document is byte for byte the one the later op alone gives, one extra record traces the overwritten write. Nothing is bypassed: the settable table checks the first segment on both sides (`constructor` as a field: `unknown field`), no value is read through a prototype, the integrity gate runs on the final document (test `an_inherited_member_name_is_a_plain_key_and_bypasses_nothing`, five member names). Accepted by the owner 2026-10-23 | | a `set` field path ending in `__proto__` (`name.__proto__`) | the prototype setter swallowed the write: change accepted, document unchanged, one record | the key is written and the strict document gate refuses it (the R5.2 family: a name of `Object.prototype` is a plain key) | | an authored key starting with NUL (`"\u0000touched"`) in a value | refused at the strict document gate when the value reaches the document, accepted when a later op overwrites or removes it | refused at parse (`reserved key`): the prefix is reserved for the applier's transient touch marks | -Observation, not a difference (reproduced as is): `applyRecord` of the migration map reads -`record.op` whatever `record.target` is, so a cascade record of a view, zone, category or -region rename whose target is an annotation of the legacy id map would be replayed as an -annotation rename, and a reference rename cascading into a finding's citations would be -replayed as a finding rename. The committed changeset renames findings only (33) and no -reference, so neither path runs; a fix would be a declared behaviour change of both -implementations, the owner's call. +Technical debt, kept during R5 (owner decision 2026-10-23, reproduced as is, never corrected +silently): `applyRecord` of the migration map reads `record.op` whatever `record.target` is. +A cascade record of a view, zone, category or region rename whose target is an annotation +of the legacy id map is replayed as a rename of that annotation; a reference rename's +cascade on a cited finding is replayed as a rename of the finding's path. Measured on the +mini sheet (TypeScript values, frozen; test +`the_migration_map_replays_cascade_records_by_their_op_as_the_typescript_does`): renaming +`view:main` sends `legacy-hip` and `legacy-neck` to `principal` and the legacy `head` view +to `principal`; renaming a cited reference sends `hip/hip-dysfunctions/mobility` away, and +re-homing brings it back, counted as 3 re-homed instead of 3 identical; combined with a +finding rename (`renames-cascade`), `hip/hip-dysfunctions/mobility` maps to null where +`hip-joint/hip-dysfunctions/motion` is the right target. The committed changeset renames +findings only (33) and no reference, so the equine map is untouched. A future changeset +renaming a view, zone, region, category or a cited reference must not be published before +the fix (a declared behaviour change on both sides, after R5). Kept as the TypeScript: invalid UTF-8 in the two curation files becomes U+FFFD (`readFile(path, 'utf8')`, as R5.2 kept for the catalog); a repeated key in either file keeps its first position and its last @@ -2956,7 +2966,7 @@ never matches; `String(x)` of a missing id is `"undefined"`; the error messages (`unknown target "…" (no "…")`, `position N out of range`, …) are the TypeScript ones with the change id and op index prefixed. -Tests: `cargo test -p faunosteo-core --features curation` (draft 8, importer 5, formatting +Tests: `cargo test -p faunosteo-core --features curation` (draft 10, importer 5, formatting 6, plus the crate's suites); anatomy 142 (the generator added); `pnpm rust:check` green. Gates: security auditor PASS (semgrep 0 findings, 6 LOW; fixed: the head view box read with