docs(docs): R5.3 fourth final round declared (RUST.md §11.12-11.13), security final4, verify record

This commit is contained in:
bchanot committed 2026-10-23 10:32:45 +02:00
1 parent 07d63076d3
commit 3ece1a64b1
3 files changed
+211 -12

No files matched your search

@@ -52,3 +52,9 @@ Contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`. Fresh verifier,
- Verifier ECARTS(2), C3 + C13, one family: a null or primitive among the children of an added item. TS: TypeError where walked (refused), skipped by a reference rename's citation walk (accepted if later removed), gate if kept. Rust: tolerated by touch/walk (accepted), refused by rename's item_at_mut. Three splits; §11.12 claimed "both refuse" for the null child, false. Differential: 82,000 random changesets, 13,758 accepted identical (document, records, stats, map), 63 schema + 34 references probes agree. C1, C2, C4 to C12 MET (19 tests).
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final3.md`), LOW only: reads follow folder links out of the seed; shape comparison ignores array length; a folder appearing after the claim accepted; seed guard wiring untested; `set name.fr.x` records `before: null`; lint allows (context).
- Fixed: non-object children refused at the add (3 TS-frozen cases, 1 declared refusal); reads confined to the canonical seed; array length compared; only folders this run created; `claim_destination(typed, seed)` + fixture unit test. Declared: replaced-string `before` (TS same), memory factor ~2x TS.
## Fourth final round on 6c0fe88
- Verifier ECARTS(2), C3 + C13: records differ (document identical) on array index keys in authored values or field segments (JSON.parse puts them first) and on integers beyond 2^53 (JSON.parse rounds; a Rust record held 9007199254740993 while document.json printed ...992); §11.12 declared-case count wrong. C1, C2, C4 to C12 MET; 25,000 random changesets: 0 accept/refuse splits, 4,331 accepted identical.
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final4.md`): MEDIUM a present non-list `children` replaced by `[]` (author data lost); LOW `--check` lists through a top-level folder link, mount alias of the seed, walk cost understated (8.2 MB file ~36 s, cap-bound worst case ~45 min), fixture in shared temp, context items.
- Fixed: `text::js_normalize` (key order + big integers at parse) and `text::js_insert` (field path writes); non-list `children`/`categories` refused at add, `children_path` never replaces; `--check` requires each generated folder to be the seed's own; `inside_by_identity` (dev, ino of every ancestor); `MAX_DOCUMENT_VALUES` 200,000; fixture under target/. Cases frozen from TS: 2 key-order, 1 integer, 1 declared non-list children; Rust tests: big integer from raw text, value bound.
@@ -0,0 +1,170 @@
```
SECURITY — VERDICT: PASS
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 35 files
BLOCKING:
(none)
REPORTED (non-blocking):
- [MEDIUM] author-data-silently-replaced (non-array `children` of an added item) — crates/core/src/curation/draft/ops.rs:252 + crates/core/src/curation/draft/targets.rs:246
- [LOW] check-lists-through-top-level-folder-link — crates/core/src/bin/import_equine.rs:193 (listed, :164)
- [LOW] seed-containment-is-path-based (bind-mount alias) — crates/core/src/bin/import_equine.rs:230 (inside, :253)
- [LOW] declared-depth-walk-cost-understates-cap-bound-worst-case — crates/core/src/curation/draft/apply.rs:110 / docs/development/RUST.md:3046
- [LOW] unit-test-fixture-in-shared-temp-with-fixed-name — crates/core/src/bin/import_equine.rs:383
- [LOW] context: clippy purity lint allowed in a new test file (project convention, not a scanner suppression) — crates/core/tests/curation_draft.rs:7
- [LOW] context: seed SQL rendered by string formatting from pipeline values — crates/core/src/curation/draft/seed.rs:29
PROOF: semgrep 129 rules on 35 files → 0 findings; checklist 9 checks → 7 findings (1 MEDIUM, 6 LOW)
```
# R5.3 draft, security gate (fresh, final 4)
Branch `feature/rust-r5-pipeline-core`, HEAD `6c0fe88`, scope `git diff 5e114da..HEAD`
(35 files, 11 commits), with attention to `86273b6`. Read whole: `bin/import_equine.rs`,
`curation/draft/{shape,ops,apply,schema,targets,fields,records,rename,stats,migration_map,seed,pipeline,mod}.rs`,
`curation/mod.rs`, `scripts/import-equine.sh`, the `package.json` line, the binary and
integration tests that touch the file system, the oracle generator's write guard
(`MAKE_R5_ORACLES=1` only). Every probe ran in
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/sec4.j5fliv`
on a copy of the seed, through a probe binary that is `import_equine.rs` byte for byte except
the `SEED` constant (pointed at the copy). Nothing ran `--out` against the repository, nothing
was deleted, `git status` is clean after the run.
## Semgrep
`semgrep scan --config p/security-audit --config p/secrets --config p/owasp-top-ten
--metrics=off` on the 35 existing files of the diff: 129 rules run, 0 findings, 0 errors.
The rulesets carry almost no Rust rules; the Rust code was covered by reading and probing.
## Checklist
| Check | Result |
|---|---|
| Hardcoded secret / token / key / auth URL | none (grep over the 35 files) |
| SQL by string interpolation | `seed.rs` renders the seed file by formatting: LOW context below, not a runtime query |
| Unsanitized render (innerHTML, eval…) | not applicable (Rust CLI and library, no DOM) |
| Sensitive endpoint without authz | not applicable (no endpoint) |
| Stack trace / internal path to an end user | not applicable (errors go to the local developer's stderr) |
| Secret / PII in a log | none (stdout/stderr carry paths, counts and refusals) |
| Tracked `.env` / credential file | only `.env.example` tracked |
| Guardrails scanner (`--domain api,supabase`) | 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER`, `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, outside the diff, dropped), 1 MANUAL |
| New suppressions (anti-gaming) | no `nosemgrep`, `nosec`, `eslint-disable`, `ts-ignore`, `noqa`; one clippy allow in a new test file (LOW context below) |
## Owner requirements, verified
- Seed never modified automatically: every read goes through `read()` (canonical path inside
the canonical seed, regular file, 16 MiB cap). Probed: `curation/` as a folder link out of
the seed and `changes.json` as a file link out of it are both refused (`leads outside the
seed folder`); a folder link that stays inside the seed is accepted.
- `--check` writes nothing: only `canonicalize`, `metadata`, `read`, `read_dir` on that path.
Probe copy: 12 files, 0 differences, nothing new under the fixture seed (`find -newer`).
- `--out` refusals, probed on the fixture seed: a link to the seed and to `draft/` (as the
destination and as its parent), a link chain `p/q` ending in a seed link, the seed path
itself plus `/new`, `legacy-import/new`, `INIT_CWD` set inside the seed, `..`, an empty path,
`/`: all refused, nothing created. A relative `INIT_CWD` is ignored. A valid destination
gets 12 files identical to the seed (`diff -rq`), a second run on it is refused `not empty`.
Files are `create_new`, folders `create_dir` tracked in `created`, so an entry that appears
after the claim is refused. File names are pipeline constants.
- Strict changeset validation, depth and resource caps: `deny_unknown_fields`, the `op`
discriminator, non-empty strings, the shape comparison (now with array length), reserved
NUL keys at parse and in field paths, 8 field segments, serde_json's 128 parse depth, 64
document levels after each op (iterative walk), 16 MiB per file. Recursive walks
(`reserved_key`, `first_mismatch`, `strip_marks`, `walk`, `touch_tree`, `js_deep_equal`) are
all bounded by the 128 parse depth or the 64-level check.
- `86273b6`: `assert_object_children` refuses a non-object entry of a `children` or
`categories` list (explicit stack); the length arm of `first_mismatch` sits before the
element arm; `claim_destination(typed, seed)` with its fixture test refuses before any
`create_dir`. All correct. One gap left next to the first: MEDIUM below.
## Findings
### MEDIUM: a non-array `children` on an added item is silently replaced by `[]`
`crates/core/src/curation/draft/ops.rs:252` (`assert_object_children` skips a `children`
value that is not an array) and `crates/core/src/curation/draft/targets.rs:246`
(`children_path` overwrites any present non-array `children` with `[]`). An `add` whose value
carries `"children"` as an object, a string or `null`, followed by an `add` (or a `move`)
into that item, is accepted: the strict gate never sees the author's value, which is gone
from `document.json` and the migration map. Probed on the real equine changeset plus one
level-A change: `"children": {"author-note": {"id": "kept", ...}}` (a whole nested finding
written as an object instead of an array), `"children": "author text"` and
`"children": null` all give `probe-node.children = [probe-child]`, exit 0, 12 files written.
Only the add record's `after` keeps the value; no R5.3 output shows it (the CHANGELOG is
R5.4). Parity with the TypeScript `childrenOf`, and not declared: §11.12 says "a lean leaf
gets `children: []` appended on demand", which covers an absent key, not a present value
replaced. This breaks the owner's "author data never dropped silently" through a plausible
typo (object in place of array). Non-blocking: no security boundary is crossed.
Hint: in `assert_object_children`, refuse a present `children` or `categories` key whose
value is not an array (same message family, same place as the entry check), or make
`children_path` refuse a present non-array value and append `[]` only when the key is absent.
Declare the split in the §11.12 table with a frozen TypeScript case.
### LOW: `--check` lists through a top-level folder link
`crates/core/src/bin/import_equine.rs:193` calls `listed(&target, ...)` on
`SEED/legacy-import` or `SEED/draft` without checking the folder itself: `read_dir` follows
it when it is a link. Probed with `legacy-import` pointing out of the seed: the reads are
refused (`missing: ...`), but `not produced: private-name.txt` and `not produced:
sub/deep.txt` print the names of files outside the seed, subfolders included. Read-only,
names only, same user. §11.12's "`--check` lists a link without walking into it" holds for
entries, not for the generated folder itself.
Hint: canonicalize the target and require `inside(&canonical, &seed)` before listing, or
refuse a target whose `symlink_metadata` is a link.
### LOW: the seed guard is path-based, a mount alias passes it
`crates/core/src/bin/import_equine.rs:230` compares canonical paths (`starts_with`).
`canonicalize` resolves links, not mounts: if the seed (or the repository) is reachable
through a second path (bind mount, the same NAS share mounted twice), `--out
<alias>/.../equine/new` passes `inside` and creates a new folder inside the official seed.
Only new files (never an overwrite, never a delete); the user must type the alias. Reasoned,
not probed (a mount needs privileges).
Hint: also compare `(dev, ino)` (`std::os::unix::fs::MetadataExt`) of the canonical parent and
each of its ancestors with the seed folder's; a bind mount root shares the source's pair.
### LOW: the declared depth-walk cost understates the cap-bound worst case
`crates/core/src/curation/draft/apply.rs:110` runs `assert_depth` over the whole document
after every op; RUST.md:3046 declares it with one sample ("a 2.4 MB changeset takes about
10 s"). The sample is right (probe: 2.3 MB, one 1 M-node add then 2,000 sets, 12.4 s,
455 MB). The worst case inside the 16 MiB read cap is two orders larger: 8.2 MB, one 4 M-node
add then 1,000 sets, 35.7 s and 1.7 GB, so about 30 ms per op, and the remaining 8 MB holds
about 94,000 more ops: roughly 45 minutes of CPU before the final gate refuses. Twenty
thousand plain sets on the real document take 0.9 s, so ordinary changesets are not affected.
Local developer tool, interruptible. The declaration is not wrong, it is incomplete.
Hint: state the cap-bound figure in §11.13, or bound the work: a total node cap on the
document after each op (also bounds memory), or a depth check limited to the subtree the op
wrote.
### LOW: unit-test fixture at a fixed name in the shared temp folder
`crates/core/src/bin/import_equine.rs:383`: `env::temp_dir().join("faunosteo-import-equine-claim")`
with `create_dir_all`. On a multi-user host another account can pre-create or link that
name. The test only runs refusals, so the worst outcome is a failed test or an empty folder
created where the link points. The integration test already uses the private
`CARGO_TARGET_TMPDIR`.
Hint: put the fixture under the target folder (for example
`concat!(env!("CARGO_MANIFEST_DIR"), "/../../target/...")`) or a per-run name created
exclusively.
### LOW (context): clippy purity lint allowed in a new test file
`crates/core/tests/curation_draft.rs:7` `#![allow(clippy::disallowed_methods,
clippy::unwrap_used, clippy::expect_used)]`. A determinism and robustness lint (no I/O, no
clock in the core), not a security scanner suppression. The same line heads the ten other
core test files. The binary's `#![allow(clippy::disallowed_methods)]` was already present at
`5e114da`. Not blocking.
### LOW (context): seed SQL rendered by string formatting
`crates/core/src/curation/draft/seed.rs:29` (`sql_text`) and `:90` (`document_literal`). Every
interpolated value is a pipeline constant, a sha256 hex, a derived uuid or a byte count;
quotes are doubled (correct with `standard_conforming_strings` on, the PostgreSQL default);
the document is embedded in a `$equine_doc$` literal and refused if it holds the tag; the
formatted JSON always ends with `}`, so no tag can form at the closing boundary. This
generates a reviewed seed file, it is not a runtime query. No action.
## Residual risks declared in RUST.md §11.13
Memory without a cap beyond 16 MiB per file: confirmed (1.7 GB from an 8.2 MB file), in line
with the declaration. Single-user check-then-use window on reads and `--out`: confirmed as
described, every file stays `create_new`. Lossy UTF-8 and repeated keys: as declared. The
quadratic walk: LOW above.
+35 -12
View File
@@ -2885,7 +2885,7 @@ until the switch; no fingerprint change; the cleaned `horse.svg` is read as froz
re-cleaned; the review drawings and the two Markdown reports are R5.4. Contract
`.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`.
`crates/core/src/curation/draft` (2,877 lines, feature `curation`):
`crates/core/src/curation/draft` (2,912 lines, feature `curation`):
| Module | Ports |
|---|---|
@@ -2916,18 +2916,19 @@ Parity, every expectation written by the TypeScript (`tests/curation_draft.rs`,
- the 612 legacy leaves through the core's `read_report_document` on the produced map: 442
identical, 141 moved, 29 removed, every resolved leaf a leaf of the produced document, the
29 flagged paths exactly those the map sends to null, the FR snapshots kept;
- 77 synthetic changesets on the mini sheet (`oracles/r5/curation-synthetic.json`,
generator `packages/anatomy/tests/r5.curation-synthetic-oracles.test.ts`): 11 accepted
- 81 synthetic changesets on the mini sheet (`oracles/r5/curation-synthetic.json`,
generator `packages/anatomy/tests/r5.curation-synthetic-oracles.test.ts`): 14 accepted
cases (every op kind, numeric and anchored positions, subtree scopes with skipped equal
values, the review rule on levels A, B and C, the five rename cascades and a reference
rename, re-homing, two chained changes, the two migration-map debt cases below) byte for
rename, re-homing, two chained changes, the two migration-map debt cases below, the three
`JSON.parse` cases of the aligned paragraph) byte for
byte on the formatted document, the records, the stats, the migration map and the seed
SQL; 63 refusals (23 schema, `null` on `scope`, `position` and `field` and the shape
family below included, 40 applier, integrity and shape gates included) refused alike, with
the same reason: the same message for an op or change refusal (change id and op index
included), the same stage for a schema or gate refusal; 7 references files refused alike;
two declared
differences below (one refusal, one acceptance), each held by its own test.
four declared synthetic cases below (three refusals, one acceptance), each held by its own
test (14 compared + 63 refused alike + 4 declared = 81).
Differences declared (none reachable by the committed files, since the folders are
reproduced byte for byte):
@@ -2943,6 +2944,7 @@ reproduced byte for byte):
| a `set` field path ending in `__proto__` (`name.__proto__`) | the prototype setter swallowed the write: change accepted, one record, the document unchanged (or, when the holder was absent, `description: {}` added by `writeField`) | the key is written and the strict document gate refuses it (the R5.2 family: a name of `Object.prototype` is a plain key) |
| an authored key starting with NUL (`"\u0000touched"`) in a value, or a field path segment starting with NUL (`name.\u0000touched`) | refused at the strict document gate when the key reaches the document, accepted when a later op overwrites or removes it | refused (`reserved key`): at parse for a value, at the op for a field path; the prefix is reserved for the applier's transient touch marks, which the cleanup strips at every depth, so a written key would be lost while a record claims it (found by the final gates, test `a_reserved_or_unbounded_field_path_is_refused_without_writing`) |
| a null, a string, a number or a boolean among the `children` or `categories` of an added item | a TypeError where a walk met it (`descendants`, the review marking: refused), silently skipped by a rename's citation walk (accepted when a later op removed the item), refused by the gate when it stayed | refused at the add (`… entry that is not an object`), whatever follows (three cases frozen from the TypeScript, one declared: `declared-children-string-renamed-then-removed`) |
| a `children` or `categories` of an added item that is present but not a list (an object, a string, `null`) | replaced by `[]` when a later op added under the item (`childrenOf`): the author's value lost without a word | refused at the add (`… that is not a list`); `children_path` never replaces a present value (declared case `declared-children-object-then-added-to`) |
| a `set` whose path goes through a string (`name.fr.x`): `writeField` replaces the string by an object | `before` recorded as undefined (null), the replaced string in no record; refused by the gate unless a later op overwrites the field | the same (kept, recorded here) |
| a field path of more than 8 segments (the deepest settable path has 2) | accepted, the gate refused the stray keys | refused at the op: a resource bound (20,000 segments overflowed the stack before it) |
| a document nested more than 64 levels after an op (the equine document has 10), by repeated `add` or `move`, or a deep value | accepted when valid | refused at the op: a resource bound checked after every op with an iterative walk (test `the_document_cannot_be_nested_past_the_bound_by_repeated_adds`) |
@@ -2963,6 +2965,14 @@ findings only (33) and no reference, so the equine map is untouched. A future ch
renaming a view, zone, region, category or a cited reference must not be published before
the fix (a declared behaviour change on both sides, after R5).
Aligned on `JSON.parse`, not a difference: an authored value keeps JavaScript's key order (array
index keys such as `"1"` first, ascending, then the others in insertion order) and an integer
beyond 2^53 is read as the double JavaScript holds (`9007199254740993` → `9007199254740992`),
at parse (`text::js_normalize`) and when a field path writes a key (`text::js_insert`), so a
record never holds a value its own document lacks (found by the fourth final verifier; cases
`js-index-key-in-value-overwritten`, `js-index-segment-overwritten`, test
`an_integer_beyond_2_53_is_read_as_the_double_json_parse_gives`).
Aligned on Zod, not a difference: serde's derives read a struct from a positional array
(`"title": ["Titre", "Title"]`, a whole change or reference as an array) and a unit enum
variant from a one-key map (`"level": {"A": null}`, `scope`, `review_status`, `laterality`,
@@ -3030,7 +3040,15 @@ seed guard tested only as a bare function, `before: null` over a replaced string
non-object children refused at the add; reads confined to the seed by canonical path; array
length compared; only folders this run created are used; `claim_destination` takes the seed
and is tested on a fixture (`the_claim_refuses_a_destination_inside_the_seed_and_creates_nothing`).
Declared: the replaced-string `before`, the memory factor.
Declared: the replaced-string `before`, the memory factor. Fourth final round on 6c0fe88:
verifier ECARTS(2) (C1, C2, C4 to C12 met, 0 accept/refuse split in 25,000 random changesets;
records differing from the TypeScript on array index keys and on integers beyond 2^53; the
declared-case count wrong) and security PASS with one MEDIUM (a present non-list `children`
silently replaced by `[]`) and LOWs (`--check` listing through a generated folder that is a
link, a mount alias of the seed, the walk cost understated, the claim test fixture in the
shared temporary folder). Fixed: `js_normalize` and `js_insert`; non-list `children` and
`categories` refused at the add and never replaced; `--check` confined to the seed's folders;
the identity guard; a 200,000-value bound; the fixture under `target/`.
### 11.13 Checkpoint R5.3
@@ -3040,16 +3058,21 @@ the readers of imported PDFs (`archive/read.rs`, `report/read.rs`, `pdf/import.r
accept an object written as an array or an enum written as a map where the frozen Zod reader
refused it. Not probed on those readers, not changed: a fix there needs the owner's per-file go.
Residual risks, measured by the final security gate and kept: memory has no cap beyond the
16 MiB read bound (a hostile 211 KB changeset of subtree sets on a 2,000-node addition asks
for more than 2 GB; the port clones the op into every record where the TypeScript shared it,
about twice its memory); the depth check walks the document
after every op (a 2.4 MB changeset takes about 10 s); `--out` assumes a single-user machine
Residual risks, measured by the final security gates and kept: the document is bounded (64
levels, 200,000 values, checked after every op; the equine document has 10 and 6,374), but
the records are not: a hostile changeset of subtree sets with a large value still costs
memory in proportion to ops × items × value size (211 KB asked for more than 2 GB before the
value bound; the port clones the op into every record where the TypeScript shared it, about
twice its memory); the bound walk runs after every op (at most 200,000 values each, a few
milliseconds; a 16 MiB changeset of tiny ops could take minutes); `--out` assumes a single-user machine
(a concurrent process able to write the destination's parent could still swap a component
between a check and its use, never overwriting: every file is `create_new`); invalid UTF-8
and repeated keys are read as the TypeScript read them. Residual risks of the slice: the
`--check` allowance for the six R5.4 files hides nothing about the
produced files but means the draft folder is not yet compared whole (R5.4 removes it); the
seed guard of `--out` compares canonical paths and file identity (device, inode) of every
ancestor, so a second mount path of the seed is refused too; `--check` refuses a generated
folder that is not itself a folder of the seed (a link there is never listed through); the
touch marks are a reserved prefix (a NUL character starts it), refused in any authored value
at parse and in any field path segment at the op, and removed after each change, so no mark
reaches a record or the document; the migration-map replay quirk above