docs(docs): R5.3 third final round declared (RUST.md §11.12-11.13), security final3, verify record

This commit is contained in:
bchanot committed 2026-10-23 10:11:51 +02:00
1 parent 86273b6206
commit 6c0fe88725
3 files changed
+217 -9

No files matched your search

@@ -46,3 +46,9 @@ Contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`. Fresh verifier,
- Verifier ECARTS(3): C2, C3 (one family) and C13. serde's derives accept a struct written as a positional array and a unit enum written as a one-key map, in changes.json, references.json and the curated document (gate through the R2 derives); Zod refused all. 55,000-changeset random differential (TS generator, Rust probe): 7,436 accepted on both sides byte-identical, 335 mismatches, all this family but one message (`document has no "categories"` vs `document has no list "categories"`). C1, C4 to C12 MET (18 tests, 203 crate tests, `--check` 0/12, refusal probes, floor clean). Note: the same derives sit in the R2 report and archive contracts (unverified there).
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final2.md`): MEDIUM memory amplification (no cap); LOW quadratic depth walk, `--check` and reads following symlinks, residual TOCTOU, `write_exclusive` any component, lossy normalisation, test aimed at the real seed, suppressions context, lint list gaps.
- Fixed: `draft/shape.rs` (typed reading serialised back, compared by shape) on both authored files and the gate; 10 changesets + 2 references files frozen from TS (all refused by TS, refused alike); missing-list message aligned; reads: regular file, 16 MiB cap; `listed` does not follow links; `write_exclusive` plain names, root re-checked; seed guard as pure `inside` + unit test, integration test no longer targets the seed. Kept, declared in §11.13: memory amplification, quadratic walk, single-user TOCTOU residual, lossy UTF-8 and repeated keys. Raised to the owner: the R2 derive leniency in the gated readers.
## Third final round on 42bbc48
- Verifier ECARTS(2), C3 + C13, one family: a null or primitive among the children of an added item. TS: TypeError where walked (refused), skipped by a reference rename's citation walk (accepted if later removed), gate if kept. Rust: tolerated by touch/walk (accepted), refused by rename's item_at_mut. Three splits; §11.12 claimed "both refuse" for the null child, false. Differential: 82,000 random changesets, 13,758 accepted identical (document, records, stats, map), 63 schema + 34 references probes agree. C1, C2, C4 to C12 MET (19 tests).
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final3.md`), LOW only: reads follow folder links out of the seed; shape comparison ignores array length; a folder appearing after the claim accepted; seed guard wiring untested; `set name.fr.x` records `before: null`; lint allows (context).
- Fixed: non-object children refused at the add (3 TS-frozen cases, 1 declared refusal); reads confined to the canonical seed; array length compared; only folders this run created; `claim_destination(typed, seed)` + fixture unit test. Declared: replaced-string `before` (TS same), memory factor ~2x TS.
@@ -0,0 +1,188 @@
# Security gate (fresh, audit mode): R5.3 curated draft, third final round
Branch `feature/rust-r5-pipeline-core`, HEAD `42bbc48`. Scope: `git diff 5e114da..HEAD`, 34 files,
with the fix commit `776c709` read whole (shape.rs, schema.rs, apply.rs, targets.rs,
bin/import_equine.rs, tests/curation_draft.rs), plus every other changed product file under
`crates/core/src/curation/draft/` and `scripts/import-equine.sh`.
```
SECURITY — VERDICT: PASS
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 34 files
BLOCKING:
(none)
REPORTED (non-blocking):
- [LOW] reads-follow-directory-symlinks — crates/core/src/bin/import_equine.rs:74
- [LOW] shape-array-length-not-enforced — crates/core/src/curation/draft/shape.rs:47
- [LOW] write-exclusive-accepts-a-new-folder — crates/core/src/bin/import_equine.rs:282
- [LOW] seed-guard-wiring-untested — crates/core/src/bin/import_equine.rs:223
- [LOW] write-field-before-null-on-replaced-holder — crates/core/src/curation/draft/fields.rs:131
- [LOW] clippy-allow-new-in-test (context) — crates/core/tests/curation_draft.rs:7
- [LOW] clippy-allow-pre-existing (context) — crates/core/src/bin/import_equine.rs:15
PROOF: semgrep 163 rules on 34 files → 0 findings; checklist 14 checks → 7 findings
```
## Method
- semgrep 1.168.0, the three pinned packs, `--metrics=off --quiet --json`, on the 34 existing
files of the diff (16 `.rs`, 13 `.md`, 2 `.json`, 1 `.ts`, 1 `.sh`, 1 `.toml`). Exit 0, 0
errors, 0 results; the `--time` rerun counts 163 rules. No `EIO_BACKEND=posix` was needed.
These packs carry few Rust rules, so the verdict rests on reading the code and on the probes.
- Checklist (14 checks): hardcoded secrets, SQL built by formatting, unsafe render, authz,
errors to the user, secrets in logs, tracked `.env` / credentials, guardrails scanner,
new suppressions, `--out` destination containment, read containment, shape and strictness
gate, depth and resource caps, author data loss and audit trail.
- Probes, all under
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/probe.m1yh/`:
a copy of the seed (`seedroot/sheets/equine`), a throwaway crate linking `faunosteo-core`
with `curation`: `ie` = `import_equine.rs` with only the `SEED` constant pointed at the
copy (diff = those lines only), and `shape` = direct calls to `parse_changeset` and
`assert_same_shape`. No `--out` ever targeted the repository. Nothing was deleted; the
scratch artifacts stay in place.
| Probe | Result |
|---|---|
| `ie --check` on the unmodified copy | legacy-import 6 files 0 differences, draft 6 files 0 differences, 6 pending R5.4, exit 0 |
| `--out` parent = symlink into the seed (`seedlink/new`, `//seedlink//new`) | refused, inside the seed folder |
| `--out` = symlink to the seed (`seedlink`, `seedlink/`) | refused, a symbolic link |
| `--out` = seed itself / empty folder inside seed / new folder inside seed / `sheets/./equine/new` | refused (not empty / inside the seed folder) |
| `--out /`, `--out ""` | refused |
| `INIT_CWD=<seed>` with a relative `--out` | refused, inside the seed folder |
| FIFO at a committed file name in `draft/` | not opened, reported, exit 1, no hang |
| 17 MiB sparse input in `legacy-sources/` | refused, larger than 16777216 bytes, exit 2 |
| `add` of a value nested 118 levels; `add` of a 60-level children chain (level B) | refused, nests more than 64 levels, exit 2, no stack overflow |
| changeset value nested 120 / 140 levels | parsed / refused by serde's recursion limit |
| title as array, level as map, scope as map, change as array, position as array | all refused (shape gate or untagged enum) |
| `draft/assets` and `curation` replaced by directory symlinks to outside folders | see L1: read through both links |
## Findings (all LOW, non-blocking)
**L1. Reads follow a symbolic link in any folder component. `import_equine.rs:64-75`
(read at :74), callers :84, :96-97, :123-127, :178.**
`read` checks the final component with `symlink_metadata` (:67), then `fs::read(path)` (:74)
resolves the whole path again. A directory symlink inside the seed is followed. Measured: with
`draft/assets` and `curation` swapped for directory symlinks to folders outside the seed copy,
`--out` wrote its 12 files, exit 0, no warning, built from the changeset and the cleaned drawing
read through the links. `--check` read `assets/horse.svg` through the link without calling it
missing, and only flagged `not produced: assets`, exit 1. Between :67 and :74 a swap of the
final component is also followed, and a FIFO swapped in would block the read. Impact is small:
read-only, regular files under 16 MiB, output only to a fresh folder or a comparison, and the
symlink must already sit in the user's own repository. But the docstring at :64 ("never a
symbolic link") and RUST.md:3017 ("every read a regular file under 16 MiB, never a link")
claim more than the code does, and the owner requirement says no unintended access through
symbolic links. The §11.13 single-user residual names `--out` only.
Fix: open once and check the handle. Either canonicalise the path and require it to equal the
lexical path under the canonical seed (no link in any component), then `File::open`,
`file.metadata()` for `is_file` and length, and read through `take(MAX_FILE_BYTES + 1)`. Or
open with `O_NOFOLLOW` per component. Otherwise, correct :64 and RUST.md:3017 to "final
component only" and add reads to the §11.13 residual.
**L2. The shape gate does not enforce array length. `shape.rs:47-53`.**
When two arrays differ in length, the length guard on the array arm (:47) fails, and the
fall-through at :52 returns `None` because both are arrays. Probe:
`assert_same_shape({"a":[1,2,3]}, {"a":[1]})` returns `Ok(())`. The module doc (:5) and
RUST.md:2969 promise "an array an array of the same length". This is unreachable today. No
deserializer in `draft/schema.rs` or `domain/sheet/schema.rs` changes an array's length:
`non_empty_strings`, `stable_ids` and `at_least_three` keep every element, and `Point` reads
an exact 2-tuple. So it is a latent gap: a future filtering or deduplicating deserializer
would drop elements without the gate noticing.
Fix: `(Value::Array(w), Value::Array(t)) if w.len() != t.len() => Some(format!("expected
{} elements at {path}, received {}", t.len(), w.len()))` ahead of the equal-length arm.
**L3. `write_exclusive` accepts a folder that appeared after the claim.
`import_equine.rs:280-284`, docstring :261-263, RUST.md:2902.**
The docs say an entry that appeared since the claim, "file, folder or symbolic link, is
refused". Line 282 accepts any existing real directory. It has to, for the folders this run
created itself (`legacy-import/` for its second file, `assets/`). This is harmless: it stays
inside the claimed root, a symlink is still refused (`symlink_metadata`), and files stay
`create_new`, so nothing is overwritten. It falls within the declared single-user residual.
Fix: correct the docs to "a file or a symbolic link". Or record the folders this run created
and refuse any other one that already exists.
**L4. The seed guard is tested as a predicate, never through its call site.
`import_equine.rs:223` (wiring), :365-380 (test).**
The unit test pins `inside()` on literal paths. No test runs `claim_destination` against a
seed. A regression at the call site would pass the whole suite: comparing the lexical `out`
instead of the canonical `parent`, or dropping :222-225. That guard backs the owner's first
requirement. The probes above show today's wiring is correct.
Fix: make the seed a parameter of `claim_destination` (the binary passes `SEED`). Then test it
on a temporary "seed" folder: a parent symlinked into it, the seed itself, a new folder under
it. This never touches the real seed.
**L5. Replacing a non-object intermediate records `before: null`.
`fields.rs:126-139` (replace at :131-133), `read_field` :111-122, used at `ops.rs:116-118`.**
`set name.fr.x` turns the string `name.fr` into `{x: …}`. `read_field` returns `None` for
that path, so the record says `before: null` and the original string appears in no record.
The TypeScript does the same (`fields.ts:62-80`). The final document is safe: the strict gate
refuses the stray object unless a later op overwrites the field. In that case the CHANGELOG
trail lacks the original value. This is the family of the declared inherited-member row
(§11.12) and is not declared.
Fix: declare it in the §11.12 table, or refuse a write through an existing non-object
intermediate (always an author error).
**L6. New lint allow in a test (context). `crates/core/tests/curation_draft.rs:7`.**
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]` on an
integration test that reads fixtures and runs the built binary. This is not a security-scanner
suppression, and product code keeps the lint. No `nosemgrep`, `nosec`, `eslint-disable`,
`ts-ignore` or `noqa` was added anywhere in the diff.
**L7. Pre-existing lint allow (context). `crates/core/src/bin/import_equine.rs:15`.**
`#![allow(clippy::disallowed_methods)]` was already present at `5e114da` (line 11). This file
is the one place of the tool allowed the file system.
## Checklist results
| # | Check | Result |
|---|---|---|
| 1 | Hardcoded secret, token, key, auth URL (diff incl. docs, `.claude`, oracles) | none |
| 2 | SQL by formatting | `seed.rs`: fixed and pipeline values through `sql_text` (quotes doubled), the document in `$equine_doc$` with the tag refused (:91); no injection path. Nothing new |
| 3 | innerHTML / eval / unsafe render | none in code (prose in memory files only) |
| 4 | Endpoint without authz | N/A (local tool, no endpoint) |
| 5 | Internals surfaced to a user | N/A (the operator's own CLI) |
| 6 | Secret / PII in a log | none |
| 7 | Tracked `.env` / credentials | `.env.example` only |
| 8 | guardrails `scan.py --domain api,supabase` | 0 FAIL; 1 WARN `SB-POLICY-NOT-OWNER` at `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, outside the delta, not counted; 1 MANUAL |
| 9 | New suppressions (anti-gaming) | none un-gated; L6, L7 context |
| 10 | `--out` containment | passes every probe; L3, L4 |
| 11 | Read containment | L1 |
| 12 | Shape and strictness gate | Zod's two refusals reproduced; L2 latent |
| 13 | Depth and resource caps | 16 MiB reads, serde depth 128, 8 field segments, document depth 64: all hold under probes |
| 14 | Author data loss, audit trail | NUL keys refused, marks stripped; L5 |
## Owner requirements
- Official seed never modified automatically: met. `--out` under or at the seed is refused
on canonical paths (8 probe paths), `--check` writes nothing, the binary test aims only at
`CARGO_TARGET_TMPDIR`, and the TypeScript generator writes oracles only under
`MAKE_R5_ORACLES=1`. Test gap: L4.
- `--check` writes nothing: met. Its code path holds only `symlink_metadata`, `read`,
`read_dir`.
- `--out` never deletes or overwrites: met. No remove call; `create_dir` and `create_new`.
- Dangerous paths and invalid destinations refused: met (empty, `..`, missing parent, `/`,
symlink with `/` or `/.`, file, non-empty folder, inside the seed).
- No unintended access through symbolic links: met on the write side (outside the declared
residual window). Partly met on the read side: L1.
- Strict changeset validation: met. L2 is latent.
- Depth and resource caps: met.
- Author data never dropped silently: met for the document. L5 concerns the record trail only.
## Declared residuals (RUST.md §11.13) checked
- Memory: no new cap. One correction: the Rust records clone the whole op per record
(`records.rs:70`), while the TypeScript shared `ctx.op` (`types.ts:41-48`). A subtree `set`
therefore holds its value twice per record, against once in the TypeScript: about twice the
TypeScript, so "as the TypeScript's clones would" (RUST.md:3032) is low by about 2x. The
conclusion (unbounded, over 2 GB) stands. Not material.
- Depth walk after every op (`apply.rs:110`, :126-139): matches the code. Not re-timed.
- `--out` check-then-use window: consistent. Reads have the same window (L1).
- Lossy UTF-8 (`import_equine.rs:79-81`) and repeated keys (`schema.rs:387-389`):
consistent.
## Permission note
One probe command was refused by the permission system. It was a compound command: write the
shape probe, `cd` into the scratch crate, export `CARGO_TARGET_DIR`, source
`scripts/rust-env.sh`, then `cargo build`. No rule text came back. I did not retry that form
and did not source the env file. The probe file was written alone. The build then ran as a
plain `cargo build --manifest-path <scratch crate>`, with `cargo` already on PATH, and the
permission system allowed it.
+23 -9
View File
@@ -2885,7 +2885,7 @@ until the switch; no fingerprint change; the cleaned `horse.svg` is read as froz
re-cleaned; the review drawings and the two Markdown reports are R5.4. Contract
`.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`.
`crates/core/src/curation/draft` (2,846 lines, feature `curation`):
`crates/core/src/curation/draft` (2,877 lines, feature `curation`):
| Module | Ports |
|---|---|
@@ -2916,14 +2916,14 @@ Parity, every expectation written by the TypeScript (`tests/curation_draft.rs`,
- the 612 legacy leaves through the core's `read_report_document` on the produced map: 442
identical, 141 moved, 29 removed, every resolved leaf a leaf of the produced document, the
29 flagged paths exactly those the map sends to null, the FR snapshots kept;
- 74 synthetic changesets on the mini sheet (`oracles/r5/curation-synthetic.json`,
- 77 synthetic changesets on the mini sheet (`oracles/r5/curation-synthetic.json`,
generator `packages/anatomy/tests/r5.curation-synthetic-oracles.test.ts`): 11 accepted
cases (every op kind, numeric and anchored positions, subtree scopes with skipped equal
values, the review rule on levels A, B and C, the five rename cascades and a reference
rename, re-homing, two chained changes, the two migration-map debt cases below) byte for
byte on the formatted document, the records, the stats, the migration map and the seed
SQL; 61 refusals (23 schema, `null` on `scope`, `position` and `field` and the shape
family below included, 38 applier, integrity and shape gates included) refused alike, with
SQL; 63 refusals (23 schema, `null` on `scope`, `position` and `field` and the shape
family below included, 40 applier, integrity and shape gates included) refused alike, with
the same reason: the same message for an op or change refusal (change id and op index
included), the same stage for a schema or gate refusal; 7 references files refused alike;
two declared
@@ -2942,6 +2942,8 @@ reproduced byte for byte):
| a `set` field path ending in an inherited `Object.prototype` member (`name.constructor`, `name.toString`, `name.valueOf`…) | `readField` returned the inherited function and the record's `structuredClone` threw (`… could not be cloned`): change refused at the op | the member is an ordinary missing key: written, `before` recorded as null. Kept alone, the shape gate refuses it (every object of the sheet contract is strict). Replaced by a later op of the run (a `set name` on the same item), the change is accepted: the document is byte for byte the one the later op alone gives, one extra record traces the overwritten write. Nothing is bypassed: the settable table checks the first segment on both sides (`constructor` as a field: `unknown field`), no value is read through a prototype, the integrity gate runs on the final document (test `an_inherited_member_name_is_a_plain_key_and_bypasses_nothing`, five member names). Accepted by the owner 2026-10-23 |
| a `set` field path ending in `__proto__` (`name.__proto__`) | the prototype setter swallowed the write: change accepted, one record, the document unchanged (or, when the holder was absent, `description: {}` added by `writeField`) | the key is written and the strict document gate refuses it (the R5.2 family: a name of `Object.prototype` is a plain key) |
| an authored key starting with NUL (`"\u0000touched"`) in a value, or a field path segment starting with NUL (`name.\u0000touched`) | refused at the strict document gate when the key reaches the document, accepted when a later op overwrites or removes it | refused (`reserved key`): at parse for a value, at the op for a field path; the prefix is reserved for the applier's transient touch marks, which the cleanup strips at every depth, so a written key would be lost while a record claims it (found by the final gates, test `a_reserved_or_unbounded_field_path_is_refused_without_writing`) |
| a null, a string, a number or a boolean among the `children` or `categories` of an added item | a TypeError where a walk met it (`descendants`, the review marking: refused), silently skipped by a rename's citation walk (accepted when a later op removed the item), refused by the gate when it stayed | refused at the add (`… entry that is not an object`), whatever follows (three cases frozen from the TypeScript, one declared: `declared-children-string-renamed-then-removed`) |
| a `set` whose path goes through a string (`name.fr.x`): `writeField` replaces the string by an object | `before` recorded as undefined (null), the replaced string in no record; refused by the gate unless a later op overwrites the field | the same (kept, recorded here) |
| a field path of more than 8 segments (the deepest settable path has 2) | accepted, the gate refused the stray keys | refused at the op: a resource bound (20,000 segments overflowed the stack before it) |
| a document nested more than 64 levels after an op (the equine document has 10), by repeated `add` or `move`, or a deep value | accepted when valid | refused at the op: a resource bound checked after every op with an iterative walk (test `the_document_cannot_be_nested_past_the_bound_by_repeated_adds`) |
@@ -2987,8 +2989,9 @@ seed SQL built by string formatting as the TypeScript, from pipeline values only
suite's clippy allow line), `.claude/audits/2026-10-22-rust-r5-draft-security.md`. Fresh
verifier, three passes (`.claude/audits/2026-10-22-rust-r5-draft-verify.md`): pass 1 found
four unreachable divergences, two aligned (`String(id)` for finding targets, the exact
JavaScript `trimEnd` set), two kept as both sides refuse (an added item with a `null` child,
refused at a later stage; a position beyond 2^64, a different number printed); pass 2 found
JavaScript `trimEnd` set), one kept as both sides refuse (a position beyond 2^64, a different number printed; the
second, an added item with a `null` child, proved wrong by the third final round and
replaced by the non-object child row of the table); pass 2 found
one reachable divergence, `null` accepted on the optional `scope`, `position` and `field`
where Zod refused it, now refused alike (three synthetic cases added), and aligned the
re-homing candidates on a set (a path met twice counts once); outcome-identical details
@@ -3014,10 +3017,20 @@ authored files and the gate, plus a missing-list message) and security PASS (MED
LOW: quadratic depth walk, `--check` and reads following symbolic links, residual
check-then-use window, `write_exclusive` accepting any component, lossy normalisation, a test
aimed at the real seed). Fixed: the shape comparison; `document has no list` for a missing
list; every read a regular file under 16 MiB, never a link; `--check` lists a link without
list; every read a regular file under 16 MiB whose canonical path lies in the seed folder (a
link inside the seed is followed only if it stays there); `--check` lists a link without
walking into it; `write_exclusive` takes plain names only and re-checks the claimed folder;
the seed refusal tested on the pure check (`inside`), no test targets the real seed. Kept and
declared: the measured residual risks above.
declared: the measured residual risks above. Third final round on 42bbc48: verifier ECARTS(2)
(C1, C2, C4 to C12 met; 82,000 random changesets, 13,758 accepted identical; one family: a
non-object child of an added item, three splits, and the §11.12 sentence on the `null` child
wrong) and security PASS with LOWs only (reads following folder links out of the seed, the
shape comparison not enforcing array length, a folder appearing after the claim accepted, the
seed guard tested only as a bare function, `before: null` over a replaced string). Fixed:
non-object children refused at the add; reads confined to the seed by canonical path; array
length compared; only folders this run created are used; `claim_destination` takes the seed
and is tested on a fixture (`the_claim_refuses_a_destination_inside_the_seed_and_creates_nothing`).
Declared: the replaced-string `before`, the memory factor.
### 11.13 Checkpoint R5.3
@@ -3029,7 +3042,8 @@ refused it. Not probed on those readers, not changed: a fix there needs the owne
Residual risks, measured by the final security gate and kept: memory has no cap beyond the
16 MiB read bound (a hostile 211 KB changeset of subtree sets on a 2,000-node addition asks
for more than 2 GB, as the TypeScript's clones would); the depth check walks the document
for more than 2 GB; the port clones the op into every record where the TypeScript shared it,
about twice its memory); the depth check walks the document
after every op (a 2.4 MB changeset takes about 10 s); `--out` assumes a single-user machine
(a concurrent process able to write the destination's parent could still swap a component
between a check and its use, never overwriting: every file is `create_new`); invalid UTF-8