docs(docs): R5.3 second final round: shape alignment, residual risks measured, R2 derive leniency raised to the owner; audits, TODO

This commit is contained in:
bchanot committed 2026-10-23 01:33:02 +02:00
1 parent 776c7095b3
commit 42bbc48fcc
4 files changed
+258 -7

No files matched your search

@@ -40,3 +40,9 @@ Contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`. Fresh verifier,
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final.md`): MEDIUM-1 a 20,000-segment field path (40 KB changes.json) overflowed the stack; MEDIUM-2 `--out` writes not exclusive after the claim; LOW-1 the NUL field path (same as the verifier); LOW-2 `..` and parents created before a failed claim; LOW-3 a destination inside the seed accepted; LOW-4 INIT_CWD and direct-run base; LOW-5 non-UTF-8 argument panic; LOW-6 clippy disallowed list partial (pre-existing); LOW-7 the test allow line (convention).
- Fixed: NUL segment refused at the op (`reserved key`); field paths capped at 8 segments; document depth capped at 64 levels after every op (iterative walk); `--out`: `..` refused, parent must exist, destination inside the seed refused, destination printed before writing, every folder `create_dir`, every file `create_new`; `args_os`. Documented: direct-run base (LOW-4). Kept: LOW-6 (pre-existing lint scope, out of R5.3), LOW-7 (test convention). Tests added: field-path refusals and 20,000 segments refused without crash, 40 nested adds refused at the bound, 8 `--out` refusals against the built binary with nothing written.
- Next: a fresh verifier and a fresh security audit on the fixed commit.
## Second final round on 314cf04
- Verifier ECARTS(3): C2, C3 (one family) and C13. serde's derives accept a struct written as a positional array and a unit enum written as a one-key map, in changes.json, references.json and the curated document (gate through the R2 derives); Zod refused all. 55,000-changeset random differential (TS generator, Rust probe): 7,436 accepted on both sides byte-identical, 335 mismatches, all this family but one message (`document has no "categories"` vs `document has no list "categories"`). C1, C4 to C12 MET (18 tests, 203 crate tests, `--check` 0/12, refusal probes, floor clean). Note: the same derives sit in the R2 report and archive contracts (unverified there).
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final2.md`): MEDIUM memory amplification (no cap); LOW quadratic depth walk, `--check` and reads following symlinks, residual TOCTOU, `write_exclusive` any component, lossy normalisation, test aimed at the real seed, suppressions context, lint list gaps.
- Fixed: `draft/shape.rs` (typed reading serialised back, compared by shape) on both authored files and the gate; 10 changesets + 2 references files frozen from TS (all refused by TS, refused alike); missing-list message aligned; reads: regular file, 16 MiB cap; `listed` does not follow links; `write_exclusive` plain names, root re-checked; seed guard as pure `inside` + unit test, integration test no longer targets the seed. Kept, declared in §11.13: memory amplification, quadratic walk, single-user TOCTOU residual, lossy UTF-8 and repeated keys. Raised to the owner: the R2 derive leniency in the gated readers.
@@ -0,0 +1,211 @@
# Security gate (fresh): R5.3 draft port, final 2
Branch `feature/rust-r5-pipeline-core`, HEAD `314cf04`, scope `git diff 5e114da..HEAD`
(32 files), focus on `9f11499`. Fresh run: no earlier verdict consulted.
```
SECURITY — VERDICT: PASS
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 32 files
BLOCKING:
(none)
REPORTED (non-blocking):
- [MEDIUM] resource-cap/memory-amplification — crates/core/src/curation/draft/records.rs:68
- [LOW] resource-cap/cpu-quadratic — crates/core/src/curation/draft/apply.rs:109
- [LOW] check-follows-symlinks — crates/core/src/bin/import_equine.rs:148
- [LOW] out-toctou-residual — crates/core/src/bin/import_equine.rs:203
- [LOW] write-exclusive-components — crates/core/src/bin/import_equine.rs:246
- [LOW] silent-input-normalisation — crates/core/src/bin/import_equine.rs:66
- [LOW] test-targets-real-seed — crates/core/tests/curation_draft.rs:681
- [LOW] suppression-context (new, test crate, documented policy) — crates/core/tests/curation_draft.rs:7
- [LOW] suppression-context (pre-existing) + lint list gaps — crates/core/src/bin/import_equine.rs:15
PROOF: semgrep 129 rules on 32 files → 0 findings; checklist 10 checks → 9 findings
```
## Tool notes
- semgrep 1.168.0, pinned packs only, `--metrics=off`. The first run died inside semgrep-core
(`io_uring_queue_init: Cannot allocate memory`, sandbox memlock limit). The rerun with
`EIO_BACKEND=posix` scanned all 32 files: 0 errors, 129 rules run (163 loaded), 0 findings.
- These packs carry almost no Rust rules. The Rust analysis below therefore rests on a full
read of every changed product file plus runtime probes, not on semgrep.
- Guardrails scanner (`api,supabase`): 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER` in
`supabase/migrations/20261008110000_anatomical_sheets.sql`, outside the diff, dropped).
## Checklist
| # | Check | Result |
|---|---|---|
| 1 | Hardcoded secret, token, key, auth URL (diff incl. docs, `.claude`, oracle JSON) | none (only `https://example.org/a`) |
| 2 | SQL by concatenation / interpolation | none: `seed.rs` interpolates constants only, `sql_text` doubles quotes, the document goes into `$equine_doc$…$equine_doc$`, and a document holding the tag is refused. The trimmed document always ends with `}`, so no prefix of the tag can join the closing tag. `config.toml` `sql_paths` lists literal files, so an `--out` under `supabase/seed/` is never executed by `db reset` |
| 3 | Unsanitised render / `eval` | none (no rendering; the TS oracle test has no sink) |
| 4 | Sensitive endpoint without authz | N/A (dev tool, no endpoint) |
| 5 | Stack trace / internal path to end users | none (operator CLI only) |
| 6 | Secret / PII in a log | none |
| 7 | Tracked `.env` / credential file | none (`.env.example` only, outside the diff) |
| 8 | Guardrails scanner | 0 findings in diff files |
| 9 | New suppressions (anti-gaming) | no `nosemgrep` / `nosec` / `eslint-disable` / `ts-ignore` / `noqa`; one clippy allow (L7) |
| 10 | Brief-specific: seed protection, `--check` read-only, `--out` semantics, caps, strict validation, author data | M1, L1 to L6 |
## Findings
### M1 [MEDIUM] No memory cap: hostile `changes.json` amplifies about 10,000× (non-blocking)
`records.rs:68-75`: every record deep-clones the whole `Op`, value included, plus
`before`/`after`. `ops.rs:145-164`: a subtree `set` writes the value into every entry and
records it once per entry. `ops.rs:193,197`: a `rename` records the whole item twice, subtree
included. `import_equine.rs:60-68`: input reads have no size bound. The depth and segment
caps of `9f11499` bound the stack. Nothing bounds memory, the document size or the record
volume.
Measured with the release probe under `ulimit -v 3000000`:
| input | size | peak RSS | outcome |
|---|---|---|---|
| real `changes.json` | 227 KB | 14 MB | OK, 825 records |
| add 1,000 nodes + subtree `set` of a 100 KB description | 154 KB | 1.10 GB | OK, accepted and rendered |
| same, 2,000 nodes | 211 KB | > 2.38 GB | abort: `memory allocation of 804175406 bytes failed` |
| 5,000-node category + 200 rename ping-pongs | 300 KB | 2.13 GB | OK |
| same, 500 renames | 326 KB | > 3 GB | abort |
Growth is linear in (ops × item size): one 90-byte rename op on that category costs about
10 MB. A 1 MB hostile file asks for tens of GB. This host has 14 GB of RAM with its 3 GB of
swap already full, so the result is an OOM kill or thrashing, during `--check` as well. No
data is lost and nothing is written. The input is an authored repository file, which can come
from a contributor PR on an MIT project. The TypeScript cloned the same way, so this is a
parity issue. It still falls short of the owner's explicit "depth and resource caps"
requirement.
Fix direction:
- Cap the input files in the binary (for example `changes.json` ≤ 4 MiB; the real file is
227 KB).
- Bound the curated document next to `assert_depth`: node count or serialised size, plus
string length.
- Optionally, records reference the op (index or `Rc<Op>`) instead of cloning it.
### L1 [LOW] Quadratic CPU: whole-document depth walk after every op
`apply.rs:109,125-147`: `depth_of` (new in `9f11499`) walks the whole document after each op.
A 2.4 MB input (20,000-node added subtree, then 10,000 one-field `set` ops) takes 9.9 s. The
cost is O(ops × document). Fix direction: the input cap of M1. Otherwise compute depth
incrementally (the depth of the touched item's path plus the depth of the written or inserted
value).
### L2 [LOW] `--check` and input reads follow symbolic links of the committed seed
`import_equine.rs:148`: `listed` decides recursion with `entry.path().is_dir()`, which follows
links. `import_equine.rs:60-62,163`: `fs::read` follows links and has no size bound. Two
consequences:
- A committed `draft/x -> /` makes `--check` walk the filesystem and print file names to
stderr, bounded only by ELOOP and time.
- `curation/changes.json -> /dev/zero`, or a FIFO, ends in OOM or a hang.
Both stay read-only and local, with no exfiltration. Traced by reading: no symlink was planted
in the repository. Fix direction: use `entry.file_type()` (no follow) and report a symlink as a
difference; read inputs only after a `symlink_metadata` regular-file check and under a size
cap.
### L3 [LOW] Residual check-then-use window in `--out`
The path is canonicalised at `import_equine.rs:203`, then the folder is created at `:216`.
Each intermediate folder is checked with `symlink_metadata` at `:252`, then the file is opened
at `:257-261`. All of these are path-based. A concurrent local actor with write access to the
destination's parent, or the owner of a pre-existing empty destination, could swap a
component for a symlink between check and use. The impact is bounded: every file is
`create_new` (O_EXCL) with a fixed name, so nothing is overwritten, truncated or deleted. The
worst case is new fixed-name files in another folder the user can write. Fix direction, only
if wanted: hold the claimed root as a directory fd and create children with
`openat`/`mkdirat` + `O_NOFOLLOW` (rustix or cap-std). Otherwise document the single-user
machine assumption.
### L4 [LOW] `write_exclusive` accepts any path component
`import_equine.rs:244-255` pushes each component of `relative` without refusing `RootDir`,
`ParentDir` or `Prefix`. Today every name is a compile-time constant (`render_artifacts`,
`render_draft_artifacts`, `assets/Horse.svg`), so this is not exploitable. It becomes a
traversal if R5.4 derives a file name from data, for example review drawings named after view
ids. Fix direction: refuse any component other than `Component::Normal`.
### L5 [LOW] Silent input normalisation, against "author data never dropped silently"
- `import_equine.rs:66-68`: invalid UTF-8 in `changes.json` or `references.json` silently
becomes U+FFFD.
- `schema.rs:382-384` (`js::parse_json`): a duplicated key keeps its last value, and the first
is dropped without notice.
Both are declared parity with `readFile(..., 'utf8')` and `JSON.parse`. Fix direction: refuse
invalid UTF-8 (`String::from_utf8`) and duplicate keys in the authored files, or warn.
### L6 [LOW] A test runs the real binary against the official seed path
`curation_draft.rs:676-701` runs `import-equine --out <seed>/new-folder`. If the seed guard
regressed, the test itself would create `new-folder/` with 12 files inside the official seed
before its assertion fails (no overwrite). For the same reason this gate did not run
`cargo test --test curation_draft`: the brief forbids `--out` against a repository path. Fix
direction: assert the seed refusal through the pure claim logic with an injectable seed root,
or keep it and accept the regression risk knowingly.
### L7 [LOW, context] New lint allow in a test crate: not a security suppression
`curation_draft.rs:7` adds
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. It sits on an
integration-test crate, uses the same line as 10 existing test files, and follows the policy
written in `clippy.toml` ("Tests … allow these lints locally where they own I/O"). It silences
no security scanner and no security finding, so it is not blocking.
### L8 [LOW, context] Pre-existing allow, and gaps in the I/O ban list
`import_equine.rs:15` `#![allow(clippy::disallowed_methods)]` was already present at
`5e114da`. Separately, `clippy.toml` bans only `fs::read`, `read_to_string`, `write`,
`File::open`/`create`, `env::var`/`var_os` and the clocks. `fs::create_dir*`,
`OpenOptions::open`, `read_dir`, `canonicalize`, `remove_*`, `env::current_dir`,
`env::args_os` and `process::Command` would pass the lint inside the library. A grep confirms
that `crates/core/src` outside `src/bin` uses none of them today. Fix direction: extend the
list.
## Verified, no finding
Stack and validation. The probe linked `faunosteo-core` with the `curation` feature and ran
in debug on a 2 MiB thread, in debug on the main thread, and in release:
- a 20,000-segment field and a 9-segment field are refused; the 8-segment field is refused
later by the strict shape check;
- `name.\0touched` is refused; a `\0touched` key inside a value is refused at parse;
- a value at serde's 128 nesting limit is refused with "recursion limit exceeded";
- a 122-deep object value, a 58-level children chain and 40 chained adds (refused at op 29)
all end with "nests more than 64 levels";
- moving an item into itself is refused;
- no crash, no panic in any mode.
Margins on real data: document depth 10 against the cap of 64; field segments at most 2
against the cap of 8.
`--check`, as briefed: `legacy-import` 6 files, 0 differences; `draft` 6 files, 0 differences
(6 pending R5.4). `git status` clean afterwards.
`--out`, 20 cases in a fresh `mktemp -d`. A copy of the binary was built there; the
repository stayed clean.
- Written:
- an absent folder: 12 files, `document.json` byte-identical to the seed;
- an existing empty folder;
- a parent symlink: the files land at the canonical path;
- an interior `./`;
- a relative path with an absolute `INIT_CWD`; a relative `INIT_CWD` is ignored.
- Refused with exit code 2 and nothing created:
- an existing non-empty folder; a file;
- a symlink, `symlink/`, `symlink/.`, a dangling symlink (its target is not created);
- a missing parent; `..`; `""`; `/`; an extra argument; a non-UTF-8 argument.
Scratch total: 74 files = 6 × 12 + 2 fixtures.
Seed protection, traced by reading and not run against the repository:
- the parent is canonicalised, then compared with `starts_with` component by component
against the canonical seed;
- a root equal to the seed, or to any ancestor, is non-empty and refused;
- a final-component symlink is refused;
- `folders()` is computed before the claim, so a refused changeset creates nothing.
The library performs no file system, environment or process access. The Rust ownership model
rules out reference cycles from `move`.
Probe artefacts (scratch only):
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/secfinal2.SbMNbq`