diff --git a/.claude/audits/2026-10-22-rust-r5-draft-verify.md b/.claude/audits/2026-10-22-rust-r5-draft-verify.md index cfc709fe..1d9f2664 100644 --- a/.claude/audits/2026-10-22-rust-r5-draft-verify.md +++ b/.claude/audits/2026-10-22-rust-r5-draft-verify.md @@ -58,3 +58,9 @@ Contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`. Fresh verifier, - Verifier ECARTS(2), C3 + C13: records differ (document identical) on array index keys in authored values or field segments (JSON.parse puts them first) and on integers beyond 2^53 (JSON.parse rounds; a Rust record held 9007199254740993 while document.json printed ...992); §11.12 declared-case count wrong. C1, C2, C4 to C12 MET; 25,000 random changesets: 0 accept/refuse splits, 4,331 accepted identical. - Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final4.md`): MEDIUM a present non-list `children` replaced by `[]` (author data lost); LOW `--check` lists through a top-level folder link, mount alias of the seed, walk cost understated (8.2 MB file ~36 s, cap-bound worst case ~45 min), fixture in shared temp, context items. - Fixed: `text::js_normalize` (key order + big integers at parse) and `text::js_insert` (field path writes); non-list `children`/`categories` refused at add, `children_path` never replaces; `--check` requires each generated folder to be the seed's own; `inside_by_identity` (dev, ino of every ancestor); `MAX_DOCUMENT_VALUES` 200,000; fixture under target/. Cases frozen from TS: 2 key-order, 1 integer, 1 declared non-list children; Rust tests: big integer from raw text, value bound. + +## Fifth final round on 3ece1a6 + +- Verifier ECARTS(2), C3 + C13: a `__proto__` segment in a field path: at the end, TS records `before: {}` (setter ignored the primitive) where Rust recorded null and kept a plain key; in the middle, TS polluted Object.prototype and refused, Rust accepted when overwritten; RUST.md claimed the inherited-member family refused by TS at the op, false for `__proto__`; test count stale (15 vs 13). C1, C2, C4 to C12 MET; 120,000 random changesets: 40,494 accepted identical, no other undeclared split. +- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final5.md`): LOW identity guard covers the seed folder only (a subfolder alias passes), quadratic id comparisons (parse_references 31 s at 160,000), seed SQL formatting (checked), test allow (context). +- Fixed: `__proto__` segment refused at the op (reserved key), declared case frozen from TS (the non-polluting one; the polluting one tested in Rust only, it would corrupt the generator's process); identity guard over every folder of the seed tree; HashSet in assert_declared, distinct, duplicates; docs rows and counts. diff --git a/.claude/audits/2026-10-23-rust-r5-draft-security-final5.md b/.claude/audits/2026-10-23-rust-r5-draft-security-final5.md new file mode 100644 index 00000000..0eae8a3c --- /dev/null +++ b/.claude/audits/2026-10-23-rust-r5-draft-security-final5.md @@ -0,0 +1,141 @@ +SECURITY — VERDICT: PASS +TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten +SCOPE: 37 files +BLOCKING: + (none) +REPORTED (non-blocking): + - [LOW] seed-guard-identity-coverage — crates/core/src/bin/import_equine.rs:263-273 (claim at :236; claim in docs/development/RUST.md §11.13) + - [LOW] quadratic-id-comparisons — crates/core/src/curation/draft/ops.rs:355-383, :404-408; schema.rs:369-380 + - [LOW] sql-built-by-formatting (checked, not injectable) — crates/core/src/curation/draft/seed.rs:29-135 + - [LOW] new-lint-allow-in-test (context, not a scanner suppression) — crates/core/tests/curation_draft.rs:7 +PROOF: semgrep 129 rules on 37 files → 0 findings; checklist 14 checks → 4 findings + +# Security gate, R5.3 curated draft, fifth final round (fresh, blind) + +- Repository: /home/bchanot/Documents/faunosteo, branch `feature/rust-r5-pipeline-core`, HEAD `3ece1a6`. +- Scope: `git diff 5e114da..HEAD` (37 files, all present), focus on `07d6307`. +- Product files read in full: `crates/core/src/bin/import_equine.rs`, `crates/core/src/curation/text.rs`, + `crates/core/src/curation/draft/{apply,fields,migration_map,mod,ops,pipeline,records,rename,schema,seed,shape,stats,targets}.rs`, + `crates/core/src/curation/mod.rs` (diff), `crates/core/Cargo.toml`, `package.json`, `scripts/import-equine.sh`; + test files and the TypeScript oracle generator inspected for file system and process calls. + +## Semgrep + +`EIO_BACKEND=posix semgrep scan --config p/security-audit --config p/secrets --config p/owasp-top-ten --metrics=off --json <37 files>`: +129 rules run (multilang 41, json 3, ts 84, bash 1), 37 targets, 0 findings, 0 errors. These packs carry no Rust +rule: the Rust code is covered by the checklist and the probes below, not by semgrep. + +## Checklist (14 checks) + +| # | Check | Result | +|---|---|---| +| 1 | Hardcoded secret, token, key, auth URL (diff incl. docs, `.claude`, oracle JSON) | none (oracle holds only `https://example.org/a`) | +| 2 | SQL built by string formatting | seed.rs generator, quoting verified: LOW 3 | +| 3 | Unsanitized render, `eval`, `unsafe` | none (no web render, no `unsafe` in scope) | +| 4 | Sensitive endpoint without authz | not applicable (no endpoint; dev CLI and library) | +| 5 | Stack trace, internal path, DB error to an end user | none (CLI messages go to the developer running the tool) | +| 6 | Secret or PII in a log | none (stdout/stderr carry paths, counts, refusals) | +| 7 | Tracked `.env` or credential file | none in the diff (`.env.example` only, pre-existing) | +| 8 | Guardrails scanner (`--domain api,supabase`) | 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER`, `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`) outside the diff, filtered out; 1 MANUAL not reported | +| 9 | New suppressions (anti-gaming) | no `nosemgrep`, `nosec`, `eslint-disable`, `ts-ignore`, `noqa`; one clippy allow in a test file: LOW 4 | +| 10 | `--check` writes nothing | proven: strace of the built binary, 28 `openat`, none with `O_WRONLY`/`O_RDWR`/`O_CREAT`/`O_TRUNC`, no `mkdir`/`unlink`/`rename`/`truncate`/`symlink`/`link`/`chmod`; seed file list, sizes and mtimes identical before and after; `git status` clean; output 12 files, 0 differences | +| 11 | `--out` destinations | 14 probes in a fresh `mktemp -d` under the scratchpad, all as designed (below) | +| 12 | Seed guard of `--out` (canonical path + device/inode) | covers the seed folder and its aliases; not an alias of a seed subfolder: LOW 1 | +| 13 | Resource caps (depth, value bound, non-list children, memory, CPU) | depth, value bound and children refusals hold; memory matches the §11.13 declaration; CPU has an undeclared quadratic term: LOW 2 | +| 14 | `js_insert` / `js_normalize` | behave as `JSON.parse`/assignment (index keys ascending first, `"01"` and `"4294967295"` named, existing key keeps its place, integers past 2^53 rounded, past u64 already f64); recursion bounded by the parser's 128 levels; only called on parsed values and on field-path writes | + +### Probes run (scratchpad only, never `--out` into the repository, no recursive delete) + +Binary built with `CARGO_TARGET_DIR` in the scratchpad (`cargo build --release --locked --offline -p faunosteo-core --features curation --bin import-equine`); +a throwaway library probe linking `faunosteo-core` with `curation`. + +- `--out`: absent folder (written, 12 files); same folder again (`not empty, nothing written`); empty folder with trailing `/` (written); + parent through a symlink to another scratch folder (parent resolved, canonical path printed before the first write); final symlink, + with `/` and as a dangling link (`a symbolic link, nothing written`); `./new/.` (normalised, written); `""` (usage error); `/` + (`not a destination folder`); relative path with `INIT_CWD` unset, relative (ignored) and absolute (typed from it). No entry outside + the probe folder appeared. +- Non-list `children` / `categories` in an added value: object, `null`, string, nested object, nested `null` entry, each refused at the add. +- Depth: an added or set value nested 60 to 122 levels is refused by the 64-level bound after the op; 122+ is refused by the parser's + limit; a 58-level `children` chain (the `touch_tree` recursion) refused by the bound, no stack overflow. +- Memory: one `set` of a 976 KiB array peaks at 156 MB RSS, a 3.9 MiB array at 610 MB (about 156 times the input, refused by the + 200,000-value bound after the op). A 16 MiB changeset therefore asks for about 2.5 GB in one op, in line with the §11.13 row + ("memory in proportion to ops × items × value size", "more than 2 GB"): declaration accurate, not reported. + +## Findings (all non-blocking) + +### LOW 1: the identity guard covers the seed folder, not an alias of one of its subfolders + +`crates/core/src/bin/import_equine.rs:263-273` (`inside_by_identity`, called at `:236`). The guard compares the (device, inode) of +the seed folder with every ancestor of the canonical parent. A bind mount of the seed itself (or of an ancestor) is refused, as +intended. A bind mount of a seed subfolder, for example `draft/` mounted at `/mnt/d`, has the inode of `draft/`, not of the +seed: `--out /mnt/d/x` passes both checks and creates `draft/x/legacy-import/…` and `draft/x/draft/…` inside the official seed. +The same applies to a network share mounted twice with distinct superblocks (distinct `st_dev`). Traced by reading; not probed +(a mount needs privileges and is the user's to run). + +Why only LOW: it needs a mount the owner made by hand; every write is `create_dir`/`create_new` into a new or empty folder, so +nothing is overwritten or deleted, and the stray folder shows up in `--check` as `not produced`. What is materially off is +the wording: the code comment says "a share mounted twice is caught too" and RUST.md §11.13 says "a second mount path of +the seed is refused too". Both hold for the seed folder only. + +Suggested fix: collect the (device, inode) of every folder of the seed tree once (a few dozen) and refuse a parent whose +ancestor matches any of them; or keep the code and narrow the comment and §11.13 to "a second mount path of the seed folder or +of an ancestor". + +### LOW 2: id comparisons are quadratic, a CPU cost the residual-risk declaration does not name + +- `ops.rs:355-372` `assert_declared`: `declared.contains` on a `Vec` costs O(ids × declared), per cite op and per change + (`apply.rs:104`). The TypeScript used a `Set` here (`ops.ts:185`). +- `ops.rs:375-383` `distinct`: O(n²), called once per scope entry inside the cite loop (`:404`). The TypeScript used `new Set`. +- `ops.rs:406` `before.contains` per added id: O(ids × existing), same as the TypeScript's `includes`. +- `schema.rs:369-380` `duplicates`: O(n²) on change and reference ids, same order as the TypeScript's `indexOf`. + +Measured (release build): `parse_references` takes 0.24 s at 20,000 references, 1.9 s at 40,000, 6.0 s at 80,000 and 31 s at +160,000 (10.5 MiB). A subtree cite of every reference on 5 items takes 1.16 s at 16,000 ids and 4.7 s at 32,000, where the +200,000-value bound then refuses. The value bound caps the apply phase near 30,000 references, so a hostile 16 MiB pair of +files stays in minutes, the same order as the declared "a 16 MiB changeset of tiny ops could take minutes". The §11.13 +CPU sentence names only the bound walk. + +Suggested fix: a `HashSet<&str>` for `assert_declared`, `distinct` and the `duplicates` scan (output order kept by walking +the original list). Or add one clause to §11.13 ("id comparisons are quadratic, minutes at the 16 MiB cap"). + +### LOW 3: seed SQL built by string formatting, checked and not injectable + +`crates/core/src/curation/draft/seed.rs:29-135`. A checklist hit on "SQL built by interpolation", but this code writes a +generated seed file (run by `supabase db reset` and the PGlite harness). It is not a runtime query, and no untrusted text +reaches it in an injectable form: + +- the `sql_text` values are pipeline constants, the sheet id from `sheet_id_for(EQUINE_SPECIES_ID, EQUINE_SHEET_VERSION)`, + sha256 hex, a uuid from the sha256 and the fixed `horse.svg`, all quoted by doubling `'` (`standard_conforming_strings` is on + by default in both runners); +- the authored document is dollar-quoted, and a text holding `$equine_doc$` is refused (`:91`). The text is `format_json` + of an object, so it starts with `{` and ends with `}` after `trimEnd`: no tag can form at a junction, and a `$` + escape is decoded and re-emitted as a literal `$`, then caught. + +Same construction as the TypeScript. No action needed. If touched later, keep the tag check on the final text. + +### LOW 4 (context): new lint allow in an integration test + +`crates/core/tests/curation_draft.rs:7` `#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. +This allows the project's no-I/O/no-clock lint and the unwrap lints in a test file, as every other core test file does +(`anatomy_*`, `pdf_*`, `curation_format.rs`, `curation_legacy.rs`, …). It does not silence a security scanner, so it is not +a gated suppression. The binary's `#![allow(clippy::disallowed_methods)]` (`import_equine.rs:15`) was already present at `5e114da`. + +## Owner requirements, checked + +- The official seed is never modified automatically: `--check` is proven read-only. The `--out` seed guard holds for the seed + folder and its aliases (exception: LOW 1). +- `--out` never deletes (no remove call in the binary) and never overwrites (`create_dir`, `create_new`, the claimed folder + re-checked before each file). Refusals probed for an empty path, `..`, a missing parent, a symlink (trailing `/`, `/.`, + dangling), a non-folder and a non-empty folder. +- Strict changeset validation: strict shapes plus the shape comparison; NUL keys refused at parse and in field segments; + non-list or non-object children refused at the add, never replaced (`targets.rs:245-256`). +- Depth and resource caps: parser 128, field path 8 segments, document 64 levels and 200,000 values after each op, files + under 16 MiB. Memory matches the declaration; the undeclared CPU term is LOW 2. +- Author data is never dropped silently in the final document: an overwritten value is recorded as `before`. A path through + a non-object replaces it (`before: null`), which matches the TypeScript (`isItem` excludes arrays too) and is declared in §11.12. + +## Not done + +- Semgrep has no Rust rules in the pinned packs. Rust coverage comes from the reading, the checklist and the probes. +- Mount aliases (LOW 1) were traced by reading, not reproduced. +- The test suites were not run (the verifier's job); only the release binary and the probe were built, outside the repository. diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 58c501ec..8bfc0803 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -2901,7 +2901,7 @@ re-cleaned; the review drawings and the two Markdown reports are R5.4. Contract | `pipeline.rs` | `buildEquineDraft` (CHG-000 from the frozen asset's sha256, byte size and uuid, the head view box parsed from the legacy SVG; the reserved id refused), `renderDraftArtifacts` for the R5.3 files (`document.json`, `assets.json`, `sheet-ref.json` with `document_sha256` = sha256 of the `document.json` bytes, `report-migration-map.json`, `equine_draft.sql`), `parseViewBox` | | `bin/import_equine.rs` | both folders; `--check` compares the produced files with the committed seed and lists the six R5.4 files of the draft as pending (`shortcut:` comment, removed at R5.4); `--out