chore(memory): R5.3 contract, security and verifier audits, TODO, plan status, journal

This commit is contained in:
bchanot committed 2026-10-23 00:11:06 +02:00
1 parent 94280a4c56
commit 8010661de6
6 files changed
+303 -3

No files matched your search

@@ -0,0 +1,157 @@
# Security gate: Rust R5.3 draft pipeline (2026-10-22)
Mode: audit (delta list). Branch `feature/rust-r5-pipeline-core`, uncommitted working tree.
Scope: the R5.3 change set (8 modified files, 17 new files), read whole; `--check` run once
(read-only, exit 0, seed folder unchanged). No code was edited.
```
SECURITY — VERDICT: PASS
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 25 files
BLOCKING:
(none)
REPORTED (non-blocking):
- [LOW] symlink check covers the final component only — crates/core/src/bin/import_equine.rs:182-203
- [LOW] check-then-write is not atomic (TOCTOU) — crates/core/src/bin/import_equine.rs:213-228
- [LOW] serde_json::Map indexed with [] (panics on a missing key) — crates/core/src/bin/import_equine.rs:111
- [LOW] INIT_CWD trusted from the environment — crates/core/src/bin/import_equine.rs:206-211
- [LOW] seed SQL assembled by string formatting — crates/core/src/curation/draft/seed.rs:28-29,91-101
- [LOW] new test file carries the suite's clippy allow line (convention, not a scanner suppression) — crates/core/tests/curation_draft.rs:7
- [MEDIUM] guardrails SB-POLICY-NOT-OWNER, outside the R5.3 delta, pre-existing — supabase/migrations/20261008110000_anatomical_sheets.sql:1177
PROOF: semgrep 725 rules on 25 files → 0 findings; checklist 9 checks → 6 findings (all LOW); guardrails scan.py --domain api,supabase → 0 FAIL, 1 WARN (out of delta), 1 MANUAL; anti-gaming → 0 new un-gated suppressions
```
## Files scanned (25)
Modified: `crates/core/src/bin/import_equine.rs`, `crates/core/src/curation/mod.rs`,
`crates/core/Cargo.toml`, `package.json`, `CLAUDE.md`, `CHANGELOG.md`,
`docs/development/RUST.md`, `.claude/tasks/TODO.md`.
New: `crates/core/src/curation/draft/{mod,schema,targets,fields,ops,rename,apply,records,migration_map,pipeline,seed,stats}.rs`,
`crates/core/tests/curation_draft.rs`, `crates/core/tests/oracles/r5/curation-synthetic.json`,
`packages/anatomy/tests/r5.curation-synthetic-oracles.test.ts`, `scripts/import-equine.sh`,
`.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`.
## Findings in detail
### 1. [LOW] Symlink check covers the final component only
`crates/core/src/bin/import_equine.rs:182-203`. `writable_destination` calls
`fs::symlink_metadata(out)` and refuses when `out` itself is a symbolic link. An ancestor
of `out` that is a symlink is not resolved: `--out <typed-parent>/link/new` with
`link -> <elsewhere>` creates `new/legacy-import` and `new/draft` under the real target of
`link`, and the success line prints the typed path. Because the destination must be absent
or an empty folder, nothing existing can be overwritten or deleted; the effect is new files
in a folder other than the one typed. Fix direction: `fs::canonicalize` the nearest existing
ancestor of `out` and refuse when it differs from the typed ancestor, or at least print the
canonical destination in the success line.
### 2. [LOW] Check-then-write is not atomic
`crates/core/src/bin/import_equine.rs:213-228`. `write` validates with
`writable_destination`, then runs `fs::create_dir_all` and `fs::write` per file. A process
that swaps `out` (or a created subfolder) for a symbolic link between the check and the
writes would have the writes follow it. A development tool on the owner's machine has no
such adversary; reported for completeness since the brief names TOCTOU. Fix direction:
when `out` is absent, claim it with `fs::create_dir(out)` (fails if something appears) and
re-run `symlink_metadata` on the result before writing; a `std::fs::DirBuilder` without
`recursive` for the two folder levels achieves the same.
### 3. [LOW] `serde_json::Map` indexed with `[]`
`crates/core/src/bin/import_equine.rs:111`:
`result.view_boxes[HEAD_VIEW_ID].as_str()`. `Index` on `serde_json::Map` panics on a
missing key (unlike `Index` on `Value`). The legacy importer inserts both `main` and
`head` unconditionally (`crates/core/src/curation/legacy/importer.rs:365-370`), so the
panic is unreachable today; the same pattern was replaced by `get` in commit 6afc1e0 after
a previous gate. Same line in `crates/core/tests/curation_draft.rs:72` (test code, allowed).
Fix direction: `result.view_boxes.get(HEAD_VIEW_ID).and_then(Value::as_str)` with a
typed error.
### 4. [LOW] `INIT_CWD` trusted from the environment
`crates/core/src/bin/import_equine.rs:206-211`. `base_dir` takes `INIT_CWD` as the base
of a relative `--out` without checking it is absolute or exists. pnpm sets it to the
invocation folder, which is the intent; a stale or foreign value exported in the shell
silently retargets a relative `--out`. Mitigations already present: the destination must
be absent or empty, the resolved path is printed, an absolute `--out` ignores the base
(`PathBuf::join` replaces it). Fix direction: accept `INIT_CWD` only when absolute, else
fall back to `current_dir`.
### 5. [LOW] Seed SQL assembled by string formatting
`crates/core/src/curation/draft/seed.rs:28-29` (`sql_text`, doubles single quotes) and
`:91-101` (`document_literal`, dollar-quoted with the tag collision refused). The checklist
flags SQL built by concatenation; here the output is a committed seed file reviewed by diff
and run by `supabase db reset`, not a runtime query. Every interpolated value is derived by
the pipeline: the uuid sheet id, the sha256 hex and byte size of the frozen asset, the
constant file name, the constant change note, and the Prettier-formatted document inside
`$equine_doc$ ... $equine_doc$`. No request or user-typed value reaches it. Parity with
`packages/anatomy/src/curation/seed.ts`. Not a parameterization case; listed so it is not
silently dropped.
### 6. [LOW] Clippy allow line in the new test file
`crates/core/tests/curation_draft.rs:7`:
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. A new
file, so new to the tree, but the identical line heads all 31 committed
`crates/core/tests/*.rs` files and `crates/clippy.toml` states the rule ("tests and the
worker binary allow these lints locally where they own I/O"). Test code, not a security
scanner suppression. The binary's pre-existing `#![allow(clippy::disallowed_methods)]`
(`import_equine.rs:15`) is an unchanged context line and the documented exception for the
two files allowed `std::fs`.
### 7. [MEDIUM, outside the delta] Guardrails `SB-POLICY-NOT-OWNER`
`supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, policy
`species_select_active` on `species` not tied to `auth.uid()`. The scanner runs on the
project root; this file is not part of the R5.3 change set and the policy predates it.
`species` is shared reference data, so an owner scope is likely not the intent; confirm and
record the exception rather than leave it to be re-flagged. `SB-RLS-LIVE` is MANUAL (not
reported here).
## Considered, no finding
- Non-ASCII slicing: `targets.rs:121` slices at `find(':')` and `migration_map.rs:68` at
`rfind('/')`, both ASCII bytes and therefore char boundaries; `migration_map.rs:55-59`
(`strip`) works on UTF-16 units with `from_utf16_lossy`; `schema.rs:319` uses
`strip_prefix`; `fields.rs:84` splits on `'.'`.
- Indexing on parsed JSON in library code: `targets.rs:250,252,253,282,415` read
`ids[0]`, `ids[1]`, `ids[2..]` after `parse_target` guarantees one id (collections) or at
least two (findings); `fields.rs:92` reads `[0]` of a `split('.')` that always yields one
element; `pipeline.rs:81-84` reads four numbers after `len() != 4` is refused;
`migration_map.rs:185` is a slice pattern; `ops.rs:277,304,309,259` remove/insert at
indices produced from the same list moments before (`insertion_index` refuses
`index > len`, `u64 -> usize` saturates). No `unwrap`, `expect`, `panic!`, `unreachable!`,
`unsafe` or `Map[...]` in `crates/core/src/curation/draft/*`.
- Panics from a hand-written `changes.json` / `references.json`: both parsed with strict
serde (`deny_unknown_fields`), custom validators return errors; every op failure refuses
the whole changeset with the change id and op index (`apply.rs:103-109`); the final strict
parse into `AnatomicalSheetDocument` and `validate_sheet_document` refuse shape and
integrity errors (`apply.rs:128-144`); `CHG-000` reserved id refused (`pipeline.rs:127`).
Recursion in `walk`, `touch_tree`, `strip_marks` is bounded by document depth (serde_json
caps parsed nesting at 128; only repeated authored `add`s could deepen it; a stack overflow
would abort a developer tool, nothing more).
- `--check` is read-only: `fs::read`, `fs::read_dir`, `println!`/`eprintln!` only.
Ran once from `crates/core` on the native toolchain: exit 0,
`legacy-import: 6 files, 0 differences`, `draft: 6 files, 0 differences`, six R5.4 files
listed as pending; `git status` shows the seed folder untouched.
- `--out` never deletes: no `remove_*` call in the binary; refuses an empty path, a
symbolic link, a non-folder, a non-empty folder; file names come from constants and the
render functions (no `..`); parents created with `create_dir_all`.
- `scripts/import-equine.sh`: `set -euo pipefail`, `"$@"` quoted, `cargo run --locked`;
`scripts/rust-env.sh` only appends an absolute `$CARGO_HOME/bin` when `cargo` is absent.
- TypeScript generator `packages/anatomy/tests/r5.curation-synthetic-oracles.test.ts`:
writes only `crates/core/tests/oracles/r5/curation-synthetic.json`
(`OUT = resolve(HERE, '../../../crates/core/tests/oracles/r5')`), and only under
`MAKE_R5_ORACLES=1`; otherwise it reads and compares.
- Secrets: none in the change set; no tracked `.env` or credential file; the seed SQL
carries no credential (`creator_user_id null`, `visibility 'private'`).
- Logging: stderr lines carry paths of the developer's own repository; a CLI for the
developer, not an end-user surface.
- Semgrep: 0 results, 0 errors, 0 skipped paths across the three pinned packs
(725 code rules loaded; Rust coverage in these packs is thin, so the manual checklist
carries the weight for `crates/`).
- Observation outside the change set: `.claude/tasks/plans/2026-10-22-rust-r5-pipeline-core.md`
appeared as modified (1 line) during the audit; not produced by `--check` (no write path).
REPORT_WRITTEN: /home/bchanot/Documents/faunosteo/.claude/audits/2026-10-22-rust-r5-draft-security.md
@@ -0,0 +1,35 @@
# Verification: Rust R5.3 draft pipeline (2026-10-22)
Contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`. Fresh verifier, report-only (persisted by the orchestrator: the agent has no write tool).
## Pass 1: ECARTS(1)
- C1 to C12 MET, each with code references and proofs run by the verifier: `cargo test --features curation --test curation_draft` 6/6, `curation_legacy` 5/5, the whole crate with `--features curation --locked` 22 suites green, clippy `-D warnings` clean, fmt clean, `import-equine --check` 12 files 0 differences (git status unchanged), `--out` into a `mktemp -d` folder then `diff -r` (legacy identical; draft identical but the three R5.4 entries), produced `document_sha256` = `27c7383c…2660` = sha256 of the produced `document.json`, 8 refusal and success probes of `--out`, anatomy 142/142, domain 262/262, `tsc --noEmit` clean.
- C13 partial by design: `pnpm check:all` and the security gate were the orchestrator's (both green / PASS); docs present (RUST.md §11.12 and §11.13, CHANGELOG, CLAUDE.md).
- Gap: the floor scan flagged the plan heading of §B.3.5 (`.claude/tasks/plans/2026-10-22-rust-r5-pipeline-core.md:94`, pre-existing at HEAD, R5.1), whose status words read as an unfinished marker. Resolved: heading reworded to "(approved at A.5, applied at R5.6)", the status BDR-068 records.
- Semantic fidelity (beyond the declared `set` without `value`): no reachable divergence. Four unreachable ones: (1) `id_text` printed a non-string id as JSON text; aligned on `js_string` (`String()`); (2) `trimEnd` stripped U+0085; aligned on the core's exact JavaScript white-space set (`is_js_space`); (3) an added item with a `null` child: TypeScript throws at the op, Rust refuses at the shape gate (both refuse; kept, declared in RUST.md §11.12); (4) a position beyond 2^64: both refuse "out of range", the printed number differs (kept, declared).
- Reproduced quirks: `strip` slices eight UTF-16 units of `op.to`; `apply_record` reads `record.op` whatever the record target (declared observation, RUST.md §11.12).
## Pass 2: ECARTS(3)
- C1, C3 to C12 MET (proofs rerun: fmt, clippy, curation_draft 6 + curation_legacy 5, `--check` 12 files 0 differences, 11 `--out` probes incl. a symlinked parent printing the canonical path and a dangling symlink refused, `diff -r`, anatomy 142, domain 262).
- C2 NOT MET: JSON `null` on the optional `scope` (set, cite), `position` (add, move) and `field` (add) parsed as absent by serde; Zod `.optional()` refused it. Not reached by the committed changeset. C13 NOT MET on the same root (undeclared). Fixed: those fields deserialised through `js::present` (absent accepted, `null` refused); three refused synthetic cases `schema-null-{scope,position,field}` added to the generator (58 cases: 9 compared, 49 refused alike).
- Floor: the pass 1 record above quoted the flagged words; reworded.
- Fidelity details, outcome-identical: re-homing candidates on a `Vec` where TS used a `Set` (aligned: deduplicated); `annotation_names` reads a non-string name as empty, duplicate ids first-wins (kept: the legacy import writes strings and unique ids); unlisted counter keys keep insertion order (kept: enum strings); `parseViewBox` error quoting (kept: both refuse).
- Doc line count corrected (2,689).
## Pass 3: ECARTS(4) (first attempt stopped by an API credit error before any check; rerun on Opus)
- C1, C4 to C10, C12 MET (fmt, clippy, curation_draft 6 + curation_legacy 5, `--check` 12/0, `--out` + `diff -r`, anatomy 142, floor clean, a TypeScript probe of 14 changesets and 2 references files).
- C2: three accept/refuse splits at parse, undeclared: (a) a repeated key outside `ops` refused by serde's derive where `JSON.parse` keeps the last value; (d) a lone surrogate escape; (e) nesting deeper than 128. Fixed (a): both files parsed through `js::parse_json` then read into the typed shape (test `a_repeated_key_keeps_its_last_value_as_json_parse_does`); declared (d), (e) in RUST.md §11.12.
- C3: (b) a field path ending in `__proto__`: TypeScript swallowed the write, Rust refuses at the gate: declared. (c) an authored `"\u0000touched"` key: TypeScript refused at the gate, Rust took it for a touch mark and the cleanup dropped it silently: a defect, fixed by refusing any NUL-prefixed key in the changeset at parse (test `a_reserved_touch_mark_key_in_an_authored_value_is_refused`).
- C11: `--out <symlink>/` (trailing slash) followed the link into an empty folder: fixed, the destination is rebuilt from its components before the symlink check (probed: `alink/` and `alink//` refused, rc 2, target left empty).
- C13: the splits above undeclared, and §11.13 claimed a leftover mark would be refused by the gate while the cleanup removed it: both corrected.
- The verifier asked the orchestrator to run a probe harness it had written in the scratchpad (its own run was refused by the permission layer); not run: the cases are covered by the tests above and the declared table.
## Pass 4: ECARTS(1), on C13 only
- C1 to C12 MET (fmt, clippy, curation_draft 8 + curation_legacy 5, `--check` 12/0, `--out` + `diff -r`, refusal probes incl. `symlink/` and `symlink/.` rc 2, `pnpm r5:import-equine --check` rc 0, anatomy 142, floor clean).
- C13: undeclared (a) invalid UTF-8 in changes.json / references.json: TS lossy, Rust refused. Fixed: `read_text` lossy, as R5.2 kept for the catalog. Declared: (b) `1e400` in a discarded value, (c) an inherited `Object.prototype` member in a field path; corrected (c') the NUL-key row, (d) test and line counts (8 tests, 2,718 lines).
- Notes kept: synthetic refusals assert `is_err` only (the contract asks no reason); `scripts/import-equine.sh` called directly without INIT_CWD resolves a relative `--out` from the repository root (pnpm sets INIT_CWD).
- Not re-verified by a fresh verifier after these corrections (docs plus the one-line lossy read); `pnpm check:all` rerun green on the final code (00:10:03), `--check` 12 files 0 differences.