chore(memory): R5.3 closed on c6e0d85 (verifier CONFORME 13/13, security PASS), journal, TODO, security final6

This commit is contained in:
bchanot committed 2026-10-23 11:12:22 +02:00
1 parent c6e0d851a9
commit 0a78492d64
4 files changed
+164 -1

No files matched your search

@@ -64,3 +64,10 @@ Contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`. Fresh verifier,
- Verifier ECARTS(2), C3 + C13: a `__proto__` segment in a field path: at the end, TS records `before: {}` (setter ignored the primitive) where Rust recorded null and kept a plain key; in the middle, TS polluted Object.prototype and refused, Rust accepted when overwritten; RUST.md claimed the inherited-member family refused by TS at the op, false for `__proto__`; test count stale (15 vs 13). C1, C2, C4 to C12 MET; 120,000 random changesets: 40,494 accepted identical, no other undeclared split.
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final5.md`): LOW identity guard covers the seed folder only (a subfolder alias passes), quadratic id comparisons (parse_references 31 s at 160,000), seed SQL formatting (checked), test allow (context).
- Fixed: `__proto__` segment refused at the op (reserved key), declared case frozen from TS (the non-polluting one; the polluting one tested in Rust only, it would corrupt the generator's process); identity guard over every folder of the seed tree; HashSet in assert_declared, distinct, duplicates; docs rows and counts.
## Sixth final round on c6e0d85: CONFORME 13/13 (R5.3 closed)
- Verifier CONFORME: C1 to C13 MET on the committed state c6e0d85 (23 curation tests, `--check` 12/0 via cargo and pnpm, `--out` + `diff -r`, 21 destination probes, anatomy 142, floor clean; TS-vs-Rust differential on 70,073 changesets: 9,397 accepted identical, 0 Rust-only accepts, TS-only accepts all declared families).
- Security PASS (`.claude/audits/2026-10-23-rust-r5-draft-security-final6.md`), LOW only: destination folder itself not identity-checked (needs an empty seed folder + a hand mount; none exists), one linear membership in cite (doc precision), seed SQL formatting (checked), test allow (context).
- check:all green 10:54:50 on c6e0d85.
- Carried into R5.4 (not applied to the verified state): destination identity check, cite membership hashed, "seed folder unreadable" message, stale test comment, doc nits (inherited row: remove form too; 200,000-value row in the table; line count 2,908; set-without-value counted among schema refusals).
@@ -0,0 +1,151 @@
SECURITY — VERDICT: PASS
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 38 files
BLOCKING:
(none)
REPORTED (non-blocking):
- [LOW] seed-guard-destination-identity — crates/core/src/bin/import_equine.rs:236 (destination branch :243-253)
- [LOW] cite-membership-linear (doc precision, RUST.md §11.13) — crates/core/src/curation/draft/ops.rs:401-405
- [LOW] sql-built-by-formatting (checked, not injectable) — crates/core/src/curation/draft/seed.rs:29-135
- [LOW] new-lint-allow-in-test (context, not a scanner suppression) — crates/core/tests/curation_draft.rs:7
PROOF: semgrep 163 rules on 38 files → 0 findings; checklist 14 checks → 4 findings
# Security gate, R5.3 curated draft, sixth final round (fresh, blind)
- Repository: /home/bchanot/Documents/faunosteo, branch `feature/rust-r5-pipeline-core`, HEAD `c6e0d85`.
- Scope: `git diff 5e114da..HEAD`, 38 files, all present. Focus on `f245b3e`.
- Product files read in full: `crates/core/src/bin/import_equine.rs`,
`crates/core/src/curation/draft/{apply,fields,migration_map,mod,ops,pipeline,records,rename,schema,seed,shape,stats,targets}.rs`,
`crates/core/src/curation/{mod,text}.rs`, `scripts/import-equine.sh`, and the diffs of `crates/core/Cargo.toml` and
`package.json`. The test files and the TypeScript oracle generator were checked for file system and process calls.
## Semgrep
`EIO_BACKEND=posix semgrep scan --config p/security-audit --config p/secrets --config p/owasp-top-ten --metrics=off --quiet --json <38 files>`:
163 rules run (javascript 73, generic 57, typescript 11, python 6, package managers 5, json 3, java 3, scala 3, bash 1,
ruby 1), 38 files scanned, 0 findings, 0 errors. The pinned packs hold no Rust rule, so the Rust code is covered by the
reading, the checklist and the probes below.
## Checklist (14 checks)
| # | Check | Result |
|---|---|---|
| 1 | Hardcoded secret, token, key, auth URL (whole diff incl. docs, `.claude`, oracle JSON) | none |
| 2 | SQL built by string formatting | `seed.rs` generator, quoting verified: LOW 3 |
| 3 | Unsanitized render, `eval`, `unsafe`, process spawn | none in scope |
| 4 | Sensitive endpoint without authz | not applicable (dev CLI and library, no endpoint) |
| 5 | Stack trace, internal path, DB error to an end user | none (CLI messages go to the developer running the tool) |
| 6 | Secret or PII in a log | none (stdout and stderr carry paths, counts, refusals) |
| 7 | Tracked `.env` or credential file | none (`.env.example` only, pre-existing, outside the diff) |
| 8 | Guardrails scanner (`--domain api,supabase`) | 0 FAIL, 1 WARN `SB-POLICY-NOT-OWNER` at `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, outside the diff, filtered out; 1 MANUAL not reported |
| 9 | New suppressions (anti-gaming) | no `nosemgrep`, `nosec`, `eslint-disable`, `ts-ignore`, `noqa`; one clippy allow in a test file: LOW 4 |
| 10 | `--check` writes nothing | built binary run: 12 files, 0 differences, 6 R5.4 files listed as pending; seed file list and mtimes identical before and after; `git status` clean |
| 11 | `--out` destinations | 11 probes in a fresh `mktemp -d` under the scratchpad, all as designed (below) |
| 12 | Seed guard of `--out` (canonical path, then device/inode over the seed tree) | holds for every parent path; the destination folder itself is not identity-checked: LOW 1 |
| 13 | `__proto__` field segments | refused at every position, before any write (probe below) |
| 14 | Resource caps and id comparisons | parser depth, 8 segments, 64 levels, 200,000 values hold; hash sets in `distinct`, `assert_declared`, `duplicates` keep the old semantics; one linear membership left in `cite`: LOW 2 |
### Probes (scratchpad only; no `--out` into the repository; no recursive delete)
The binary was built with `CARGO_TARGET_DIR` in the scratchpad (`cargo build --release --locked --offline -p faunosteo-core --features curation --bin import-equine`).
Two throwaway probes ran there: one linking `faunosteo-core` with `curation`, and one compiling lines 213-293 of
`import_equine.rs` verbatim against a scratch seed tree.
- `split_field`: `__proto__`, `name.__proto__`, `name.__proto__.x` and `name.fr.__proto__` are all refused with
`reserved key "__proto__"`. `name.__proto__ ` (trailing space), `name.__PROTO__`, `name.constructor` and `name.prototype` are
plain keys, as in JavaScript on a plain object. A NUL segment, 9 segments and an empty segment are refused.
`apply_changeset` with `set name.__proto__…` followed by an overwriting `set name` refuses at `CHG-001 op 0` in all four
variants. `is_settable` calls `split_field` before `item_at_mut`, so nothing is touched or written first.
- Identity guard (verbatim copy, scratch tree `seed/{a/b/c,d}` plus a link to `/` and a link loop inside it): the walk
records 5 identities and follows neither link. `seed/a/b/new`, `seed/new`, a parent link into `seed/a/b` and a parent link
to `seed` are all refused as `inside the seed folder`, with nothing created. A final link, a trailing `/` and a trailing
`/.` are refused as `a symbolic link`. A sibling folder is accepted. With one seed folder made unreadable (`chmod 000`,
restored afterwards), every claim is refused: the walk fails closed.
- Real binary `--out`: absent folder (12 files written, then `diff -r` against the seed shows only the 6 R5.4 files);
the same folder again (`not empty`, exit 2); an empty folder (written); a regular file (`not a folder`); a final link, with
`/` and `/.` (`a symbolic link`, exit 2, link target left empty); `empty/../x` (`..` refused); a missing parent; `""`
(usage error); a relative path with `INIT_CWD` unset (typed from the current folder, the canonical path printed before the
first write). No entry appeared outside the probe folder.
- Cite cost (release build): citing 28,000 declared ids twice on one finding takes 0.92 s, four times 2.7 s. The value bound
then refuses a larger registry plus its citations.
## Findings (all non-blocking)
### LOW 1: the destination folder itself is never identity-checked
`crates/core/src/bin/import_equine.rs:236` runs `inside_by_identity(&parent, seed)`, which checks the ancestors of the
canonical parent. When the destination already exists (`:247-253`), only its emptiness is checked, never its own
(device, inode). Suppose an existing empty folder is a second mount path (a bind mount, or a share mounted twice on the
same device) of an empty folder of the seed. The claim accepts it, and `write` then creates `legacy-import/…` and
`draft/…` inside the seed tree. A mount alias of a non-empty seed folder is refused as `not empty`.
Why only LOW: this needs an empty folder under the seed (none in the checkout: `find … -type d -empty` returns nothing,
and git tracks no empty folder) plus a mount made by hand at the destination. Every write is `create_dir` or `create_new`,
so nothing is overwritten or deleted. Also, the §11.13 sentence "a second mount path of the seed or of any of its folders
on the same device is refused too" holds for every parent, but not for the destination itself. Traced by reading, not
reproduced: a mount needs privileges and is the user's to run.
Suggested fix: in the existing-folder branch, also refuse when `inside_by_identity(&root, seed)` is true (`root.ancestors()`
starts with `root` itself). Alternatively, run the identity check on `root` once it exists, created or not. Or narrow the
§11.13 sentence to the ancestors of the destination.
### LOW 2: one cite comparison is still linear, while §11.13 says cite uses hash sets
`crates/core/src/curation/draft/ops.rs:401-405`: for each scope entry, every distinct cited id is checked with
`before.contains(&Value::from(id))` on the item's existing citations. That costs O(distinct × existing) per item. `distinct`
and `assert_declared` are hashed now, but this membership is not, and §11.13 reads "id comparisons use hash sets (cite,
declared references, duplicate ids)". Measured: about 0.9 s per repeated op at the largest registry the 200,000-value bound
admits (about 28,000 references, each serialised as 6 values). Spread over a subtree, the worst case is about twice that.
A 16 MiB changeset therefore stays in minutes, the same order as the declared CPU residual. So this is a precision issue
in the declaration, not a new risk.
Suggested fix: build a `HashSet<&str>` of the string entries of `before` once per item and keep the output order by
walking `distinct`. Or reword §11.13 ("cite deduplication and the declared check are hashed; the check against existing
citations is linear, bounded by the value bound").
### LOW 3: seed SQL built by string formatting, checked and not injectable
`crates/core/src/curation/draft/seed.rs:29-135`. This is a checklist hit on "SQL built by interpolation", but the code
writes a generated seed file, not a runtime query. The `sql_text` values are pipeline values: the sheet id from the legacy
import, sha256 hex, the asset id derived from it, the fixed file name and the constant change note. All of them are quoted
by doubling `'`. The authored document goes into a `$equine_doc$` dollar-quoted literal, and a document text that holds the
tag is refused (`:91`). The text is `format_json` of an object, so it begins with `{` and ends with `}` after `trimEnd`, and
no tag can form at a junction. Same construction as the TypeScript. No action needed. If this is touched later, keep the
tag check on the final text.
### LOW 4 (context): lint allow in a new integration test
`crates/core/tests/curation_draft.rs:7` `#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`.
Every other core test file carries the same allow, and `clippy.toml` documents it ("Tests and the worker binary allow these
lints locally where they own I/O"). It silences no security scanner, so it is not a gated suppression. The binary's
`#![allow(clippy::disallowed_methods)]` (`import_equine.rs:15`) was already present at `5e114da`. The core library itself has
no file system, environment, clock or process call (grep over `crates/core/src` outside `bin/`).
## Owner requirements, checked
- The official seed is never modified automatically. `--check` was run and is read-only. `--out` refuses every parent
inside the seed by canonical path and by identity over the whole seed tree (exception: LOW 1, the destination itself).
- `--out` never deletes (the binary has no remove call) and never overwrites (`create_dir`, `create_new`, the claimed
folder re-checked before each file). It refuses an empty path, `..`, a missing parent, a symbolic link (trailing `/` and
`/.` included), a non-folder and a non-empty folder.
- Strict changeset validation: strict shapes plus the shape comparison; NUL keys refused at parse and in field segments;
`__proto__` segments refused at the op; non-list or non-object children refused at the add.
- Depth and resource caps: parser 128 levels, field path 8 segments, document 64 levels and 200,000 values after each op,
source files under 16 MiB. The seed walk is iterative and follows no link.
- Author data is not dropped silently. An overwritten value is recorded as `before`, and the cases with no faithful reading
(`__proto__`, NUL keys, non-list children) refuse.
- The residual risks declared in §11.13 (record memory, the per-op bound walk, the single-user check-then-use window, lossy
UTF-8 and repeated keys, a share mounted twice on distinct devices) match the code. The exceptions are the two wording
points in LOW 1 and LOW 2.
## Notes (no severity)
- When the seed walk fails, the refusal reads `inside the seed folder` (`import_equine.rs:287-289`). That fails closed, but
the message is misleading. Something like "seed folder unreadable" would point at the real cause.
- The doc comment of `crates/core/tests/curation_draft.rs:520-526` lists `__proto__` among the ordinary keys, then says it
is refused. It is stale wording in a test.
## Not done
- The pinned semgrep packs have no Rust rules: Rust coverage comes from the reading, the checklist and the probes.
- LOW 1 was traced by reading. No mount was made.
+4
View File
@@ -330,3 +330,7 @@ Owner validated R5.1, decided D-R5 = B `sha256-rfc8785-validated-v1` + aliases,
## 2026-10-22/23 (R5.3 curated draft in Rust)
Owner validated R5.1b/R5.2, go R5.3, named `pnpm r5:import-equine`. e0f149b: `crates/core/src/curation/draft` (2,718 lines: schema, targets, fields, records, ops, rename, apply, migration_map, stats, seed, pipeline) + `import-equine` both folders + `scripts/import-equine.sh`; draft 5 texts + frozen horse.svg byte-exact (`--check` 12/0), 825 records, 46 changes, stats, migration stats 792/571/180/7/34, 612 leaves 442/141/29, 58 synthetic TS cases (`oracles/r5/curation-synthetic.json`, 9 byte-exact, 49 refused alike). Security PASS (6 LOW, 3 fixed). Verifier 4 passes: found null-on-optional, repeated keys, NUL touch-mark drop, symlink trailing slash, lossy UTF-8; all fixed or declared (RUST.md §11.12); pass 4 C1-C12 met, C13 doc gaps fixed after, not re-verified. check:all green 00:10:03. documentSha256 unchanged. STOP at R5.3 checkpoint; R5.4 on owner go.
## 2026-10-23 (R5.3 closed after 6 final gate rounds)
Owner validated R5.3 functionally, asked a fresh verifier on the final state, Object.prototype characterisation, migration-map debt kept + characterised, LRN/EVAL. 9901106/8654905: Object.prototype member = plain key, safe, test; debt test from TS-frozen cases; refusal reasons compared by stage; LRN-072, LRN-073, EVAL-028. Then 6 rounds (verifier + security + check:all each) on the final state, each finding real edge families via TS-vs-Rust fuzzing (up to 150,000 changesets): NUL field segment silent drop, stack overflow (8 segments, 64 levels, 200,000 values), non-exclusive --out, serde struct-as-array / enum-as-map (draft/shape.rs; R2 derives raised to owner, gated readers), non-list/non-object children, JSON.parse key order + big ints, __proto__ segments, seed identity guard, reads confined to seed. Sixth round c6e0d85: CONFORME 13/13, security PASS LOW only, check:all green. LOW leftovers carried into R5.4. Next: R5.4 (owner go conditional on these checks: met).
+2 -1
View File
@@ -309,12 +309,13 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
- [x] owner decisions (2026-10-22, BDR-068): D-R5 = B `sha256-rfc8785-validated-v1` (applied at R5.6 only), aliases from trusted frozen documents only, F1 go, byte-exact by default + declared differences, historical SVG bytes kept, `.prettierignore` on Rust-written paths + reproducible check, `ureq` preferred at R5.5; stop after R5.2
- [x] R5.1b F1 fix (aed6b1e): `zmij` digits; V8 oracle `oracles/r5/numbers.json` 44,322 rows (1,552 mismatches before, 0 after); 840.1692504882812 test; `zod_oracle` checks oracle bytes vs core compact print (red on old printer); consumers audited R2-R4: rust:check + wasm + check green, no oracle/golden/digest moved
- [x] R5.2 legacy importer in Rust (feature `curation`, `crates/core/src/curation/**`, 3,187 lines) + `import-equine --check | --out`: legacy-import/ 4 texts + 2 SVG byte-exact, pipeline.json legacy parts equal, raw report = pre-Prettier digest, determinism, 10 synthetic TS cases (2 byte-exact, 8 refused alike), format probes (968 JSON, 10 tables, slugs, uuids, Number, Math.round, key order); declared differences RUST.md §11.10; contract `.claude/tasks/contracts/2026-10-22-rust-r5-f1-importer.md` → checkpoint, STOP before R5.3
- [x] R5.3 changeset applier, migration map, stats, seed SQL (done e0f149b, checkpoint delivered, STOP before R5.4; owner go 2026-10-22 « Validation R5.1b / R5.2 et autorisation R5.3 »: `pnpm r5:import-equine --check | --out <dest>` names the Rust binary, `pnpm import:equine` kept; no fingerprint switch; `horse.svg` frozen bytes read, never re-cleaned; R5.4 files (review SVGs, CHANGELOG, REVIEW-REPORT) out of scope; stop at the checkpoint)
- [x] R5.3 changeset applier, migration map, stats, seed SQL (CLOSED 2026-10-23 on c6e0d85: fresh verifier CONFORME 13/13 after 6 final rounds, security PASS LOW only, check:all green; owner go 2026-10-22 « Validation R5.1b / R5.2 et autorisation R5.3 »: `pnpm r5:import-equine --check | --out <dest>` names the Rust binary, `pnpm import:equine` kept; no fingerprint switch; `horse.svg` frozen bytes read, never re-cleaned; R5.4 files (review SVGs, CHANGELOG, REVIEW-REPORT) out of scope; stop at the checkpoint)
- [x] R5.3a `crates/core/src/curation/draft/{schema,targets,fields,ops,rename,apply}.rs`: changeset + references files parsed strictly (Zod shape order on re-emit, `priority` default, level C question, duplicate ids), target grammar, settable fields, the six ops on the plain JSON document (`serde_json::Value`, insertion order), rename propagation, review rule (touched items → `to_confirm`, identity through a transient marker key), shape + integrity gate through the R2 core (`AnatomicalSheetDocument` + `validate_sheet_document`)
- [x] R5.3b `draft/{migration_map,stats,seed,pipeline}.rs`: replay of rename/remove/move records over the 792 legacy paths, re-homing, stats; `documentStats` (17 fields, JS key order); `renderDraftSeedSql` byte-exact; `build_equine_draft` (CHG-000 from the frozen `horse.svg` bytes and the head view box) + `render_draft_artifacts` (document, assets, sheet-ref with sha256 of the document bytes, migration map, seed SQL)
- [x] R5.3c `import-equine`: both folders (`<dest>/legacy-import`, `<dest>/draft`), `--check` compares the produced files and lists the R5.4 files as pending (not counted), `--out` refuses a non-empty folder, a symlink, a file, an empty path; `scripts/import-equine.sh` + root script `r5:import-equine`
- [x] R5.3d proofs (draft 5 files byte-exact, 12/12 `--check`, 46 changes + 825 records + stats equal, 612 leaves 442/141/29, 58 synthetic cases: 9 byte-exact + 49 refused alike + 5 references files, rust:check green): `tests/curation_draft.rs` (draft files byte-exact vs committed, `pipeline.json` draft section: 46 changes, 825 records, stats, migration stats, assets, issues; determinism; 612-leaf walk through the core's migration on the produced map); TS generator `r5.curation-synthetic-oracles.test.ts` → `oracles/r5/curation-synthetic.json` (mini sheet: every op kind, positions, subtree scopes, review rule B/C, renames with cascades, re-homing, refusals) run by the Rust suite; `pnpm rust:check`, `pnpm check:all`
- [x] R5.3e (security PASS 6 LOW / 3 fixed; verifier 4 passes, last C1-C12 met + C13 docs fixed after; check:all green 00:10:03) gates: security auditor (diff), independent verifier on the contract `.claude/tasks/contracts/2026-10-22-rust-r5-draft.md`; docs RUST.md §11.12-11.13, CHANGELOG, CLAUDE.md commands; memory → checkpoint report, STOP before R5.4
- [x] R5.3g final rounds 1-6 (verifier + security + check:all each): NUL field segments, depth/segment/value bounds, exclusive --out writes, shape family, JSON.parse normalisation, non-list/non-object children, __proto__ segments, seed identity guard, reads confined to the seed; all fixed or declared (RUST.md §11.12-11.13)
- [x] R5.3f owner 2026-10-23: Object.prototype member characterised (safe, accepted, test), migration-map debt kept + characterisation test, refusal reasons compared by stage; final fresh verifier + security on the R5.3 state
- [ ] debt (owner 2026-10-23, after R5): migration map replays cascade records by their op (`applyRecord` / `apply_record`): a view, zone, region, category or cited-reference rename corrupts the map; fix both sides as a declared change before any changeset uses such a rename (RUST.md §11.12)
- [ ] OWNER DECISION (found 2026-10-23 by the R5.3 final verifier): serde derives of the R2 contracts read a struct from an array and a unit enum from a one-key map, where Zod refused; likely reachable by the gated readers of imported PDFs (`archive/read.rs`, `report/read.rs`, `pdf/import.rs`); probe the readers, then a per-file go for a strict deserialisation (RUST.md §11.13)