diff --git a/.claude/audits/2026-10-23-rust-r5-review-security.md b/.claude/audits/2026-10-23-rust-r5-review-security.md new file mode 100644 index 00000000..d1b9c908 --- /dev/null +++ b/.claude/audits/2026-10-23-rust-r5-review-security.md @@ -0,0 +1,231 @@ +``` +SECURITY — VERDICT: PASS +TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten +SCOPE: 25 files +BLOCKING: + (none) +REPORTED (non-blocking): + - [MEDIUM] check/declared-tree-rule-too-wide — crates/core/src/curation/draft/svg_compare.rs:104 + - [LOW] check/unchecked-xml-characters — crates/core/src/curation/draft/app_review.rs:33 + - [LOW] check/markdown-structure-injection — crates/core/src/curation/draft/review_report.rs:64 + - [LOW] check/quadratic-renderers — crates/core/src/curation/draft/review_report.rs:223 + - [LOW] anti-gaming/test-clippy-allow (context) — crates/core/tests/curation_review.rs:6 +PROOF: semgrep 162 rules on 25 files → 0 findings; checklist 8 checks + anti-gaming + 5 brief probes → 5 findings +``` + +# R5.4 security gate (fresh), 2026-10-23 + +Repository `/home/bchanot/Documents/faunosteo`, branch `feature/rust-r5-pipeline-core`, HEAD +`0a79c52`, scope `git diff c6e0d85..HEAD` (25 existing files). Every changed product file was +read whole: `crates/core/src/curation/draft/{labels,changelog,review_report,drawing,overlay, +app_review,svg_compare,pipeline,stats,ops,mod}.rs`, `crates/core/src/bin/import_equine.rs`, +plus `crates/core/src/curation/markdown.rs`, `anatomy/svg/{serialize,tree}.rs` and +`anatomy/compose.rs` where the new code relies on them. + +## Method + +- semgrep 1.168.0 (`EIO_BACKEND=posix`), `--config p/security-audit --config p/secrets + --config p/owasp-top-ten --metrics=off --quiet --json` on the 25 files: 162 rules, 0 + results, 0 errors. These packs cover Rust thinly, so the manual review carries the weight. +- Checklist (CLAUDE.md defaults): secrets, SQL concatenation, unsanitized render, authz, + error exposure, secrets in logs, tracked `.env`, guardrails scanner + (`python3 -B ~/.claude/skills/guardrails/scripts/scan.py --domain api,supabase`: + 0 FAIL, 1 WARN, 1 MANUAL; the WARN `SB-POLICY-NOT-OWNER` is in + `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, outside the delta, so it + is not reported here). +- Anti-gaming: the diff was scanned for new suppressions. +- Probes, all in the scratchpad: a throwaway crate linking `faunosteo-core` (feature + `curation`) at + `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/probe.DaWdIB`, + and `import-equine` built with `CARGO_TARGET_DIR` in the scratchpad. `--check` ran + read-only on the repository. `--out` ran only inside the fresh + `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/out.6SamF5`. + Nothing was deleted, and `git status` stayed clean. + +## Findings + +### 1. [MEDIUM] The tree fallback hides more than the declared trigger-name rule + +`crates/core/src/curation/draft/svg_compare.rs:104-123` (`compare_drawing`), used by +`crates/core/src/bin/import_equine.rs:196-215` for `review/app-*.svg`. + +When the bytes differ, both files are parsed with `parse_svg`. That function builds the tree +of the root element only, and it keeps only elements and text +(`anatomy/svg/tree.rs:155, 182-193`). Everything the tree does not carry is therefore never +compared: + +- the prolog: XML declaration and its `encoding`, processing instructions, comments; +- comments and processing instructions inside the root and after it; +- entity and character-reference forms, CDATA against escaped text, attribute quoting, + `` against ``; +- attribute-value whitespace that the parser normalises. + +Measured on the real committed `draft/review/app-main.svg` against the produced one, with +the rule applied: + +| Committed file changed by | `compare_drawing` | +|---|---| +| nothing | `Ok(TreeAfterRule)` | +| `` and a comment in the prolog, a comment and a PI right after ``, a trailing comment | `Ok(TreeAfterRule)` | +| also `encoding="UTF-8"` replaced by `encoding="ISO-8859-1"` | `Ok(TreeAfterRule)` | + +In all three cases `--check` prints the same `declared: review/app-main.svg (trees equal +under the R3 trigger-name rule)` line it prints today and exits 0. The same holds for +`app-head.svg`, which is byte-identical today: a comment-only change would turn it into a +"declared" file. The only visible sign would be the extra line. + +Impact: the owner requirement "`--check` fails on any unauthorised difference" is not met +for the two application drawings. A stylesheet processing instruction is honoured by +browsers that open a standalone SVG: it could restyle or hide markers in the review drawing +and fetch an external URL, and `--check` would still pass. Exploiting it needs write access +to the repository. The files are dev review artifacts: the app, the bundle and the uploads +never read them (grep of `apps/app`, `vite`, `packages/supabase`). Hence MEDIUM, not +blocking. + +Three declarations are materially wrong: +- `svg_compare.rs:7` says "Any other difference is reported". +- `import_equine.rs:185-186` and the printed "declared" line suggest the rule alone explains + the difference. +- RUST.md §11.15 says the comparison "accepts exactly the trigger-name rule and nothing + else". §11.13's R5.4 condition did mention "declaration, prefixes and whitespace + declared", but §11.15 does not carry that through, and it never covered processing + instructions or comments. + +Fix direction: once the rule has been applied to the committed tree, require the committed +prolog to be byte-equal to the produced one. Then require +`prolog + serialize(committed tree after rule)` to equal the produced text byte for byte. +The core serializer writes the produced file, and `app-head.svg` already matches it byte for +byte. At minimum: compare the prologs and the text after the root, refuse any comment or PI +node in the committed document, and print "declared" only when the rule actually changed +the committed tree. Then correct the three declarations. + +### 2. [LOW] XML-invalid characters from authored strings reach the review drawings unchecked + +`crates/core/src/curation/draft/app_review.rs:33-36` calls `render_layer` directly. The R3 +composition passes the layer through `checked` / `xml_safe` (`anatomy/compose.rs:221-236`, +"refuse one XML cannot carry"), and this path skips that step. The same gap exists in +`overlay.rs:194-198` (path `d`, ids in titles and labels). + +Measured: a changeset that sets `annotation:hock` `name` to `{"fr":"Jarret\u0001&","en":"Hock\u000b"}` +is accepted, and the produced `review/app-main.svg` is not well-formed (roxmltree: "a +non-XML character '\u{1}'"). A path geometry `d` holding U+0001 in an overlay gives the same +result. This is not markup injection: the serializer escaping holds (see "Verified"). It is +a malformed output written by `--out` without any refusal. `--check` then fails, closed. + +Fix direction: run `is_xml_text` over the produced layer and overlay (reuse compose's +`checked`) and refuse. This changes nothing on the committed data. + +### 3. [LOW] Markdown structure injection from authored text + +- `review_report.rs:64-85` (`change_line`: `title.fr`, `rationale`, `question` written raw + in list items); +- `review_report.rs:295-320` (questions); +- `review_report.rs:245-278` (`citation`, `url`, `doi`); +- `changelog.rs:80, 85` (table cells through `markdown.rs:43-49` `cell()`). `cell()` escapes + `|` and `\n`, but not `\r`, and not a backslash in front of a pipe. + +Measured with an added change: +- a rationale `"Ligne un.\n\n## 10. Décisions réservées au professionnel\n\nAucune question ouverte. "` + forges a second section 10 inside section 6 of `REVIEW-REPORT.md` (lines 92 and 343) and + passes raw HTML through; +- a `\r` in a title reaches `CHANGELOG.md` raw. GFM treats a CR as a line ending, so the + table row breaks; +- an authored `\|` becomes `\\|`. `format_markdown` and micromark both read that as an + escaped backslash followed by a real delimiter, so the row gains a ninth column under an + eight-column header. + +Context: the text comes from the repository owner, and the code matches the TypeScript +(`markdown.ts`). The current data holds none of this: no LF, CR, pipe or backtick in any +title, rationale or question. The one `<0697:toih>` in a DOI and its URL renders as text, +because a scheme cannot start with a digit. Gitea strips raw HTML. The code spans hold only +constrained strings: `CHG-\d{3,}`, stable ids, resolved targets and allowlisted fields. +§11.11's "Markdown risks" are about Prettier differences, so this risk is not declared +anywhere. + +Fix direction: refuse CR, LF and control characters in single-line authored fields at +`parse_changeset` and `parse_references` (title, question, citation, doi). Either refuse +them in the rationale too, or render newlines as spaces in list items. Escape `\` in front of +`|` in `cell()`. Optionally escape `<`. + +### 4. [LOW] Quadratic renderers on authored volume + +- `review_report.rs:223-234` `citation_count` rebuilds `all_findings(document)`, one + allocated `FindingEntry` per finding, for every reference: O(R × F). +- `overlay.rs:87-143`: linear `find` over `previous` and `kept.contains`, O(A²) per view. +- `review_report.rs:47-54` `subjects`: changes × records. + +Measured with `render_review_report` on synthetic documents: + +| References | Leaves | Time | +|---|---|---| +| 1,000 | 1,000 | 0.35 s | +| 4,000 | 4,000 | 5.9 s | +| 8,000 | 8,000 | 22.6 s | + +Extrapolated, not measured: about 3 to 4 minutes for a balanced document at the 200,000-value +bound. The equine data (48 references, about 600 findings) renders in milliseconds: +`--check` takes 0.13 s wall time and 18 MB RSS. This is the same class as the declared +"minutes for a 16 MiB changeset" risk (§11.13), but these renderers are not declared. + +Fix direction: count citations in one pass into a `HashMap<&str, usize>`, index `previous` +and `kept` with a `HashMap` / `HashSet`, and group records by change id once. + +### 5. [LOW, context] New clippy allow in a test crate + +`crates/core/tests/curation_review.rs:6` +`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. This is an +integration test, outside the library, with the same header as all 33 files of +`crates/core/tests`. It is a lint allow, not a security-scanner suppression. No new +`nosemgrep`, `nosec`, `eslint-disable`, `ts-ignore` or `noqa` appears in the diff. Not +blocking. + +## Verified, no finding + +- SVG markup injection: element and attribute names are literals. Values go through the + core serializer, which escapes `& < > "` and `\t \n \r` in attributes and `& < > \r` in + text. Probe: an annotation name `J">&`, an English name + `]]>`, a view name `T` and an overlay path + `d="M0 0…\"/>"`. The output is well-formed (apart from finding 2's + control character), with 0 `script` elements and every payload escaped. +- File names built from view ids: views are `stable_id` (`[a-z0-9][a-z0-9_-]{0,63}`), + enforced by the typed parse in `pipeline.rs:206` before any write. `write_exclusive` + also refuses any component that is not `Normal`. The `--check` reads stay canonical + inside the seed. +- Panics on authored input: `compact_value` slices only past 120 UTF-16 units and decodes + lossily. The prolog slice ends at roxmltree's root start (`<`, a char boundary). + `style()[0]` is a static slice. Polygon centroid: at least 3 points (typed), NaN otherwise, + never a panic. No `unwrap`, `expect` or unchecked index on data in the product files. + Recursion in `first_difference` and `TriggerNames::apply` is bounded by the reader's + depth cap (64). +- `--check` is read-only: no file in `legacy-import/` or `draft/` changed and `git status` + stayed clean. Output: `legacy-import: 6 files, 0 differences`, `draft: declared: + review/app-main.svg …`, `draft: 12 files, 0 differences`. A file that is missing, not + produced or not UTF-8 fails. A generated folder must canonicalise to the seed's child. + Files other than `review/app-*` have no rule and must be byte-equal. `horse.svg` is + compared with itself, but its sha256 drives `assets.json`, `document.json` and the seed + SQL, so a change to it fails the check. +- `--out`, probed in the mktemp folder: + - refused, nothing written: a non-empty folder, a symbolic link (with and without a + trailing `/`), a file, `x/../fresh`, a missing parent, an empty path; + - written: a fresh relative path through `INIT_CWD` (18 files), and a fresh folder under a + parent link inside the scratchpad (the parent is resolved as documented); + - refused: a second run on the same destination ("not empty"); + - `diff -r` against the seed: `legacy-import` identical, `draft` identical except + `review/app-main.svg`, the declared trigger names. +- R5.3 carry-overs: a seed walk that fails is now an error, and the claim refuses ("the + seed folder cannot be read"). An existing destination folder is identity-checked + (device, inode) against every folder of the seed. Checked by reading only: a second + mount path needs a bind mount (root), and a symlink to the repository was not used, + because the brief forbids `--out` into the repository. +- `ops.rs` cite: the `HashSet` keeps `before.includes(id)` semantics (string equality only). +- No secret, token, auth URL or private key in the diff (the `.claude` audit texts only cite + the semgrep command). No SQL is built in the diff. stdout and stderr carry paths, counts + and refusals only. The only tracked env file is `.env.example`. + +## Residual risks already declared (RUST.md §11.13, §11.15) + +Checked and accurate, apart from the §11.15 sentence corrected by finding 1: +- the emoji refusal of the alignment; +- the single-user TOCTOU of `--out`, mitigated by `create_new`; +- records memory and the bound-walk time; +- the frozen `horse.svg`. diff --git a/.claude/audits/2026-10-23-rust-r5-review-verify.md b/.claude/audits/2026-10-23-rust-r5-review-verify.md new file mode 100644 index 00000000..b718b00c --- /dev/null +++ b/.claude/audits/2026-10-23-rust-r5-review-verify.md @@ -0,0 +1,13 @@ +# Verification: Rust R5.4 review files (2026-10-23) + +Contract `.claude/tasks/contracts/2026-10-23-rust-r5-review.md`. Fresh verifier, report-only (persisted by the orchestrator). + +## Round 1 on 0a79c52: verifier ECARTS(2), security PASS (1 MEDIUM, 4 LOW) + +- Verifier: C1, C4 to C9 MET (30 tests, `--check` 0/12 + 1 declared, `--out` ×2 identical, anatomy 143, floor clean; 140 targeted + 10,500 random changesets). C2 NOT MET: `format_markdown` reproduces only Prettier's table alignment; authored Markdown syntax or whitespace (`*`, `_`, double spaces, tabs, line breaks in list items, stray backticks, `\|`) is rewritten by Prettier and not by the port (931 accepted / 909 differing in formatted Markdown on a special-character alphabet, 0 on a safe alphabet); a row with an extra cell lost a cell silently. C3 NOT MET: a CR in a label gives an undeclared tree difference (xmldom raw CR vs core ` `). Notes: emoji refusal set wider than emoji; tree fallback ignores prolog/comments/PIs; stale test comment; quadratic renderers. +- Security (`.claude/audits/2026-10-23-rust-r5-review-security.md`): MEDIUM tree fallback too wide (stylesheet PI, comments, encoding change accepted as "declared"); LOW XML-invalid characters reach the drawings unchecked; LOW Markdown structure injection (rationale forging a section, CR, `\|`); LOW quadratic renderers; LOW test allow (context). +- Fixed: Markdown guard (`UnsupportedMarkdown`) built from a TypeScript sweep `oracles/r5/markdown-sweep.json` (220 probes: 187 identical, 33 refused, 0 silent; equine-safe probes never refused; `**strong**` spans kept as the import report needs), single-line check of list-item fields, table cell-count check; strict drawing comparison (committed file must rebuild from its tree, rule must change it, produced equals rebuilt-after-rule byte for byte; attack test); drawings refuse XML-invalid characters and CR; renderers indexed; stale comment; docs §11.14-11.15 (guard rules, declared table, emoji breadth, residual risks). + +## Round 2 + +Fresh verifier and security on the fixed commit: below. diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 2a29a2e4..38d680b1 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -3151,8 +3151,41 @@ existing citations is hashed; the stale test comment fixed; the §11.12 nits (th covers a later removal; the 200,000-value row; the line count at the close; the set-without-value case counted among the schema refusals). -Tests: `cargo test -p faunosteo-core --features curation` (review 6, draft 16, importer 5, -binary 3, formatting 6, plus the crate's suites); anatomy 143 (the generator added). +Markdown beyond the table alignment (found by the first R5.4 gates: the fresh verifier's +differential of 10,500 changesets showed that Prettier also rewrites Markdown in authored text, +the security gate that it could break the reports' structure). The port reproduces only the +table alignment, so it refuses, with the closed kind `UnsupportedMarkdown`, every text Prettier +would rewrite, never writing a report that differs silently. The rules come from a sweep written +by the TypeScript (`oracles/r5/markdown-sweep.json`, generator +`packages/anatomy/tests/r5.markdown-sweep-oracles.test.ts`: 31 characters in six positions, +whitespace patterns and Markdown constructs, each placed in a title, a rationale, a question, a +citation and a changed label; 220 probes): outside code spans, `*` outside a well-formed +`**strong**` span, an underscore not between two letters or digits, a backslash before a space or +at the end or an authored one before the cell escape `\|`, two spaces in a row, a tab, a carriage +return, an odd number of backticks, an empty code span or one with an outer space; a table row +whose cell count differs from its header; a control character (a line break included) in a +title, rationale, question, citation, URL, DOI or vet-flag value written in a list item. Result +(test `every_probe_is_formatted_as_the_typescript_or_refused`): 187 probes byte for byte with the +TypeScript, 33 refused, 0 silent difference; a probe made only of what the equine reports use is +never refused. The equine reports and the R5.2 import report pass the guard unchanged. + +| Input | TypeScript | Rust | +|---|---|---| +| authored text with Markdown Prettier rewrites (`*x*`, `_x`, `a b`, a tab, a line break in a list item, `` `` ``, `a\|b`…) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines; a row with an extra cell) | refused (`UnsupportedMarkdown`) | +| a pictograph in a table cell (an emoji, and ©, ®, ™, ✓, ★, ⚠ and the other characters `text::is_emoji_candidate` lists) | measured with Prettier's tables | refused (`UnsupportedWidth`, R5.2): the set is wider than the emoji, the refusal stays loud | +| a label, id or path holding a code point XML 1.0 refuses, or a carriage return, in a review drawing | written (a non-well-formed file; a raw CR read back as a line feed) | refused: the drawing is not written | + +The drawing comparison hides nothing (security gate, MEDIUM): the rule fallback of `--check` +requires the committed file to rebuild byte for byte from its tree (so no comment, processing +instruction, encoding or entity form can escape the comparison), the rule to change that tree, +and the rebuilt text after the rule to equal the produced one byte for byte; `declared:` is +printed only then (test `the_rule_fallback_refuses_what_the_tree_does_not_carry`: a stylesheet +processing instruction, comments, a trailing comment and another declared encoding refused). +The renderers are indexed (citation counts in one walk, previous markers and kept ids hashed, +subjects grouped per change). + +Tests: `cargo test -p faunosteo-core --features curation` (review 8, Markdown 1, draft 16, +importer 5, binary 3, formatting 6, plus the crate's suites); anatomy 144 (two generators added). ### 11.15 Checkpoint R5.4 @@ -3160,8 +3193,12 @@ Graphic differences for the owner's validation: none in shapes, colours, positio one text difference in `app-main.svg` (the trigger's English name and tooltip), already decided at R3 and identical to the editor canvas. -Residual risks: the Markdown alignment refuses an emoji (declared since R5.2); the comparison of -the application's drawings accepts exactly the trigger-name rule and nothing else; the frozen +Residual risks: the reports refuse what Prettier would rewrite (emphasis, escapes, whitespace +runs, line breaks in list items) and the pictographs the width table does not reproduce: a +future changeset using them stops the pipeline, visibly, until the port grows a Markdown printer +or the text changes; raw HTML and links in authored text are kept as the TypeScript kept them +(GFM renders them; Gitea strips raw HTML); the comparison of the application's drawings accepts +exactly the trigger-name rule, byte for byte; the frozen `horse.svg` is read, never regenerated, so a future change of the drawing needs its own decision; the residual risks of §11.13 stand.