diff --git a/.claude/audits/2026-10-23-rust-r5-review-security-2.md b/.claude/audits/2026-10-23-rust-r5-review-security-2.md new file mode 100644 index 00000000..82f346ef --- /dev/null +++ b/.claude/audits/2026-10-23-rust-r5-review-security-2.md @@ -0,0 +1,195 @@ +``` +SECURITY — VERDICT: PASS +TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten +SCOPE: 32 files +BLOCKING: + (none) +REPORTED (non-blocking): + - [MEDIUM] markdown-guard-bypass-via-ids — crates/core/src/curation/draft/review_report.rs:367 (also markdown.rs:149, ops.rs:286, ops.rs:197, changelog.rs:81) + - [MEDIUM] uncapped-recursive-parse — crates/core/src/curation/draft/svg_compare.rs:108 (called at :129) + - [MEDIUM] silent-prettier-difference-and-wrong-declaration — crates/core/src/curation/markdown.rs:115 (docs/development/RUST.md §11.14, §11.15) + - [LOW] overlay-non-finite-number — crates/core/src/curation/draft/overlay.rs:163 + - [LOW] new-lint-allow-in-tests (context) — crates/core/tests/curation_markdown.rs:6, crates/core/tests/curation_review.rs:6 +PROOF: semgrep 128 rules on 32 files → 0 findings; checklist 9 checks (8 defaults + anti-gaming) → 1 finding (LOW); targeted probes (68 authored changesets through the Rust renderers, 136 report comparisons against Prettier 3.8.3; compare_drawing at depths 60 to 100000; one read-only --check) → 4 findings +``` + +Gate on `git diff c6e0d85..HEAD` (HEAD 68dec25, branch feature/rust-r5-pipeline-core), focus on fix commit 94f8b35. +No HIGH or CRITICAL. Three MEDIUMs: one makes a declared guard leak, one regresses a depth cap +the project documents as an invariant, one shows the RUST.md parity declarations are wrong. None is +reachable from outside the repository: every input is authored in the owner's own seed folder. + +## Method + +- semgrep 1.168.0 (`EIO_BACKEND=posix`, `--metrics=off`, the three pinned packs) on the 32 diff + files that still exist: 128 rules, 0 findings, 0 errors. +- Checklist (CLAUDE.md defaults) over the diff: secrets, SQL concatenation, unsanitized render, + authz, error exposure, secrets in logs, tracked `.env`, guardrails scanner; anti-gaming suppressions. +- Guardrails `scan.py --domain api,supabase`: 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER`, + `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`). That file is not in the diff, so the WARN is filtered out. +- I read whole: `markdown.rs`, `review_report.rs`, `svg_compare.rs`, `drawing.rs`, `overlay.rs`, + `app_review.rs`, `changelog.rs`, `labels.rs`, `stats.rs`, `pipeline.rs`, `bin/import_equine.rs`. + I also read the parts of `ops.rs`, `apply.rs`, `schema.rs`, `targets.rs`, `fields.rs`, + `anatomy/svg/{serialize,tree}.rs` and roxmltree 0.21.1 `tokenizer.rs` that these files rely on. +- Probes, all in the scratchpad (`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/`): + a throwaway crate linking `faunosteo-core` (feature `curation`). It feeds 68 authored changesets + on the mini sheet of `oracles/r5/markdown-sweep.json` through `apply_changeset`, + `render_changelog`, `render_review_report` and `format_markdown`. A Node script then runs the + repository's Prettier 3.8.3 (the options of `packages/anatomy/src/format.ts`) on the same raw + texts and compares (136 comparisons: 118 identical, 4 refused, 14 silent differences). A second + entry point calls `compare_drawing` on nested input. Finally `import-equine --check` ran once + (target dir in the scratchpad, `--locked`): `legacy-import: 6 files, 0 differences`, + `draft: declared: review/app-main.svg`, `draft: 12 files, 0 differences`. The working tree was + clean before and after. `--out` was never run. + +## Findings + +### 1. [MEDIUM] Markdown guard bypassed through ids: forged headings, sections and tables, raw CR in table rows + +Where: `review_report.rs:367` (`assert_list_items_single_line`), `markdown.rs:149` (code parts +checked only for emptiness and outer spaces), `ops.rs:286` (`add` takes any string id), +`ops.rs:197` (`rename` takes any string id), `fields.rs:115` (only the first field segment is +checked), `changelog.rs:81` (`cell()` turns `\n` into a space but leaves `\r`). + +The single-line check covers titles, rationales, questions, citations, URLs, DOIs and vet-flag +values. It misses the other authored texts the review report writes into list items. These are: + +- the record subjects (op target plus field), at lines 97, 167 and 181; +- the change `reference_ids`, at lines 98 and 339. + +Ids are only checked as stable ids at the end of the changeset (`apply.rs:210`, +`curated_issues`). So an item added with an arbitrary id and then removed, or renamed away, leaves +its raw id in the records while the final document stays valid. The same holds for a field path +`name.` set on such an item. A reference added and removed the same way can be cited in +`reference_ids`, because `assert_declared` runs before the ops. + +`guard_line` checks line by line. A newline inside a code span, with one backtick on each side, +therefore yields lines that each pass. Measured, all passing the whole guard (`format_markdown` +Ok on both reports): + +- Add then remove annotation `x`␊`## 11. Section forgée`␊`y` (written with a backtick after `x` + and before `y`). REVIEW-REPORT §2 and §3 then read: + ``` + - `CHG-001` Titre (niveau A, Correction, confirmation requise : non). Raison. Objets : `annotation:x` + ## 11. Section forgée + `y`. Références : aucune. + ``` + The report gets an eleventh section. A `kind: translation` change forges an H1 in §7. A + reference id forges a heading in the `Références :` part. Prettier puts blank lines around the + heading, so the Rust file also differs silently from the TypeScript's. +- An id carrying the lines `| Forgé | Tableau |`, `| --- | --- |`, `| a | b |` produces a table + inside REVIEW-REPORT, which `format_markdown` aligns. Prettier keeps it indented inside the list + item, so this is a silent difference again. +- Id `x\ry`. CR is refused in plain text but not inside a code span. The CHANGELOG row + `| `CHG-001` Titre | `annotation:x\ry` | ...` keeps a raw CR. GFM (goldmark on Gitea) reads it + as a line ending, which breaks the table. Prettier normalizes it to LF, splitting the row in its + own output. The same raw CR shows up in the REVIEW-REPORT list item. + +Impact: the reports, the CHANGELOG and the review document the osteopath signs off on, can carry +structure that never came from the generator. The owner's contract ("refused, never a report that +differs silently") does not hold for this channel. It takes a deliberately crafted changeset in +the repository (add then remove, or a rename chain). No accidental path was found. + +Fix direction: +- At the root: check ids as stable ids when the op runs. That covers the added item's `id` (and + the key field), `rename.id`, and every target id segment and field segment. The final document + requires stable ids anyway, so refusing earlier loses nothing that could be published. +- Defence in depth: `guard_text` refuses any control character inside a code part, and + `assert_list_items_single_line` also checks record subjects and change `reference_ids`. +- Add these probes to the sweep, then declare the refusal in RUST.md. + +### 2. [MEDIUM] Uncapped recursive XML parse of the committed drawing, ahead of the prescan + +Where: `svg_compare.rs:108`. `outside_root` calls `roxmltree::Document::parse` with default +options. Its caller is line 129, before `parse_svg`, which is the function that runs +`prescan(SVG_CAPS)`. This ordering is new in 94f8b35. + +roxmltree's `parse_element` → `parse_content` → `parse_element` recurses once per nesting level. +The workspace manifest itself says the core prescan must cap depth before any parse. Measured with +`compare_drawing(produced, committed, Some(rule))`: depths 60, 70 and 1000 are refused cleanly. +At depth 100000 (a 300 KB committed text, far below the 16 MiB read cap) the process dies with +`thread 'main' has overflowed its stack`, exit 134. In `--check` this happens whenever a +`review/app-*.svg` differs from the produced bytes. + +Impact: an abort, not a pass. It fails closed, writes nothing and exits non-zero. Still, it breaks +the "depth and resource caps" requirement and the project's own prescan-first invariant. +`drawing.rs` orders the same two calls correctly (`read_svg` first). + +Fix direction: call `parse_svg(committed)` before `outside_root`, or run `prescan(text, &SVG_CAPS)` +at the top of `outside_root`. Add a nested-input case to `curation_review.rs`. + +### 3. [MEDIUM] Authored plain text that passes the guard but which Prettier rewrites; RUST.md §11.14 and §11.15 overstate the guarantee + +Where: `markdown.rs:115` (the `_` rule tests `is_alphanumeric` on both neighbours), and the guard +having no rule for link syntax. + +Measured, each passing the guard, then differing from Prettier with no refusal: + +- `中_中`, `ア_ア`, `한_한` in a title, rationale or changed label. Prettier's text splitter makes + each CJK, kana or Hangul character its own word, so the `_` sits on a word boundary. In tables + it becomes `中\_中`. In a review list item that holds two such texts, Prettier reads + `_中 … 中_` as emphasis and prints `中*中 … 中*中`. Rust writes the raw text. +- `[a]( https://x.org )` in a changed label: Prettier prints `[a](https://x.org)`, Rust keeps the + spaces. + +These do not break the Rust output's structure. They break parity: TypeScript-era files and Rust +files differ without a refusal. Two declarations are therefore wrong as written: +- §11.14 says the port "refuses ... every text Prettier would rewrite, never writing a report that + differs silently". That holds for the 220 sweep probes only. +- §11.15 says "links in authored text are kept as the TypeScript kept them". The TypeScript output + was Prettier's, which normalizes this link form. + +Fix direction: +- Refuse `_` when either neighbour is a character Prettier's splitter isolates (CJK, kana, + Hangul), or more simply whenever a neighbour is not an ASCII or Latin letter or digit. +- Refuse a link destination with surrounding whitespace, or refuse `](` outright, since the equine + reports use none. +- Add these probes to the sweep and correct both declarations. + +### 4. [LOW] Overlay prints an overflowed coordinate as `null` + +Where: `overlay.rs:163`. `text_of` special-cases NaN only. `js_number` prints a non-finite value +as `null`, where `String(Infinity)` printed `Infinity`. It is reachable with finite but extreme +document values (a view box `min_y` near `f64::MAX`, or polygon points around `1e308`, whose sum +overflows). The XML stays well-formed and nothing is injected. It is a silent parity gap only. +Fix: print `Infinity` / `-Infinity` in `text_of`. + +### 5. [LOW] New lint allows in the two new test files (context, anti-gaming) + +Where: `curation_markdown.rs:6` and `curation_review.rs:6`, both +`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. This is the +pattern `clippy.toml` documents for tests that do their own I/O; 34 existing test files carry it. +It is not a security suppression. There is no `nosemgrep`, `nosec`, `eslint-disable`, +`ts-ignore` or new allow in library code. + +## Checked and clean + +- Strict SVG comparison logic (apart from finding 2). The fallback accepts only when + `produced == prolog + serialize(rule(parse(committed))) + epilog` and the committed file + rebuilds byte for byte from its own tree. So the produced file can differ from the committed one + only where the rule changed the tree. The rule's values come from the produced document's view + names, and `document.json` is compared byte for byte. A changed French name makes the rule + change nothing, which is refused. A DTD is refused by roxmltree's default options. Comments, + PIs, entity forms and other encodings fail the rebuild check. I found no way to get an + unauthorized difference accepted. +- Markup injection in the drawings: the serializer escapes `& < > "`, plus `\t \n \r` in + attributes and `& < > \r` in text. Element and attribute names are fixed by code. + `drawing_text` refuses non-XML characters and CR before writing. +- Table cells: `|` is always escaped. An authored `\|`, a trailing backslash and a backslash before + a space are refused. Pipes and a trailing backslash inside code spans format exactly like + Prettier. The per-cell backtick check and the cell-count check in `align` hold. +- Control characters (NUL, FF, VT, NEL, U+2028), NBSP, BOM, U+3000, zero-width space, `#`, `-`, + `+`, `1.`, `>`, `---`, `~x~`, `~~x~~`, backslash escapes before punctuation, entities, autolinks, + images, footnote syntax and raw HTML at the start of a cell: all identical to Prettier. +- Panics on authored input: none found in the new library code. There is no `unwrap`, `expect` or + unchecked index, and the slices fall on ASCII or roxmltree boundaries. Tree recursion + (`first_difference`, `TriggerNames::apply`, `serialize`) runs on trees from `parse_svg`, which + is depth-capped at 64. `first_unwritable` uses an explicit stack. +- Resources: the guard and the alignment are linear per line. Renderers are indexed (subjects + grouped, citation counts in one walk, hashed previous markers). Record memory is a declared + residual (§11.13). +- `--check`: read-only, as declared. Reads go through the canonical-path, regular-file and + 16 MiB checks. A file that is missing, not produced or different fails. `declared:` is printed + only after the strict match. `--out` claim logic: the only change is that an existing + destination is now checked against the seed by identity. Sound. +- Secrets, SQL built from strings, authz, error exposure, logging of secrets or PII, tracked + `.env`: nothing in the diff (only `.env.example` is tracked). diff --git a/.claude/audits/2026-10-23-rust-r5-review-verify.md b/.claude/audits/2026-10-23-rust-r5-review-verify.md index b718b00c..99f2bc28 100644 --- a/.claude/audits/2026-10-23-rust-r5-review-verify.md +++ b/.claude/audits/2026-10-23-rust-r5-review-verify.md @@ -11,3 +11,13 @@ Contract `.claude/tasks/contracts/2026-10-23-rust-r5-review.md`. Fresh verifier, ## Round 2 Fresh verifier and security on the fixed commit: below. + +## Round 2 on 68dec25: verifier ECARTS(2), security PASS (3 MEDIUM, 2 LOW) + +- Verifier: C1, C3 to C6, C8, C9 MET. C2: silent Prettier rewrites the guard missed (inline link destinations and titles, autolinks with an entity, `[[ a ]]`, character references decoding to `*`/`_`), shown end to end on the equine pipeline with one appended change; §11.15 sentence on links wrong. C7: `markdown-sweep.json` not excepted by the contract. Differential: 2,430 targeted + 4,500 random cases, 0 raw-text or drawing differences. +- Security (`.claude/audits/2026-10-23-rust-r5-review-security-2.md`): MEDIUM ids with line breaks (added then removed, renamed, nested, field segments) forging report sections through code spans, raw CR in code spans; MEDIUM uncapped roxmltree parse of the committed drawing before the prescan (stack overflow at depth 100,000); MEDIUM `中_中` / `[a]( x )` silent and §11.14-11.15 overstated; LOW Infinity printed `null`; LOW test allows (context). +- Fixed: construct rules (`](`, `][`, padded `[[ ]]`, autolinks, character references) from new sweep probes; underscore rule ASCII-only (the sweep showed `é_é` rewritten too); CR/TAB refused anywhere in a line; ids validated at the op (add, nested, rename) and field segments without control characters or backticks; core reader before roxmltree in the comparison; `Infinity`; contract clarification for the second oracle; docs (counts 247/187/60, declared rows, §11.15). Tests: forged ids, field segments, 100,000-level drawing. + +## Round 3 + +Fresh verifier and security on the fixed commit: below. diff --git a/.claude/tasks/contracts/2026-10-23-rust-r5-review.md b/.claude/tasks/contracts/2026-10-23-rust-r5-review.md index 4f68d464..296913e2 100644 --- a/.claude/tasks/contracts/2026-10-23-rust-r5-review.md +++ b/.claude/tasks/contracts/2026-10-23-rust-r5-review.md @@ -50,3 +50,11 @@ identity-checked, cite membership hashed, a failed seed walk named as such, the C9. **Gates.** `pnpm rust:check` and `pnpm check:all` green; security auditor PASS on the diff; this contract CONFORME by a fresh verifier; RUST.md §11.14 (R5.4) and §11.15 (checkpoint), CHANGELOG, CLAUDE.md. No merge, no R5.5, no fingerprint switch, no release, no deploy. + +## Clarifications + +- 2026-10-23 (gate round 2, C7): `crates/core/tests/oracles/r5/markdown-sweep.json` is a second new oracle of this slice, + written by the TypeScript generator `packages/anatomy/tests/r5.markdown-sweep-oracles.test.ts` (`MAKE_R5_ORACLES=1`). It + freezes what Prettier does to authored Markdown, from which the port's `UnsupportedMarkdown` guard is derived (C2 read as + "byte for byte with the TypeScript, or a loud refusal declared in RUST.md §11.14"). It is added, never a frozen file + changed; C7 excepts it with `review-synthetic.json`. diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 38d680b1..a02ce06c 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -3116,7 +3116,7 @@ LOWs only, §11.12). Contract `.claude/tasks/contracts/2026-10-23-rust-r5-review | `pipeline.rs` | `render_draft_artifacts` complete, in the TypeScript order: document, assets, sheet-ref, migration map, the overlays and the application's drawings per view, CHANGELOG, REVIEW-REPORT, seed SQL | | `bin/import_equine.rs` | `--check` compares the 6 + 12 files with explicit criteria (byte identity; the application's drawings may fall back on parsed trees under the declared rule, printed as `declared:`); a file missing, not produced or different fails; the pending list and its `shortcut:` are gone; `--out` writes the 18 files | -Parity, every expectation written by the TypeScript (`tests/curation_review.rs`, 6 tests): +Parity, every expectation written by the TypeScript (`tests/curation_review.rs`, 8 tests): - `CHANGELOG.md` (1,114,533 bytes) and `REVIEW-REPORT.md` of the committed draft byte for byte; their raw texts equal the ledger's pre-Prettier digests, the aligned texts its digests; @@ -3141,8 +3141,8 @@ Parity, every expectation written by the TypeScript (`tests/curation_review.rs`, without the rule. `pnpm r5:import-equine --check`: `legacy-import: 6 files, 0 differences`; `draft: declared: -review/app-main.svg (trees equal under the R3 trigger-name rule)`; `draft: 12 files, 0 -differences`. `--out` then `diff -r`: legacy identical; draft identical but `app-main.svg`. +review/app-main.svg (byte-identical once the R3 trigger-name rule is applied)`; `draft: 12 +files, 0 differences`. `--out` then `diff -r`: legacy identical; draft identical but `app-main.svg`. R5.3 carry-over (final gate LOWs, not applied to the verified state): an existing destination folder is itself identity-checked against the seed's folders; a failed seed walk reads `the seed @@ -3159,20 +3159,27 @@ would rewrite, never writing a report that differs silently. The rules come from by the TypeScript (`oracles/r5/markdown-sweep.json`, generator `packages/anatomy/tests/r5.markdown-sweep-oracles.test.ts`: 31 characters in six positions, whitespace patterns and Markdown constructs, each placed in a title, a rationale, a question, a -citation and a changed label; 220 probes): outside code spans, `*` outside a well-formed -`**strong**` span, an underscore not between two letters or digits, a backslash before a space or +citation and a changed label; 247 probes): a carriage return or a tab anywhere, code spans +included; outside code spans, an inline link, image or +reference (`](`, `][`), a `[[ … ]]` with inner spaces, an autolink (``, `<…@…>`), a +character reference (`&name;`, `&#n;`, `&#xh;`; the last three refused whole although Prettier +keeps some of them), `*` outside a well-formed +`**strong**` span, an underscore not between two ASCII letters or digits (Prettier escapes one +next to an accented letter or a CJK, kana or Hangul character), a backslash before a space or at the end or an authored one before the cell escape `\|`, two spaces in a row, a tab, a carriage return, an odd number of backticks, an empty code span or one with an outer space; a table row whose cell count differs from its header; a control character (a line break included) in a title, rationale, question, citation, URL, DOI or vet-flag value written in a list item. Result (test `every_probe_is_formatted_as_the_typescript_or_refused`): 187 probes byte for byte with the -TypeScript, 33 refused, 0 silent difference; a probe made only of what the equine reports use is -never refused. The equine reports and the R5.2 import report pass the guard unchanged. +TypeScript, 60 refused, 0 silent difference; a probe made only of what the equine reports use, +outside the constructs, is never refused (a polygon's `[[10,10],…]` and the DOI's +`<0697:toih>` pass). The equine reports and the R5.2 import report pass the guard unchanged. | Input | TypeScript | Rust | |---|---|---| -| authored text with Markdown Prettier rewrites (`*x*`, `_x`, `a b`, a tab, a line break in a list item, `` `` ``, `a\|b`…) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines; a row with an extra cell) | refused (`UnsupportedMarkdown`) | +| authored text with Markdown Prettier rewrites (`*x*`, `_x`, `a b`, a tab, a line break in a list item, `` `` ``, `a\|b`, `[a]()`, `[a](b "t")`, `[[ a ]]`, `*`…) or one of the constructs refused whole (links, autolinks, character references) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines; a row with an extra cell) | refused (`UnsupportedMarkdown`) | | a pictograph in a table cell (an emoji, and ©, ®, ™, ✓, ★, ⚠ and the other characters `text::is_emoji_candidate` lists) | measured with Prettier's tables | refused (`UnsupportedWidth`, R5.2): the set is wider than the emoji, the refusal stays loud | +| an id an `add` writes (the item's, and those nested in its `children` or `categories`) or a `rename` id that is not a stable id (a region code for a region), or a field segment holding a control character or a backtick | accepted when a later op removed or renamed the item: the raw id stayed in the records and reached the reports' code spans (a line break there forged a section, security gate R5.4 round 2) | refused at the op, the check the final document applies anyway (test `ids_and_field_segments_that_could_forge_report_structure_are_refused`) | | a label, id or path holding a code point XML 1.0 refuses, or a carriage return, in a review drawing | written (a non-well-formed file; a raw CR read back as a line feed) | refused: the drawing is not written | The drawing comparison hides nothing (security gate, MEDIUM): the rule fallback of `--check` @@ -3181,8 +3188,10 @@ instruction, encoding or entity form can escape the comparison), the rule to cha and the rebuilt text after the rule to equal the produced one byte for byte; `declared:` is printed only then (test `the_rule_fallback_refuses_what_the_tree_does_not_carry`: a stylesheet processing instruction, comments, a trailing comment and another declared encoding refused). -The renderers are indexed (citation counts in one walk, previous markers and kept ids hashed, -subjects grouped per change). +The committed drawing is read by the core's reader before any other parse, so its depth cap +holds (a 100,000-level file is refused, never a stack overflow). The overlay prints a coordinate +that overflowed as `Infinity`, as `String(number)` did. The renderers are indexed (citation +counts in one walk, previous markers and kept ids hashed, subjects grouped per change). Tests: `cargo test -p faunosteo-core --features curation` (review 8, Markdown 1, draft 16, importer 5, binary 3, formatting 6, plus the crate's suites); anatomy 144 (two generators added). @@ -3196,8 +3205,9 @@ at R3 and identical to the editor canvas. Residual risks: the reports refuse what Prettier would rewrite (emphasis, escapes, whitespace runs, line breaks in list items) and the pictographs the width table does not reproduce: a future changeset using them stops the pipeline, visibly, until the port grows a Markdown printer -or the text changes; raw HTML and links in authored text are kept as the TypeScript kept them -(GFM renders them; Gitea strips raw HTML); the comparison of the application's drawings accepts +or the text changes; inline links, autolinks and character references are refused as well, even +where Prettier would keep them; raw HTML in authored text (`…`) is kept as the TypeScript +kept it, byte for byte (GFM renders it; Gitea strips it); the comparison of the application's drawings accepts exactly the trigger-name rule, byte for byte; the frozen `horse.svg` is read, never regenerated, so a future change of the drawing needs its own decision; the residual risks of §11.13 stand.