From aff3a96d55c2310c659002e9ec233571cd752606 Mon Sep 17 00:00:00 2001 From: bchanot Date: Fri, 23 Oct 2026 13:02:22 +0200 Subject: [PATCH] docs(docs): R5.4 fourth gate round declared, guarantee restated exactly, audits --- .../2026-10-23-rust-r5-review-security-4.md | 166 ++++++++++++++++++ .../2026-10-23-rust-r5-review-verify.md | 5 + docs/development/RUST.md | 42 +++-- 3 files changed, 196 insertions(+), 17 deletions(-) create mode 100644 .claude/audits/2026-10-23-rust-r5-review-security-4.md diff --git a/.claude/audits/2026-10-23-rust-r5-review-security-4.md b/.claude/audits/2026-10-23-rust-r5-review-security-4.md new file mode 100644 index 00000000..30f320c4 --- /dev/null +++ b/.claude/audits/2026-10-23-rust-r5-review-security-4.md @@ -0,0 +1,166 @@ +# Security gate R5.4, round 4 (fresh): feature/rust-r5-pipeline-core @ 6d4fdd6 + +Scope: `git diff c6e0d85..HEAD` (36 existing files), focus on 124d954 (`curation/report_guard.rs`, +`curation/markdown.rs`, `draft/{pipeline,fields,ops}.rs`). Product files read whole: report_guard.rs, +markdown.rs, changelog.rs, review_report.rs, overlay.rs, drawing.rs, app_review.rs, svg_compare.rs, +labels.rs, stats.rs, bin/import_equine.rs, plus the R5.4 hunks of fields.rs, ops.rs, pipeline.rs, mod.rs. + +``` +SECURITY — VERDICT: BLOCK(3) +TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten +SCOPE: 36 files +BLOCKING: + 1. [HIGH] markdown-guard/character-reference — crates/core/src/curation/report_guard.rs:43 — a character reference without its `;`, or with a name over 32 characters, passes the guard and Prettier rewrites it: `a * b` → `a \* b`, `*` → `\*`, `_` → `\_`, `a_b` → `a_b`, `�…042;` (35-character name) → `\*`; a non-terminated reference followed later by a bare `&` before `,` or `"` makes Prettier overwrite that `&`: `< x &` → `< x <`, `A x &` → `A x A` — hint: outside code spans refuse `&#` whatever follows, and `&` followed by a legacy (non-terminated) entity name with or without `;`; keep `&` before other text admitted (the equine REVIEW-REPORT holds `&pid=S1019` in a URL) + 2. [HIGH] markdown-guard/image-reference — crates/core/src/curation/report_guard.rs:33 — `![…]` is not among the refused constructs; Prettier parses it as an image reference and prints the alt from its decoded, unescaped value: `![a\"b]` → `![a"b]`, `![a\|b]` → `![a|b]`, `![x|y]` in a CHANGELOG cell, `![&]` → `![&]`, `![<]` → `![<]`, `![*]` → `![*]`; the `![` and its `]` can come from two insertions on one list line (title, rationale, question) — hint: refuse `![` outside code spans (the equine reports hold none) + 3. [HIGH] markdown-guard/wiki-link — crates/core/src/curation/report_guard.rs:34 — the padded-wiki check splits on `[[`, so the text between a `[[` and the first `]]` is never examined when another `[[` sits in between; Prettier's tokenizer (`/^\[\[(?.+?)\]\]/su`, contents trimmed) matches across it: `[[ a [[b]]` → `[[a [[b]]`, also `[[ é [[中]]`, `[[ 1 [[2]]`, `[[a]] [[ b [[c]]`, `[[ x [ [[y]]` — hint: for every `[[` occurrence (overlapping `[[[` included) take the text up to the first following `]]` and refuse it when it starts or ends with whitespace, or refuse `[[` followed by whitespace and whitespace followed by `]]` +REPORTED (non-blocking): + - [MEDIUM] declaration-wrong (guarantee overstated) — docs/development/RUST.md:3164, 3169, 3184, 3214 + - [LOW] resource/quadratic-underscore-check — crates/core/src/curation/report_guard.rs:88 + - [LOW] lint-allow in test files (convention, not a scanner suppression) — crates/core/tests/curation_review.rs:6, crates/core/tests/curation_markdown.rs:6 +PROOF: semgrep 128 rules on 36 files → 0 findings; checklist 13 checks → 6 findings +``` + +## Severity basis + +The three blocking findings break the owner requirement this guard exists for: a report is byte for +byte what the TypeScript+Prettier pipeline wrote, or it is refused loudly. Each one is a silent +difference reachable through the normal authoring channel (a title, rationale or question in +`changes.json`, a citation in `references.json`, a label set by an op), with no refusal and a clean +exit. There is no confidentiality or availability impact and no actor beyond the author of the +curation files; the classification rests on the explicit integrity requirement and on the fact that +RUST.md states the opposite (finding M1). In two families the TypeScript output is the one that alters +the author's text (`< x &` became `< x <`), so the fix is a refusal, never an imitation of +Prettier. + +## Method and evidence (differential, all in the scratchpad) + +- TypeScript side: `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/ts/sweep.mts`, + the sweep generator's exact `changesFile(probe)` on the mini sheet, run through `packages/anatomy/src` + (`applyChangeset`, `renderChangelog`, `renderReviewReport`) and `formatMarkdown` (Prettier 3.8.3, + options of `packages/anatomy/src/format.ts`). +- Rust side: `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/rs/src/main.rs`, + a throwaway binary linking `faunosteo-core` (`curation`), same changeset through `apply_changeset`, + `render_changelog`, `render_review_report`, `format_report`. Raw texts matched the TypeScript in every + case (no RAWDIFF), so every difference is the guard admitting what Prettier rewrites. +- Batches: 79 hand-picked probes (20 silent), 2,500 seeded random strings over the admitted alphabet + plus `&#`, `&`, `<`, `![`, `[[` atoms (151 silent, every one holding `&#`, a legacy entity or + `![`), 3,000 seeded random strings without `&`, `![` or `[[` (0 silent), 55 targeted edges (7 silent), + 13 entity variants (2 silent). Total 5,647 probes, 180 silent differences, all in the three families. + Probe lists and results: `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/out/`. +- Prettier internals confirmed in `prettier@3.8.3/plugins/markdown.js`: wiki tokenizer + `/^\[\[(?.+?)\]\]/su` with `.trim()`, placed before `link`; the unescape restore keeps the + source of a one-character text node except `*` and `_` (`Tf=/^\\?.$/su`), which is why `*` and + `_` come out as `\*` and `\_`; `imageReference` prints `node.alt`, the decoded value. + +Sample (review list item, probe in the title): + +``` +rust: - `CHG-001` Titre a * b (niveau C, … +ts: - `CHG-001` Titre a \* b (niveau C, … +rust: - `CHG-001` Titre ![a\"b] (niveau C, … +ts: - `CHG-001` Titre ![a"b] (niveau C, … +rust: - `CHG-001` Titre [[ a [[b]] (niveau C, … +ts: - `CHG-001` Titre [[a [[b]] (niveau C, … +rust: … `probe-ref` Citation < x &, cité par 1 nœud(s). +ts: … `probe-ref` Citation < x <, cité par 1 nœud(s). +``` + +In the CHANGELOG the same probes also shift the table alignment (Prettier pads on the rewritten text). + +## Findings in detail + +### H1 [HIGH] character references not terminated by `;` (report_guard.rs:43-53) + +`reference` requires `after.contains(';')` and `name.len() <= 32`. The parser Prettier uses decodes +non-terminated numeric references (`*`, `*`, `*`) and the legacy named ones (`<`, `&`, +`©`…), and numeric references take any number of leading zeros, so the 32 cap lets +`*` through. A reference decoding to `*` or `_` is escaped +(`\*`, `\_`) or merged (`a_b` → `a_b`); a non-terminated reference followed later by a bare `&` (seen +before `,` and `"`, i.e. at the end of a citation or a JSON value) gets that `&` replaced by the decoded +character. Fix direction: refuse `&#` outside code spans unconditionally, and `&` followed by a legacy +entity name (the non-terminated set) regardless of `;`; keep the existing `&name;` rule; keep `&` before +anything else admitted, since the equine REVIEW-REPORT carries `&pid=S1019` inside a URL. Add the +reproducers above to `r5.markdown-sweep-oracles.test.ts`. + +### H2 [HIGH] image references (report_guard.rs:33-55) + +`rewritten_construct` refuses `](` and `][` but not a shortcut image reference `![…]`. Its alt is printed +from the decoded, unescaped value, so any `\"`, `\|` (including the `\|` that `cell()` adds in the +CHANGELOG) or reference inside it is rewritten. `![a]` with plain text stays identical, which is why the +sweep's `![img](x.png)` (refused through `](`) never showed it. Fix direction: refuse `![` outside code +spans. + +### H3 [HIGH] wiki links whose contents hold a `[[` (report_guard.rs:34-41) + +`plain.split("[[")` yields the segments between occurrences, not the text the regex matches: for +`[[ a [[b]]` the first segment ` a ` holds no `]]` and is skipped, the second `b]]` is unpadded, so the +line is admitted while Prettier matches ` a [[b` and trims it. Fix direction: emulate the lazy match +from each `[[` position (a `]]` at offset 1 or more), or refuse `[[` followed by whitespace and +whitespace followed by `]]`. + +### M1 [MEDIUM] declarations overstated (docs/development/RUST.md) + +- Line 3164: "a `[[ … ]]` padded inside" is refused: false for H3. +- Line 3169: "so an unknown form is refused by default": false; the allowlist is per character, and + constructs built from admitted characters (`&`, `#`, `!`, `[`, `]`) are admitted. +- Line 3184: "the constructs refused whole (links, HTML tags, autolinks, character references)": false for + non-terminated or long references (H1), and image references are not refused at all (H2). +- Line 3214: the residual risk says an uncovered combination "could still differ" and that the last rounds + "found none": three families with 180 reproducers were found in one round. The declaration must be + rewritten with the fixes, and the sweep extended with these families. + +### L1 [LOW] quadratic underscore check (report_guard.rs:88) + +For each `_`, `chars[i + 1..].iter().collect::()` copies the rest of the text to test the URL +starts: O(n²) per authored text. Measured with the probe binary: 10 KB of `a_` 14 ms, 40 KB 221 ms, 80 KB +0.91 s (x4 per doubling), so about 2.5 min per occurrence at 1 MB and hours near the 16 MiB file cap; a +title or rationale is checked in up to four places. Owner-authored input of a dev tool, hence LOW. Fix +direction: compare the slice in place (`chars[i + 1..].starts_with(&['h','t','t','p',…])` or byte offsets +on the `&str`). + +### L2 [LOW] lint allows in the new test files (context) + +`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]` heads +`curation_review.rs` and `curation_markdown.rs`, the same header as the pre-existing `curation_draft.rs`: +test-only file access and unwraps, not a security-scanner suppression. No `nosemgrep`, `nosec`, +`eslint-disable`, `ts-ignore` or new allow in library code. + +## Checklist (13 checks) + +| # | Check | Result | +|---|---|---| +| 1 | Hardcoded secret, token, key, auth URL (added lines, incl. docs and oracle JSON) | none | +| 2 | SQL built by concatenation | none in the diff (the seed SQL writer is R5.3, unchanged) | +| 3 | Unsanitized render (innerHTML, eval, raw) | none; SVG text goes through the R3 serializer (`& < > "` and tab/newline escaped in attributes) after `first_unwritable` (XML 1.0 code points, CR refused) | +| 4 | Sensitive endpoint without authz | not applicable (offline dev tool, no endpoint) | +| 5 | Stack trace or internal path to a user | not applicable (CLI stderr for the developer) | +| 6 | Secret or PII in a log | none (stdout and stderr carry file names, counts, refusals) | +| 7 | Tracked `.env` or credential file | only `.env.example`; nothing in the diff | +| 8 | Guardrails scanner (`--domain api,supabase`) | 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER`, `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`) outside the diff, filtered | +| 9 | New suppressions (anti-gaming) | none in library code; L2 | +| 10 | Markdown guard differential (5,647 probes) | H1, H2, H3, M1 | +| 11 | Strict SVG comparison | holds (below) | +| 12 | Resource use of the new code | L1; renderers indexed, document bound unchanged | +| 13 | `--check` / `--out` behaviour | holds (below) | + +## What holds + +- Channels: both draft reports pass every raw line through `format_report` (pipeline.rs:273-274); no + authored text reaches a heading; list-item fields with a control character are refused + (`assert_list_items_single_line`); ids that reach code spans are stable ids or region codes, checked at + the op (ops.rs `assert_stable_id`), field segments refuse controls (NUL included) and backticks; a line + with an odd backtick count is refused, so no code span can hide a line break. The R5.2 IMPORT-REPORT + stays on `format_markdown` with legacy sources only. View ids in file names are stable ids, and + `write_exclusive` refuses any non-normal path component. +- Strict SVG comparison (`svg_compare.rs`), 11 crafted cases against the committed `app-main.svg`: the + legitimate trigger-name difference is the only acceptance; an extra comment, trailing text, a prolog + change, another English name, a renamed attribute, a DOCTYPE, an empty or 100,000-level produced file, + a 100,000-level committed file and a stylesheet PI in the committed file are all refused, none panics. +- `--check` (binary built in the scratchpad, read-only): `legacy-import: 6 files, 0 differences`, + `draft: declared: review/app-main.svg`, `draft: 12 files, 0 differences`, exit 0. +- `--out` into a fresh `mktemp -d` under the scratchpad: 18 files written; a second run refused + `not empty, nothing written` (exit 2); legacy identical to the seed, draft identical except + `app-main.svg`. Working tree clean afterwards. +- Panics: none found in the new code (slices after an ASCII `**` match, `chars[i + 1..]` with `i < len`, + `compact_value` cut bounded by the length test, overlay NaN paths printed as `String(number)`). +- semgrep: 128 rules (p/security-audit, p/secrets, p/owasp-top-ten) on 36 files, 0 findings, 0 errors. diff --git a/.claude/audits/2026-10-23-rust-r5-review-verify.md b/.claude/audits/2026-10-23-rust-r5-review-verify.md index 24002324..372eadeb 100644 --- a/.claude/audits/2026-10-23-rust-r5-review-verify.md +++ b/.claude/audits/2026-10-23-rust-r5-review-verify.md @@ -31,3 +31,8 @@ Fresh verifier and security on the fixed commit: below. ## Round 4 Fresh verifier and security on the fixed commit: below. + +## Round 4 on 6d4fdd6: security BLOCK(3) (verifier pending at fix time) + +- Security (`.claude/audits/2026-10-23-rust-r5-review-security-4.md`): HIGH non-terminated / long character references (`* b` → `\*`, `< x &` → `< x <`), HIGH image references `![…]` (alt reprinted decoded), HIGH nested padded wiki links (`[[ a [[b]]`); MEDIUM §11.14/§11.15 overstated ("unknown form refused by default", "refused whole", "found none"); LOW quadratic underscore check. Control: 3,000 random strings without `&`, `![`, `[[`: 0 silent. +- Fixed: `&` admitted only before a space or as a URL query separator `&name=` (name not a legacy-reference prefix, the 106 HTML legacy names); `![` refused; padded `[[`/`]]` refused when the matching half exists; URL checks without allocation; sweep 782 probes (new families, random alphabet with `&#`, `&`, `<`, `![`, `[[`, `]]`): 279 identical, 503 refused, 0 silent; docs restated exactly. diff --git a/docs/development/RUST.md b/docs/development/RUST.md index ef3d2df6..08785c72 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -3157,24 +3157,30 @@ also rewrites Markdown in authored text (escapes, emphasis, whitespace, links); reproduces only the table alignment. After a denylist that leaked round after round, the guard of the draft's two reports (`curation/report_guard.rs`, `format_report`; the R5.2 import report keeps `format_markdown`, verified at R5.2) is an allowlist: outside code spans, letters and digits, -single spaces (none at a line end), the punctuation `.,;:!?'"«»()[]{}/-–—…%°≠=+#&<>’‘“”±→×≤≥|`, an -underscore between two ASCII letters or digits and not glued to a URL (`http://`, `https://`, -`www.`), a backslash before `"` or `|` (a JSON quote, the cell escape), `**word**` with one run of -letters between spaces or line edges (the import report's form), and no inline construct (an -inline link, image or reference `](`, `][`; a `[[ … ]]` padded inside; `<` before a letter, `/`, -`!` or `?`: an HTML tag or an autolink; a character reference `&name;`, `&#n;`); inside code spans -(generated ids and targets), anything but a carriage return or a tab, never empty, no outer space; -a table row with another cell count than its header and a control character in a list-item field -(title, rationale, question, citation, URL, DOI, vet-flag value) are refused too. Everything else -is refused with `UnsupportedMarkdown`, so an unknown form is refused by default. +single spaces (none at a line end), the punctuation `.,;:!?'"«»()[]{}/-–—…%°≠=+#<>’‘“”±→×≤≥|`, an +`&` before a space or as a URL's query separator (`&name=` in a bare URL, the name not starting +with one of HTML's 106 legacy references, which Prettier decodes without a `;`), an underscore +between two ASCII letters or digits and not glued to a URL (`http://`, `https://`, `www.`), a +backslash before `"` or `|` (a JSON quote, the cell escape), `**word**` with one run of letters +between spaces or line edges (the import report's form), and no inline construct (an inline link, +image or reference `](`, `][`; an image reference `![`; a `[[` before a space with a `]]` after +it, or a `]]` after a space with a `[[` before it; `<` before a letter, `/`, `!` or `?`: an HTML +tag or an autolink; a character reference `&name;`, `&#n;`); inside code spans (generated ids and +targets), anything but a carriage return or a tab, never empty, no outer space; a table row with +another cell count than its header and a control character in a list-item field (title, +rationale, question, citation, URL, DOI, vet-flag value) are refused too. Everything else is +refused with `UnsupportedMarkdown`: a character outside the set is refused by default; the forms +built from admitted characters are ruled one by one, so a form no rule names and no probe covers +could still pass. The guarantee is empirical, not a proof of Prettier's behaviour: the TypeScript sweep (`oracles/r5/markdown-sweep.json`, generator `packages/anatomy/tests/r5.markdown-sweep-oracles.test.ts`, -digests of the raw and formatted reports) holds 762 probes: 48 characters in six positions, -whitespace patterns, Markdown constructs, the families each gate round found, and 400 strings from a -seeded generator, each placed in a title, a rationale, a question, a citation, a label after a +digests of the raw and formatted reports) holds 782 probes: 48 characters in six positions, +whitespace patterns, Markdown constructs, the families each gate round found (strong pairs, URLs, +non-terminated references, image references, nested padded wiki links), and 400 strings from a +seeded generator over an alphabet that includes `&#`, `&`, `<`, `![`, `[[` and `]]`, each placed in a title, a rationale, a question, a citation, a label after a word and a label alone (a cell start). Result (test `every_probe_is_formatted_as_the_typescript_or_refused`): -304 probes byte for byte with the TypeScript, 458 refused, 0 silent difference; a probe made only +279 probes byte for byte with the TypeScript, 503 refused, 0 silent difference; a probe made only of what the equine reports use, outside the constructs, is never refused (a polygon's `[[10,10],…]`, the DOI's `<0697:toih>` and the URLs' inner underscores pass). The equine reports pass the guard unchanged. @@ -3210,9 +3216,11 @@ Residual risks: the reports admit an allowlist of characters and forms and refus including text Prettier would have kept (links, HTML, character references, symbols, combining marks, non-ASCII spaces, most uses of `*` and `_`): a future changeset using them stops the pipeline, visibly, until the allowlist is widened from new TypeScript probes or the port grows a -Markdown printer; the allowlist itself is proven by probes, not by a model of Prettier, so a -combination of admitted characters no probe covered could still differ (the sweep's random part -and the three gate differentials found none in their last rounds); the comparison of the application's drawings accepts +Markdown printer; the allowlist is proven by probes, not by a model of Prettier: each gate round +of R5.4 found a family the previous rules missed (Markdown syntax, constructs, strong pairs and +URLs, then non-terminated references, image references and nested padded wiki links), each now +ruled and frozen in the sweep, so a combination of admitted characters no probe covers could still +differ; the comparison of the application's drawings accepts exactly the trigger-name rule, byte for byte; the frozen `horse.svg` is read, never regenerated, so a future change of the drawing needs its own decision; the residual risks of §11.13 stand.