diff --git a/.claude/audits/2026-10-23-rust-r5-review-security-6.md b/.claude/audits/2026-10-23-rust-r5-review-security-6.md new file mode 100644 index 00000000..bb27375c --- /dev/null +++ b/.claude/audits/2026-10-23-rust-r5-review-security-6.md @@ -0,0 +1,153 @@ +# Security gate R5.4, round 6 (fresh, blind) + +Branch `feature/rust-r5-pipeline-core`, HEAD `aeb5050`. Scope: +`git diff c6e0d85..HEAD -- crates/core packages/anatomy/tests docs/development/RUST.md` +(25 files). Mode: audit (this file is the only write). The brief carried a list of earlier +findings; per the auditor contract it was not used as a checklist. The scan below is complete +on its own and covers those areas anyway. + +``` +SECURITY — VERDICT: PASS +TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten +SCOPE: 25 files +BLOCKING: + (none) +REPORTED (non-blocking): + - [MEDIUM] resource/changelog-table-amplification — crates/core/src/curation/draft/changelog.rs:100 (via crates/core/src/curation/markdown.rs:133) + - [LOW] resource/per-view-drawing-amplification — crates/core/src/curation/draft/pipeline.rs:204 + - [LOW] resource/guard-underscore-pairing-quadratic-in-isolation — crates/core/src/curation/report_guard.rs:216 + - [LOW] markdown/invisible-hangul-fillers-admitted — crates/core/src/curation/report_guard.rs:159 + - [LOW] markdown/unpadded-wiki-link-in-table-cell (context, by design) — crates/core/src/curation/report_guard.rs:58 + - [LOW] anti-gaming/clippy-allow-in-new-test-file (context, convention) — crates/core/tests/curation_markdown.rs:6 + - [LOW] anti-gaming/clippy-allow-in-new-test-file (context, convention) — crates/core/tests/curation_review.rs:6 +PROOF: semgrep 162 rules on 25 files → 0 findings; checklist 13 checks → 7 findings +``` + +## What was run + +- semgrep 1.168.0, the three pinned packs, `--metrics=off`, on the 25 existing files of the + diff: 162 rules, 25 files scanned, 0 results, 0 errors. +- Checklist greps on the added lines: secrets, SQL concatenation, unsafe render, logging, + suppressions, tracked `.env` / credential files (only `.env.example` is tracked). Nothing + found. No endpoint in scope (authz check N/A). +- Guardrails scanner (`--domain api,supabase`): 0 FAIL, 1 WARN, 1 MANUAL. The WARN + (`SB-POLICY-NOT-OWNER`, `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`) is + outside the diff, not counted. +- `cargo test -p faunosteo-core --features curation`: every suite green. `cargo clippy + --all-targets -D warnings` (curation): clean. No `std::fs`, `std::env`, clock or process use + in the new core modules (only the binary, which already carried its documented allow). +- `import-equine --check`: legacy 6 files 0 differences; draft 12 files 0 differences, one + `declared:` line for `review/app-main.svg`. +- Differential fuzz of `report_guard::format_report` against Prettier 3.8.3 (the TypeScript + formatter, same options as `packages/anatomy/src/format.ts`), through a scratch binary on the + core library. 10,201 generated reports (seeded random strings over the admitted alphabet plus + Markdown tokens: URL starts, entities, `**`, `[[`, `](`, ` "`, `\t \n \r` as + references in attributes); XML-invalid code points and CR refused (`drawing.rs:41`); file + names built from stable view ids. +- `--check`: the tree fallback only for `review/app-*`, only with the declared rule, and only + when the committed file rebuilds from its tree byte for byte, the rule changes it and the + result equals the produced bytes; the committed file goes through the core reader's prescan + first. +- `--out`: `..` refused, final symbolic link refused (trailing slash handled), parent + canonicalised, seed refused by path and by (device, inode) of every seed folder, existing + destination identity-checked and must be empty, folders `create_dir`, files `create_new`, + components all `Normal`. The swap window between check and use stays as declared (§11.13). diff --git a/.claude/audits/2026-10-23-rust-r5-review-verify.md b/.claude/audits/2026-10-23-rust-r5-review-verify.md index 26a4b415..bca96b8b 100644 --- a/.claude/audits/2026-10-23-rust-r5-review-verify.md +++ b/.claude/audits/2026-10-23-rust-r5-review-verify.md @@ -49,6 +49,12 @@ Fresh verifier and security on the fixed commit: below. - Security (`.claude/audits/2026-10-23-rust-r5-review-security-5.md`): MEDIUM F2 the same wiki link across fields; MEDIUM F1 an authored field segment (`name.x]`) inside a generated code span closing an author's `[`; LOW docs "code spans are the generator's alone" wrong (segments are authored); LOW quadratic `in_url_token` and padded-wiki scans; LOW test allows (context). - Fixed: field segments limited to `[A-Za-z0-9_-]` at the op (declared refusal; TypeScript accepted any); code spans admit only the generator alphabet `[A-Za-z0-9_:/. -]`; list-item fields refuse `[[`, `]]`, `{%`, `%}` whole; plain parts refuse `{%`/`%}`; URL-token membership in one pass, padded wiki through one find/rfind; sweep 798 probes (eight cross-field construct pairs): 268 identical, 530 refused, 0 silent; segment refusal test with its reason; docs counts (draft 4,385, gain 1,477, report_guard.rs 258) and rules restated. -## Round 6 +## Round 6 on aeb5050: verifier C1 to C8 MET (C9 unverifiable: gates run in parallel; 8 FLOOR lines = keywords quoted in audit prose), security PASS (1 MEDIUM, 6 LOW); `pnpm check:all` green 14:02:00 + +- Verifier: 0 silent over 100,000 guard probes × 3 contexts, 45,000 cross-field pairs (28 guard-level, all stopped first by `assert_single_line`), 32,000 end-to-end changesets TS vs Rust, 148,921 letter/digit code points in a cell. Observations: stale `draft/mod.rs` header; RUST.md: `format_markdown` gained the cell-count refusal (import report too), "combining marks" wording (alphabetic marks pass, aligned alike); declared emoji case skipped its review report and drawings; TODO R5.4a cites `format_markdown`. +- Security (`.claude/audits/2026-10-23-rust-r5-review-security-6.md`): MEDIUM CHANGELOG table amplification (rows × widest cell: +155 KB changeset → 1.09 GB, 2.2 GB RSS; TypeScript alike); LOW per-view drawings bounded only by the value bound (≈2 GB estimate); LOW `underscore_pairs_later` O(parts × length) in isolation, unreachable (≤2 authored parts per line); LOW invisible Hangul fillers admitted (Prettier keeps them); LOW unpadded `[[x]]` in a cell by design; LOW test allows (convention). +- Fixed: aligned table bounded at 32 Mi display columns before any row is built (`Guard`, test `an_aligned_table_past_the_bound_is_refused_before_it_is_built`; equine widest ≈0.5 MB), declared in §11.14; residual risks (drawings bound, Hangul fillers) in §11.15; `draft/mod.rs` header; RUST.md wording (format_markdown shared refusals, alphabetic marks); declared emoji case now holds its review report (same text or declared refusal) and drawings; TODO R5.4a. + +## Round 7 Fresh verifier and security on the fixed commit: below. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 0e71c600..7ce74378 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -321,7 +321,7 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo - [ ] OWNER DECISION (found 2026-10-23 by the R5.3 final verifier): serde derives of the R2 contracts read a struct from an array and a unit enum from a one-key map, where Zod refused; likely reachable by the gated readers of imported PDFs (`archive/read.rs`, `report/read.rs`, `pdf/import.rs`); probe the readers, then a per-file go for a strict deserialisation (RUST.md §11.13) - [ ] chore (pre-existing, security gates R5.3): `clippy.toml` disallowed list misses `fs::create_dir*`, `OpenOptions::open`, `read_dir`, `canonicalize`, `remove_*`, `env::current_dir`, `env::args_os`, `process::Command`; extend so the core's no-I/O rule is enforced, not only observed - [ ] R5.4 review overlays + app drawings, CHANGELOG + REVIEW-REPORT, --check over the whole draft (owner go 2026-10-23, conditional on the R5.3 final checks: met on c6e0d85; contract `.claude/tasks/contracts/2026-10-23-rust-r5-review.md`; stop at the checkpoint, no R5.5) - - [x] R5.4a `draft/{labels,changelog,review_report}.rs` + `stats::stat_rows`: CHANGELOG and REVIEW-REPORT raw text = ledger pre-Prettier digests, formatted text byte-exact (`markdown::format_markdown`); compactValue on UTF-16 units + - [x] R5.4a `draft/{labels,changelog,review_report}.rs` + `stats::stat_rows`: CHANGELOG and REVIEW-REPORT raw text = ledger pre-Prettier digests, formatted text byte-exact (`report_guard::format_report`: allowlist guard + `markdown::format_markdown` alignment); compactValue on UTF-16 units - [x] R5.4b `draft/{overlay,app_review}.rs` on the core tree (`read_svg`, frozen horse.svg, prolog kept verbatim): review overlays and app drawings; bytes vs xmldom where reproducible, parsed trees otherwise; the R3 trigger-name decision declared for app-main - [x] R5.4c `render_draft_artifacts` complete (TS order); `import-equine --check` covers the 12 draft files with explicit criteria (bytes; declared tree rule), PENDING list and `shortcut:` removed; fails on any missing or unauthorised difference - [x] R5.4d proofs: `tests/curation_review.rs` (the 6 files, ledger digests, determinism); TS generator `r5.review-synthetic-oracles.test.ts` → `oracles/r5/review-synthetic.json` (changelog raw + formatted, review report, overlays, app drawings on mini-sheet cases: Unicode, pipes, newlines, long values, emoji refused, every status); R5.3 LOW carry-over (destination identity, cite membership, unreadable-seed message, stale comment, doc nits) diff --git a/crates/core/src/curation/draft/mod.rs b/crates/core/src/curation/draft/mod.rs index d2d28d0f..dc447e88 100644 --- a/crates/core/src/curation/draft/mod.rs +++ b/crates/core/src/curation/draft/mod.rs @@ -1,8 +1,8 @@ -//! The draft side of the pipeline (R5.3): the authored changeset applied on the legacy -//! baseline, the report migration map, the document statistics and the seed SQL -//! (`packages/anatomy/src/curation`, without the review drawings and the Markdown reports of -//! R5.4). Parity with the TypeScript is proven by `tests/curation_draft.rs` against the -//! committed `draft/` folder, `oracles/r5/pipeline.json` and `oracles/r5/curation-synthetic.json`. +//! The draft side of the pipeline (`packages/anatomy/src/curation`): the authored changeset +//! applied on the legacy baseline, the report migration map, the document statistics and the +//! seed SQL (R5.3), the CHANGELOG, the review report and the review drawings (R5.4). Parity with +//! the TypeScript is proven by `tests/curation_draft.rs` and `tests/curation_review.rs` against +//! the committed `draft/` folder and the `oracles/r5` files. pub mod app_review; pub mod apply; diff --git a/crates/core/src/curation/markdown.rs b/crates/core/src/curation/markdown.rs index f07aae09..bf17e3a2 100644 --- a/crates/core/src/curation/markdown.rs +++ b/crates/core/src/curation/markdown.rs @@ -130,6 +130,12 @@ pub(crate) fn split_row(line: &str) -> Vec { .collect() } +/// The largest aligned table written, in display columns (rows × padded row width): every row +/// is padded to the widest cell of each column, so one long authored cell multiplies by the row +/// count (an 80 KB rationale over 14,000 records asked for 1 GB). The equine CHANGELOG's widest +/// table holds about 0.5 MB. +pub const MAX_TABLE_COLUMNS_TOTAL: usize = 32 * 1024 * 1024; + fn align(lines: &[&str]) -> Result, PipelineError> { let rows: Vec> = lines.iter().map(|line| split_row(line)).collect(); let columns = rows.first().map_or(0, Vec::len); @@ -144,6 +150,13 @@ fn align(lines: &[&str]) -> Result, PipelineError> { widths[column] = widths[column].max(display_width(text)?); } } + let row_width = widths.iter().sum::() + 3 * columns + 1; + if row_width.saturating_mul(rows.len()) > MAX_TABLE_COLUMNS_TOTAL { + return Err(PipelineError::Guard(format!( + "an aligned table of {} rows × {row_width} columns exceeds the {MAX_TABLE_COLUMNS_TOTAL} bound", + rows.len() + ))); + } let mut out = Vec::with_capacity(rows.len()); for (index, row) in rows.iter().enumerate() { out.push(if index == 1 { diff --git a/crates/core/tests/curation_format.rs b/crates/core/tests/curation_format.rs index a4bcc375..86da6e5c 100644 --- a/crates/core/tests/curation_format.rs +++ b/crates/core/tests/curation_format.rs @@ -77,6 +77,20 @@ fn tables_align_as_prettier_aligns_them() { assert_eq!(refused, vec![5], "only the emoji table is refused"); } +/// One long cell pads every row of its column: the aligned table is bounded before any row is +/// built (security gate R5.4 round 6: an 80 KB rationale over 14,000 records asked for 1 GB). +#[test] +fn an_aligned_table_past_the_bound_is_refused_before_it_is_built() { + let mut raw = vec!["| a | b |".to_owned(), "| --- | --- |".to_owned()]; + raw.push(format!("| {} | x |", "w".repeat(2_000))); + raw.extend((0..20_000).map(|_| "| a | b |".to_owned())); + let error = format_markdown(&raw.join("\n")).unwrap_err(); + assert!( + matches!(&error, PipelineError::Guard(message) if message.contains("exceeds the")), + "{error}" + ); +} + #[test] fn slugs_and_uuids_match_the_typescript() { let semantics = oracle("js-semantics.json"); diff --git a/crates/core/tests/curation_review.rs b/crates/core/tests/curation_review.rs index a969a5f6..14fff308 100644 --- a/crates/core/tests/curation_review.rs +++ b/crates/core/tests/curation_review.rs @@ -361,18 +361,21 @@ fn the_synthetic_review_cases_match_the_typescript_or_are_refused_as_declared() ), "{name}" ); - continue; + } else { + assert_eq!( + rendered.changelog.unwrap(), + case["changelog"].as_str().unwrap(), + "{name}" + ); + } + // The review report and the drawings of a declared case are held too: the same text + // or a declared refusal, never another text. + match rendered.review { + Ok(review) => assert_eq!(review, case["review"].as_str().unwrap(), "{name}"), + Err(PipelineError::UnsupportedWidth(_) | PipelineError::UnsupportedMarkdown(_)) + if DECLARED_REFUSALS.contains(&name) => {} + Err(other) => panic!("{name}: {other}"), } - assert_eq!( - rendered.changelog.unwrap(), - case["changelog"].as_str().unwrap(), - "{name}" - ); - assert_eq!( - rendered.review.unwrap(), - case["review"].as_str().unwrap(), - "{name}" - ); let views = typed(&case_document(&frozen, case)).views.len(); assert_eq!(rendered.drawings.len(), 2 * views, "{name}"); let rule = TriggerNames::of(&typed(&case_document(&frozen, case))); diff --git a/docs/development/RUST.md b/docs/development/RUST.md index 49bd32cd..c359a72a 100644 --- a/docs/development/RUST.md +++ b/docs/development/RUST.md @@ -3156,7 +3156,8 @@ three security probes, each round finding a Prettier rewrite the previous rules also rewrites Markdown in authored text (escapes, emphasis, whitespace, links); the port reproduces only the table alignment. After a denylist that leaked round after round, the guard of the draft's two reports (`curation/report_guard.rs`, `format_report`; the R5.2 import report keeps -`format_markdown`, verified at R5.2) is an allowlist: outside code spans, letters and digits, +`format_markdown`, verified at R5.2, which gained at R5.4 the cell-count refusal and the table +bound below, both shared with the import report) is an allowlist: outside code spans, letters and digits, single spaces (none at a line end), the punctuation `.,;:!?'"«»()[]{}/-–—…%°≠=+#<>’‘“”±→×≤≥|`, an `&` before a space or as a URL's query separator (`&name=` in a bare URL, the name not starting with one of HTML's 106 legacy references, which Prettier decodes without a `;`), an underscore @@ -3200,8 +3201,9 @@ pass the guard unchanged. | Input | TypeScript | Rust | |---|---|---| -| authored text outside the allowlist: Markdown Prettier rewrites (`*x*`, `_x`, `x_**a**`, `**a****b**`, `Voir_https://…`, `a b`, a non-ASCII space, a tab, a line break in a list item, `` `` ``, `a\|b`, `[a]()`, `[[ a ]]`, `*`…), the constructs refused whole (links, HTML tags, autolinks, character references) and any character outside the admitted set (combining marks, pictographs, symbols) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines, normalised links; a row with an extra cell), or kept | refused (`UnsupportedMarkdown`), even where Prettier would have kept the text | +| authored text outside the allowlist: Markdown Prettier rewrites (`*x*`, `_x`, `x_**a**`, `**a****b**`, `Voir_https://…`, `a b`, a non-ASCII space, a tab, a line break in a list item, `` `` ``, `a\|b`, `[a]()`, `[[ a ]]`, `*`…), the constructs refused whole (links, HTML tags, autolinks, character references) and any character outside the admitted set (pictographs, symbols, combining marks without the Alphabetic property; an alphabetic mark such as U+0345 is a letter and Prettier aligns it alike) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines, normalised links; a row with an extra cell), or kept | refused (`UnsupportedMarkdown`), even where Prettier would have kept the text | | a pictograph in a table cell (an emoji, and ©, ®, ™, ✓, ★, ⚠ and the other characters `text::is_emoji_candidate` lists) | measured with Prettier's tables | refused (`UnsupportedWidth`, R5.2): the set is wider than the emoji, the refusal stays loud | +| an aligned report table past 32 Mi display columns in all (rows × padded row width; the equine CHANGELOG's widest table is about 0.5 MB): one long authored cell pads every row of its level, so an 80 KB rationale over 14,000 records asked for 1 GB (security gate R5.4 round 6) | Prettier aligned it (the same amplification) | refused before any row is built (`Guard`, test `an_aligned_table_past_the_bound_is_refused_before_it_is_built`), import report included | | an id an `add` writes (the item's, and those nested in its `children` or `categories`) or a `rename` id that is not a stable id (a region code for a region), or a field segment outside `[A-Za-z0-9_-]` (a control character, NUL included, a backtick, a `]`, a space, a non-ASCII letter) | accepted when a later op removed or renamed the item: the raw id stayed in the records and reached the reports' code spans (a line break there forged a section, security gate R5.4 round 2) | refused at the op, the check the final document applies anyway (test `ids_and_field_segments_that_could_forge_report_structure_are_refused`) | | a label, id or path holding a code point XML 1.0 refuses, or a carriage return, in a review drawing | written (a non-well-formed file; a raw CR read back as a line feed) | refused: the drawing is not written | @@ -3238,7 +3240,10 @@ ruled and frozen in the sweep, so a combination of admitted characters no probe differ; the comparison of the application's drawings accepts exactly the trigger-name rule, byte for byte; the frozen `horse.svg` is read, never regenerated, so a future change of the drawing needs its own -decision; the residual risks of §11.13 stand. +decision; the review drawings are bounded only through the document's 200,000 values (two drawings +of about 137 KB per view, about 2 GB at the bound, an estimate of the security gate R5.4 round 6, +not measured), and the allowlist admits the invisible Hangul fillers (U+3164, U+115F, U+1160, +U+FFA0, letters Prettier keeps, no structure); the residual risks of §11.13 stand. Conditions for R5.5: the owner's validation of this checkpoint; the upload tool (`ureq`, the service key from the environment, bucket `anatomical-sheet-assets`, key `.svg`,