docs(docs): R5.4 emoji refusal named at the guard, round 12 residuals declared

This commit is contained in:
bchanot committed 2026-10-23 17:13:30 +02:00
1 parent c4e717da86
commit 88e42749ec
4 files changed
+152 -6

No files matched your search

@@ -0,0 +1,129 @@
# Security gate R5.4 (round 12): review files in Rust
Date: 2026-10-23. Mode: audit (fresh, blind). Branch `feature/rust-r5-pipeline-core`, HEAD
`c4e717d`. Scope: `git diff c6e0d85..HEAD -- crates/core packages/anatomy/tests docs/development/RUST.md`
(29 files, none deleted).
```
SECURITY — VERDICT: PASS
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 29 files
BLOCKING:
(none)
REPORTED (non-blocking):
- [LOW] quadratic-scan (latent, unreachable today) — crates/core/src/curation/report_guard.rs:216
- [LOW] quadratic-scan (bounded, ~2 s at the document bound) — crates/core/src/curation/draft/pipeline.rs:256
- [LOW] new lint allowance (context, documented convention) — crates/core/tests/curation_markdown.rs:6, crates/core/tests/curation_review.rs:6
- [LOW] declaration precision (R5.3 code, outside the diff) — crates/core/src/curation/draft/schema.rs:430
PROOF: semgrep 128 rules on 29 files → 0 findings; checklist 9 checks → 0 findings (guardrails 0 FAIL, 1 WARN outside the diff); differential probes 38,378 admitted template reports + 898 admitted real-pipeline reports → 0 silent differences; display width 1,112,032 code points → 0 differences
```
## Tool run
- `semgrep scan --config p/security-audit --config p/secrets --config p/owasp-top-ten --metrics=off`
on the 29 scope files: 128 rules applicable, 0 results, 0 errors.
- Guardrails scanner (`--domain api,supabase`, project root): 0 FAIL, 1 WARN
(`SB-POLICY-NOT-OWNER`, `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, not in
the diff, dropped), 1 MANUAL (not reported).
## Checklist (9 checks)
| Check | Result |
|---|---|
| hardcoded secret, token, key, auth URL in added lines | none |
| SQL by concatenation | none in the diff (`seed.rs` untouched) |
| unsanitized render, `eval`, shell | none; SVG output escaped (below) |
| sensitive endpoint without authz | N/A (offline CLI, no endpoint) |
| stack trace or DB error to an end user | N/A (operator CLI; refusals print `{:?}`, control characters escaped) |
| secret or PII in a log | none |
| tracked `.env` or credential file | none (`git ls-files`) |
| guardrails scanner | run, nothing in the diff |
| new suppression (anti-gaming) | no `nosemgrep`, `nosec`, `eslint-disable`; two lint allowances in test files (LOW 3) |
## What was verified, by probe or by reading
Markdown guard (`report_guard.rs`) against Prettier 3.8.3 with the repository options, from a
scratch harness (Prettier imported from the repository's `node_modules`, the Rust guard through a
scratch crate depending on `faunosteo-core` with `curation`):
- template reports reproducing the ten list-item and cell positions of the two reports (title,
rationale, question, citation, URL, DOI, vet-flag before/after, label cell): 1,197 hand probes
(reference shapes `[a] [b]`, `[a] (b)`, `\|` outside tables, `+`/`>`/`1.`/`#` mid-line, URL
trailing punctuation and `&name=`, `_` near URLs, `**word**`), 27,000 random single-field, 10,000
random cross-field and 36,000 URL-centric cases: 38,378 admitted, 0 differ from Prettier;
- the real generators: 7,200 random changesets run through the TypeScript (`applyChangeset`,
`renderChangelog`, `renderReviewReport`, `formatMarkdown`) and through the Rust
(`apply_changeset`, `render_changelog`, `render_review_report`, `format_report`): 898 admitted,
6,302 refused, 0 raw-text differences, 0 silent formatted differences;
- `text::display_width` against Prettier's `util.getStringWidth` for `"a" + c`, every scalar value
from U+0020 to U+10FFFF: 0 differences wherever the port measures (it refuses the rest).
Report structure: authored texts carry no backtick (`assert_authored` on every authored field
of both reports), list-item fields refuse control characters, cells turn `\n` into a space and
escape `|`; ids written by `add`/`rename` are stable ids or region codes, field segments
`[A-Za-z0-9_-]`, targets resolve only to existing stable ids, so no author byte starts a line or
writes a code span. Email autolinks need `@`, which the allowlist refuses.
SVG drawings: the serializer escapes `& < >` in text and `& < > " \t \n \r` in attributes
(`]]>` cannot appear); `drawing::first_unwritable` refuses non-XML characters and CR; element and
attribute names are constants; colours are `#rrggbb` by schema; view ids (file names) are stable
ids. Drawings projection: names copied at most four times per annotation, worst escape `&` → 5
bytes per JSON byte, so `24 × (json + longest name)` holds.
Records bound: every record site goes through `Recorder::record` or `mark_to_confirm`, both
charged before the clone. Aligned tables: CHANGELOG projected from cell bytes (bytes ≥ display
width, numbers ≤ 25), every table rechecked in `align` before padding.
CLI `import_equine.rs`: reads only canonical regular files inside the canonical seed, 16 MiB
cap; `--check` refuses a generated folder that is not the seed's own, lists without following
links, compares bytes (app drawings only through the strict rebuild rule); `--out` refuses `..`,
symlinks, non-empty folders and any path inside the seed by prefix or by (dev, inode), writes
with `create_dir`/`create_new`, deletes nothing; file names from views are stable ids and
`write_exclusive` accepts only normal components. The check/use race is declared (§11.13).
Timing of the guard on adversarial lines (linear): 400,000 `& ` in 7.9 ms, 400 KB of `[[x`,
`x ]]`, `**ab**`, `<`, URL `&a=1` runs under 10 ms, a 1.4 MB 100,000-column table in 95 ms.
## Findings (non-blocking)
### LOW 1: `underscore_pairs_later` is quadratic, latent
`crates/core/src/curation/report_guard.rs:216`. For every plain part holding `_`, the function
rescans all later code parts and re-collects each into a `Vec<char>`. A synthetic line of 100,000
`` `ab` x_y `` (900 KB) took 53.7 s. Not reachable today: authored text holds no backtick, so the
generator alone interleaves code spans and plain parts, and on every line of both reports at most
two plain parts can carry an authored `_` (title with rationale, question), which keeps the scan
linear. A future line that puts authored text between generated spans would turn it quadratic.
Hint: scan the parts right to left once, keeping a "a later code part closes" flag.
### LOW 2: drawings projection walks views × annotations before the bound
`crates/core/src/curation/draft/pipeline.rs:256`. `review_drawings_bytes` runs `layer_bytes` over
every annotation for every view, and the view count is not yet bounded there (only by the
200,000-value document bound). Measured on a synthetic `DraftResult`: 10,000 × 10,000 in 1.8 s,
20,000 × 20,000 in 7.6 s; the document bound allows about 11,000 of each, so about 2 s. Hint:
group annotations by `view_id` once, or stop as soon as the running total passes the bound.
### LOW 3: lint allowances in the two new test files (context)
`crates/core/tests/curation_markdown.rs:6`, `crates/core/tests/curation_review.rs:6`:
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. The lint is the
core's no-clock, no-file-system rule; `clippy.toml` documents that tests allow it locally, and
every existing integration test file carries the same line. Not a security-rule suppression.
### LOW 4: §11.15 names `changes.json` alone for the parse amplification
`crates/core/src/curation/draft/schema.rs:430`. `parse_references` goes through the same
`json_parse` as `parse_changeset`, so `references.json` under its 16 MiB cap should hold the same
order of memory (factor not measured here). R5.3 code, outside this diff; the residual line could
name both files.
## Notes
- The working tree changed during the run (comment and doc lines only, by another agent):
`.claude/audits/2026-10-23-rust-r5-review-verify.md`, `docs/development/RUST.md`,
`packages/anatomy/tests/r5.review-synthetic-oracles.test.ts`. HEAD stayed `c4e717d`; this gate
judges the committed range.
- Declared residuals (§11.15, including the ~65× `changes.json` parse) not re-reported.
- Probes and harnesses live in the session scratchpad only; no tracked file touched, the
`import-equine` binary never run; the scratch crates read only the committed sweep oracle.
@@ -82,6 +82,14 @@ Fresh verifier and security on the fixed commit: below.
- Security (`.claude/audits/2026-10-23-rust-r5-review-security-11.md`): PASS. 146,000 random cases vs Prettier, 44,330 admitted, 0 differ; drawings wrote 0.12 to 0.41 of the projection on hostile documents. LOW test allows (convention); LOW bare URLs autolinked in any authored field (visible text = target, TypeScript alike: owner choice).
- Fixed: 4,096 bytes per drawing for its fixed parts (`pipeline::DRAWING_FIXED_BYTES`; equine projection 1,406,812 for 564,239 written), test `the_drawings_projection_bounds_what_is_written` (equine and no annotations); RUST.md counts (4,606 lines, review 12 tests); test comment sections.
## Round 12
## Round 12 on c4e717d: verifier C1 to C8 MET, C9 NOT-MET (1 doc gap) + 12 FLOOR (audit prose, reported apart); `pnpm check:all` green 16:41:34
- Verifier: own differential 37,264 cases (32,000 random biased to the admitted alphabet, 5,264 handcrafted placements), 0 silent, 0 raw difference; drawings projection ≥ written on 8 adversarial drafts (tightest margin 15,477 B, no annotations); every figure of §11.14/§11.15 checked exact. C9: the emoji refusal is named at the aligner (`UnsupportedWidth`) in RUST.md (synthetic case line, pictograph row) and the TS generator comment, while the report guard refuses first (`UnsupportedMarkdown`); `UnsupportedWidth` applies in the draft reports only to alphanumeric candidates (Ⓜ, 🅰) and in IMPORT-REPORT.
- Fixed: the two RUST.md passages and the generator comment (oracle unchanged, generator green in check mode).
- Security (`.claude/audits/2026-10-23-rust-r5-review-security-12.md`): PASS. 7,200 random changesets through both real pipelines (898 admitted, 0 raw and 0 formatted differences), 73,000 template cases (38,378 admitted, 0 differ), display width on every code point U+0020 to U+10FFFF, guard linear on stress lines. LOW `underscore_pairs_later` quadratic, unreachable (≤2 authored plain parts per line); LOW `review_drawings_bytes` views × annotations (≈2 s at the value bound); LOW test allows (convention); LOW §11.15 parse residual names `changes.json` only (`references.json` same parser).
- Declared: the four LOWs in §11.15 (no code change: a fix would restart the gates; owner decision).
## Round 13
Fresh verifier on the documentation commit: below.
Fresh verifier and security on the fixed commit: below.
+11 -3
View File
@@ -3136,7 +3136,9 @@ Parity, every expectation written by the TypeScript (`tests/curation_review.rs`,
moved, an added and a polygon marker, a path marker on a detail view framed by its view box,
a sheet with nothing but a level A change: raw and aligned Markdown byte for byte, the
overlays byte for byte, the application's drawings byte for byte or under the declared rule;
an emoji in a cell gives the same raw text and the declared refusal at the alignment (R5.2);
an emoji in a cell gives the same raw text and a declared refusal: the report guard refuses it
first (`UnsupportedMarkdown`, a character outside the allowlist), before the aligner's
`UnsupportedWidth` (R5.2) is reached;
- two runs byte-equal; an undeclared difference (a colour, a coordinate) refused with and
without the rule.
@@ -3202,7 +3204,7 @@ pass the guard unchanged.
| Input | TypeScript | Rust |
|---|---|---|
| authored text outside the allowlist: Markdown Prettier rewrites (`*x*`, `_x`, `x_**a**`, `**a****b**`, `Voir_https://…`, `a b`, a non-ASCII space, a tab, a line break in a list item, `` `` ``, `a\|b`, `[a](<b>)`, `[[ a ]]`, `&#42;`…), the constructs refused whole (links, HTML tags, autolinks, character references) and any character outside the admitted set (pictographs, symbols, combining marks without the Alphabetic property; an alphabetic mark such as U+0345 is a letter and Prettier aligns it alike) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines, normalised links; a row with an extra cell), or kept | refused (`UnsupportedMarkdown`), even where Prettier would have kept the text |
| a pictograph in a table cell (an emoji, and ©, ®, ™, ✓, ★, ⚠ and the other characters `text::is_emoji_candidate` lists) | measured with Prettier's tables | refused (`UnsupportedWidth`, R5.2): the set is wider than the emoji, the refusal stays loud |
| a pictograph in a table cell (an emoji, and ©, ®, ™, ✓, ★, ⚠ and the other characters `text::is_emoji_candidate` lists) | measured with Prettier's tables | refused, loudly: in the draft's two reports by the guard first (`UnsupportedMarkdown`, outside the allowlist), the aligner's `UnsupportedWidth` (R5.2) applying there only to the alphanumeric candidates (Ⓜ, 🅰) and in IMPORT-REPORT; the set is wider than the emoji |
| aligned report tables past 32 Mi display columns in all (rows × padded row width, the CHANGELOG's three level tables summed; the equine CHANGELOG's widest table is about 0.5 MB): one long authored cell pads every row of its level, so an 80 KB rationale over 14,000 records asked for 1 GB (security gate R5.4 round 6) | Prettier aligned it (the same amplification) | refused (`Guard`): the CHANGELOG's tables are projected from their cells (bytes bound the display width) before the raw table is joined (test `a_changelog_whose_alignment_would_explode_is_refused_before_it_is_built`, through the pipeline), every other table in the aligner before any padded row is built (test `an_aligned_table_past_the_bound_is_refused_before_it_is_built`), import report included |
| records weighing more than 16 Mi units in all (`records::record_weight`, about the bytes in memory: 64 per JSON value and per string, plus one per byte of text, for the values, the change and the op, which each record clones; the equine records weigh 1,966,163; an add or a move also clones its position anchor): a rename or a remove records the whole item and one op, or the B/C marking, records once per item it reaches, so 400 renames of a 5,000-child item asked for 4.3 GB (round 7), a level B add of 10,000 children with a 128 KiB rationale 1.37 GB inside one marking, a 64 KiB value set over a subtree 2.0 GB (round 8), 200,000 one-character reference ids cloned into each record 948 MB at one unit per byte (round 9) | recorded (the same copies) | refused at the record that passes the bound, charged before the change and the op are cloned into it (`records::charge`, tests `the_records_cannot_outgrow_their_bound`, `the_records_bound_holds_inside_one_op_and_the_marking`); the raw report rows, built from the records, are bounded with them; the 200,000-value document bound now holds the baseline (test from a padded baseline) |
| review drawings past 64 MiB in all, projected as an upper bound before any is rendered (`pipeline::review_drawings_bytes`: per view, the cleaned drawing twice with 4,096 bytes each for the drawing's fixed parts (layer group, overlay group and legend), plus for each annotation of the view in the overlay's two documents and in the application's drawing 1,024 bytes and 24 times its JSON text and the longest view name, four copies escaped up to six times longer; the equine draft projects 1,406,812 bytes for 564,239 written): 2,000 views asked for 547 MB (security gate R5.4 round 8), unnamed triggers repeating a long opened view's names 131 MB from a 485 KB changeset (round 10) | written | refused before rendering (`pipeline::MAX_REVIEW_DRAWINGS_BYTES`, tests `review_drawings_past_their_bound_are_refused_before_rendering`, `unnamed_triggers_repeating_a_long_view_name_are_refused_before_rendering`; the projection bounds what is written, a draft without annotations included: test `the_drawings_projection_bounds_what_is_written`) |
@@ -3248,7 +3250,13 @@ value into each item before recording it, so the document holds at most as many
records bound admits; the comparison of `--check` looks up each trigger's view linearly (views ×
triggers, the views bounded through the drawings bound); parsing `changes.json` (R5.3 code) holds
about 65 times the file in memory (about 1 GB at the 16 MiB read cap, security gate R5.4 round 9),
before any bound of the applier applies; and the allowlist admits the invisible Hangul fillers (U+3164, U+115F, U+1160,
before any bound of the applier applies, and `references.json` goes through the same parser (the
same order, not measured); two scans are quadratic but out of reach or bounded (security gate
R5.4 round 12): `report_guard::underscore_pairs_later` on a line of many code spans and plain
underscores (53.7 s on a synthetic 900 KB line; the generator writes at most two plain parts
that can carry an authored `_` per line, so it stays linear), and `pipeline::review_drawings_bytes`
walking views × annotations before its bound applies (about 2 s at the 200,000-value document
bound); and the allowlist admits the invisible Hangul fillers (U+3164, U+115F, U+1160,
U+FFA0, letters Prettier keeps, no structure); the residual risks of §11.13 stand.
Conditions for R5.5: the owner's validation of this checkpoint; the upload tool (`ureq`, the
@@ -219,7 +219,8 @@ const CASES: Case[] = [
},
},
{
// Prettier measures an emoji with its RGI table; the port refuses (declared, R5.2).
// Prettier measures an emoji with its RGI table; the port refuses (declared: the report
// guard first, UnsupportedMarkdown, before the aligner's UnsupportedWidth of R5.2).
name: 'declared-emoji-in-a-cell',
changes_file: {
changes: [