docs(docs): R5.4 fourth gate round declared, guarantee restated exactly, audits
This commit is contained in:
1 parent
dcef2eb79c
commit
aff3a96d55
3 files changed
+196
-17
No files matched your search
@@ -0,0 +1,166 @@
|
||||
# Security gate R5.4, round 4 (fresh): feature/rust-r5-pipeline-core @ 6d4fdd6
|
||||
|
||||
Scope: `git diff c6e0d85..HEAD` (36 existing files), focus on 124d954 (`curation/report_guard.rs`,
|
||||
`curation/markdown.rs`, `draft/{pipeline,fields,ops}.rs`). Product files read whole: report_guard.rs,
|
||||
markdown.rs, changelog.rs, review_report.rs, overlay.rs, drawing.rs, app_review.rs, svg_compare.rs,
|
||||
labels.rs, stats.rs, bin/import_equine.rs, plus the R5.4 hunks of fields.rs, ops.rs, pipeline.rs, mod.rs.
|
||||
|
||||
```
|
||||
SECURITY — VERDICT: BLOCK(3)
|
||||
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
|
||||
SCOPE: 36 files
|
||||
BLOCKING:
|
||||
1. [HIGH] markdown-guard/character-reference — crates/core/src/curation/report_guard.rs:43 — a character reference without its `;`, or with a name over 32 characters, passes the guard and Prettier rewrites it: `a * b` → `a \* b`, `*` → `\*`, `_` → `\_`, `a_b` → `a_b`, `�…042;` (35-character name) → `\*`; a non-terminated reference followed later by a bare `&` before `,` or `"` makes Prettier overwrite that `&`: `< x &` → `< x <`, `A x &` → `A x A` — hint: outside code spans refuse `&#` whatever follows, and `&` followed by a legacy (non-terminated) entity name with or without `;`; keep `&` before other text admitted (the equine REVIEW-REPORT holds `&pid=S1019` in a URL)
|
||||
2. [HIGH] markdown-guard/image-reference — crates/core/src/curation/report_guard.rs:33 — `![…]` is not among the refused constructs; Prettier parses it as an image reference and prints the alt from its decoded, unescaped value: `![a\"b]` → `![a"b]`, `![a\|b]` → `![a|b]`, `![x|y]` in a CHANGELOG cell, `![&]` → `![&]`, `![<]` → `![<]`, `![*]` → `![*]`; the `![` and its `]` can come from two insertions on one list line (title, rationale, question) — hint: refuse `![` outside code spans (the equine reports hold none)
|
||||
3. [HIGH] markdown-guard/wiki-link — crates/core/src/curation/report_guard.rs:34 — the padded-wiki check splits on `[[`, so the text between a `[[` and the first `]]` is never examined when another `[[` sits in between; Prettier's tokenizer (`/^\[\[(?<linkContents>.+?)\]\]/su`, contents trimmed) matches across it: `[[ a [[b]]` → `[[a [[b]]`, also `[[ é [[中]]`, `[[ 1 [[2]]`, `[[a]] [[ b [[c]]`, `[[ x [ [[y]]` — hint: for every `[[` occurrence (overlapping `[[[` included) take the text up to the first following `]]` and refuse it when it starts or ends with whitespace, or refuse `[[` followed by whitespace and whitespace followed by `]]`
|
||||
REPORTED (non-blocking):
|
||||
- [MEDIUM] declaration-wrong (guarantee overstated) — docs/development/RUST.md:3164, 3169, 3184, 3214
|
||||
- [LOW] resource/quadratic-underscore-check — crates/core/src/curation/report_guard.rs:88
|
||||
- [LOW] lint-allow in test files (convention, not a scanner suppression) — crates/core/tests/curation_review.rs:6, crates/core/tests/curation_markdown.rs:6
|
||||
PROOF: semgrep 128 rules on 36 files → 0 findings; checklist 13 checks → 6 findings
|
||||
```
|
||||
|
||||
## Severity basis
|
||||
|
||||
The three blocking findings break the owner requirement this guard exists for: a report is byte for
|
||||
byte what the TypeScript+Prettier pipeline wrote, or it is refused loudly. Each one is a silent
|
||||
difference reachable through the normal authoring channel (a title, rationale or question in
|
||||
`changes.json`, a citation in `references.json`, a label set by an op), with no refusal and a clean
|
||||
exit. There is no confidentiality or availability impact and no actor beyond the author of the
|
||||
curation files; the classification rests on the explicit integrity requirement and on the fact that
|
||||
RUST.md states the opposite (finding M1). In two families the TypeScript output is the one that alters
|
||||
the author's text (`< x &` became `< x <`), so the fix is a refusal, never an imitation of
|
||||
Prettier.
|
||||
|
||||
## Method and evidence (differential, all in the scratchpad)
|
||||
|
||||
- TypeScript side: `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/ts/sweep.mts`,
|
||||
the sweep generator's exact `changesFile(probe)` on the mini sheet, run through `packages/anatomy/src`
|
||||
(`applyChangeset`, `renderChangelog`, `renderReviewReport`) and `formatMarkdown` (Prettier 3.8.3,
|
||||
options of `packages/anatomy/src/format.ts`).
|
||||
- Rust side: `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/rs/src/main.rs`,
|
||||
a throwaway binary linking `faunosteo-core` (`curation`), same changeset through `apply_changeset`,
|
||||
`render_changelog`, `render_review_report`, `format_report`. Raw texts matched the TypeScript in every
|
||||
case (no RAWDIFF), so every difference is the guard admitting what Prettier rewrites.
|
||||
- Batches: 79 hand-picked probes (20 silent), 2,500 seeded random strings over the admitted alphabet
|
||||
plus `&#`, `&`, `<`, `![`, `[[` atoms (151 silent, every one holding `&#`, a legacy entity or
|
||||
`![`), 3,000 seeded random strings without `&`, `![` or `[[` (0 silent), 55 targeted edges (7 silent),
|
||||
13 entity variants (2 silent). Total 5,647 probes, 180 silent differences, all in the three families.
|
||||
Probe lists and results: `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/out/`.
|
||||
- Prettier internals confirmed in `prettier@3.8.3/plugins/markdown.js`: wiki tokenizer
|
||||
`/^\[\[(?<linkContents>.+?)\]\]/su` with `.trim()`, placed before `link`; the unescape restore keeps the
|
||||
source of a one-character text node except `*` and `_` (`Tf=/^\\?.$/su`), which is why `*` and
|
||||
`_` come out as `\*` and `\_`; `imageReference` prints `node.alt`, the decoded value.
|
||||
|
||||
Sample (review list item, probe in the title):
|
||||
|
||||
```
|
||||
rust: - `CHG-001` Titre a * b (niveau C, …
|
||||
ts: - `CHG-001` Titre a \* b (niveau C, …
|
||||
rust: - `CHG-001` Titre ![a\"b] (niveau C, …
|
||||
ts: - `CHG-001` Titre ![a"b] (niveau C, …
|
||||
rust: - `CHG-001` Titre [[ a [[b]] (niveau C, …
|
||||
ts: - `CHG-001` Titre [[a [[b]] (niveau C, …
|
||||
rust: … `probe-ref` Citation < x &, cité par 1 nœud(s).
|
||||
ts: … `probe-ref` Citation < x <, cité par 1 nœud(s).
|
||||
```
|
||||
|
||||
In the CHANGELOG the same probes also shift the table alignment (Prettier pads on the rewritten text).
|
||||
|
||||
## Findings in detail
|
||||
|
||||
### H1 [HIGH] character references not terminated by `;` (report_guard.rs:43-53)
|
||||
|
||||
`reference` requires `after.contains(';')` and `name.len() <= 32`. The parser Prettier uses decodes
|
||||
non-terminated numeric references (`*`, `*`, `*`) and the legacy named ones (`<`, `&`,
|
||||
`©`…), and numeric references take any number of leading zeros, so the 32 cap lets
|
||||
`*` through. A reference decoding to `*` or `_` is escaped
|
||||
(`\*`, `\_`) or merged (`a_b` → `a_b`); a non-terminated reference followed later by a bare `&` (seen
|
||||
before `,` and `"`, i.e. at the end of a citation or a JSON value) gets that `&` replaced by the decoded
|
||||
character. Fix direction: refuse `&#` outside code spans unconditionally, and `&` followed by a legacy
|
||||
entity name (the non-terminated set) regardless of `;`; keep the existing `&name;` rule; keep `&` before
|
||||
anything else admitted, since the equine REVIEW-REPORT carries `&pid=S1019` inside a URL. Add the
|
||||
reproducers above to `r5.markdown-sweep-oracles.test.ts`.
|
||||
|
||||
### H2 [HIGH] image references (report_guard.rs:33-55)
|
||||
|
||||
`rewritten_construct` refuses `](` and `][` but not a shortcut image reference `![…]`. Its alt is printed
|
||||
from the decoded, unescaped value, so any `\"`, `\|` (including the `\|` that `cell()` adds in the
|
||||
CHANGELOG) or reference inside it is rewritten. `![a]` with plain text stays identical, which is why the
|
||||
sweep's `` (refused through `](`) never showed it. Fix direction: refuse `![` outside code
|
||||
spans.
|
||||
|
||||
### H3 [HIGH] wiki links whose contents hold a `[[` (report_guard.rs:34-41)
|
||||
|
||||
`plain.split("[[")` yields the segments between occurrences, not the text the regex matches: for
|
||||
`[[ a [[b]]` the first segment ` a ` holds no `]]` and is skipped, the second `b]]` is unpadded, so the
|
||||
line is admitted while Prettier matches ` a [[b` and trims it. Fix direction: emulate the lazy match
|
||||
from each `[[` position (a `]]` at offset 1 or more), or refuse `[[` followed by whitespace and
|
||||
whitespace followed by `]]`.
|
||||
|
||||
### M1 [MEDIUM] declarations overstated (docs/development/RUST.md)
|
||||
|
||||
- Line 3164: "a `[[ … ]]` padded inside" is refused: false for H3.
|
||||
- Line 3169: "so an unknown form is refused by default": false; the allowlist is per character, and
|
||||
constructs built from admitted characters (`&`, `#`, `!`, `[`, `]`) are admitted.
|
||||
- Line 3184: "the constructs refused whole (links, HTML tags, autolinks, character references)": false for
|
||||
non-terminated or long references (H1), and image references are not refused at all (H2).
|
||||
- Line 3214: the residual risk says an uncovered combination "could still differ" and that the last rounds
|
||||
"found none": three families with 180 reproducers were found in one round. The declaration must be
|
||||
rewritten with the fixes, and the sweep extended with these families.
|
||||
|
||||
### L1 [LOW] quadratic underscore check (report_guard.rs:88)
|
||||
|
||||
For each `_`, `chars[i + 1..].iter().collect::<String>()` copies the rest of the text to test the URL
|
||||
starts: O(n²) per authored text. Measured with the probe binary: 10 KB of `a_` 14 ms, 40 KB 221 ms, 80 KB
|
||||
0.91 s (x4 per doubling), so about 2.5 min per occurrence at 1 MB and hours near the 16 MiB file cap; a
|
||||
title or rationale is checked in up to four places. Owner-authored input of a dev tool, hence LOW. Fix
|
||||
direction: compare the slice in place (`chars[i + 1..].starts_with(&['h','t','t','p',…])` or byte offsets
|
||||
on the `&str`).
|
||||
|
||||
### L2 [LOW] lint allows in the new test files (context)
|
||||
|
||||
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]` heads
|
||||
`curation_review.rs` and `curation_markdown.rs`, the same header as the pre-existing `curation_draft.rs`:
|
||||
test-only file access and unwraps, not a security-scanner suppression. No `nosemgrep`, `nosec`,
|
||||
`eslint-disable`, `ts-ignore` or new allow in library code.
|
||||
|
||||
## Checklist (13 checks)
|
||||
|
||||
| # | Check | Result |
|
||||
|---|---|---|
|
||||
| 1 | Hardcoded secret, token, key, auth URL (added lines, incl. docs and oracle JSON) | none |
|
||||
| 2 | SQL built by concatenation | none in the diff (the seed SQL writer is R5.3, unchanged) |
|
||||
| 3 | Unsanitized render (innerHTML, eval, raw) | none; SVG text goes through the R3 serializer (`& < > "` and tab/newline escaped in attributes) after `first_unwritable` (XML 1.0 code points, CR refused) |
|
||||
| 4 | Sensitive endpoint without authz | not applicable (offline dev tool, no endpoint) |
|
||||
| 5 | Stack trace or internal path to a user | not applicable (CLI stderr for the developer) |
|
||||
| 6 | Secret or PII in a log | none (stdout and stderr carry file names, counts, refusals) |
|
||||
| 7 | Tracked `.env` or credential file | only `.env.example`; nothing in the diff |
|
||||
| 8 | Guardrails scanner (`--domain api,supabase`) | 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER`, `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`) outside the diff, filtered |
|
||||
| 9 | New suppressions (anti-gaming) | none in library code; L2 |
|
||||
| 10 | Markdown guard differential (5,647 probes) | H1, H2, H3, M1 |
|
||||
| 11 | Strict SVG comparison | holds (below) |
|
||||
| 12 | Resource use of the new code | L1; renderers indexed, document bound unchanged |
|
||||
| 13 | `--check` / `--out` behaviour | holds (below) |
|
||||
|
||||
## What holds
|
||||
|
||||
- Channels: both draft reports pass every raw line through `format_report` (pipeline.rs:273-274); no
|
||||
authored text reaches a heading; list-item fields with a control character are refused
|
||||
(`assert_list_items_single_line`); ids that reach code spans are stable ids or region codes, checked at
|
||||
the op (ops.rs `assert_stable_id`), field segments refuse controls (NUL included) and backticks; a line
|
||||
with an odd backtick count is refused, so no code span can hide a line break. The R5.2 IMPORT-REPORT
|
||||
stays on `format_markdown` with legacy sources only. View ids in file names are stable ids, and
|
||||
`write_exclusive` refuses any non-normal path component.
|
||||
- Strict SVG comparison (`svg_compare.rs`), 11 crafted cases against the committed `app-main.svg`: the
|
||||
legitimate trigger-name difference is the only acceptance; an extra comment, trailing text, a prolog
|
||||
change, another English name, a renamed attribute, a DOCTYPE, an empty or 100,000-level produced file,
|
||||
a 100,000-level committed file and a stylesheet PI in the committed file are all refused, none panics.
|
||||
- `--check` (binary built in the scratchpad, read-only): `legacy-import: 6 files, 0 differences`,
|
||||
`draft: declared: review/app-main.svg`, `draft: 12 files, 0 differences`, exit 0.
|
||||
- `--out` into a fresh `mktemp -d` under the scratchpad: 18 files written; a second run refused
|
||||
`not empty, nothing written` (exit 2); legacy identical to the seed, draft identical except
|
||||
`app-main.svg`. Working tree clean afterwards.
|
||||
- Panics: none found in the new code (slices after an ASCII `**` match, `chars[i + 1..]` with `i < len`,
|
||||
`compact_value` cut bounded by the length test, overlay NaN paths printed as `String(number)`).
|
||||
- semgrep: 128 rules (p/security-audit, p/secrets, p/owasp-top-ten) on 36 files, 0 findings, 0 errors.
|
||||
@@ -31,3 +31,8 @@ Fresh verifier and security on the fixed commit: below.
|
||||
## Round 4
|
||||
|
||||
Fresh verifier and security on the fixed commit: below.
|
||||
|
||||
## Round 4 on 6d4fdd6: security BLOCK(3) (verifier pending at fix time)
|
||||
|
||||
- Security (`.claude/audits/2026-10-23-rust-r5-review-security-4.md`): HIGH non-terminated / long character references (`* b` → `\*`, `< x &` → `< x <`), HIGH image references `![…]` (alt reprinted decoded), HIGH nested padded wiki links (`[[ a [[b]]`); MEDIUM §11.14/§11.15 overstated ("unknown form refused by default", "refused whole", "found none"); LOW quadratic underscore check. Control: 3,000 random strings without `&`, `![`, `[[`: 0 silent.
|
||||
- Fixed: `&` admitted only before a space or as a URL query separator `&name=` (name not a legacy-reference prefix, the 106 HTML legacy names); `![` refused; padded `[[`/`]]` refused when the matching half exists; URL checks without allocation; sweep 782 probes (new families, random alphabet with `&#`, `&`, `<`, `![`, `[[`, `]]`): 279 identical, 503 refused, 0 silent; docs restated exactly.
|
||||
+25
-17
@@ -3157,24 +3157,30 @@ also rewrites Markdown in authored text (escapes, emphasis, whitespace, links);
|
||||
reproduces only the table alignment. After a denylist that leaked round after round, the guard of
|
||||
the draft's two reports (`curation/report_guard.rs`, `format_report`; the R5.2 import report keeps
|
||||
`format_markdown`, verified at R5.2) is an allowlist: outside code spans, letters and digits,
|
||||
single spaces (none at a line end), the punctuation `.,;:!?'"«»()[]{}/-–—…%°≠=+#&<>’‘“”±→×≤≥|`, an
|
||||
underscore between two ASCII letters or digits and not glued to a URL (`http://`, `https://`,
|
||||
`www.`), a backslash before `"` or `|` (a JSON quote, the cell escape), `**word**` with one run of
|
||||
letters between spaces or line edges (the import report's form), and no inline construct (an
|
||||
inline link, image or reference `](`, `][`; a `[[ … ]]` padded inside; `<` before a letter, `/`,
|
||||
`!` or `?`: an HTML tag or an autolink; a character reference `&name;`, `&#n;`); inside code spans
|
||||
(generated ids and targets), anything but a carriage return or a tab, never empty, no outer space;
|
||||
a table row with another cell count than its header and a control character in a list-item field
|
||||
(title, rationale, question, citation, URL, DOI, vet-flag value) are refused too. Everything else
|
||||
is refused with `UnsupportedMarkdown`, so an unknown form is refused by default.
|
||||
single spaces (none at a line end), the punctuation `.,;:!?'"«»()[]{}/-–—…%°≠=+#<>’‘“”±→×≤≥|`, an
|
||||
`&` before a space or as a URL's query separator (`&name=` in a bare URL, the name not starting
|
||||
with one of HTML's 106 legacy references, which Prettier decodes without a `;`), an underscore
|
||||
between two ASCII letters or digits and not glued to a URL (`http://`, `https://`, `www.`), a
|
||||
backslash before `"` or `|` (a JSON quote, the cell escape), `**word**` with one run of letters
|
||||
between spaces or line edges (the import report's form), and no inline construct (an inline link,
|
||||
image or reference `](`, `][`; an image reference `![`; a `[[` before a space with a `]]` after
|
||||
it, or a `]]` after a space with a `[[` before it; `<` before a letter, `/`, `!` or `?`: an HTML
|
||||
tag or an autolink; a character reference `&name;`, `&#n;`); inside code spans (generated ids and
|
||||
targets), anything but a carriage return or a tab, never empty, no outer space; a table row with
|
||||
another cell count than its header and a control character in a list-item field (title,
|
||||
rationale, question, citation, URL, DOI, vet-flag value) are refused too. Everything else is
|
||||
refused with `UnsupportedMarkdown`: a character outside the set is refused by default; the forms
|
||||
built from admitted characters are ruled one by one, so a form no rule names and no probe covers
|
||||
could still pass.
|
||||
|
||||
The guarantee is empirical, not a proof of Prettier's behaviour: the TypeScript sweep
|
||||
(`oracles/r5/markdown-sweep.json`, generator `packages/anatomy/tests/r5.markdown-sweep-oracles.test.ts`,
|
||||
digests of the raw and formatted reports) holds 762 probes: 48 characters in six positions,
|
||||
whitespace patterns, Markdown constructs, the families each gate round found, and 400 strings from a
|
||||
seeded generator, each placed in a title, a rationale, a question, a citation, a label after a
|
||||
digests of the raw and formatted reports) holds 782 probes: 48 characters in six positions,
|
||||
whitespace patterns, Markdown constructs, the families each gate round found (strong pairs, URLs,
|
||||
non-terminated references, image references, nested padded wiki links), and 400 strings from a
|
||||
seeded generator over an alphabet that includes `&#`, `&`, `<`, `![`, `[[` and `]]`, each placed in a title, a rationale, a question, a citation, a label after a
|
||||
word and a label alone (a cell start). Result (test `every_probe_is_formatted_as_the_typescript_or_refused`):
|
||||
304 probes byte for byte with the TypeScript, 458 refused, 0 silent difference; a probe made only
|
||||
279 probes byte for byte with the TypeScript, 503 refused, 0 silent difference; a probe made only
|
||||
of what the equine reports use, outside the constructs, is never refused (a polygon's
|
||||
`[[10,10],…]`, the DOI's `<0697:toih>` and the URLs' inner underscores pass). The equine reports
|
||||
pass the guard unchanged.
|
||||
@@ -3210,9 +3216,11 @@ Residual risks: the reports admit an allowlist of characters and forms and refus
|
||||
including text Prettier would have kept (links, HTML, character references, symbols, combining
|
||||
marks, non-ASCII spaces, most uses of `*` and `_`): a future changeset using them stops the
|
||||
pipeline, visibly, until the allowlist is widened from new TypeScript probes or the port grows a
|
||||
Markdown printer; the allowlist itself is proven by probes, not by a model of Prettier, so a
|
||||
combination of admitted characters no probe covered could still differ (the sweep's random part
|
||||
and the three gate differentials found none in their last rounds); the comparison of the application's drawings accepts
|
||||
Markdown printer; the allowlist is proven by probes, not by a model of Prettier: each gate round
|
||||
of R5.4 found a family the previous rules missed (Markdown syntax, constructs, strong pairs and
|
||||
URLs, then non-terminated references, image references and nested padded wiki links), each now
|
||||
ruled and frozen in the sweep, so a combination of admitted characters no probe covers could still
|
||||
differ; the comparison of the application's drawings accepts
|
||||
exactly the trigger-name rule, byte for byte; the frozen
|
||||
`horse.svg` is read, never regenerated, so a future change of the drawing needs its own
|
||||
decision; the residual risks of §11.13 stand.
|
||||
|
||||
Reference in new issue
Block a user