docs(docs): R5.4 first gate round declared (Markdown guard, strict comparison), audits
This commit is contained in:
1 parent
94f8b35105
commit
68dec25e5e
3 files changed
+285
-4
No files matched your search
@@ -0,0 +1,231 @@
|
||||
```
|
||||
SECURITY — VERDICT: PASS
|
||||
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
|
||||
SCOPE: 25 files
|
||||
BLOCKING:
|
||||
(none)
|
||||
REPORTED (non-blocking):
|
||||
- [MEDIUM] check/declared-tree-rule-too-wide — crates/core/src/curation/draft/svg_compare.rs:104
|
||||
- [LOW] check/unchecked-xml-characters — crates/core/src/curation/draft/app_review.rs:33
|
||||
- [LOW] check/markdown-structure-injection — crates/core/src/curation/draft/review_report.rs:64
|
||||
- [LOW] check/quadratic-renderers — crates/core/src/curation/draft/review_report.rs:223
|
||||
- [LOW] anti-gaming/test-clippy-allow (context) — crates/core/tests/curation_review.rs:6
|
||||
PROOF: semgrep 162 rules on 25 files → 0 findings; checklist 8 checks + anti-gaming + 5 brief probes → 5 findings
|
||||
```
|
||||
|
||||
# R5.4 security gate (fresh), 2026-10-23
|
||||
|
||||
Repository `/home/bchanot/Documents/faunosteo`, branch `feature/rust-r5-pipeline-core`, HEAD
|
||||
`0a79c52`, scope `git diff c6e0d85..HEAD` (25 existing files). Every changed product file was
|
||||
read whole: `crates/core/src/curation/draft/{labels,changelog,review_report,drawing,overlay,
|
||||
app_review,svg_compare,pipeline,stats,ops,mod}.rs`, `crates/core/src/bin/import_equine.rs`,
|
||||
plus `crates/core/src/curation/markdown.rs`, `anatomy/svg/{serialize,tree}.rs` and
|
||||
`anatomy/compose.rs` where the new code relies on them.
|
||||
|
||||
## Method
|
||||
|
||||
- semgrep 1.168.0 (`EIO_BACKEND=posix`), `--config p/security-audit --config p/secrets
|
||||
--config p/owasp-top-ten --metrics=off --quiet --json` on the 25 files: 162 rules, 0
|
||||
results, 0 errors. These packs cover Rust thinly, so the manual review carries the weight.
|
||||
- Checklist (CLAUDE.md defaults): secrets, SQL concatenation, unsanitized render, authz,
|
||||
error exposure, secrets in logs, tracked `.env`, guardrails scanner
|
||||
(`python3 -B ~/.claude/skills/guardrails/scripts/scan.py <root> --domain api,supabase`:
|
||||
0 FAIL, 1 WARN, 1 MANUAL; the WARN `SB-POLICY-NOT-OWNER` is in
|
||||
`supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, outside the delta, so it
|
||||
is not reported here).
|
||||
- Anti-gaming: the diff was scanned for new suppressions.
|
||||
- Probes, all in the scratchpad: a throwaway crate linking `faunosteo-core` (feature
|
||||
`curation`) at
|
||||
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/probe.DaWdIB`,
|
||||
and `import-equine` built with `CARGO_TARGET_DIR` in the scratchpad. `--check` ran
|
||||
read-only on the repository. `--out` ran only inside the fresh
|
||||
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/out.6SamF5`.
|
||||
Nothing was deleted, and `git status` stayed clean.
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. [MEDIUM] The tree fallback hides more than the declared trigger-name rule
|
||||
|
||||
`crates/core/src/curation/draft/svg_compare.rs:104-123` (`compare_drawing`), used by
|
||||
`crates/core/src/bin/import_equine.rs:196-215` for `review/app-*.svg`.
|
||||
|
||||
When the bytes differ, both files are parsed with `parse_svg`. That function builds the tree
|
||||
of the root element only, and it keeps only elements and text
|
||||
(`anatomy/svg/tree.rs:155, 182-193`). Everything the tree does not carry is therefore never
|
||||
compared:
|
||||
|
||||
- the prolog: XML declaration and its `encoding`, processing instructions, comments;
|
||||
- comments and processing instructions inside the root and after it;
|
||||
- entity and character-reference forms, CDATA against escaped text, attribute quoting,
|
||||
`<g/>` against `<g></g>`;
|
||||
- attribute-value whitespace that the parser normalises.
|
||||
|
||||
Measured on the real committed `draft/review/app-main.svg` against the produced one, with
|
||||
the rule applied:
|
||||
|
||||
| Committed file changed by | `compare_drawing` |
|
||||
|---|---|
|
||||
| nothing | `Ok(TreeAfterRule)` |
|
||||
| `<?xml-stylesheet type="text/css" href="https://attacker.invalid/hide.css"?>` and a comment in the prolog, a comment and a PI right after `<svg …>`, a trailing comment | `Ok(TreeAfterRule)` |
|
||||
| also `encoding="UTF-8"` replaced by `encoding="ISO-8859-1"` | `Ok(TreeAfterRule)` |
|
||||
|
||||
In all three cases `--check` prints the same `declared: review/app-main.svg (trees equal
|
||||
under the R3 trigger-name rule)` line it prints today and exits 0. The same holds for
|
||||
`app-head.svg`, which is byte-identical today: a comment-only change would turn it into a
|
||||
"declared" file. The only visible sign would be the extra line.
|
||||
|
||||
Impact: the owner requirement "`--check` fails on any unauthorised difference" is not met
|
||||
for the two application drawings. A stylesheet processing instruction is honoured by
|
||||
browsers that open a standalone SVG: it could restyle or hide markers in the review drawing
|
||||
and fetch an external URL, and `--check` would still pass. Exploiting it needs write access
|
||||
to the repository. The files are dev review artifacts: the app, the bundle and the uploads
|
||||
never read them (grep of `apps/app`, `vite`, `packages/supabase`). Hence MEDIUM, not
|
||||
blocking.
|
||||
|
||||
Three declarations are materially wrong:
|
||||
- `svg_compare.rs:7` says "Any other difference is reported".
|
||||
- `import_equine.rs:185-186` and the printed "declared" line suggest the rule alone explains
|
||||
the difference.
|
||||
- RUST.md §11.15 says the comparison "accepts exactly the trigger-name rule and nothing
|
||||
else". §11.13's R5.4 condition did mention "declaration, prefixes and whitespace
|
||||
declared", but §11.15 does not carry that through, and it never covered processing
|
||||
instructions or comments.
|
||||
|
||||
Fix direction: once the rule has been applied to the committed tree, require the committed
|
||||
prolog to be byte-equal to the produced one. Then require
|
||||
`prolog + serialize(committed tree after rule)` to equal the produced text byte for byte.
|
||||
The core serializer writes the produced file, and `app-head.svg` already matches it byte for
|
||||
byte. At minimum: compare the prologs and the text after the root, refuse any comment or PI
|
||||
node in the committed document, and print "declared" only when the rule actually changed
|
||||
the committed tree. Then correct the three declarations.
|
||||
|
||||
### 2. [LOW] XML-invalid characters from authored strings reach the review drawings unchecked
|
||||
|
||||
`crates/core/src/curation/draft/app_review.rs:33-36` calls `render_layer` directly. The R3
|
||||
composition passes the layer through `checked` / `xml_safe` (`anatomy/compose.rs:221-236`,
|
||||
"refuse one XML cannot carry"), and this path skips that step. The same gap exists in
|
||||
`overlay.rs:194-198` (path `d`, ids in titles and labels).
|
||||
|
||||
Measured: a changeset that sets `annotation:hock` `name` to `{"fr":"Jarret\u0001<b>&","en":"Hock\u000b"}`
|
||||
is accepted, and the produced `review/app-main.svg` is not well-formed (roxmltree: "a
|
||||
non-XML character '\u{1}'"). A path geometry `d` holding U+0001 in an overlay gives the same
|
||||
result. This is not markup injection: the serializer escaping holds (see "Verified"). It is
|
||||
a malformed output written by `--out` without any refusal. `--check` then fails, closed.
|
||||
|
||||
Fix direction: run `is_xml_text` over the produced layer and overlay (reuse compose's
|
||||
`checked`) and refuse. This changes nothing on the committed data.
|
||||
|
||||
### 3. [LOW] Markdown structure injection from authored text
|
||||
|
||||
- `review_report.rs:64-85` (`change_line`: `title.fr`, `rationale`, `question` written raw
|
||||
in list items);
|
||||
- `review_report.rs:295-320` (questions);
|
||||
- `review_report.rs:245-278` (`citation`, `url`, `doi`);
|
||||
- `changelog.rs:80, 85` (table cells through `markdown.rs:43-49` `cell()`). `cell()` escapes
|
||||
`|` and `\n`, but not `\r`, and not a backslash in front of a pipe.
|
||||
|
||||
Measured with an added change:
|
||||
- a rationale `"Ligne un.\n\n## 10. Décisions réservées au professionnel\n\nAucune question ouverte. <img …>"`
|
||||
forges a second section 10 inside section 6 of `REVIEW-REPORT.md` (lines 92 and 343) and
|
||||
passes raw HTML through;
|
||||
- a `\r` in a title reaches `CHANGELOG.md` raw. GFM treats a CR as a line ending, so the
|
||||
table row breaks;
|
||||
- an authored `\|` becomes `\\|`. `format_markdown` and micromark both read that as an
|
||||
escaped backslash followed by a real delimiter, so the row gains a ninth column under an
|
||||
eight-column header.
|
||||
|
||||
Context: the text comes from the repository owner, and the code matches the TypeScript
|
||||
(`markdown.ts`). The current data holds none of this: no LF, CR, pipe or backtick in any
|
||||
title, rationale or question. The one `<0697:toih>` in a DOI and its URL renders as text,
|
||||
because a scheme cannot start with a digit. Gitea strips raw HTML. The code spans hold only
|
||||
constrained strings: `CHG-\d{3,}`, stable ids, resolved targets and allowlisted fields.
|
||||
§11.11's "Markdown risks" are about Prettier differences, so this risk is not declared
|
||||
anywhere.
|
||||
|
||||
Fix direction: refuse CR, LF and control characters in single-line authored fields at
|
||||
`parse_changeset` and `parse_references` (title, question, citation, doi). Either refuse
|
||||
them in the rationale too, or render newlines as spaces in list items. Escape `\` in front of
|
||||
`|` in `cell()`. Optionally escape `<`.
|
||||
|
||||
### 4. [LOW] Quadratic renderers on authored volume
|
||||
|
||||
- `review_report.rs:223-234` `citation_count` rebuilds `all_findings(document)`, one
|
||||
allocated `FindingEntry` per finding, for every reference: O(R × F).
|
||||
- `overlay.rs:87-143`: linear `find` over `previous` and `kept.contains`, O(A²) per view.
|
||||
- `review_report.rs:47-54` `subjects`: changes × records.
|
||||
|
||||
Measured with `render_review_report` on synthetic documents:
|
||||
|
||||
| References | Leaves | Time |
|
||||
|---|---|---|
|
||||
| 1,000 | 1,000 | 0.35 s |
|
||||
| 4,000 | 4,000 | 5.9 s |
|
||||
| 8,000 | 8,000 | 22.6 s |
|
||||
|
||||
Extrapolated, not measured: about 3 to 4 minutes for a balanced document at the 200,000-value
|
||||
bound. The equine data (48 references, about 600 findings) renders in milliseconds:
|
||||
`--check` takes 0.13 s wall time and 18 MB RSS. This is the same class as the declared
|
||||
"minutes for a 16 MiB changeset" risk (§11.13), but these renderers are not declared.
|
||||
|
||||
Fix direction: count citations in one pass into a `HashMap<&str, usize>`, index `previous`
|
||||
and `kept` with a `HashMap` / `HashSet`, and group records by change id once.
|
||||
|
||||
### 5. [LOW, context] New clippy allow in a test crate
|
||||
|
||||
`crates/core/tests/curation_review.rs:6`
|
||||
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. This is an
|
||||
integration test, outside the library, with the same header as all 33 files of
|
||||
`crates/core/tests`. It is a lint allow, not a security-scanner suppression. No new
|
||||
`nosemgrep`, `nosec`, `eslint-disable`, `ts-ignore` or `noqa` appears in the diff. Not
|
||||
blocking.
|
||||
|
||||
## Verified, no finding
|
||||
|
||||
- SVG markup injection: element and attribute names are literals. Values go through the
|
||||
core serializer, which escapes `& < > "` and `\t \n \r` in attributes and `& < > \r` in
|
||||
text. Probe: an annotation name `J"><script>alert(1)</script>&`, an English name
|
||||
`]]><!--x-->`, a view name `T</title><script>y</script>` and an overlay path
|
||||
`d="M0 0…\"/><script>x</script>"`. The output is well-formed (apart from finding 2's
|
||||
control character), with 0 `script` elements and every payload escaped.
|
||||
- File names built from view ids: views are `stable_id` (`[a-z0-9][a-z0-9_-]{0,63}`),
|
||||
enforced by the typed parse in `pipeline.rs:206` before any write. `write_exclusive`
|
||||
also refuses any component that is not `Normal`. The `--check` reads stay canonical
|
||||
inside the seed.
|
||||
- Panics on authored input: `compact_value` slices only past 120 UTF-16 units and decodes
|
||||
lossily. The prolog slice ends at roxmltree's root start (`<`, a char boundary).
|
||||
`style()[0]` is a static slice. Polygon centroid: at least 3 points (typed), NaN otherwise,
|
||||
never a panic. No `unwrap`, `expect` or unchecked index on data in the product files.
|
||||
Recursion in `first_difference` and `TriggerNames::apply` is bounded by the reader's
|
||||
depth cap (64).
|
||||
- `--check` is read-only: no file in `legacy-import/` or `draft/` changed and `git status`
|
||||
stayed clean. Output: `legacy-import: 6 files, 0 differences`, `draft: declared:
|
||||
review/app-main.svg …`, `draft: 12 files, 0 differences`. A file that is missing, not
|
||||
produced or not UTF-8 fails. A generated folder must canonicalise to the seed's child.
|
||||
Files other than `review/app-*` have no rule and must be byte-equal. `horse.svg` is
|
||||
compared with itself, but its sha256 drives `assets.json`, `document.json` and the seed
|
||||
SQL, so a change to it fails the check.
|
||||
- `--out`, probed in the mktemp folder:
|
||||
- refused, nothing written: a non-empty folder, a symbolic link (with and without a
|
||||
trailing `/`), a file, `x/../fresh`, a missing parent, an empty path;
|
||||
- written: a fresh relative path through `INIT_CWD` (18 files), and a fresh folder under a
|
||||
parent link inside the scratchpad (the parent is resolved as documented);
|
||||
- refused: a second run on the same destination ("not empty");
|
||||
- `diff -r` against the seed: `legacy-import` identical, `draft` identical except
|
||||
`review/app-main.svg`, the declared trigger names.
|
||||
- R5.3 carry-overs: a seed walk that fails is now an error, and the claim refuses ("the
|
||||
seed folder cannot be read"). An existing destination folder is identity-checked
|
||||
(device, inode) against every folder of the seed. Checked by reading only: a second
|
||||
mount path needs a bind mount (root), and a symlink to the repository was not used,
|
||||
because the brief forbids `--out` into the repository.
|
||||
- `ops.rs` cite: the `HashSet` keeps `before.includes(id)` semantics (string equality only).
|
||||
- No secret, token, auth URL or private key in the diff (the `.claude` audit texts only cite
|
||||
the semgrep command). No SQL is built in the diff. stdout and stderr carry paths, counts
|
||||
and refusals only. The only tracked env file is `.env.example`.
|
||||
|
||||
## Residual risks already declared (RUST.md §11.13, §11.15)
|
||||
|
||||
Checked and accurate, apart from the §11.15 sentence corrected by finding 1:
|
||||
- the emoji refusal of the alignment;
|
||||
- the single-user TOCTOU of `--out`, mitigated by `create_new`;
|
||||
- records memory and the bound-walk time;
|
||||
- the frozen `horse.svg`.
|
||||
@@ -0,0 +1,13 @@
|
||||
# Verification: Rust R5.4 review files (2026-10-23)
|
||||
|
||||
Contract `.claude/tasks/contracts/2026-10-23-rust-r5-review.md`. Fresh verifier, report-only (persisted by the orchestrator).
|
||||
|
||||
## Round 1 on 0a79c52: verifier ECARTS(2), security PASS (1 MEDIUM, 4 LOW)
|
||||
|
||||
- Verifier: C1, C4 to C9 MET (30 tests, `--check` 0/12 + 1 declared, `--out` ×2 identical, anatomy 143, floor clean; 140 targeted + 10,500 random changesets). C2 NOT MET: `format_markdown` reproduces only Prettier's table alignment; authored Markdown syntax or whitespace (`*`, `_`, double spaces, tabs, line breaks in list items, stray backticks, `\|`) is rewritten by Prettier and not by the port (931 accepted / 909 differing in formatted Markdown on a special-character alphabet, 0 on a safe alphabet); a row with an extra cell lost a cell silently. C3 NOT MET: a CR in a label gives an undeclared tree difference (xmldom raw CR vs core ` `). Notes: emoji refusal set wider than emoji; tree fallback ignores prolog/comments/PIs; stale test comment; quadratic renderers.
|
||||
- Security (`.claude/audits/2026-10-23-rust-r5-review-security.md`): MEDIUM tree fallback too wide (stylesheet PI, comments, encoding change accepted as "declared"); LOW XML-invalid characters reach the drawings unchecked; LOW Markdown structure injection (rationale forging a section, CR, `\|`); LOW quadratic renderers; LOW test allow (context).
|
||||
- Fixed: Markdown guard (`UnsupportedMarkdown`) built from a TypeScript sweep `oracles/r5/markdown-sweep.json` (220 probes: 187 identical, 33 refused, 0 silent; equine-safe probes never refused; `**strong**` spans kept as the import report needs), single-line check of list-item fields, table cell-count check; strict drawing comparison (committed file must rebuild from its tree, rule must change it, produced equals rebuilt-after-rule byte for byte; attack test); drawings refuse XML-invalid characters and CR; renderers indexed; stale comment; docs §11.14-11.15 (guard rules, declared table, emoji breadth, residual risks).
|
||||
|
||||
## Round 2
|
||||
|
||||
Fresh verifier and security on the fixed commit: below.
|
||||
@@ -3151,8 +3151,41 @@ existing citations is hashed; the stale test comment fixed; the §11.12 nits (th
|
||||
covers a later removal; the 200,000-value row; the line count at the close; the
|
||||
set-without-value case counted among the schema refusals).
|
||||
|
||||
Tests: `cargo test -p faunosteo-core --features curation` (review 6, draft 16, importer 5,
|
||||
binary 3, formatting 6, plus the crate's suites); anatomy 143 (the generator added).
|
||||
Markdown beyond the table alignment (found by the first R5.4 gates: the fresh verifier's
|
||||
differential of 10,500 changesets showed that Prettier also rewrites Markdown in authored text,
|
||||
the security gate that it could break the reports' structure). The port reproduces only the
|
||||
table alignment, so it refuses, with the closed kind `UnsupportedMarkdown`, every text Prettier
|
||||
would rewrite, never writing a report that differs silently. The rules come from a sweep written
|
||||
by the TypeScript (`oracles/r5/markdown-sweep.json`, generator
|
||||
`packages/anatomy/tests/r5.markdown-sweep-oracles.test.ts`: 31 characters in six positions,
|
||||
whitespace patterns and Markdown constructs, each placed in a title, a rationale, a question, a
|
||||
citation and a changed label; 220 probes): outside code spans, `*` outside a well-formed
|
||||
`**strong**` span, an underscore not between two letters or digits, a backslash before a space or
|
||||
at the end or an authored one before the cell escape `\|`, two spaces in a row, a tab, a carriage
|
||||
return, an odd number of backticks, an empty code span or one with an outer space; a table row
|
||||
whose cell count differs from its header; a control character (a line break included) in a
|
||||
title, rationale, question, citation, URL, DOI or vet-flag value written in a list item. Result
|
||||
(test `every_probe_is_formatted_as_the_typescript_or_refused`): 187 probes byte for byte with the
|
||||
TypeScript, 33 refused, 0 silent difference; a probe made only of what the equine reports use is
|
||||
never refused. The equine reports and the R5.2 import report pass the guard unchanged.
|
||||
|
||||
| Input | TypeScript | Rust |
|
||||
|---|---|---|
|
||||
| authored text with Markdown Prettier rewrites (`*x*`, `_x`, `a b`, a tab, a line break in a list item, `` `` ``, `a\|b`…) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines; a row with an extra cell) | refused (`UnsupportedMarkdown`) |
|
||||
| a pictograph in a table cell (an emoji, and ©, ®, ™, ✓, ★, ⚠ and the other characters `text::is_emoji_candidate` lists) | measured with Prettier's tables | refused (`UnsupportedWidth`, R5.2): the set is wider than the emoji, the refusal stays loud |
|
||||
| a label, id or path holding a code point XML 1.0 refuses, or a carriage return, in a review drawing | written (a non-well-formed file; a raw CR read back as a line feed) | refused: the drawing is not written |
|
||||
|
||||
The drawing comparison hides nothing (security gate, MEDIUM): the rule fallback of `--check`
|
||||
requires the committed file to rebuild byte for byte from its tree (so no comment, processing
|
||||
instruction, encoding or entity form can escape the comparison), the rule to change that tree,
|
||||
and the rebuilt text after the rule to equal the produced one byte for byte; `declared:` is
|
||||
printed only then (test `the_rule_fallback_refuses_what_the_tree_does_not_carry`: a stylesheet
|
||||
processing instruction, comments, a trailing comment and another declared encoding refused).
|
||||
The renderers are indexed (citation counts in one walk, previous markers and kept ids hashed,
|
||||
subjects grouped per change).
|
||||
|
||||
Tests: `cargo test -p faunosteo-core --features curation` (review 8, Markdown 1, draft 16,
|
||||
importer 5, binary 3, formatting 6, plus the crate's suites); anatomy 144 (two generators added).
|
||||
|
||||
### 11.15 Checkpoint R5.4
|
||||
|
||||
@@ -3160,8 +3193,12 @@ Graphic differences for the owner's validation: none in shapes, colours, positio
|
||||
one text difference in `app-main.svg` (the trigger's English name and tooltip), already decided
|
||||
at R3 and identical to the editor canvas.
|
||||
|
||||
Residual risks: the Markdown alignment refuses an emoji (declared since R5.2); the comparison of
|
||||
the application's drawings accepts exactly the trigger-name rule and nothing else; the frozen
|
||||
Residual risks: the reports refuse what Prettier would rewrite (emphasis, escapes, whitespace
|
||||
runs, line breaks in list items) and the pictographs the width table does not reproduce: a
|
||||
future changeset using them stops the pipeline, visibly, until the port grows a Markdown printer
|
||||
or the text changes; raw HTML and links in authored text are kept as the TypeScript kept them
|
||||
(GFM renders them; Gitea strips raw HTML); the comparison of the application's drawings accepts
|
||||
exactly the trigger-name rule, byte for byte; the frozen
|
||||
`horse.svg` is read, never regenerated, so a future change of the drawing needs its own
|
||||
decision; the residual risks of §11.13 stand.
|
||||
|
||||
|
||||
Reference in new issue
Block a user