docs(docs): R5.4 first gate round declared (Markdown guard, strict comparison), audits

This commit is contained in:
bchanot committed 2026-10-23 11:51:42 +02:00
1 parent 94f8b35105
commit 68dec25e5e
3 files changed
+285 -4

No files matched your search

@@ -0,0 +1,231 @@
```
SECURITY — VERDICT: PASS
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 25 files
BLOCKING:
(none)
REPORTED (non-blocking):
- [MEDIUM] check/declared-tree-rule-too-wide — crates/core/src/curation/draft/svg_compare.rs:104
- [LOW] check/unchecked-xml-characters — crates/core/src/curation/draft/app_review.rs:33
- [LOW] check/markdown-structure-injection — crates/core/src/curation/draft/review_report.rs:64
- [LOW] check/quadratic-renderers — crates/core/src/curation/draft/review_report.rs:223
- [LOW] anti-gaming/test-clippy-allow (context) — crates/core/tests/curation_review.rs:6
PROOF: semgrep 162 rules on 25 files → 0 findings; checklist 8 checks + anti-gaming + 5 brief probes → 5 findings
```
# R5.4 security gate (fresh), 2026-10-23
Repository `/home/bchanot/Documents/faunosteo`, branch `feature/rust-r5-pipeline-core`, HEAD
`0a79c52`, scope `git diff c6e0d85..HEAD` (25 existing files). Every changed product file was
read whole: `crates/core/src/curation/draft/{labels,changelog,review_report,drawing,overlay,
app_review,svg_compare,pipeline,stats,ops,mod}.rs`, `crates/core/src/bin/import_equine.rs`,
plus `crates/core/src/curation/markdown.rs`, `anatomy/svg/{serialize,tree}.rs` and
`anatomy/compose.rs` where the new code relies on them.
## Method
- semgrep 1.168.0 (`EIO_BACKEND=posix`), `--config p/security-audit --config p/secrets
--config p/owasp-top-ten --metrics=off --quiet --json` on the 25 files: 162 rules, 0
results, 0 errors. These packs cover Rust thinly, so the manual review carries the weight.
- Checklist (CLAUDE.md defaults): secrets, SQL concatenation, unsanitized render, authz,
error exposure, secrets in logs, tracked `.env`, guardrails scanner
(`python3 -B ~/.claude/skills/guardrails/scripts/scan.py <root> --domain api,supabase`:
0 FAIL, 1 WARN, 1 MANUAL; the WARN `SB-POLICY-NOT-OWNER` is in
`supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, outside the delta, so it
is not reported here).
- Anti-gaming: the diff was scanned for new suppressions.
- Probes, all in the scratchpad: a throwaway crate linking `faunosteo-core` (feature
`curation`) at
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/probe.DaWdIB`,
and `import-equine` built with `CARGO_TARGET_DIR` in the scratchpad. `--check` ran
read-only on the repository. `--out` ran only inside the fresh
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/out.6SamF5`.
Nothing was deleted, and `git status` stayed clean.
## Findings
### 1. [MEDIUM] The tree fallback hides more than the declared trigger-name rule
`crates/core/src/curation/draft/svg_compare.rs:104-123` (`compare_drawing`), used by
`crates/core/src/bin/import_equine.rs:196-215` for `review/app-*.svg`.
When the bytes differ, both files are parsed with `parse_svg`. That function builds the tree
of the root element only, and it keeps only elements and text
(`anatomy/svg/tree.rs:155, 182-193`). Everything the tree does not carry is therefore never
compared:
- the prolog: XML declaration and its `encoding`, processing instructions, comments;
- comments and processing instructions inside the root and after it;
- entity and character-reference forms, CDATA against escaped text, attribute quoting,
`<g/>` against `<g></g>`;
- attribute-value whitespace that the parser normalises.
Measured on the real committed `draft/review/app-main.svg` against the produced one, with
the rule applied:
| Committed file changed by | `compare_drawing` |
|---|---|
| nothing | `Ok(TreeAfterRule)` |
| `<?xml-stylesheet type="text/css" href="https://attacker.invalid/hide.css"?>` and a comment in the prolog, a comment and a PI right after `<svg …>`, a trailing comment | `Ok(TreeAfterRule)` |
| also `encoding="UTF-8"` replaced by `encoding="ISO-8859-1"` | `Ok(TreeAfterRule)` |
In all three cases `--check` prints the same `declared: review/app-main.svg (trees equal
under the R3 trigger-name rule)` line it prints today and exits 0. The same holds for
`app-head.svg`, which is byte-identical today: a comment-only change would turn it into a
"declared" file. The only visible sign would be the extra line.
Impact: the owner requirement "`--check` fails on any unauthorised difference" is not met
for the two application drawings. A stylesheet processing instruction is honoured by
browsers that open a standalone SVG: it could restyle or hide markers in the review drawing
and fetch an external URL, and `--check` would still pass. Exploiting it needs write access
to the repository. The files are dev review artifacts: the app, the bundle and the uploads
never read them (grep of `apps/app`, `vite`, `packages/supabase`). Hence MEDIUM, not
blocking.
Three declarations are materially wrong:
- `svg_compare.rs:7` says "Any other difference is reported".
- `import_equine.rs:185-186` and the printed "declared" line suggest the rule alone explains
the difference.
- RUST.md §11.15 says the comparison "accepts exactly the trigger-name rule and nothing
else". §11.13's R5.4 condition did mention "declaration, prefixes and whitespace
declared", but §11.15 does not carry that through, and it never covered processing
instructions or comments.
Fix direction: once the rule has been applied to the committed tree, require the committed
prolog to be byte-equal to the produced one. Then require
`prolog + serialize(committed tree after rule)` to equal the produced text byte for byte.
The core serializer writes the produced file, and `app-head.svg` already matches it byte for
byte. At minimum: compare the prologs and the text after the root, refuse any comment or PI
node in the committed document, and print "declared" only when the rule actually changed
the committed tree. Then correct the three declarations.
### 2. [LOW] XML-invalid characters from authored strings reach the review drawings unchecked
`crates/core/src/curation/draft/app_review.rs:33-36` calls `render_layer` directly. The R3
composition passes the layer through `checked` / `xml_safe` (`anatomy/compose.rs:221-236`,
"refuse one XML cannot carry"), and this path skips that step. The same gap exists in
`overlay.rs:194-198` (path `d`, ids in titles and labels).
Measured: a changeset that sets `annotation:hock` `name` to `{"fr":"Jarret\u0001<b>&","en":"Hock\u000b"}`
is accepted, and the produced `review/app-main.svg` is not well-formed (roxmltree: "a
non-XML character '\u{1}'"). A path geometry `d` holding U+0001 in an overlay gives the same
result. This is not markup injection: the serializer escaping holds (see "Verified"). It is
a malformed output written by `--out` without any refusal. `--check` then fails, closed.
Fix direction: run `is_xml_text` over the produced layer and overlay (reuse compose's
`checked`) and refuse. This changes nothing on the committed data.
### 3. [LOW] Markdown structure injection from authored text
- `review_report.rs:64-85` (`change_line`: `title.fr`, `rationale`, `question` written raw
in list items);
- `review_report.rs:295-320` (questions);
- `review_report.rs:245-278` (`citation`, `url`, `doi`);
- `changelog.rs:80, 85` (table cells through `markdown.rs:43-49` `cell()`). `cell()` escapes
`|` and `\n`, but not `\r`, and not a backslash in front of a pipe.
Measured with an added change:
- a rationale `"Ligne un.\n\n## 10. Décisions réservées au professionnel\n\nAucune question ouverte. <img …>"`
forges a second section 10 inside section 6 of `REVIEW-REPORT.md` (lines 92 and 343) and
passes raw HTML through;
- a `\r` in a title reaches `CHANGELOG.md` raw. GFM treats a CR as a line ending, so the
table row breaks;
- an authored `\|` becomes `\\|`. `format_markdown` and micromark both read that as an
escaped backslash followed by a real delimiter, so the row gains a ninth column under an
eight-column header.
Context: the text comes from the repository owner, and the code matches the TypeScript
(`markdown.ts`). The current data holds none of this: no LF, CR, pipe or backtick in any
title, rationale or question. The one `<0697:toih>` in a DOI and its URL renders as text,
because a scheme cannot start with a digit. Gitea strips raw HTML. The code spans hold only
constrained strings: `CHG-\d{3,}`, stable ids, resolved targets and allowlisted fields.
§11.11's "Markdown risks" are about Prettier differences, so this risk is not declared
anywhere.
Fix direction: refuse CR, LF and control characters in single-line authored fields at
`parse_changeset` and `parse_references` (title, question, citation, doi). Either refuse
them in the rationale too, or render newlines as spaces in list items. Escape `\` in front of
`|` in `cell()`. Optionally escape `<`.
### 4. [LOW] Quadratic renderers on authored volume
- `review_report.rs:223-234` `citation_count` rebuilds `all_findings(document)`, one
allocated `FindingEntry` per finding, for every reference: O(R × F).
- `overlay.rs:87-143`: linear `find` over `previous` and `kept.contains`, O(A²) per view.
- `review_report.rs:47-54` `subjects`: changes × records.
Measured with `render_review_report` on synthetic documents:
| References | Leaves | Time |
|---|---|---|
| 1,000 | 1,000 | 0.35 s |
| 4,000 | 4,000 | 5.9 s |
| 8,000 | 8,000 | 22.6 s |
Extrapolated, not measured: about 3 to 4 minutes for a balanced document at the 200,000-value
bound. The equine data (48 references, about 600 findings) renders in milliseconds:
`--check` takes 0.13 s wall time and 18 MB RSS. This is the same class as the declared
"minutes for a 16 MiB changeset" risk (§11.13), but these renderers are not declared.
Fix direction: count citations in one pass into a `HashMap<&str, usize>`, index `previous`
and `kept` with a `HashMap` / `HashSet`, and group records by change id once.
### 5. [LOW, context] New clippy allow in a test crate
`crates/core/tests/curation_review.rs:6`
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]`. This is an
integration test, outside the library, with the same header as all 33 files of
`crates/core/tests`. It is a lint allow, not a security-scanner suppression. No new
`nosemgrep`, `nosec`, `eslint-disable`, `ts-ignore` or `noqa` appears in the diff. Not
blocking.
## Verified, no finding
- SVG markup injection: element and attribute names are literals. Values go through the
core serializer, which escapes `& < > "` and `\t \n \r` in attributes and `& < > \r` in
text. Probe: an annotation name `J"><script>alert(1)</script>&amp;`, an English name
`]]><!--x-->`, a view name `T</title><script>y</script>` and an overlay path
`d="M0 0…\"/><script>x</script>"`. The output is well-formed (apart from finding 2's
control character), with 0 `script` elements and every payload escaped.
- File names built from view ids: views are `stable_id` (`[a-z0-9][a-z0-9_-]{0,63}`),
enforced by the typed parse in `pipeline.rs:206` before any write. `write_exclusive`
also refuses any component that is not `Normal`. The `--check` reads stay canonical
inside the seed.
- Panics on authored input: `compact_value` slices only past 120 UTF-16 units and decodes
lossily. The prolog slice ends at roxmltree's root start (`<`, a char boundary).
`style()[0]` is a static slice. Polygon centroid: at least 3 points (typed), NaN otherwise,
never a panic. No `unwrap`, `expect` or unchecked index on data in the product files.
Recursion in `first_difference` and `TriggerNames::apply` is bounded by the reader's
depth cap (64).
- `--check` is read-only: no file in `legacy-import/` or `draft/` changed and `git status`
stayed clean. Output: `legacy-import: 6 files, 0 differences`, `draft: declared:
review/app-main.svg …`, `draft: 12 files, 0 differences`. A file that is missing, not
produced or not UTF-8 fails. A generated folder must canonicalise to the seed's child.
Files other than `review/app-*` have no rule and must be byte-equal. `horse.svg` is
compared with itself, but its sha256 drives `assets.json`, `document.json` and the seed
SQL, so a change to it fails the check.
- `--out`, probed in the mktemp folder:
- refused, nothing written: a non-empty folder, a symbolic link (with and without a
trailing `/`), a file, `x/../fresh`, a missing parent, an empty path;
- written: a fresh relative path through `INIT_CWD` (18 files), and a fresh folder under a
parent link inside the scratchpad (the parent is resolved as documented);
- refused: a second run on the same destination ("not empty");
- `diff -r` against the seed: `legacy-import` identical, `draft` identical except
`review/app-main.svg`, the declared trigger names.
- R5.3 carry-overs: a seed walk that fails is now an error, and the claim refuses ("the
seed folder cannot be read"). An existing destination folder is identity-checked
(device, inode) against every folder of the seed. Checked by reading only: a second
mount path needs a bind mount (root), and a symlink to the repository was not used,
because the brief forbids `--out` into the repository.
- `ops.rs` cite: the `HashSet` keeps `before.includes(id)` semantics (string equality only).
- No secret, token, auth URL or private key in the diff (the `.claude` audit texts only cite
the semgrep command). No SQL is built in the diff. stdout and stderr carry paths, counts
and refusals only. The only tracked env file is `.env.example`.
## Residual risks already declared (RUST.md §11.13, §11.15)
Checked and accurate, apart from the §11.15 sentence corrected by finding 1:
- the emoji refusal of the alignment;
- the single-user TOCTOU of `--out`, mitigated by `create_new`;
- records memory and the bound-walk time;
- the frozen `horse.svg`.
@@ -0,0 +1,13 @@
# Verification: Rust R5.4 review files (2026-10-23)
Contract `.claude/tasks/contracts/2026-10-23-rust-r5-review.md`. Fresh verifier, report-only (persisted by the orchestrator).
## Round 1 on 0a79c52: verifier ECARTS(2), security PASS (1 MEDIUM, 4 LOW)
- Verifier: C1, C4 to C9 MET (30 tests, `--check` 0/12 + 1 declared, `--out` ×2 identical, anatomy 143, floor clean; 140 targeted + 10,500 random changesets). C2 NOT MET: `format_markdown` reproduces only Prettier's table alignment; authored Markdown syntax or whitespace (`*`, `_`, double spaces, tabs, line breaks in list items, stray backticks, `\|`) is rewritten by Prettier and not by the port (931 accepted / 909 differing in formatted Markdown on a special-character alphabet, 0 on a safe alphabet); a row with an extra cell lost a cell silently. C3 NOT MET: a CR in a label gives an undeclared tree difference (xmldom raw CR vs core `&#13;`). Notes: emoji refusal set wider than emoji; tree fallback ignores prolog/comments/PIs; stale test comment; quadratic renderers.
- Security (`.claude/audits/2026-10-23-rust-r5-review-security.md`): MEDIUM tree fallback too wide (stylesheet PI, comments, encoding change accepted as "declared"); LOW XML-invalid characters reach the drawings unchecked; LOW Markdown structure injection (rationale forging a section, CR, `\|`); LOW quadratic renderers; LOW test allow (context).
- Fixed: Markdown guard (`UnsupportedMarkdown`) built from a TypeScript sweep `oracles/r5/markdown-sweep.json` (220 probes: 187 identical, 33 refused, 0 silent; equine-safe probes never refused; `**strong**` spans kept as the import report needs), single-line check of list-item fields, table cell-count check; strict drawing comparison (committed file must rebuild from its tree, rule must change it, produced equals rebuilt-after-rule byte for byte; attack test); drawings refuse XML-invalid characters and CR; renderers indexed; stale comment; docs §11.14-11.15 (guard rules, declared table, emoji breadth, residual risks).
## Round 2
Fresh verifier and security on the fixed commit: below.
+41 -4
View File
@@ -3151,8 +3151,41 @@ existing citations is hashed; the stale test comment fixed; the §11.12 nits (th
covers a later removal; the 200,000-value row; the line count at the close; the
set-without-value case counted among the schema refusals).
Tests: `cargo test -p faunosteo-core --features curation` (review 6, draft 16, importer 5,
binary 3, formatting 6, plus the crate's suites); anatomy 143 (the generator added).
Markdown beyond the table alignment (found by the first R5.4 gates: the fresh verifier's
differential of 10,500 changesets showed that Prettier also rewrites Markdown in authored text,
the security gate that it could break the reports' structure). The port reproduces only the
table alignment, so it refuses, with the closed kind `UnsupportedMarkdown`, every text Prettier
would rewrite, never writing a report that differs silently. The rules come from a sweep written
by the TypeScript (`oracles/r5/markdown-sweep.json`, generator
`packages/anatomy/tests/r5.markdown-sweep-oracles.test.ts`: 31 characters in six positions,
whitespace patterns and Markdown constructs, each placed in a title, a rationale, a question, a
citation and a changed label; 220 probes): outside code spans, `*` outside a well-formed
`**strong**` span, an underscore not between two letters or digits, a backslash before a space or
at the end or an authored one before the cell escape `\|`, two spaces in a row, a tab, a carriage
return, an odd number of backticks, an empty code span or one with an outer space; a table row
whose cell count differs from its header; a control character (a line break included) in a
title, rationale, question, citation, URL, DOI or vet-flag value written in a list item. Result
(test `every_probe_is_formatted_as_the_typescript_or_refused`): 187 probes byte for byte with the
TypeScript, 33 refused, 0 silent difference; a probe made only of what the equine reports use is
never refused. The equine reports and the R5.2 import report pass the guard unchanged.
| Input | TypeScript | Rust |
|---|---|---|
| authored text with Markdown Prettier rewrites (`*x*`, `_x`, `a b`, a tab, a line break in a list item, `` `` ``, `a\|b`…) | written, then rewritten by Prettier (escapes, `_emphasis_`, collapsed spaces, indented continuation lines; a row with an extra cell) | refused (`UnsupportedMarkdown`) |
| a pictograph in a table cell (an emoji, and ©, ®, ™, ✓, ★, ⚠ and the other characters `text::is_emoji_candidate` lists) | measured with Prettier's tables | refused (`UnsupportedWidth`, R5.2): the set is wider than the emoji, the refusal stays loud |
| a label, id or path holding a code point XML 1.0 refuses, or a carriage return, in a review drawing | written (a non-well-formed file; a raw CR read back as a line feed) | refused: the drawing is not written |
The drawing comparison hides nothing (security gate, MEDIUM): the rule fallback of `--check`
requires the committed file to rebuild byte for byte from its tree (so no comment, processing
instruction, encoding or entity form can escape the comparison), the rule to change that tree,
and the rebuilt text after the rule to equal the produced one byte for byte; `declared:` is
printed only then (test `the_rule_fallback_refuses_what_the_tree_does_not_carry`: a stylesheet
processing instruction, comments, a trailing comment and another declared encoding refused).
The renderers are indexed (citation counts in one walk, previous markers and kept ids hashed,
subjects grouped per change).
Tests: `cargo test -p faunosteo-core --features curation` (review 8, Markdown 1, draft 16,
importer 5, binary 3, formatting 6, plus the crate's suites); anatomy 144 (two generators added).
### 11.15 Checkpoint R5.4
@@ -3160,8 +3193,12 @@ Graphic differences for the owner's validation: none in shapes, colours, positio
one text difference in `app-main.svg` (the trigger's English name and tooltip), already decided
at R3 and identical to the editor canvas.
Residual risks: the Markdown alignment refuses an emoji (declared since R5.2); the comparison of
the application's drawings accepts exactly the trigger-name rule and nothing else; the frozen
Residual risks: the reports refuse what Prettier would rewrite (emphasis, escapes, whitespace
runs, line breaks in list items) and the pictographs the width table does not reproduce: a
future changeset using them stops the pipeline, visibly, until the port grows a Markdown printer
or the text changes; raw HTML and links in authored text are kept as the TypeScript kept them
(GFM renders them; Gitea strips raw HTML); the comparison of the application's drawings accepts
exactly the trigger-name rule, byte for byte; the frozen
`horse.svg` is read, never regenerated, so a future change of the drawing needs its own
decision; the residual risks of §11.13 stand.