docs(docs): R5.4 third gate round declared (allowlist guard, empirical guarantee), audits

This commit is contained in:
bchanot committed 2026-10-23 12:39:18 +02:00
1 parent 124d954a0d
commit 6d4fdd65b2
3 files changed
+231 -34

No files matched your search

@@ -0,0 +1,182 @@
# Security gate (fresh, round 3): R5.4 review files, `c6e0d85..077d91b`
Branch `feature/rust-r5-pipeline-core`, HEAD `077d91b`, attention on `df82424`. Blind run: no
earlier verdict or finding list was read.
```
SECURITY — VERDICT: BLOCK(2)
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
SCOPE: 34 files
BLOCKING:
1. [HIGH] markdown-guard/strong-span — crates/core/src/curation/markdown.rs:79-106 (outside_strong), :157-163 — the `**strong**` model accepts pairings Prettier reads or escapes differently: a report differs from the TypeScript+Prettier output with no refusal — hint: treat `**` as a text boundary (refuse `_` or `\` next to a delimiter, a `*` run longer than 2, strong content starting or ending with a non-ASCII-space whitespace, Cf or Cc character); add these classes and a seeded random set to the TS sweep
2. [HIGH] markdown-guard/underscore-before-autolink-literal — crates/core/src/curation/markdown.rs:157-163, :109-145 — `_` between a letter and `http://`, `https://` or `www.` passes the neighbour test, Prettier escapes it (`Voir_https://…` → `Voir\_https://…`): silent difference — hint: refuse `_` directly before or after a bare URL literal (`http://`, `https://`, `www.`), add the probe to the sweep
REPORTED (non-blocking):
- [LOW] field-segment/nul-mid-segment — crates/core/src/curation/draft/fields.rs:104
- [LOW] markdown-guard/trailing-space-latent — crates/core/src/curation/markdown.rs:169
- [LOW] anti-gaming/test-allow (context) — crates/core/tests/curation_markdown.rs:6, crates/core/tests/curation_review.rs:6
PROOF: semgrep 128 rules on 34 files → 0 findings; checklist 9 checks → 5 findings
```
## Method
- semgrep 1.168.0, `EIO_BACKEND=posix`, the three pinned packs, on the 34 files of
`git diff --name-only c6e0d85..HEAD` that exist: 128 rules ran (41 multilang on 34 files, 84
TypeScript on 2, 3 JSON on 2), 0 findings, 0 errors. The packs carry no Rust rule: the Rust
product files were read whole and probed by hand.
- Differential probing, all under the fresh
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/probe.x3u0rG/`:
a throwaway binary linking `faunosteo-core` (feature `curation`) calls `format_markdown` and
the real renderers; a Node script runs the repository's Prettier 3.8.3 with the options of
`packages/anatomy/src/format.ts`; a `tsx` script runs the TypeScript pipeline
(`applyChangeset`, `renderChangelog`, `renderReviewReport`, `formatMarkdown`) on the same
changesets. About 52,000 report texts compared (targeted sweeps plus two seeded random
fuzzes), then 9 changesets end to end through both pipelines.
- `import-equine --check` rebuilt in the scratchpad target dir and run (read-only). `--out` was
not run; its write path was read. `git status` clean after every run.
- Test suites `curation_draft` (16), `curation_markdown` (1), `curation_review` (9): pass.
## Blocking findings
### 1. [HIGH] The strong-span model of the Markdown guard lets silent differences through
`outside_strong` (markdown.rs:79-106) removes every `**…**` pair it judges well formed and
`plain_rewrite` (:147-173) then checks the text left. Four input classes pass the guard while
Prettier 3.8.3 (remark-parse 7 underneath) rewrites them:
| Class | Example authored text | Rust writes | TypeScript + Prettier writes |
|---|---|---|---|
| content made only of whitespace, format or control characters other than U+0020 | `** **` (also U+1680, U+2000-200A, U+2028/2029, U+202F, U+205F, U+3000, U+FEFF, U+200B-200D, U+2060, U+00AD, U+180E, C0/C1 controls) | `** **` | `\*\* \*\*` |
| a backslash right before either delimiter | `\**a**`, `x\**a**`, `**1&é.\**-` | as written | `\*\*a\*\*` |
| touching pairs | `**a****b**` | as written | `**a\*\***b\*\*` |
| underscore next to a delimiter, judged on the text with the delimiters removed (`x_a`) | `x_**a**`, `**a**_x` | as written | `x\_**a**`, `**a**\_x` |
Every channel that writes plain text is affected: title, rationale, question, citation, URL,
DOI, vet-flag values in the review list items, and label, description and value texts in the
CHANGELOG cells, where the table widths change as well.
End-to-end proof on the mini sheet of `oracles/r5/markdown-sweep.json`
(`probe.x3u0rG/e2e_in.json`, results `e2e_out.json` for Rust and `ts_out.json` for the
TypeScript, comparator `js/cmp_e2e.py`): the raw texts are identical in all 8 changesets. In 7
of them Rust accepts a final CHANGELOG different from the TypeScript's, and in 6 a different
REVIEW-REPORT. Excerpt (changeset 2, rationale `Raison x_**important**.`):
```
rust: …confirmation requise : oui). Raison x_**important**. Objets…
ts: …confirmation requise : oui). Raison x\_**important**. Objets…
```
Impact: no structure is forged. The Rust output is the authored raw text, which equals the
TypeScript raw text: no line, item or cell is created. Rendered meaning is mostly unchanged.
The broken property is the owner's requirement for this tool: "byte-identical or loudly
refused". The guarantee written in RUST.md §11.14 (line 3158 "never writing a report that
differs silently", line 3174 "0 silent difference") holds only for the 247 sweep probes, so
that declaration is wrong. Today `--check` would catch such a change against the committed
TypeScript output. After the switch, the report is written and committed silently, and a later
`pnpm format` (`prettier --write .`, the draft folder is not in `.prettierignore`) rewrites it,
so `--check` then fails far from the cause.
Why HIGH: this guard is the only thing that enforces an explicit owner integrity requirement,
the documented guarantee is false, and inputs reach it through ordinary authoring. Beyond that
requirement there is no exploit.
Fix direction: treat each `**` as the edge of a text node. Refuse `_` or `\` next to a
delimiter, a run of three or more `*`, and strong content whose first or last character is a
Unicode whitespace, a format (Cf) or a control (Cc) character. Then grow the TypeScript sweep
with these four classes plus a seeded random combination set. The random fuzz here found them
within a minute: the sweep's hand-picked probe list is the weak point.
### 2. [HIGH] An underscore glued to a bare URL passes the guard and is escaped by Prettier
The underscore rule (markdown.rs:157-163) accepts `_` between two ASCII letters or digits.
Before `http://`, `https://` or `www.`, the next letter belongs to a link that remark-parse 7
opens at the URL. The underscore then ends a text node, and Prettier escapes it.
`rewritten_construct` (:109-145) has no rule for bare URLs.
| Authored text | Rust writes | TypeScript + Prettier writes |
|---|---|---|
| `Voir_https://example.org/x` (title) | `Voir_https://example.org/x` | `Voir\_https://example.org/x` |
| `Voir_www.example.org` (citation) | `Voir_www.example.org` | `Voir\_www.example.org` |
| `&b_http://x.y1`, `b_www.a.b>#` (random fuzz) | as written | `\_` inserted |
End to end: changesets 1 and 6 of `e2e_in.json`. The raw texts are identical, both reports
differ, Rust raises no refusal. `mailto:` did not trigger the difference. Impact and severity
are those of finding 1.
Fix direction: refuse `_` directly before or after `http://`, `https://` or `www.`, and add
the probe to the sweep.
## Reported, non-blocking
- [LOW] field-segment/nul-mid-segment, `crates/core/src/curation/draft/fields.rs:104`.
`c.is_control() && c != '\0'` exempts NUL everywhere, while line 110 refuses it only as the
first character of a segment. The ops `set name.f\0r` then `set name {fr,en}` give a valid
final document and leave a NUL in the record subject and in both reports' code spans. This
was verified end to end: the TypeScript writes the same bytes and Prettier keeps them, so
there is no divergence. It contradicts the comment at line 101 and the RUST.md §11.14 row at
line 3182 ("a field segment holding a control character … refused"). Fix: refuse `\0`
anywhere in a segment.
- [LOW] markdown-guard/trailing-space-latent, `crates/core/src/curation/markdown.rs:169`. The
guard never refuses a space that ends a line, which Prettier strips. Only a question reaches a
line end (`change_line`, review_report.rs:79). Today it is refused by accident: section 10
(`professional_decisions`, :319) reprints every question mid-line, where the trailing space
becomes a double space (end-to-end changeset 0). A layout change of section 10 would reopen
it. Fix: refuse trailing whitespace at the end of a guarded line.
- [LOW] anti-gaming/test-allow (context). The new
`#![allow(clippy::disallowed_methods, clippy::unwrap_used, clippy::expect_used)]` in
`crates/core/tests/curation_markdown.rs:6` and `crates/core/tests/curation_review.rs:6` is a
test-only allow for reading oracle files, the same pattern as the pre-existing
`crates/core/tests/curation_draft.rs:7`. It is not a security suppression. No `nosemgrep`,
`nosec`, `eslint-disable`, `ts-ignore` or `noqa` was added.
## Checklist (9 checks)
| Check | Result |
|---|---|
| Hardcoded secret, token, key, auth URL | none in added lines (audit texts only name the rulesets) |
| SQL built by concatenation | no SQL change in the diff |
| Unsanitized render of authored text | Markdown: findings 1, 2, LOW trailing space; SVG: clean (below) |
| Sensitive endpoint without authz | none (offline dev tool) |
| Stack trace, internal path, DB error to a user | none (CLI messages to the operator) |
| Secret or PII in a log | none (stdout and stderr carry file names, counts, refusals) |
| Tracked `.env` or credential file | only `.env.example` tracked, not in the diff |
| Guardrails scanner (`--domain api,supabase`) | 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER`, a migration outside the diff, dropped), 1 MANUAL |
| New suppressions (anti-gaming) | LOW context only |
## Verified clean
- Ids at the op (`df82424`): `add` checks the item's key and every nested `children` /
`categories` id, `rename` checks the new id (a region code for a region). Record targets
resolve to existing, validated ids, change ids match `CHG-\d{3,}`, and reference ids come from
the strictly parsed registry or a validated add. I found no remaining path for a line break or
a backtick into a code span.
- Code spans: 739 probes (NBSP, U+2000, U+3000, U+FEFF, zero-width characters, controls and NUL
at the edges and inside, `\`, `|`, `\|` in table code), 0 divergence.
- Display width: about 110 pictograph, combining, format and wide characters in cells. Each one
is either equal to Prettier or refused (`UnsupportedWidth`).
- List items: `assert_single_line` covers the title, rationale, question, citation, URL, DOI and
vet values. CHANGELOG cells turn `\n` into a space and escape `|`, and the guard refuses CR
and TAB anywhere.
- Drawings: the serialiser escapes `& < > "` and `\t \n \r` in attributes and `& < > \r` in
text, identical to xmldom 0.9.12 on 11 probe values. `first_unwritable` refuses non-XML code
points and CR. The overlay prints `Infinity`, `-Infinity` and `NaN` as `String(number)`.
- `compare_drawing`: `parse_svg` (prescan: depth 64, 20,000 elements) runs before any roxmltree
parse. The committed file must rebuild to itself byte for byte, the rule must change the tree,
and the ruled text must equal the produced one byte for byte. Only `review/app-*` may use the
rule; the overlays compare as bytes only. I found no path that accepts a difference other than
the declared trigger-name rule. A trigger whose own English name equals the view's French name
is refused (a false refusal, and a loud one).
- Panics and resources: slicing only at ASCII `**` offsets, `get` elsewhere. The two indexings
(`style()[0]`, `split_field(..)?[0]`) cannot be empty. The guard is linear (splits into
pieces, never suffixes) and the renderers are indexed. Record memory stays as declared in
§11.13.
- `import-equine --check`: `legacy-import: 6 files, 0 differences`; `draft: declared:
review/app-main.svg (byte-identical once the R3 trigger-name rule is applied)`; `draft: 12
files, 0 differences`. `--out` (read): `create_dir` / `create_new` only, `Normal` components
only, file names built from validated view ids, seed identity checks as declared.
## Reproduction
`/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/probe.x3u0rG/`:
`rs/` (probe crate, binaries `gateprobe` and `e2e`), `js/fmt.mjs` (Prettier),
`js/ts_e2e.mts` (TypeScript pipeline), `js/gen*.py` (probe generators), `e2e_in.json`,
`e2e_out.json`, `ts_out.json`, `cmp1.txt`, `cmp2.txt`, `cmp4.txt` (divergence lists).