fix(anatomy): R5.4 records charged at each record, review drawings bounded before rendering
This commit is contained in:
1 parent
5880dfa959
commit
8bdb079f70
10 files changed
+357
-75
No files matched your search
@@ -0,0 +1,148 @@
|
||||
# Security gate R5.4, round 8 (fresh): 2026-10-23
|
||||
|
||||
Scope: `git diff c6e0d85..HEAD -- crates/core packages/anatomy/tests docs/development/RUST.md`,
|
||||
HEAD 5880dfa, branch `feature/rust-r5-pipeline-core`. Mode: audit (REPORT path given; this file
|
||||
is the only write). Blind run: no prior verdict or finding list read.
|
||||
|
||||
```
|
||||
SECURITY — VERDICT: BLOCK(1)
|
||||
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
|
||||
SCOPE: 28 files
|
||||
BLOCKING:
|
||||
1. [HIGH] resource-bound/records-checked-after-op — crates/core/src/curation/draft/apply.rs:140,145 (sites: apply.rs:66-96 mark_to_confirm, ops.rs:126-168 set_subtree, ops.rs:417-452 apply_cite, records.rs:61-76 Recorder::record) — the document-values and records-weight bounds only run after a whole op (and after the B/C marking); inside one op every record clones the change texts and the op value, once per item, so about 1 MB of changeset allocates GBs before any bound runs (measured: 711 KB → 1.37 GB RSS; 1.1 MB → abort at a 3 GB cap; 646 KB → 2.0 GB; 494 KB → 2.1 GB). RUST.md:3244-3245 says "tens of MB at most", which is false — hint: count the weight as each record is pushed (Recorder::record and mark_to_confirm), refuse before cloning; keep a per-op values budget for subtree writes (set/cite); or share ChangeSummary/Op (Arc) instead of cloning them; include CiteOp in record_weight; correct the RUST.md residual
|
||||
REPORTED (non-blocking):
|
||||
- [MEDIUM] resource-bound/review-drawings-per-view (declared residual, now measured) — crates/core/src/curation/draft/pipeline.rs:204-226
|
||||
- [LOW] record_weight ignores the op cloned into each record for cite/rename/remove/move — crates/core/src/curation/draft/records.rs:125-128
|
||||
- [LOW] TriggerNames::apply linear view search per trigger element (views × triggers, --check only) — crates/core/src/curation/draft/svg_compare.rs:36-50
|
||||
- [LOW] new test-file lint allows `#![allow(clippy::disallowed_methods, …)]`, test crates only, same convention as 20 earlier test files, not a security-scanner suppression — crates/core/tests/curation_markdown.rs:6, crates/core/tests/curation_review.rs:6
|
||||
- [LOW] pre-existing (context) `#![allow(clippy::disallowed_methods)]` in the CLI — crates/core/src/bin/import_equine.rs:17
|
||||
PROOF: semgrep 162 rules on 28 files → 0 findings; checklist 15 checks → 5 findings (1 HIGH, 1 MEDIUM, 3 LOW) + 1 pre-existing LOW
|
||||
```
|
||||
|
||||
## Blocking finding 1: the records and values bounds run after the op
|
||||
|
||||
What the code claims: `apply.rs` bounds the records at 16 Mi weight units, "checked after every
|
||||
op, before the next one records anything". RUST.md §11.15 states the residual as "one op still
|
||||
records up to about three copies of an item under the 200,000-value bound before the records
|
||||
bound is checked (tens of MB at most)".
|
||||
|
||||
What happens. `Recorder::record` (records.rs:61-76) clones `self.change` (title, rationale,
|
||||
question, reference ids) and `self.op` (including a set's or an add's value) into every record.
|
||||
`assert_depth` and `assert_recorded` (apply.rs:139-140) run only once `apply_op` returns, and
|
||||
`mark_to_confirm` (apply.rs:66-96) is checked only after it has pushed one record per touched
|
||||
item (apply.rs:145). One op that reaches N items therefore allocates about N × (author text or
|
||||
value) before any bound runs. N is bounded only by the 200,000-value document cap; the size of
|
||||
the text or value is bounded only by the 16 MiB read cap.
|
||||
|
||||
Three paths, measured with a scratch probe built on `faunosteo-core` (`--features curation`,
|
||||
the real legacy import, the committed changes and references, with one or two hostile changes
|
||||
appended). Each run had `ulimit -v 3000000`:
|
||||
|
||||
| Path | changes.json | Peak RSS | Outcome |
|
||||
|---|---|---|---|
|
||||
| level B `add` of a node with 10,000 children, rationale 128 KiB → `mark_to_confirm` pushes 10,001 records, each cloning the rationale | 711 KB | 1,372,972 KB | refused afterwards ("records weigh more") |
|
||||
| same, rationale 512 KiB (about 5 GB needed) | 1.1 MB | 2,996,672 KB | `memory allocation of 524288 bytes failed`, SIGABRT |
|
||||
| level A add of 10,000 children, then `set` `description` (64 KiB value) with `scope: subtree` → 10,000 writes plus 10,000 records holding the value twice | 646 KB | 2,027,252 KB | refused afterwards |
|
||||
| add of 2,000 children, then `cite` 4,000 declared references with `scope: subtree` (references.json 150 KB) → 8 M values written before `assert_depth` | 344 KB | 2,112,620 KB | refused afterwards ("more than 200000 values") |
|
||||
|
||||
The cost grows linearly in N × size, so a changeset of a few MB asks for tens to hundreds of
|
||||
GB. The documented "tens of MB at most" is wrong, and the `mark_to_confirm` path is declared
|
||||
nowhere. Impact is availability only: a hostile or careless `changes.json` run through
|
||||
`pnpm r5:import-equine --check|--out`, or through the test suites, exhausts the memory of the
|
||||
developer machine. The OOM killer may then take down other processes, such as the Docker
|
||||
Supabase stack. No integrity or confidentiality impact. It is blocking because it breaks a
|
||||
bound that both the code and the threat model of this gate require, and because the
|
||||
documented residual misstates it.
|
||||
|
||||
Fix direction:
|
||||
- Enforce the budget at push time: route `mark_to_confirm` through the same accounting, and
|
||||
make `Recorder::record` refuse once the running weight would pass the bound.
|
||||
- Bound the per-op writes of subtree `set` and `cite` (items × value size, or a running values
|
||||
budget) before writing.
|
||||
- Alternatively, stop cloning: share `ChangeSummary` and `Op` across records (`Arc`), so a
|
||||
record weighs only its own before and after.
|
||||
- Count the cite op's `reference_ids` in `record_weight`.
|
||||
- Correct RUST.md §11.15 and add a test per path (marking, subtree set, subtree cite) that
|
||||
fails before any GB-scale allocation.
|
||||
|
||||
Probe source: `/tmp/claude-1000/-home-bchanot-Documents-faunosteo/9b0edf59-ccd2-4662-b90c-e473dd78028e/scratchpad/sec8/probe/src/main.rs`
|
||||
(modes `add <rationale bytes> <children>`, `value2 <value bytes> <children>`,
|
||||
`cite <references> <children>`).
|
||||
|
||||
## Reported findings
|
||||
|
||||
MEDIUM, review drawings per view (declared in RUST.md §11.15, now measured). Each view of the
|
||||
curated document yields `review/annotations-<view>.svg` and `review/app-<view>.svg`. Each file
|
||||
holds a full copy of the cleaned drawing, and all files sit in memory before the CLI writes
|
||||
them. Level A adds of a detail view plus its trigger: 500 views → 1,011 files, 138.8 MB,
|
||||
176 MB RSS; 2,000 views → 4,011 files, 547 MB, 638 MB RSS. That is about 273 KB per view, so
|
||||
about 1.3 GB at the records bound (about 4,800 views), consistent with the declared "about
|
||||
2 GB" estimate. The multiplier is the drawing's size. The frozen `horse.svg` is 130 KB, but
|
||||
the core reader admits up to 2 MiB, so a replaced drawing would scale the same views to about
|
||||
19 GB. Not blocking because it is declared and the estimate holds. A view-count or output-bytes
|
||||
bound would close it.
|
||||
|
||||
LOW findings:
|
||||
- `record_weight` counts the op only for `set` and `add` (records.rs:125-128), but every record
|
||||
clones the op. A cite's `reference_ids` list is copied uncounted into each record, so the
|
||||
weight under-counts by about a third on that path.
|
||||
- `TriggerNames::apply` searches the view list linearly for every element carrying
|
||||
`data-anno-opens-view` (views × triggers). This runs in `--check` only, on committed files,
|
||||
in seconds at the bounds.
|
||||
- The two new test files carry the crate's usual test allow (`disallowed_methods`, `unwrap`,
|
||||
`expect`) for fixture reads. They are test crates only and do not suppress a security scanner.
|
||||
- Context, outside the diff: `clippy.toml`'s `disallowed-methods` bans `fs::read`,
|
||||
`fs::write`, `File::open|create` and `env::var`. It does not ban `OpenOptions::open`,
|
||||
`fs::create_dir`, `fs::read_dir`, `fs::remove_*` or `fs::rename`, so "the core has no I/O"
|
||||
rests on review for those. A grep of `crates/core/src/curation` finds no `std::fs`,
|
||||
`std::env`, `std::process` or clock use today.
|
||||
|
||||
## What was checked and held
|
||||
|
||||
- Markdown guard, differential against Prettier 3.8.3: `report_guard::format_report` against
|
||||
the repository's Prettier (`parser: markdown`, the repository options), on raw reports in the
|
||||
generator's three line forms: the list item of `change_line`, the 8-column CHANGELOG row via
|
||||
`cell()`, and the bibliography line. 16,395 cases (hand-picked cell-start and line constructs,
|
||||
CJK, Hangul, Arabic with harakat, Devanagari and Thai marks, URLs with `_` and `&name=`,
|
||||
`{{ }}`, wiki and strong forms, plus three seeded random sets). 4,696 were admitted. The only
|
||||
3 differences were wiki links opened in one list-item field and closed in another across code
|
||||
spans. `format_report` was called directly there; in the pipeline,
|
||||
`review_report::assert_single_line` refuses `[[`/`]]` in every list-item field first. Table
|
||||
cells parse on their own and showed no difference. No silent difference is reachable in
|
||||
this sample.
|
||||
- Structure forging: line breaks never reach a line start. List-item fields refuse control
|
||||
characters, and `cell()` maps `\n` to a space. `\\|` (an authored `\|` after escaping) is
|
||||
refused. Code spans are limited to `[A-Za-z0-9_:/. -]`, authored backticks are refused, and
|
||||
field segments are limited to `[A-Za-z0-9_-]`. Ids written by add and rename must be stable
|
||||
ids at the op.
|
||||
- SVG: hostile annotation and view names (`<a href>`, `&`, `A`, `]]>`, `<!-- -->`,
|
||||
`<?pi?>`, quotes, U+0085, U+2028, NBSP, tab, newline) and a path `d` holding
|
||||
`" onload="x` produce four well-formed drawings (xml.dom.minidom). The values stay data,
|
||||
with no `onload` attribute and no `a` or `script` element. Non-XML characters and CR are
|
||||
refused (`drawing_text`). The cleaned drawing goes through the core reader first: 2 MiB,
|
||||
20,000 elements, depth 64, DOCTYPE and ENTITY refused. `compare_drawing` parses the committed
|
||||
file with the capped reader before roxmltree and requires a byte-exact rebuild.
|
||||
- CLI file safety: `--check` passes at HEAD (legacy-import 6/0, draft 12/0, app-main declared).
|
||||
`--out` refusals were run in a scratch folder: empty path, symlink, symlink with a trailing
|
||||
`/`, non-empty folder, `..`, a destination through a symlink into the seed or into `draft/`,
|
||||
a missing parent. A fresh folder gets 18 files, and a second run refuses "not empty". Writes
|
||||
are `create_new`, folders are `create_dir` and components must be `Normal`. Reads are
|
||||
canonical, confined to the seed, regular files only, capped at 16 MiB. Same-user TOCTOU is
|
||||
declared. The working tree stayed clean.
|
||||
- Quadratic scans: the guard is linear (the `&`, `<`, `[[` and URL scans work per segment).
|
||||
`underscore_pairs_later` rescans only from the few authored plain parts. The renderers are
|
||||
hashed. No input under the caps reached a measurable guard time (max 0 ms per case).
|
||||
- Checklist: no hardcoded secret, token or auth URL in the diff or the oracles. No SQL built in
|
||||
scope; `seed.rs` is unchanged since R5.3. No HTML render, endpoint, authz surface or logging
|
||||
in scope. The CLI's error messages go to the local operator only. The only tracked env file
|
||||
is `.env.example`; reading it was refused by the permission rules (rule: Bash `cat .env.example`
|
||||
denied), it was not retried another way, and it is outside the diff. Guardrails scanner
|
||||
(`--domain api,supabase`): 0 FAIL, 1 WARN, `SB-POLICY-NOT-OWNER`
|
||||
`supabase/migrations/20261008110000_anatomical_sheets.sql:1177`, outside the diff and dropped
|
||||
per the diff-scope rule, plus 1 MANUAL.
|
||||
- Anti-gaming: no `nosemgrep`, `nosec`, `eslint-disable` or `unsafe` added. The only new allows
|
||||
are the two test-file lint allows listed above.
|
||||
|
||||
Semgrep note: the three packs carry few Rust rules (162 rules loaded on 28 files, 0 findings),
|
||||
so semgrep alone says little about this diff. The verdict rests on the checklist and the
|
||||
probes.
|
||||
Reference in new issue
Block a user