docs(docs): R5.5 gate round 2, checkpoint, memory, TODO
This commit is contained in:
1 parent
fad85c0f92
commit
dead54612b
4 files changed
+159
-6
No files matched your search
@@ -0,0 +1,122 @@
|
||||
# Security audit: R5.5 upload tool, fix commits 56593af and fad85c0
|
||||
|
||||
- date: 2026-10-23 | branch `feature/rust-r5-pipeline-core` | scope `git diff a481af5..fad85c0`
|
||||
- context read: `crates/core/src/bin/upload_sheet_assets/{main,storage}.rs`, `crates/core/src/curation/upload.rs`,
|
||||
`scripts/{upload-sheet-assets,sheet-assets-e2e,floor-clarification-check}.sh`, `crates/core/tests/upload_tool.rs`,
|
||||
contract `.claude/tasks/contracts/2026-10-23-rust-r5-upload.md`
|
||||
- mode: audit (fresh, blind; no prior verdict used). Nothing was edited but this file.
|
||||
- not run, by rule: `upload-sheet-assets` against any real stack, `scripts/sheet-assets-e2e.sh`, any service key,
|
||||
any transfer tool, any recursive delete.
|
||||
|
||||
## What was measured
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| semgrep 1.168.0, p/security-audit + p/secrets + p/owasp-top-ten, 75 rules, 11 files (the 10 diff files + `curation/upload.rs`) | 0 findings, 0 errors |
|
||||
| guardrails `scan.py --domain api,supabase` | 0 FAIL, 1 WARN (`SB-POLICY-NOT-OWNER`, `supabase/migrations/20261008110000_anatomical_sheets.sql:1177`), outside the diff, dropped |
|
||||
| `cargo test --locked -p faunosteo-core --features upload --test upload_tool` (loopback stub, made-up key, built without `SUPABASE_SERVICE_ROLE_KEY` in the env) | 12 passed, 0 failed |
|
||||
| `floor-guard.sh develop`, then the script's three regexes applied to its output | 22 findings: 21 requalified (SUPPRESS in `.claude/audits/*.md`), 1 declared (`STUB .claude/memory/evals.md:205`), 0 remaining |
|
||||
| `/proc/sys/net/ipv{4,6}/ip_nonlocal_bind` | 0 and 0 |
|
||||
| UDP bind probe (no packet sent) on this host (`enp89s0` 192.168.1.100/24, brd 192.168.1.255) | 192.168.1.100 OK; 192.168.1.255 OK; 224.0.0.1 OK; 0.0.0.0 OK; 192.168.1.0, 192.168.1.101, 10.255.255.254 refused |
|
||||
| TCP connect to 192.168.1.255:54321 | refused by the kernel, ENETUNREACH (no SYN leaves) |
|
||||
| new suppressions in added lines (`nosemgrep`, `nosec`, `eslint-disable`, `@ts-ignore`, `noqa`, `#[allow`, `shellcheck disable`, `floor-guard: allow`) | none in code; the only hits are prose in `.claude/audits/2026-10-23-rust-r5-upload-security.md` |
|
||||
| tracked env files | `.env.example` only |
|
||||
|
||||
## Targeted review of the fixes
|
||||
|
||||
**Error classification and retry** (`storage.rs:17`, `:73-87`, `:189-214`). `ATTEMPTS = 4` gives one try and three
|
||||
retries, sleeps 1.0 s, 1.5 s, 2.0 s, so contract C6 "retried three times" holds. Retried: `Io`, `Timeout`,
|
||||
`HostNotFound`, `ConnectionFailed`, `Protocol`, and a 5xx status. Every other `ureq::Error` is final.
|
||||
`BodyExceedsLimit` maps to a final `State` refusal: the ureq 3.4.2 `LimitReader` raises it wrapped in an
|
||||
`io::Error`, and `From<io::Error>` unwraps it back (`ureq-3.4.2/src/error.rs:216-227`), so the guard does fire. The
|
||||
test `upload_tool.rs:319` proves it, with exactly one object GET and no write. A retried POST stays safe, because
|
||||
`x-upsert: false` turns a replay into a 409 handled as a duplicate and the object is then re-hashed. A retried
|
||||
confirm is server-idempotent. Secret exposure: the final branch prints only a fixed text or the numeric limit. The
|
||||
retried branch prints `error.to_string()`. For `Io`, `Timeout` and `Protocol` that text carries no header value:
|
||||
ureq-proto's `BadHeader` holds the `http::Error` text, not the value, and `BadAuthorizationHeader` has no payload
|
||||
(`ureq-proto-0.6.4/src/error.rs:45-53`). Request analysis also runs before the prelude is written
|
||||
(`client/sendreq.rs:42`), so a refused key never reaches the wire. The fix is a strict improvement: before it,
|
||||
`Http`, `BadUri`, `TooManyRedirects` and the rest were echoed and retried. One residual inaccuracy is listed as L1.
|
||||
|
||||
**DEV-address ownership** (`main.rs:112-124`). `UdpSocket::bind((ip, 0))` sends nothing and is closed on drop. It
|
||||
runs before the key is read, and `check_destination` still requires a private IPv4 equal to the DEV host
|
||||
(`upload.rs:190-200`). Unspecified and multicast addresses bind but are not private, so they stay refused. The
|
||||
directed broadcast of an attached private subnet passes, but the kernel refuses the TCP connection. The test
|
||||
`upload_tool.rs:342` discriminates: without the filter the run would reach the network (exit 3), not a
|
||||
`Destination` refusal (exit 1). Residuals: L2.
|
||||
|
||||
**Keys off every argv in the e2e script** (`sheet-assets-e2e.sh:18-32`). The key reaches the binary only through a
|
||||
prefix assignment inside the `$(...)` subshell (env, not argv). The leak check feeds the patterns to
|
||||
`grep -f <(printf ...)`: `printf` is a builtin, and grep's argv holds only `/dev/fd/N`. The keys come from the
|
||||
status JSON on stdin; the `python3 -c` argv carries only the field name. `env -u SUPABASE_SERVICE_ROLE_KEY cargo
|
||||
build` puts the variable name on argv, never its value. Dropping `env -i` lets the binary inherit the caller's env.
|
||||
This is no exposure: the tool reads only `SUPABASE_URL`, `SUPABASE_SERVICE_ROLE_KEY`, `FAUNOSTEO_DEV_HOST` and
|
||||
`INIT_CWD` (the last two are unset there), it uses `proxy(None)`, and the crate has no logger. An empty key or
|
||||
status field would put an empty pattern into `-f`, which matches every output. That fails the case: fail-safe.
|
||||
|
||||
**Launcher** (`upload-sheet-assets.sh:15-16`). Under `set -euo pipefail`, a failed build stops the script before
|
||||
the `exec`. Cargo, build scripts and proc macros no longer see the key. Only the `exec`'d binary receives it, in its
|
||||
environment, with the folder and `--check` as its only argv. Residual: L3.
|
||||
|
||||
**FLOOR declared line** (`floor-clarification-check.sh:16`, `:23-27`). `DECLARED_RE` is anchored and covers one
|
||||
kind, one file and one line, with the dots escaped and a trailing space, so `:2050` cannot match. On the branch it
|
||||
matches exactly the one line the R5.4 contract names (`2026-10-23-rust-r5-review.md:69`). Nothing else is
|
||||
requalified, `requalified` does not count it, and it is printed under `DECLARED`. The negative proof still fails
|
||||
(real SUPPRESS in `src/*.ts`, `src/*.rs` and an active contract). Residual: L4.
|
||||
|
||||
## Findings
|
||||
|
||||
No CRITICAL, no HIGH, no MEDIUM, no new ungated suppression.
|
||||
|
||||
- **L1** `storage.rs:196` (with `:200-202`): `Protocol(_)` counts as transient. That bucket also holds deterministic
|
||||
local failures: `BadAuthorizationHeader` for a key with non-ASCII bytes (the `http` crate accepts bytes 0x80 and up
|
||||
in a value, ureq-proto's `to_str` refuses them), `OutputOverflow` for a huge key, and `HttpParseFail` for a
|
||||
malformed answer. Each is retried three times and ends as `Network`, exit 3. The `final_error` doc comment says
|
||||
"a key a header refuses, a malformed answer" are final, which is only true for control characters (`Http`). This
|
||||
costs about 4.5 s and a wrong exit code, and nothing is sent or printed. Hint: make `Protocol` final, or retry only
|
||||
its response-side variants, and correct the comment.
|
||||
- **L2** `main.rs:112-115`: the bind probe also accepts the subnet broadcast (measured: 192.168.1.255). With
|
||||
`FAUNOSTEO_DEV_HOST=192.168.1.255` and `SUPABASE_URL=http://192.168.1.255:54321`, the destination check passes,
|
||||
the kernel refuses the TCP connection (ENETUNREACH, measured), the key is never sent, and the run ends `Network`
|
||||
instead of `Destination`. The probe also degrades to "any private IPv4" on a host with `ip_nonlocal_bind=1`
|
||||
(0 here). Hint: after the bind, `connect((ip, port))` the UDP socket (sends nothing; a broadcast gives EACCES,
|
||||
measured) and require `local_addr().ip() == ip`, and document the sysctl dependency.
|
||||
- **L3** `upload-sheet-assets.sh:15-16` and `sheet-assets-e2e.sh:15,18`: the binary path `target/release/...` is
|
||||
hard-coded, but cargo honours `CARGO_TARGET_DIR`, `CARGO_BUILD_TARGET` and a user `build.target-dir`. If any is
|
||||
set, the build lands elsewhere and the script `exec`s a stale binary, possibly pre-fix, with the service key in its
|
||||
environment (or a missing one). None is set on this machine (measured; no `~/.cargo/config.toml`, and the workspace
|
||||
`.cargo/config.toml` has no target dir). Hint: `env -u CARGO_TARGET_DIR -u CARGO_BUILD_TARGET` on the build, or
|
||||
take the path from `cargo build --message-format=json`.
|
||||
- **L4** `floor-clarification-check.sh:16`: the exemption is pinned by line number, not content. If a
|
||||
`/prune-memory` rewrite puts another STUB on `evals.md:205`, that STUB is silently exempt, while the original
|
||||
finding fails loudly. The negative proof has no `evals.md`, so the regex's narrowness is proven only by reading.
|
||||
Since this change, the script prints OK while the finding waits for the owner's decision. That is a quality-gate
|
||||
question for the verifier and has no security effect. Hint: also match the snippet prefix (`Anomalies: LOW
|
||||
report.rs`), and add a proof case with a STUB on another `evals.md` line.
|
||||
- **L5** `upload_tool.rs:346-347`: the "not this machine" address 10.255.255.254 is the one WSL2 assigns to `lo`.
|
||||
On such a host the test would connect locally with the made-up key and fail with exit 3. This is a portability
|
||||
nit (fail-loud) with no exposure. Hint: use a TEST-NET address (192.0.2.1 is not private, so pick a private
|
||||
address checked to be non-local at run time).
|
||||
|
||||
Context only, outside the diff, not counted: `supabase/config.toml` sets no `jwt_secret` (only `jwt_expiry` at
|
||||
`:89`). The local stack therefore likely signs with the CLI's public default secret, and port 54321 is open to the
|
||||
LAN by design (CLAUDE.md "Environments and LAN access"). Anyone on the LAN who knows the public default
|
||||
service_role JWT would then hold service_role on the local stack, which makes the tool's key secrecy moot there. I
|
||||
did not verify this against the running stack. The diff's RUST.md text names the same point.
|
||||
|
||||
## Verdict block
|
||||
|
||||
```
|
||||
SECURITY — VERDICT: PASS
|
||||
TOOL: semgrep 1.168.0 — p/security-audit, p/secrets, p/owasp-top-ten
|
||||
SCOPE: 11 files (10 in a481af5..fad85c0 + crates/core/src/curation/upload.rs read as context)
|
||||
BLOCKING:
|
||||
(none)
|
||||
REPORTED (non-blocking):
|
||||
- [LOW] retry-classification (Protocol treated as transient, doc comment overstates finality) — crates/core/src/bin/upload_sheet_assets/storage.rs:196
|
||||
- [LOW] dev-address-probe (subnet broadcast binds; relies on ip_nonlocal_bind=0) — crates/core/src/bin/upload_sheet_assets/main.rs:112
|
||||
- [LOW] stale-binary-exec (hard-coded target/release vs CARGO_TARGET_DIR) — scripts/upload-sheet-assets.sh:16, scripts/sheet-assets-e2e.sh:15
|
||||
- [LOW] floor-declared-line (pinned by line number; no negative proof for DECLARED_RE) — scripts/floor-clarification-check.sh:16
|
||||
- [LOW] test-portability (10.255.255.254 is lo on WSL2) — crates/core/tests/upload_tool.rs:346
|
||||
PROOF: semgrep 75 rules on 11 files → 0 findings; checklist 9 checks (secrets, SQL concat, unsafe render, authz, error exposure, secret in log, tracked .env, guardrails, new suppressions) + 5 targeted fix reviews → 5 findings (all LOW); cargo test upload_tool 12/12 green; floor classification 22 → 21 requalified, 1 declared, 0 remaining
|
||||
```
|
||||
@@ -338,3 +338,9 @@ Owner validated R5.3 functionally, asked a fresh verifier on the final state, Ob
|
||||
## 2026-10-23 (R5.4 review files in Rust, 14 gate rounds)
|
||||
|
||||
R5.4: `draft/{labels,changelog,review_report,drawing,overlay,app_review,svg_compare}.rs` + `curation/report_guard.rs` (allowlist guard); CHANGELOG 1,114,533 B, REVIEW-REPORT, 3 drawings byte-exact, app-main.svg = R3 trigger-name rule only; `--check` 6/0 + 12/0, 1 declared. Gates: rounds 1-5 Markdown families → allowlist + 798-probe TS sweep; rounds 6-11 bounds (records 16 Mi weight charged per record, report tables 32 Mi summed, drawings 64 MiB projected per annotation + fixed parts); 12-14 docs. Final c4e717d security PASS, check:all green; eb667ad verifier 9/9 MET, only audit-prose FLOOR. LRN-074, LRN-075, EVAL-029. STOP at R5.4 checkpoint; R5.5 on owner go.
|
||||
|
||||
## 2026-10-23 (R5.5)
|
||||
- R5.4 adjustments: FLOOR clarification + negative-proof script (21 audit prose requalified, evals.md:205 STUB declared), 4 MiB cap changes/references (+39 MB parse at 3.98 MB), serde tolerance characterised, no reader changed.
|
||||
- R5.5: `upload-sheet-assets` Rust (core rules pure, bin ureq), local-only destination, plan before write, no overwrite, confirm after verify; 12 stub tests + 7 unit.
|
||||
- Blocked: own run vs real local stack refused by permission layer, not retried; owner runs `pnpm e2e:sheet-assets`. Fable gate agents out of credits → re-dispatched on opus.
|
||||
- Gates: verifier ECARTS(4)→fixed→C1-C13 MET; security PASS ×2; check:all green 21:47:11. STOP at checkpoint.
|
||||
@@ -326,13 +326,13 @@ feat/findings-tree-v2 merged (FindingsTreePicker tracked in HEAD); all work belo
|
||||
- [x] R5.4c `render_draft_artifacts` complete (TS order); `import-equine --check` covers the 12 draft files with explicit criteria (bytes; declared tree rule), PENDING list and `shortcut:` removed; fails on any missing or unauthorised difference
|
||||
- [x] R5.4d proofs: `tests/curation_review.rs` (the 6 files, ledger digests, determinism); TS generator `r5.review-synthetic-oracles.test.ts` → `oracles/r5/review-synthetic.json` (changelog raw + formatted, review report, overlays, app drawings on mini-sheet cases: Unicode, pipes, newlines, long values, emoji refused, every status); R5.3 LOW carry-over (destination identity, cite membership, unreadable-seed message, stale comment, doc nits)
|
||||
- [x] R5.4e gates: 14 rounds; c4e717d security PASS (LOW only) + check:all green 16:41:34; eb667ad (docs only over it) fresh verifier 9/9 clauses MET, ECARTS = 13 FLOOR hits on audit prose (owner Clarification); docs RUST.md §11.14-11.15, CHANGELOG, CLAUDE.md; memory LRN-074, LRN-075, EVAL-029 → checkpoint delivered, STOP before R5.5 (R5.4 closes on owner validation)
|
||||
- [ ] R5.5 `upload-sheet-assets` bin + runbook (owner go 2026-10-23 « Validation R5.4 et autorisation conditionnelle R5.5 »; local stack only; no production, no deploy; stop at the checkpoint, no R5.6)
|
||||
- [ ] R5.5 `upload-sheet-assets` bin + runbook (checkpoint delivered 2026-10-23, closes on owner validation; owner go 2026-10-23 « Validation R5.4 et autorisation conditionnelle R5.5 »; local stack only; no production, no deploy; stop at the checkpoint, no R5.6)
|
||||
- [x] R5.5-0 R5.4 closing adjustments: FLOOR clarification (contract + `scripts/floor-clarification-check.sh`, 19 audit-prose hits requalified, 1 STUB in evals.md:205 outside scope, reported), 4 MiB cap on `changes.json` / `references.json` (core `MAX_CURATION_FILE_BYTES`, bin bounded read; 222 KB + 17 KB committed, parse of a 3.98 MB changeset peaks +39 MB), `tests/curation_caps.rs` (cap, margin, stable ids of any species), serde R2 tolerance probe (characterisation only, nothing changed)
|
||||
- [ ] R5.5-1 core module `curation::upload` (pure, no I/O): manifest parse + entry rules, destination rule, key derivation, per-entry plan from observed states, closed error kinds
|
||||
- [ ] R5.5-2 bin `upload-sheet-assets` (feature `upload`, ureq): `--check` read-only and write mode, destination verified (loopback or the DEV host, http, explicit port) then handshake (private bucket) before any transfer, existing objects verified by sha256 never overwritten, confirm only after the object is verified, bounded retries, secrets never printed
|
||||
- [ ] R5.5-3 scripts: `assets:upload` points at the Rust bin (`scripts/upload-sheet-assets.sh`), TS script kept until R5.7 census; runbook SUPABASE.md §3.1, RUST.md §11.16
|
||||
- [ ] R5.5-4 tests: unit (core), bin tests against a stub HTTP server (refusals, partial failure, resume, network errors), local Supabase run (upload, rerun, anon refused, wrong key, tamper); `pnpm rust:check`, `pnpm check:all`, `r5:import-equine --check`
|
||||
- [ ] R5.5-5 gates: fresh security audit, fresh verifier on the contract `.claude/tasks/contracts/2026-10-23-rust-r5-upload.md`; memory; checkpoint report (9 points), STOP
|
||||
- [x] R5.5-1 core module `curation::upload` (pure, no I/O): manifest parse + entry rules, destination rule, key derivation, per-entry plan from observed states, closed error kinds
|
||||
- [x] R5.5-2 bin `upload-sheet-assets` (feature `upload`, ureq): `--check` read-only and write mode, destination verified (loopback or the DEV host, http, explicit port) then handshake (private bucket) before any transfer, existing objects verified by sha256 never overwritten, confirm only after the object is verified, bounded retries, secrets never printed
|
||||
- [x] R5.5-3 scripts: `assets:upload` points at the Rust bin (`scripts/upload-sheet-assets.sh`), TS script kept until R5.7 census; runbook SUPABASE.md §3.1, RUST.md §11.16
|
||||
- [x] R5.5-4 tests (real-stack e2e NOT run by Claude: permission layer refused; owner runs `pnpm e2e:sheet-assets`): unit (core), bin tests against a stub HTTP server (refusals, partial failure, resume, network errors), local Supabase run (upload, rerun, anon refused, wrong key, tamper); `pnpm rust:check`, `pnpm check:all`, `r5:import-equine --check`
|
||||
- [x] R5.5-5 gates (round 1: verifier ECARTS(4) + security PASS 1 MEDIUM 9 LOW, fixed; round 2 fad85c0: verifier C1-C13 MET, check:all green 21:47:11, security PASS 5 LOW declared RUST.md §11.16): fresh security audit, fresh verifier on the contract `.claude/tasks/contracts/2026-10-23-rust-r5-upload.md`; memory; checkpoint report (9 points), STOP
|
||||
- [ ] R5.6 D-R5 switch (convention, aliases, membership test in gated readers with per-file go, seed regenerated once by the Rust tool, `.prettierignore`, CHANGELOG)
|
||||
- [ ] R5.7 consumer census, domain fixtures moved, packages deleted, xmldom + prettier out of the lockfile, docs, ledger; `pnpm check:all`; merge only on the owner's signal
|
||||
- [ ] separate chore (D-R7): form constraints module + PGlite `pg_constraint`/config.toml parity test
|
||||
|
||||
@@ -3365,3 +3365,28 @@ pre-existing `confirm_sheet_asset_upload` that does not re-check the hash (L6, S
|
||||
waived fixture lines (L8), the serde tolerance (L9, owner decision). Outside the diff, reported by the
|
||||
audit and not verified: `supabase/config.toml` sets no JWT secret, so the local stack likely uses the
|
||||
CLI's public default keys while port 54321 is open to the LAN.
|
||||
|
||||
Gate round 2 (fad85c0): `pnpm check:all` green 21:47:11; targeted fresh verifier: C1 to C13 MET, C14
|
||||
open only on these two gates, now run; security PASS, 5 LOW kept and declared
|
||||
(`.claude/audits/2026-10-23-rust-r5-upload-security-2.md`): `Protocol` errors (also a key with
|
||||
non-ASCII bytes) are retried and end `Network`; the DEV-address bind accepts the subnet broadcast (the
|
||||
kernel then refuses the connection, the key is never sent, the run ends `Network`); the launchers run
|
||||
`target/release/…` and ignore `CARGO_TARGET_DIR`; the FLOOR declared line is pinned by line number; the
|
||||
non-local DEV-address test assumes 10.255.255.254 is not local (false on WSL2).
|
||||
|
||||
### 11.17 Checkpoint R5.5
|
||||
|
||||
Done: the upload tool in Rust with its runbook, the R5.4 closing adjustments (4 MiB curation cap, FLOOR
|
||||
clarification, serde characterisation). Not done by the assistant: the run against the real local stack
|
||||
(`pnpm e2e:sheet-assets`, by hand). Open for the owner: the STUB line `evals.md:205` (FLOOR, declared),
|
||||
the serde tolerance decision before R5.6, the local stack's likely default keys with 54321 on the LAN.
|
||||
|
||||
Prerequisites of R5.6: the owner's go; the convention `sha256-rfc8785-validated-v1` (D-R5 B, BDR-068)
|
||||
with its RFC 8785 vectors and the independent differential; the aliases of the historical fingerprints
|
||||
taken from trusted frozen documents only (the census `.claude/audits/R5_DOCUMENT_SHA256_CENSUS.md`:
|
||||
`27c7383c…2660` and the earlier revisions it lists), never from an archive or a bundle; the per-file go
|
||||
for the membership test in the gated readers (`report/rebase.rs`, `archive/read.rs`, `pdf/import.rs`),
|
||||
together with the serde decision above; the seed regenerated once by the Rust tool and the assets
|
||||
re-uploaded only if their bytes change (they do not: `horse.svg` is frozen); `.prettierignore` on the
|
||||
Rust-written paths; the migration-map cascade debt (§11.12) fixed before any changeset renames a view,
|
||||
zone, region, category or cited reference.
|
||||
Reference in new issue
Block a user