From f0bcbfef461ce1a963500ed9f0a5b7894e7849c6 Mon Sep 17 00:00:00 2001 From: bchanot Date: Sun, 18 Oct 2026 18:49:51 +0200 Subject: [PATCH] feat(guardrails): read-only live probe to confirm a hole from outside probe.py observes a running target the user owns and names, driven by a plan built from the static findings. Five probes: anon-read, cross-user, no-auth route, CORS reflection, capped rate-limit burst. GET/HEAD/OPTIONS only, --authorized gate, secrets from the environment, same-host redirect only, bounded request budget. SKILL STEP 5, checks.md reference, docs. Suite guardrails-probe.test.sh (16 checks) against a localhost fixture. --- CHANGELOG.md | 2 +- README.md | 2 +- USAGE.md | 3 +- lib/tests/guardrails-probe.test.sh | 156 +++++++++++++ skills/guardrails/SKILL.md | 62 ++++- skills/guardrails/references/checks.md | 51 ++++ skills/guardrails/scripts/probe.py | 309 +++++++++++++++++++++++++ skills/guardrails/test-prompts.json | 3 +- 8 files changed, 572 insertions(+), 16 deletions(-) create mode 100644 lib/tests/guardrails-probe.test.sh create mode 100644 skills/guardrails/scripts/probe.py diff --git a/CHANGELOG.md b/CHANGELOG.md index f6ea3c8..fb6ca0a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and this project ## [Unreleased] ### Added -- **Secure-by-default guardrails**: `rules/secure-by-default.md` (always on) makes five domains baseline on every project, plus systematic API and access hygiene (auth on every route, CORS, secrets, passwords hashed with argon2id or bcrypt, never compared in clear, logged or hardcoded): hosting (rate limit, CDN and firewall, hard spend cap), accessibility (keyboard, visible focus, alt text, field labels), Supabase (RLS on every table, owner-scoped policies, no service role key client-side), SMS and email (recorded consent, unsubscribe, STOP), cloud functions (no self-trigger, instance cap, budget stop). `/guardrails` audits them: a stdlib Python scanner (`skills/guardrails/scripts/scan.py`, FAIL / WARN / MANUAL, exit 1 on FAIL) then judgment, report in `.claude/audits/GUARDRAILS.md`. The security-auditor gate runs the scanner's `api` and `supabase` domains and blocks on FAIL. Items only a provider dashboard can set stay MANUAL, never claimed done. Suite: `lib/tests/guardrails-scan.test.sh` (clean fixture plus one mutant per rule). +- **Secure-by-default guardrails**: `rules/secure-by-default.md` (always on) makes five domains baseline on every project, plus systematic API and access hygiene (auth on every route, CORS, secrets, passwords hashed with argon2id or bcrypt, never compared in clear, logged or hardcoded): hosting (rate limit, CDN and firewall, hard spend cap), accessibility (keyboard, visible focus, alt text, field labels), Supabase (RLS on every table, owner-scoped policies, no service role key client-side), SMS and email (recorded consent, unsubscribe, STOP), cloud functions (no self-trigger, instance cap, budget stop). `/guardrails` audits them: a stdlib Python scanner (`skills/guardrails/scripts/scan.py`, FAIL / WARN / MANUAL, exit 1 on FAIL) then judgment, report in `.claude/audits/GUARDRAILS.md`. The security-auditor gate runs the scanner's `api` and `supabase` domains and blocks on FAIL. Items only a provider dashboard can set stay MANUAL, never claimed done. An optional read-only live probe (`skills/guardrails/scripts/probe.py`, `--live`) confirms a finding from outside on a target the user owns and names: anon-read, cross-user, no-auth route, CORS reflection and a capped rate-limit burst, verdicts OPEN / CLOSED / SKIPPED / ERROR. It sends GET, HEAD and OPTIONS only (one opt-in POST burst, capped at 25), requires `--authorized`, takes secrets from the environment never the command line, never follows a redirect to another host, and stops at a request budget. Suites: `lib/tests/guardrails-scan.test.sh` (clean fixture plus one mutant per rule) and `lib/tests/guardrails-probe.test.sh` (probe logic against a localhost fixture server). - **model-router mod**: `mods/model-router/`, a Claude Code mod (function-hooks plugin), routes every repo skill and agent from phase rows (`plan`, `reflect`, `orchestrate`, `escalate`, `judge`, `implement`, `write`, `verify`, `explore`, `apply`, `mechanical`; built-ins Explore on sonnet/medium, Plan on opus/xhigh). A typed skill routes the main loop to its row, and a best-tier row holds across turns in a run slot until `/route clear`, `/route off`, a user `/model` or a typed skill on a non-best row. Agents get their row's model at spawn (within the tier, upward only) and its effort on every step; explicit Agent params win. Orchestrators declare their phases through the `mcp__model-router__route` tool. `ultrathink` in a prompt sets the turn's minimum effort and `/route effort=max` holds until `/route clear`; the built-in `/effort` is not a lever inside a run. `/route show` names the run slot (`main: run `), and a `null` row in the override drops a default row. The model gets a `route` tool and the user a `/route` command (`show|clear|off|on|reload|pending|ask on|off|forget ||model= effort=|switch on|off|verbose on|off`). Optional per-machine config `~/.claude/model-router.json`, where `"enabled": false` turns it off on that machine. The spinner suffix and the status line show the route in force. It loads in every session through the tracked symlink `skills/model-router` (`model-router@skills-dir`). `make doctor` gains a Mods section; suite `make test suite=lib/tests/mods.test.sh`. Known limits: a main-loop switch runs only while the cold read it costs (the context minus what the target model's prompt cache holds) fits `upgradeMaxTokens` (200000) or `downgradeMaxTokens` (120000), and a downgrade also needs `mainModelSwitch` (on by default, off on any non-boolean value); the hooks send full model ids, so the `models` table has to follow new versions. - **model-router first-use confirmation**: `mods/model-router/routing.json` (tracked) is the single source of the phase table and of every skill and agent row, and keeps the decisions: `confirmed`, `changed` (`from`/`to`), `projects` exceptions keyed by the origin remote reduced to `host/path` (credentials and local paths never stored), and `ask`. The first use of a rowed typed skill, a rowed agent spawn or a main-loop phase declared through the `route` tool opens a dialog with context: the skill's description (first sentence of its `SKILL.md` frontmatter) or the agent's, the phase with its `about` line (a new field on each of the 11 phases) and the model id and effort the next step really runs on. A row offers Later, Keep or Change; a main-loop phase Later or Keep. Change asks the model (fable, opus, sonnet, haiku with their tier), then the effort among those the phases of that model use (skipped when there is only one), then Everywhere or This project only; the pair maps to an existing phase (rows stay phase names, the same phase counts as Keep, a pair no phase offers is added by hand as a new phase). A skill-row change toasts the model and effort it now runs on, with the `/route switch on` hint when the main loop holds back a downgrade; a free-text answer counts as Later. One dialog at a time, never in headless (`-p`) runs or inside a sub-agent. Only a dialog answer, `/route ask on|off` or `/route forget` (which takes decisions back; a changed row returns to its shipped phase, the frontmatter floors stay yours to realign) writes the file (serialized, 64 KiB cap, never created when absent); each write asks you to commit it from the config repo. `/route pending` lists unconfirmed rows and phases. Layers: routing.json < `~/.claude/model-router.json` (its `ask` wins); a project's `.claude/model-router.json` is never read. Tests: `mods/model-router/hooks/register.test.ts`. - **Manual-push mode**: `git config gitflow.autopush false` (human-set) now stops every push the gitflow lib makes, not only the post-commit / post-merge hooks. `gitflow start` and `finish` branch, commit and merge locally and push nothing; `gitflow delete` leaves the `origin/` copy in place and prints `git push origin --delete
` for the user to run. `hooks/unpushed-guard.sh` stays silent at turn end in this mode and opens each session with one `ℹ manual push mode:` line counting the commits no remote holds across every local branch; an unparseable or unreadable `gitflow.autopush` value is treated as manual push mode too, and that line names it. `hooks/push-guard.sh` (PreToolUse, `Bash|Monitor`) refuses any `git push` Claude types while `gitflow.autopush` reads false in the session cwd or in a literal `-C`/`cd` directory the command names (global config counts outside a repo); the refusal tells the user to run it with `! git push`. It reads the mode through the same lib verb as every other reader and fails closed: an unparseable or unreadable value reads as manual, and an internal error, a missing `lib/gitflow.sh`, more than 20 distinct directory tokens in one command (capped before any token is classified), a `cd`/`-C` directory token mixing quoted and unquoted parts, or a payload jq cannot parse whose raw text looks like a push refuse the push (these pathological cases fire in auto mode too). Directory tokens are read as whole shell words, adjacent quoted segments and backslash escapes included. Once the text looks like a push, the command is split into simple commands (quotes, escapes, separators, heredoc bodies dropped) and a push must be git's own subcommand in one of them: `grep "git push"`, `git log -S "git push"`, `git config --get push.default`, `git stash push`, a heredoc quoting a push or a commit message naming one are allowed; `bash -c` scripts are re-checked, wrappers (`sudo`, `env`, `eval`, `xargs`, `timeout`, `git subtree`...) and assignments keep the text verdict. The misses listed in its header fall to a new `autoMode.soft_deny` rule that no request in the turn clears. The session banner adds `🔒 push : manual (autopush=false) — ! git push` when the key reads false, and `🔒 push : manual (autopush bad) — ! git push` when the value is invalid. Skills read the mode through a new lib verb, `bash ~/.claude/lib/gitflow.sh push-mode`: it prints `auto`, `manual` or `invalid` (rc 0) and names an invalid value on stderr (printable characters only, 64 at most). It is the one reader a skill may call, since the `git config` read of the key is denied to Claude. Skills push nothing on their own, except the `/release-candidate` tag in auto-push mode on an explicit go. Every "on origin" or "not pushed" line they print comes from `git rev-list --count origin/
..
` read after the fact, with the complete `! git …` command when something is left for the user to push. An invalid value (anything but unset, true or false, or a read that fails) is manual push mode for every reader and is named where it is read (see Fixed). Tests: `lib/gitflow-test.sh` T11b (push-mode verb), T18m and T18q blocks, `lib/tests/unpushed-guard.test.sh` T10-T16, `lib/tests/push-guard.test.sh` (98 checks). diff --git a/README.md b/README.md index affc7a5..2e12012 100644 --- a/README.md +++ b/README.md @@ -228,7 +228,7 @@ a different package, ships its own conflicting `graphify` bin) — see | `/reconcile` | Confront declared status (TODO, registries) against real git/fs state — surface stale items | | `/pdf-translate` | Translate a PDF to another language, output as HTML (via Vision) | | `/explain-diff` | Explain a diff, commit, branch or PR as one self-contained HTML page (background, intuition, code walkthrough, 5-question quiz), saved outside the repo | -| `/guardrails` | Audit the five baseline guardrails (hosting, a11y, Supabase RLS, SMS/email consent, cloud functions) plus API hygiene (auth, CORS, secrets, password storage): deterministic scanner, then judgment, report in `.claude/audits/GUARDRAILS.md` | +| `/guardrails` | Audit the five baseline guardrails (hosting, a11y, Supabase RLS, SMS/email consent, cloud functions) plus API hygiene (auth, CORS, secrets, password storage): deterministic scanner, then judgment, report in `.claude/audits/GUARDRAILS.md`. `--live ` adds a read-only probe that confirms a hole from outside | | `/close` | End-of-session ritual — alias for `/capitalize --ritual` (dedup + TODO reconcile + 3-question reflection) | | `/harden` | Web hardening audit — HTTPS/TLS, HSTS, CSP, security headers | | `/web-validate` | W3C HTML/CSS validity + WCAG 2.1 accessibility audit | diff --git a/USAGE.md b/USAGE.md index c97cfc9..f11366d 100644 --- a/USAGE.md +++ b/USAGE.md @@ -121,6 +121,7 @@ Tu veux... | Traduire un PDF | `/pdf-translate` | | Comprendre un changement pas suivi | `/explain-diff` | | Auditer garde-fous (RLS, a11y, consentement, fonctions, mots de passe) | `/guardrails` | +| Sonder le projet en direct (lecture seule, cible à toi) | `/guardrails --live ` | | Changer profil skills | `/profile` | | Audit/polish design (anti-slop) | `/impeccable` | | Sweep groupé tous axes (nettoyage + sécu + reconcile + doc) | `/tour` | @@ -164,7 +165,7 @@ Tu veux... | `/client-handover` | Livraison client | Audits finaux + livrable brandé | | `/pdf-translate` | Traduire un PDF vers une autre langue | Sortie HTML fidèle (images, layout, style préservés) | | `/explain-diff` | Comprendre un commit, une branche ou une PR qu'on n'a pas suivis | Page HTML autonome hors du repo : contexte, intuition, code, quiz de 5 questions | -| `/guardrails` | Vérifier hébergement, a11y, Supabase RLS, consentement SMS/email, fonctions cloud, stockage des mots de passe | Scanner déterministe puis jugement ; rapport `.claude/audits/GUARDRAILS.md` | +| `/guardrails` | Vérifier hébergement, a11y, Supabase RLS, consentement SMS/email, fonctions cloud, stockage des mots de passe | Scanner déterministe puis jugement ; rapport `.claude/audits/GUARDRAILS.md` ; `--live ` sonde en lecture seule | | `/impeccable` | Audit/polish design + détecteur anti-slop déterministe | 23 verbes ; `npx impeccable detect` (exit 0/2) | | `/tour` | Sweep groupé sur un ou plusieurs projets | Sécu + nettoyage + reconcile + doc, boucle jusqu'à un pass propre | | `/route` | Voir ou fixer la route du mod model-router | show / clear / off / on / reload / pending / ask on\|off / / model=… effort=… / switch on\|off / classify on\|off (classifieur de prompt) / verbose on\|off (transitions seules) | diff --git a/lib/tests/guardrails-probe.test.sh b/lib/tests/guardrails-probe.test.sh new file mode 100644 index 0000000..735cc0b --- /dev/null +++ b/lib/tests/guardrails-probe.test.sh @@ -0,0 +1,156 @@ +#!/usr/bin/env bash +# lib/tests/guardrails-probe.test.sh — floor of the /guardrails live probe. +# A throwaway http.server on 127.0.0.1 serves scripted responses; the probe +# runs against it and each verdict is asserted. Nothing leaves the machine. +# The probe sends read-only methods; this suite proves the OPEN/CLOSED logic, +# the --authorized gate, the same-host redirect guard and the request budget. +set -u +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +PROBE="$ROOT/skills/guardrails/scripts/probe.py" +WORK="$(mktemp -d)" +PASS=0; FAIL=0; SRV="" +cleanup() { [ -n "$SRV" ] && kill "$SRV" 2>/dev/null; rm -rf "$WORK"; } +trap cleanup EXIT + +check() { # check