feat(lib): floor-guard, diff-scoped detector of a weakened quality bar
SUPPRESS / SKIP / DELETED_TEST / ASSERT_DROP / STUB / THRESHOLD_DOWN over git diff <base> (untracked files included), floor-guard: allow <reason> waiver printed as WAIVED, rc 0/2/3. Mandatory verifier STEP 3, documented under GATE 1 of verify-secure-loop.md. Suite: 6 kinds + WAIVED + CLEAN, flip-tested. Adapted from agent-skills constraint-driven-development.
This commit is contained in:
+34
-7
@@ -71,7 +71,28 @@ You may re-run a `CHECK:` yourself to settle a doubt (Bash is read-only, and
|
|||||||
these commands are observation). You may NOT edit the contract — an evidence
|
these commands are observation). You may NOT edit the contract — an evidence
|
||||||
line you disagree with is reported, never rewritten.
|
line you disagree with is reported, never rewritten.
|
||||||
|
|
||||||
## STEP 3 — SCOPE CHECK
|
## STEP 3 — FLOOR GUARD (mandatory, deterministic)
|
||||||
|
|
||||||
|
Run the floor guard over the diff before rendering any verdict — a red or
|
||||||
|
skipped run here is a structural gap, never a judgment call:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash ~/.claude/lib/floor-guard.sh <base> -- <pathspec>...
|
||||||
|
```
|
||||||
|
|
||||||
|
`<base>` = the branch's gitflow base (develop; main for a hotfix/release).
|
||||||
|
Parse the single `FLOOR GUARD:` line:
|
||||||
|
|
||||||
|
- `clean` (rc 0) → no finding, continue to STEP 4.
|
||||||
|
- `<n> finding(s), <m> waived` (rc 2) → each `FLOOR <KIND> <file>:<line>
|
||||||
|
<snippet>` line is a gap for STEP 5's `ECARTS` count, UNLESS the
|
||||||
|
contract's `CLARIFICATIONS` explicitly authorizes that exact weakening —
|
||||||
|
quote the authorizing sentence in the verdict instead of counting it as a
|
||||||
|
gap. `WAIVED` lines are informational only, never a gap.
|
||||||
|
- rc 3 (usage error) → a structural failure like a missing contract: retry
|
||||||
|
once (base ref or pathspec likely wrong), a second failure escalates.
|
||||||
|
|
||||||
|
## STEP 4 — SCOPE CHECK
|
||||||
|
|
||||||
List the files actually touched (`git diff --name-only` over `DIFF`).
|
List the files actually touched (`git diff --name-only` over `DIFF`).
|
||||||
Compare against the contract's `FILE SCOPE`. Report every out-of-scope
|
Compare against the contract's `FILE SCOPE`. Report every out-of-scope
|
||||||
@@ -79,7 +100,7 @@ file. Disposition is NOT your call: the orchestrator treats each one as a
|
|||||||
gap — the dev removes it or justifies it, and an accepted justification
|
gap — the dev removes it or justifies it, and an accepted justification
|
||||||
only enters the contract through a human micro-gate.
|
only enters the contract through a human micro-gate.
|
||||||
|
|
||||||
## STEP 4 — VERDICT
|
## STEP 5 — VERDICT
|
||||||
|
|
||||||
Read the contract's `ABANDON:` lines. An abandoned criterion is `ABANDONED`
|
Read the contract's `ABANDON:` lines. An abandoned criterion is `ABANDONED`
|
||||||
— never `MET`, never counted as a gap the dev can close.
|
— never `MET`, never counted as a gap the dev can close.
|
||||||
@@ -89,11 +110,12 @@ is not:
|
|||||||
|
|
||||||
1. `ERROR(<reason>)` — the contract is missing or unreadable.
|
1. `ERROR(<reason>)` — the contract is missing or unreadable.
|
||||||
2. `ECARTS(n)` — n = count(NOT-MET) + count(UNVERIFIABLE) + count(out-of-scope
|
2. `ECARTS(n)` — n = count(NOT-MET) + count(UNVERIFIABLE) + count(out-of-scope
|
||||||
files). Surface any abandonment in the same report.
|
files) + count(unauthorized FLOOR findings from STEP 3). Surface any
|
||||||
|
abandonment in the same report.
|
||||||
3. `ABANDONED(n)` — zero gaps remain, but n abandonments stand. This is NOT
|
3. `ABANDONED(n)` — zero gaps remain, but n abandonments stand. This is NOT
|
||||||
a pass and NOT a dev loop: it routes straight to the human gate.
|
a pass and NOT a dev loop: it routes straight to the human gate.
|
||||||
4. `CONFORME` — ALL criteria `MET`, zero out-of-scope files, zero
|
4. `CONFORME` — ALL criteria `MET`, zero out-of-scope files, zero
|
||||||
abandonments.
|
unauthorized FLOOR findings, zero abandonments.
|
||||||
|
|
||||||
## OUTPUT (exact format — machine-parsed by the orchestrator)
|
## OUTPUT (exact format — machine-parsed by the orchestrator)
|
||||||
|
|
||||||
@@ -106,6 +128,8 @@ CRITERIA:
|
|||||||
3. <criterion> — UNVERIFIABLE — <reason>
|
3. <criterion> — UNVERIFIABLE — <reason>
|
||||||
4. <criterion> — ABANDONED — <the reason recorded in the contract>
|
4. <criterion> — ABANDONED — <the reason recorded in the contract>
|
||||||
SCOPE: in-scope <n> files; out-of-scope: <list | none>
|
SCOPE: in-scope <n> files; out-of-scope: <list | none>
|
||||||
|
FLOOR: clean | <n> finding(s) (<m> waived) — <FLOOR lines, or the
|
||||||
|
CLARIFICATIONS sentence that authorizes each one | none>
|
||||||
PROOF: read <n> files, ran <cmd → result | nothing>, checked <n>/<n> criteria
|
PROOF: read <n> files, ran <cmd → result | nothing>, checked <n>/<n> criteria
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -123,6 +147,9 @@ PROOF: read <n> files, ran <cmd → result | nothing>, checked <n>/<n> criteria
|
|||||||
- `PROOF` is MANDATORY. A `CONFORME` without a `PROOF` line is invalid —
|
- `PROOF` is MANDATORY. A `CONFORME` without a `PROOF` line is invalid —
|
||||||
the orchestrator discards it as a structural failure (LRN-048: a pass
|
the orchestrator discards it as a structural failure (LRN-048: a pass
|
||||||
must prove it looked).
|
must prove it looked).
|
||||||
|
- STEP 3's floor-guard run is MANDATORY, every dispatch. A `CONFORME` or
|
||||||
|
`ECARTS` without a `FLOOR` line is a structural failure just like a
|
||||||
|
missing `PROOF` — the run was skipped, not the diff clean.
|
||||||
- The verdict grammar is load-bearing: exactly one `VERIFY — VERDICT:`
|
- The verdict grammar is load-bearing: exactly one `VERIFY — VERDICT:`
|
||||||
line, spelled exactly as above.
|
line, spelled exactly as above.
|
||||||
|
|
||||||
@@ -145,8 +172,8 @@ loop, never here):
|
|||||||
lifts the abandonment (the criterion was fixable after all) or accepts
|
lifts the abandonment (the criterion was fixable after all) or accepts
|
||||||
the partial delivery; the run is never reported as fully complete.
|
the partial delivery; the run is never reported as fully complete.
|
||||||
- Structural failure (`ERROR(…)`, missing/duplicated VERDICT line,
|
- Structural failure (`ERROR(…)`, missing/duplicated VERDICT line,
|
||||||
unparsable output, agent crash, `CONFORME` without `PROOF`) → retry
|
unparsable output, agent crash, `CONFORME` without `PROOF` or without
|
||||||
ONCE with a fresh verifier; a 2nd structural failure → human
|
`FLOOR`) → retry ONCE with a fresh verifier; a 2nd structural failure →
|
||||||
escalation. A mute verifier is NEVER a PASS.
|
human escalation. A mute verifier is NEVER a PASS.
|
||||||
- After a security-gate fix round: re-verify the request FIRST (this
|
- After a security-gate fix round: re-verify the request FIRST (this
|
||||||
agent), THEN re-verify security — in that order.
|
agent), THEN re-verify security — in that order.
|
||||||
|
|||||||
Executable
+297
@@ -0,0 +1,297 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# lib/floor-guard.sh — diff-scoped detector of a quietly weakened quality bar.
|
||||||
|
#
|
||||||
|
# bash ~/.claude/lib/floor-guard.sh <base-ref> [-- <pathspec>...]
|
||||||
|
#
|
||||||
|
# rc 0 = clean no floor finding in the diff
|
||||||
|
# 2 = <n> finding(s), <m> waived
|
||||||
|
# 3 = usage error (missing <base-ref>, or it does not resolve to a commit)
|
||||||
|
#
|
||||||
|
# WHY (BDR-100 class): "no weakened check in this diff" is exactly the kind
|
||||||
|
# of judgment an LLM verifier can miss, or be talked past one line at a time
|
||||||
|
# — a single added TS-ignore comment, a skipped test, a dropped assertion, a
|
||||||
|
# coverage threshold shaved by one point. This makes that judgment
|
||||||
|
# deterministic: grep the diff for the known ways a change quietly lowers
|
||||||
|
# the bar, same floor doctrine as gates.sh (contract oracles) and
|
||||||
|
# doctrine-citers.test.sh (citation census) — a mechanism, not a lesson.
|
||||||
|
#
|
||||||
|
# Adapted from addyosmani/agent-skills constraint-driven-development's
|
||||||
|
# "floor guard" to this repo's own gate model: `git diff` instead of a
|
||||||
|
# staged-diff assumption, wired into agents/verifier.md STEP 3 rather than a
|
||||||
|
# pre-commit hook.
|
||||||
|
#
|
||||||
|
# Scope: `git diff <base-ref>` — working tree included (uncommitted changes
|
||||||
|
# count) — restricted to <pathspec> when given. Every ADDED line is
|
||||||
|
# classified into one of six kinds (full pattern tables below):
|
||||||
|
# SUPPRESS a checker-silencing comment added, any file
|
||||||
|
# SKIP a test disabled or isolated, test files only
|
||||||
|
# DELETED_TEST a whole test file removed
|
||||||
|
# ASSERT_DROP a test file's assertion-line count went down
|
||||||
|
# STUB a not-implemented marker added, any file
|
||||||
|
# THRESHOLD_DOWN a numeric value lowered on the same key, config files only
|
||||||
|
#
|
||||||
|
# Waiver: an added line also carrying `floor-guard: allow <reason>` prints as
|
||||||
|
# WAIVED and does not count toward the finding total or the rc.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
_usage() {
|
||||||
|
echo "usage: floor-guard.sh <base-ref> [-- <pathspec>...]" >&2
|
||||||
|
exit 3
|
||||||
|
}
|
||||||
|
|
||||||
|
[ $# -ge 1 ] || _usage
|
||||||
|
BASE_REF="$1"; shift
|
||||||
|
PATHSPEC=()
|
||||||
|
if [ $# -gt 0 ]; then
|
||||||
|
[ "$1" = "--" ] || _usage
|
||||||
|
shift
|
||||||
|
PATHSPEC=("$@")
|
||||||
|
fi
|
||||||
|
git rev-parse --verify -q "${BASE_REF}^{commit}" >/dev/null 2>&1 || _usage
|
||||||
|
|
||||||
|
TMPDIFF="$(mktemp)" || { echo "floor-guard: mktemp failed" >&2; exit 3; }
|
||||||
|
trap 'rm -f "$TMPDIFF"' EXIT
|
||||||
|
|
||||||
|
git diff --unified=0 "$BASE_REF" -- "${PATHSPEC[@]}" > "$TMPDIFF" 2>/dev/null
|
||||||
|
FLOOR_DELETED_FILES="$(git diff --diff-filter=D --name-only \
|
||||||
|
"$BASE_REF" -- "${PATHSPEC[@]}" 2>/dev/null)"
|
||||||
|
export FLOOR_DELETED_FILES
|
||||||
|
|
||||||
|
# "working tree included" means brand-new, still-untracked files too: plain
|
||||||
|
# `git diff <ref>` never shows them (git only diffs what it already tracks),
|
||||||
|
# so a file added on this branch and never `git add`-ed would be invisible
|
||||||
|
# to every kind below. --no-index against /dev/null emits the same unified
|
||||||
|
# format as the tracked diff above (diff --git / +++ b/path / @@ hunks),
|
||||||
|
# so the parser needs no separate code path for it.
|
||||||
|
while IFS= read -r f; do
|
||||||
|
[ -n "$f" ] || continue
|
||||||
|
git diff --no-index --unified=0 -- /dev/null "$f" >> "$TMPDIFF" 2>/dev/null
|
||||||
|
done < <(git ls-files --others --exclude-standard -- "${PATHSPEC[@]}" 2>/dev/null)
|
||||||
|
|
||||||
|
python3 - "$TMPDIFF" <<'PY'
|
||||||
|
import fnmatch
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
# file classes (CLARIFICATIONS): "path contains test/spec/__tests__" is a
|
||||||
|
# superset of the explicit globs (*.test.*, *.spec.*, *_test.go, *_test.py,
|
||||||
|
# test_*.py all contain one of these substrings themselves), so one check
|
||||||
|
# covers all five.
|
||||||
|
TEST_SUBSTRINGS = ('test', 'spec', '__tests__')
|
||||||
|
CONFIG_GLOBS = ('jest.config*', 'vitest.config*', '.nycrc*', 'codecov*',
|
||||||
|
'sonar-project.properties', 'lighthouserc*', 'CONSTRAINTS.md')
|
||||||
|
|
||||||
|
# ── pattern tables — the trigger strings themselves, waived on this file's
|
||||||
|
# own diff so the guard stays clean on itself (also exercises the waiver
|
||||||
|
# path for real) ────────────────────────────────────────────────────────────
|
||||||
|
SUPPRESS_SUBSTRINGS = (
|
||||||
|
'@ts-ignore', # floor-guard: allow pattern table
|
||||||
|
'eslint-disable', # floor-guard: allow pattern table
|
||||||
|
'# noqa', # floor-guard: allow pattern table
|
||||||
|
'# type: ignore', # floor-guard: allow pattern table
|
||||||
|
'nosemgrep', # floor-guard: allow pattern table
|
||||||
|
'nosec', # floor-guard: allow pattern table
|
||||||
|
'shellcheck disable', # floor-guard: allow pattern table
|
||||||
|
)
|
||||||
|
TS_EXPECT_ERROR = '@ts-expect-error' # floor-guard: allow pattern table
|
||||||
|
|
||||||
|
SKIP_SUBSTRINGS = (
|
||||||
|
'.skip(', '.only(', 'xit(', 'xdescribe(', 'fit(', 'fdescribe(',
|
||||||
|
'it.todo(', '@pytest.mark.skip', '@unittest.skip', 't.Skip(',
|
||||||
|
)
|
||||||
|
|
||||||
|
STUB_SUBSTRINGS = (
|
||||||
|
'not implemented', # floor-guard: allow pattern table
|
||||||
|
'NotImplementedError', # floor-guard: allow pattern table
|
||||||
|
)
|
||||||
|
EMPTY_CATCH_RE = re.compile(r'catch\s*\([^)]*\)\s*\{\s*\}')
|
||||||
|
BARE_EXCEPT_RE = re.compile(r'except\b[^:\n]*:\s*pass\b')
|
||||||
|
|
||||||
|
ASSERT_SUBSTRINGS = ('expect(', 'assert', 'should', '.toBe')
|
||||||
|
|
||||||
|
KEYVAL_RE = re.compile(r'["\']?([A-Za-z0-9_.\-]+)["\']?\s*[:=]\s*(-?\d+(?:\.\d+)?)')
|
||||||
|
WAIVER_RE = re.compile(r'floor-guard:\s*allow\s+(\S.*)$')
|
||||||
|
HUNK_RE = re.compile(r'^@@ -(\d+)(?:,\d+)? \+(\d+)(?:,\d+)? @@')
|
||||||
|
|
||||||
|
|
||||||
|
def is_test_file(path):
|
||||||
|
return any(s in path for s in TEST_SUBSTRINGS)
|
||||||
|
|
||||||
|
|
||||||
|
def is_config_file(path):
|
||||||
|
base = os.path.basename(path)
|
||||||
|
return any(fnmatch.fnmatch(base, g) for g in CONFIG_GLOBS)
|
||||||
|
|
||||||
|
|
||||||
|
def is_waived(text):
|
||||||
|
return bool(WAIVER_RE.search(text))
|
||||||
|
|
||||||
|
|
||||||
|
def strip_prefix(raw):
|
||||||
|
if raw == '/dev/null':
|
||||||
|
return raw
|
||||||
|
return raw[2:] if raw[:2] in ('a/', 'b/') else raw
|
||||||
|
|
||||||
|
|
||||||
|
def _new_file_entry(files):
|
||||||
|
entry = {'old_path': None, 'new_path': None, 'adds': [], 'dels': [],
|
||||||
|
'first_new': None}
|
||||||
|
files.append(entry)
|
||||||
|
return entry
|
||||||
|
|
||||||
|
|
||||||
|
def parse_diff(lines): # → list of per-file entries (adds/dels + paths)
|
||||||
|
files, cur = [], None
|
||||||
|
old_no = new_no = 0
|
||||||
|
for raw in lines:
|
||||||
|
if raw.startswith('diff --git '):
|
||||||
|
cur = _new_file_entry(files)
|
||||||
|
elif raw.startswith('--- '):
|
||||||
|
cur['old_path'] = strip_prefix(raw[4:])
|
||||||
|
elif raw.startswith('+++ '):
|
||||||
|
cur['new_path'] = strip_prefix(raw[4:])
|
||||||
|
elif raw.startswith('@@ '):
|
||||||
|
m = HUNK_RE.match(raw)
|
||||||
|
if m:
|
||||||
|
old_no, new_no = int(m.group(1)), int(m.group(2))
|
||||||
|
if cur['first_new'] is None:
|
||||||
|
cur['first_new'] = new_no
|
||||||
|
elif raw.startswith('+') and not raw.startswith('+++'):
|
||||||
|
cur['adds'].append((new_no, raw[1:])); new_no += 1
|
||||||
|
elif raw.startswith('-') and not raw.startswith('---'):
|
||||||
|
cur['dels'].append((old_no, raw[1:])); old_no += 1
|
||||||
|
return files
|
||||||
|
|
||||||
|
|
||||||
|
def effective_path(entry):
|
||||||
|
if entry['new_path'] not in (None, '/dev/null'):
|
||||||
|
return entry['new_path']
|
||||||
|
return entry['old_path']
|
||||||
|
|
||||||
|
|
||||||
|
def suppress_kind(text):
|
||||||
|
if TS_EXPECT_ERROR in text:
|
||||||
|
after = text.split(TS_EXPECT_ERROR, 1)[1].strip()
|
||||||
|
return None if after else 'SUPPRESS'
|
||||||
|
return 'SUPPRESS' if any(p in text for p in SUPPRESS_SUBSTRINGS) else None
|
||||||
|
|
||||||
|
|
||||||
|
def stub_kind(text):
|
||||||
|
if any(p in text for p in STUB_SUBSTRINGS):
|
||||||
|
return 'STUB'
|
||||||
|
if EMPTY_CATCH_RE.search(text) or BARE_EXCEPT_RE.search(text):
|
||||||
|
return 'STUB'
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def skip_kind(text):
|
||||||
|
return 'SKIP' if any(p in text for p in SKIP_SUBSTRINGS) else None
|
||||||
|
|
||||||
|
|
||||||
|
def line_findings(path, lineno, text, test_file):
|
||||||
|
out, waived = [], is_waived(text)
|
||||||
|
for kindfn in (suppress_kind, stub_kind):
|
||||||
|
kind = kindfn(text)
|
||||||
|
if kind:
|
||||||
|
out.append((kind, path, lineno, text, waived))
|
||||||
|
if test_file:
|
||||||
|
kind = skip_kind(text)
|
||||||
|
if kind:
|
||||||
|
out.append((kind, path, lineno, text, waived))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _is_assertion(text):
|
||||||
|
return any(p in text for p in ASSERT_SUBSTRINGS)
|
||||||
|
|
||||||
|
|
||||||
|
def assert_drop_finding(path, entry):
|
||||||
|
added = sum(1 for _, t in entry['adds'] if _is_assertion(t))
|
||||||
|
removed = sum(1 for _, t in entry['dels'] if _is_assertion(t))
|
||||||
|
if removed <= added:
|
||||||
|
return None
|
||||||
|
lineno = entry['adds'][0][0] if entry['adds'] else (entry['first_new'] or 1)
|
||||||
|
waived = any(is_waived(t) for _, t in entry['adds'])
|
||||||
|
snippet = 'assertion lines %d -> %d' % (removed, added)
|
||||||
|
return ('ASSERT_DROP', path, lineno, snippet, waived)
|
||||||
|
|
||||||
|
|
||||||
|
def extract_kv(lines): # → {key: (lineno, value, raw text)} last-wins
|
||||||
|
kv = {}
|
||||||
|
for lineno, text in lines:
|
||||||
|
m = KEYVAL_RE.search(text)
|
||||||
|
if m:
|
||||||
|
kv[m.group(1)] = (lineno, float(m.group(2)), text)
|
||||||
|
return kv
|
||||||
|
|
||||||
|
|
||||||
|
def threshold_down_findings(path, entry):
|
||||||
|
removed_kv = extract_kv(entry['dels'])
|
||||||
|
added_kv = extract_kv(entry['adds'])
|
||||||
|
out = []
|
||||||
|
for key, (lineno, new_val, text) in added_kv.items():
|
||||||
|
old = removed_kv.get(key)
|
||||||
|
if old and new_val < old[1]:
|
||||||
|
out.append(('THRESHOLD_DOWN', path, lineno, text, is_waived(text)))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def classify_file(entry, deleted_paths):
|
||||||
|
path = effective_path(entry)
|
||||||
|
if path is None:
|
||||||
|
return [] # pure rename/mode-change: no --- / +++ header, no content diff
|
||||||
|
test_file = is_test_file(path)
|
||||||
|
if path in deleted_paths and test_file:
|
||||||
|
return [('DELETED_TEST', path, 1, path, False)]
|
||||||
|
findings = []
|
||||||
|
for lineno, text in entry['adds']:
|
||||||
|
findings += line_findings(path, lineno, text, test_file)
|
||||||
|
if test_file:
|
||||||
|
dropped = assert_drop_finding(path, entry)
|
||||||
|
if dropped:
|
||||||
|
findings.append(dropped)
|
||||||
|
if is_config_file(path):
|
||||||
|
findings += threshold_down_findings(path, entry)
|
||||||
|
return findings
|
||||||
|
|
||||||
|
|
||||||
|
def load_deleted_paths():
|
||||||
|
raw = os.environ.get('FLOOR_DELETED_FILES', '')
|
||||||
|
return {p for p in raw.splitlines() if p}
|
||||||
|
|
||||||
|
|
||||||
|
def snippet_of(text):
|
||||||
|
return text.strip()[:100]
|
||||||
|
|
||||||
|
|
||||||
|
def emit(findings):
|
||||||
|
ordered = sorted(findings, key=lambda f: (f[1], f[2], f[0]))
|
||||||
|
n_found = n_waived = 0
|
||||||
|
for kind, path, lineno, text, waived in ordered:
|
||||||
|
tag = 'WAIVED' if waived else 'FLOOR'
|
||||||
|
print('%s %s %s:%d %s' % (tag, kind, path, lineno, snippet_of(text)))
|
||||||
|
n_waived += 1 if waived else 0
|
||||||
|
n_found += 0 if waived else 1
|
||||||
|
if n_found:
|
||||||
|
print('FLOOR GUARD: %d finding(s), %d waived' % (n_found, n_waived))
|
||||||
|
return 2
|
||||||
|
print('FLOOR GUARD: clean')
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
with open(sys.argv[1], 'r', errors='replace') as fh:
|
||||||
|
lines = fh.read().split('\n')
|
||||||
|
deleted = load_deleted_paths()
|
||||||
|
findings = []
|
||||||
|
for entry in parse_diff(lines):
|
||||||
|
findings += classify_file(entry, deleted)
|
||||||
|
return emit(findings)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
sys.exit(main())
|
||||||
|
PY
|
||||||
|
rc=$?
|
||||||
|
exit "$rc"
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# lib/tests/floor-guard.test.sh — flip-tests for lib/floor-guard.sh: one RED
|
||||||
|
# fixture per KIND, one WAIVED fixture, one CLEAN fixture. Each fixture is a
|
||||||
|
# fresh throwaway repo under $WORK (`make test` exports
|
||||||
|
# GIT_CONFIG_GLOBAL=/dev/null; core.hooksPath is also pinned per-repo so a
|
||||||
|
# machine-wide hook never fires here). This file itself carries the trigger
|
||||||
|
# strings for every kind — the self-run criterion excludes it by pathspec.
|
||||||
|
set -u
|
||||||
|
ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
|
||||||
|
LIB="$ROOT/lib/floor-guard.sh"
|
||||||
|
WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
|
||||||
|
pass=0; fail=0
|
||||||
|
|
||||||
|
# check_kind <KIND> <rc> <want_rc> <out> <want_substr>
|
||||||
|
check_kind() {
|
||||||
|
local kind="$1" rc="$2" want_rc="$3" out="$4" want_sub="$5"
|
||||||
|
if [ "$rc" = "$want_rc" ] && printf '%s\n' "$out" | grep -qF -- "$want_sub"; then
|
||||||
|
pass=$((pass+1)); echo "PASS $kind"
|
||||||
|
else
|
||||||
|
fail=$((fail+1))
|
||||||
|
printf 'FAIL %s: rc=%s (want %s), out:\n%s\n' \
|
||||||
|
"$kind" "$rc" "$want_rc" "$(printf '%s\n' "$out" | tail -5)"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# mk_repo <name> → path to a fresh throwaway repo: one test file (3
|
||||||
|
# assertions), one vitest.config.ts (coverage.lines: 80), one source file.
|
||||||
|
mk_repo() {
|
||||||
|
local d="$WORK/$1"
|
||||||
|
mkdir -p "$d/src"
|
||||||
|
git init -q "$d"
|
||||||
|
git -C "$d" config user.email t@t
|
||||||
|
git -C "$d" config user.name t
|
||||||
|
git -C "$d" config core.hooksPath /dev/null
|
||||||
|
printf 'expect(1).toBe(1);\nexpect(2).toBe(2);\nexpect(3).toBe(3);\n' \
|
||||||
|
> "$d/sample.test.js"
|
||||||
|
printf 'export default {\n coverage: {\n lines: 80,\n },\n};\n' \
|
||||||
|
> "$d/vitest.config.ts"
|
||||||
|
printf 'function add(a, b) {\n return a + b;\n}\n' > "$d/src/index.js"
|
||||||
|
git -C "$d" add -A
|
||||||
|
git -C "$d" commit -q -m base
|
||||||
|
echo "$d"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── SUPPRESS ──────────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo suppress); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
echo '// eslint-disable-next-line no-console' >> "$d/src/index.js"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind SUPPRESS "$rc" 2 "$out" 'FLOOR SUPPRESS'
|
||||||
|
|
||||||
|
# ── SKIP ──────────────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo skip); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
echo "it.skip('later', () => {});" >> "$d/sample.test.js"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind SKIP "$rc" 2 "$out" 'FLOOR SKIP'
|
||||||
|
|
||||||
|
# ── DELETED_TEST ──────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo deleted); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
rm "$d/sample.test.js"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind DELETED_TEST "$rc" 2 "$out" 'FLOOR DELETED_TEST'
|
||||||
|
|
||||||
|
# ── ASSERT_DROP ───────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo assertdrop); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
printf 'expect(1).toBe(1);\n' > "$d/sample.test.js"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind ASSERT_DROP "$rc" 2 "$out" 'FLOOR ASSERT_DROP'
|
||||||
|
|
||||||
|
# ── STUB ──────────────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo stub); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
echo "function todo() { throw new Error('not implemented'); }" >> "$d/src/index.js"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind STUB "$rc" 2 "$out" 'FLOOR STUB'
|
||||||
|
|
||||||
|
# ── THRESHOLD_DOWN ────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo threshold); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
sed -i 's/lines: 80/lines: 60/' "$d/vitest.config.ts"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind THRESHOLD_DOWN "$rc" 2 "$out" 'FLOOR THRESHOLD_DOWN'
|
||||||
|
|
||||||
|
# ── WAIVED ────────────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo waived); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
echo '// eslint-disable-next-line no-console -- floor-guard: allow legacy shim' \
|
||||||
|
>> "$d/src/index.js"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind WAIVED "$rc" 0 "$out" 'WAIVED'
|
||||||
|
|
||||||
|
# ── CLEAN ─────────────────────────────────────────────────────────────────
|
||||||
|
d=$(mk_repo clean); base=$(git -C "$d" rev-parse HEAD)
|
||||||
|
echo '// helper' >> "$d/src/index.js"
|
||||||
|
out=$(cd "$d" && bash "$LIB" "$base" 2>&1); rc=$?
|
||||||
|
check_kind CLEAN "$rc" 0 "$out" 'FLOOR GUARD: clean'
|
||||||
|
|
||||||
|
echo "PASS=$pass FAIL=$fail"
|
||||||
|
[ "$fail" -eq 0 ]
|
||||||
@@ -55,6 +55,16 @@ Dispatch a FRESH verifier subagent (`subagent_type: verifier`, or load
|
|||||||
`TEST` command. Never pass the dev's summary, never pass a prior iteration's
|
`TEST` command. Never pass the dev's summary, never pass a prior iteration's
|
||||||
gaps — the verifier reads the contract from disk and judges blind.
|
gaps — the verifier reads the contract from disk and judges blind.
|
||||||
|
|
||||||
|
The verifier's STEP 3 (`agents/verifier.md`) runs `lib/floor-guard.sh`
|
||||||
|
against the diff before it renders any verdict — a deterministic,
|
||||||
|
diff-scoped check for a quietly weakened quality bar (a suppressed
|
||||||
|
lint/type check, a skipped or deleted test, a dropped assertion, a lowered
|
||||||
|
coverage threshold) that an LLM verdict alone can miss or be talked past
|
||||||
|
one line at a time. Its findings fold straight into that same verifier's
|
||||||
|
`ECARTS` count unless the contract's `CLARIFICATIONS` explicitly authorizes
|
||||||
|
the exact weakening; there is no separate gate and no extra dispatch, it
|
||||||
|
rides this GATE 1 call.
|
||||||
|
|
||||||
Parse its single `VERIFY — VERDICT:` line:
|
Parse its single `VERIFY — VERDICT:` line:
|
||||||
|
|
||||||
- `CONFORME` → go to GATE 2. (First-pass conforme = no loop.)
|
- `CONFORME` → go to GATE 2. (First-pass conforme = no loop.)
|
||||||
@@ -74,9 +84,9 @@ Parse its single `VERIFY — VERDICT:` line:
|
|||||||
micro-gate that appends `[gated <date>]` to the contract's FILE SCOPE;
|
micro-gate that appends `[gated <date>]` to the contract's FILE SCOPE;
|
||||||
otherwise the dev removes the file.
|
otherwise the dev removes the file.
|
||||||
- Structural failure (`ERROR(…)`, missing/duplicated VERDICT line,
|
- Structural failure (`ERROR(…)`, missing/duplicated VERDICT line,
|
||||||
unparsable, crash, `CONFORME` without `PROOF`) → retry ONCE with a fresh
|
unparsable, crash, `CONFORME` without `PROOF` or without `FLOOR`) → retry
|
||||||
verifier; a 2nd structural failure → human escalation. A mute verifier is
|
ONCE with a fresh verifier; a 2nd structural failure → human escalation.
|
||||||
NEVER a PASS.
|
A mute verifier is NEVER a PASS.
|
||||||
|
|
||||||
## GATE 2 — SECURITY (fresh security-auditor)
|
## GATE 2 — SECURITY (fresh security-auditor)
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user