forked from bchanot/claude
chore(memory): BDR-083 + LRN-141/142 + journal + CHANGELOG + TODO W0-W8
BDR-083 records what was taken from unlazy and, more usefully, what was refused and why. LRN-141: an external skill's machinery encodes its threat model, not yours — take the invariants, refuse the machinery. LRN-142: structure locks are fixed-string, so reflowing a doctrine paragraph reds them; fix the doc, not the lock.
This commit is contained in:
@@ -92,6 +92,7 @@ rules:
|
||||
| BDR-072 | 2026-07-17 | SPA: honest refuse (On-page N/A, not zero), no headless browser (R2 over R1) | accepted |
|
||||
| BDR-073 | 2026-07-17 | Scoring: LLM judges findings+severity, engine does the arithmetic (deterministic /20) | accepted |
|
||||
| BDR-080 | 2026-07-21 | Bug routing inverted: /bugfix primary, /investigate explicit-only | accepted |
|
||||
| BDR-083 | 2026-08-24 | Contract gates: deterministic floor (GATE 0) under the fresh verifier | accepted |
|
||||
|
||||
---
|
||||
|
||||
@@ -1079,3 +1080,11 @@ Opus 5 (released 2026-07-24) now backs every `model: opus` pin (BDR-076/077) + a
|
||||
|
||||
### BDR-082 — seo/geo analyzers de-prescribed for Opus 5 (C1) [accepted] (2026-08-02)
|
||||
BDR-081 N5 follow-on, user-directed apparatus (plan+3-lens challenge+census+dogfood). Method: audience×mode-range invariant — dedup ONLY verbatim same-audience (spec rule / bundle-item payload / phase-local caveat) same-mode-range repeats; cross-mode + agent↔dispatcher twins stay (standalone paths need them). Census-FIRST: lib/tests/seo-geo-contract.test.sh 71 locks (verdict grammar, sentinels, ALL STEP headers incl. interiors, item fields, score labels, envelope keys), flip-proven 7 mutations→7 FAILs, committed BEFORE reword. Shipped: self-output verification removed (":970 run twice"→conditional integrity guard; ":1217"→single-shot-scoped), 2 pre-BDR-061 vestigials fixed, caps softened (P0-rule/MANDATORY/ALWAYS→plain content rules), 2 essays compressed, checklist :1309→routing map rows verbatim (challenger caught it = routing table, NOT self-check), true same-range dups only (seo Handoff+landing-page blocks; geo ZERO — all claimed pairs distinct on inspection). FROZEN: guard-first url-guard orderings, :550 denominator-before-sampling (ordering IS the honesty mechanism), R2/NAP/COVERAGE/citation invariants, external-freshness checks (world drift ≠ self-verification). Deltas: seo 1528→1503 l ("P0 rule" 2→0, ALWAYS 1→0, MUST 5→4, NEVER 9→9 = class-B bans kept); geo 1106→1107 (MANDATORY 1→0, MUST 4→3). Plan challenged correctness FATAL / robustness FATAL(3 BLOCKER) / simplicity CONCERNS + confirmation FATAL(9) — every BLOCKER closed by named change (§5bis record). Dogfood before/after on frozen zenquality copy: judge-replay on frozen signals (zero collect variance) + templates + fresh collects + e2e judge + 42/42 assert battery BOTH sets + blind reader "interchangeable; all deltas = presentation variance both directions OR after MORE spec-conformant". Alternatives rejected: keyword dedup (challengers proved audience/range-blind — most annex "twins" were distinct obligations), FULL/aggressive dogfood (billing gate killed nested CLI; left as user option), banner/shape locks (LLM-convention layers wobble — lock strings only). Evidence: .audit/dogfood-baseline/ (18 artifacts + DOGFOOD-VERDICT.md), plan .claude/tasks/plans/2026-07-30-seo-geo-deprescription-1402.md. Branch feature/seo-geo-deprescription, UNMERGED (human gate).
|
||||
|
||||
### BDR-083 — contract gates: deterministic floor (GATE 0) under the verifier [accepted] (2026-08-24)
|
||||
User asked what to take from `unlazy` skill (Leonxlnx/unlazy 2.1.0, MIT). Verdict on its verification ARCHITECTURE: teaches nothing we lack — contract + fresh blind verifier + bounded loops + order invariant already shipped (BDR-049/050/066, LRN-083). Real gap found elsewhere: between executor and GATE 1, NO deterministic floor. GATE 1 = LLM dispatch; verifier's mandatory `PROOF:` line = a line the verifier WRITES — nothing structurally stops it being produced without executing anything (LRN-048 demands a pass prove it looked; the proof is self-reported prose). Decision: import unlazy's gate ledger INTO the existing contract, never alongside it. Palier 2, user-chosen over doctrine-only / defer.
|
||||
TAKEN: criterion carries an oracle (indented `CHECK:` cmd + `EXPECT:` success-only marker + `EVIDENCE:` slot); fail-closed = exit 0 AND marker (a nonzero process never passes because its error text carries the token); evidence persisted INTO the contract → the fresh verifier reads fact, not the executor's report; `ABANDON: <id> <non-blank reason>` = impossible criterion never deleted, blocks CONFORME, routes to human gate (new verdict token `ABANDONED(n)` — distinct routing from ECARTS ⇒ distinct token, not a sub-line to re-derive); 4 gate-authoring rules (observe the named artifact / success-only marker / positive control before any absence check / recompute supplied numbers, never copy one into EXPECT); 4-pass executor discipline (feater full; bugfixer narrowed to fix+test under "keep the fix minimal", pass 3 = negative control proving the regression test fails without the fix).
|
||||
REFUSED + why: Stop hook `decision:"block"` — contradicts "STOP + human escalation", "gone WRONG → STOP re-plan", "merge only on explicit human signal"; a hook FORCING continuation is the inverse of our gates; its 6-block release either traps the session or gives up; each block = an agent continuation = real tokens. Approval store `~/.unlazy/approved` (binds ledger+cmd+CWD+shell+timeout+platform+full PATH) — exists to execute ledgers INHERITED from untrusted repos; our contracts are authored by our own orchestrator in our own repo ⇒ biggest chunk of their 28k checker closes zero threat here. `.unlazy/<scope>/` tree (PLAN+GATES+gates/+status.log+session+hook-state+locks/) — a 4th bookkeeping tree beside .claude/tasks/{contracts,plans} + memory/ + audits/. `tree N` Depth-Tree effort arithmetic — disowned by unlazy's OWN research/validation-protocol.md (v1 six-run figures unreproducible), while the repo DESCRIPTION still advertises the retracted claim. Node checker (28k .mjs + 54k .mjs tests) — lib stack is 100% bash, Health Stack = `shellcheck *.sh hooks/*.sh lib/*.sh` would cover none of it. `OWNS:` ownership leases — deferred (Palier 3): our parallel dispatches (seo/geo, 3 plan-challengers) are read-only, the write-collision problem does not exist yet.
|
||||
Shipped: lib/gates.sh (~250 l bash; `status` never executes and never writes · `run` ALWAYS re-executes every runnable criterion — trusting written evidence is the failure being closed, so there is no incremental mode to get wrong; rc 0 MET / 2 UNMET|malformed / 3 ABANDONED; parse fails closed on partial oracle, duplicate id, unindented attribute, runnable-without-EVIDENCE, and executes nothing when the ledger is malformed). GATE 0 in lib/verify-secure-loop.md (red floor → executor re-dispatch with the NOT-MET rows, NO verifier spawned; own 3-iteration budget, separate from conformity; malformed ledger fixed in the main loop, never dispatched to a dev). Order invariant now GATE 0→1→2 on every re-loop. lib/contract-interview.md: ORACLES section + template + ABANDONMENT lifecycle + per-flow oracle weight. agents/verifier.md: oracle-consumption rules — a red or unrun oracle is NEVER overridden by reading code; a MET oracle proves the ORACLE, not the English sentence ⇒ vacuous oracle = NOT-MET, the one judgement no command can make; verifier may re-run a CHECK but never edits the contract. lib/tests/gates.test.sh 64 assertions (sentinel-proved non-execution, with its own positive control asserted first).
|
||||
Alternatives rejected: Palier 1 doctrine-only (CHECK:/EXPECT: become decorative without an executant); port the Node checker (stack break, shellcheck-blind); fold ABANDONED into ECARTS (would send a dev to fix the impossible and eat the 3-iteration budget); `status` revalidating old evidence (that trust is the failure being closed).
|
||||
Branch feature/contract-gates, UNMERGED (human gate). `make test` rc 0, shellcheck clean, e2e verified on a real contract in the documented template.
|
||||
|
||||
@@ -433,3 +433,9 @@ rules:
|
||||
|
||||
## 2026-08-02
|
||||
- C1 seo/geo de-prescription EXECUTED end-to-end: census-first 71 locks flip-proven → reword under audience×range invariant (adafa35/c7646a9) → controlled dogfood (judge-replay frozen signals + templates + fresh collects + e2e + blind reader) → 42/42 both sets, zero contract regression, recall improved. Plan survived 4 challenge passes (2 FATAL + confirmation FATAL(9), all closed by name). BDR-082 + LRN-140. Nested-CLI dogfood died on monthly spend limit → inline pipeline (canonical /seo shape). feature/seo-geo-deprescription UNMERGED (human gate). Chantiers C2-C4 pending.
|
||||
|
||||
## 2026-08-24
|
||||
- Analysed `unlazy` skill (Leonxlnx/unlazy 2.1.0) on user request. Its verification architecture teaches us nothing — contract + fresh blind verifier + bounded loops already shipped. Real gap: no deterministic floor between executor and GATE 1 (the verifier's `PROOF:` is a line it writes, not a process exit).
|
||||
- Shipped Palier 2 (user-chosen): lib/gates.sh + GATE 0 + oracle-bearing criteria + `ABANDONED(n)` verdict + 4-pass executors. Refused unlazy's Stop hook, approval store, .unlazy/ tree, tree-N arithmetic, Node checker — [[BDR-083]] records each why.
|
||||
- `make test` rc 0, shellcheck clean, 64 new assertions, e2e on a real contract. Branch feature/contract-gates UNMERGED (human gate).
|
||||
- Locks caught a reflow regression (5 red on rewrapped phrases, zero doctrine lost) → [[LRN-142]]. Skill-adoption pattern → [[LRN-141]].
|
||||
|
||||
@@ -1368,3 +1368,13 @@ rules:
|
||||
- **pattern 3 — de-prescription does NOT depress recall**: reworded collect caught -encoded phone AT COLLECT (baseline collect missed it); reworded judge found new RGPD finding + self-caught false positive + corrected collect coverage claim 21/21→20/21. Integrity/honesty invariants (kept class B) carry the discipline, not the caps.
|
||||
- **pattern 4 — lock strings, never shapes**: LLM-convention output layers (banners, fences, table columns, section order) wobble run-to-run in BOTH directions — baseline itself deviated from spec where after conformed (§0 ENTRIES, BUNDLE-before-SCORING). Stable contract = census-locked literal strings; anything unlocked drifts and MUST be tolerated by consumers (tier recognition "by intent" is the right pattern).
|
||||
- **link**: [[BDR-082]] [[BDR-081]] [[LRN-139]] [[LRN-113]].
|
||||
|
||||
## LRN-141 — adopting an external skill: take the invariants, refuse the machinery (2026-08-24)
|
||||
Context: unlazy import ([[BDR-083]]). Pattern: an external skill's MACHINERY encodes ITS threat model and ITS doctrine; only its INVARIANTS transfer. Two clean cases from one repo. (1) Approval store binding PATH/shell/platform exists because unlazy executes ledgers INHERITED from untrusted repos — importing it into a config that authors its own ledgers buys per-command approval prompts and closes zero threat. (2) Stop hook returning `decision:"block"` exists because unlazy has no human gate — importing it into a config whose spine is "STOP + escalate to human" would make the tooling fight the doctrine. Meanwhile the invariants (exit 0 AND marker; evidence persisted so the next reader gets fact not report; impossible ≠ deletable) cost ~250 l of our own bash and fit the EXISTING contract with no new tree.
|
||||
Separating test: ask WHAT THREAT / WHAT DOCTRINE does this piece assume. Answer "theirs" → refuse the piece, keep the invariant it was protecting.
|
||||
Corollary on claims: unlazy's own research/validation-protocol.md RETRACTS its v1 benchmark numbers as unreproducible while the repo DESCRIPTION still advertises them. Read a project's self-criticism before its README — the retraction is the credibility signal, the headline is not.
|
||||
Future application: any skill/plugin adoption — skills-external/, /plugin-check, install-plugins.sh.
|
||||
|
||||
## LRN-142 — structure locks are fixed-string: reflowing a doctrine paragraph reds them (2026-08-24)
|
||||
Context: contract-gates ([[BDR-083]]). Editing lib/verify-secure-loop.md rewrapped 5 locked phrases across line breaks ("Max 3 conformity iterations", "Max 3 security iterations", "re-verify the REQUEST first", "always re-checked BEFORE security", "one verifier dispatch + one security dispatch") → loops-light.test.sh 30 pass / 5 fail, though ZERO doctrine was dropped. Locks did their job: they cannot distinguish "clause deleted" from "clause rewrapped", and that conservative bias is correct — the alternative (fuzzy matching) would miss real deletions.
|
||||
Rule: when editing a doctrine file under structure locks, grep the test's lock strings FIRST, then re-flow AROUND them — each locked phrase stays on one unbroken line. Fix the DOC, not the lock, unless the doctrine genuinely changed. Under locks today: verify-secure-loop.md, contract-interview.md, verifier / security-auditor / plan-challenger agents, seo+geo (71 locks).
|
||||
|
||||
@@ -1144,3 +1144,42 @@ branch) → LOT3 mis-merge trap; + 3 doctor false-warns (LRN-047 class).
|
||||
comment anchored to measured ~11.4k (LRN-088). False "92% CRITICAL" → ~5% comfortable.
|
||||
- [x] Verify — suites green (71/13/32/19/20/13 + RC 5/5); doctor 0 false-warn; shellcheck clean.
|
||||
+docs(changelog) Unreleased entry (706abff). Gate passed on GO 2026-07-03. Finish pending.
|
||||
|
||||
## 2026-08-24 — contract gates: plancher déterministe (feature/contract-gates)
|
||||
Source: analyse du skill `unlazy` (Leonxlnx/unlazy, 2.1.0). Verdict: son
|
||||
architecture de vérification n'apprend rien (contrat+verifier frais+boucles
|
||||
bornées ⊂ déjà en place). Le trou réel: **entre l'exécuteur et GATE 1 il n'y a
|
||||
aucun plancher déterministe** — GATE 1 est un dispatch LLM, et `PROOF:` est une
|
||||
ligne que le verifier ÉCRIT (rien ne l'empêche structurellement de la produire
|
||||
sans rien exécuter). Palier 2 retenu (user, 2026-08-24).
|
||||
|
||||
PRIS d'unlazy: critère porteur d'oracle exécutable (CHECK/EXPECT/EVIDENCE),
|
||||
fail-closed (exit 0 ET marqueur), evidence pending = NOT-MET, `ABANDON: <id>
|
||||
<raison>` comme handoff visible non supprimable, les 4 règles d'écriture de
|
||||
gates falsifiables, la discipline 4 passes.
|
||||
REFUSÉ: Stop hook `decision:"block"` (contredit "STOP + escalade humaine" et
|
||||
"merge sur signal humain"), approval store `~/.unlazy/approved` (résout
|
||||
l'exécution de ledgers hérités non fiables — pas notre menace), arbre
|
||||
`.unlazy/<scope>/` (4e arbre de bookkeeping ⇒ mort de la config), `tree N`
|
||||
(désavoué par ses propres docs), le checker Node 28k (stack lib = 100% bash,
|
||||
Health Stack = shellcheck).
|
||||
|
||||
- [x] W0 branche feature/contract-gates depuis develop (via lib/gitflow.sh)
|
||||
- [x] W1 `lib/gates.sh` — parse ACCEPTANCE CRITERIA, exécute fail-closed
|
||||
(exit 0 ET EXPECT), réécrit EVIDENCE dans le contrat. Sous-commandes
|
||||
`run` (exécute+écrit) / `status` (parse seul, jamais d'exécution, jamais
|
||||
d'écriture). rc 0=MET · 2=UNMET/malformé · 3=ABANDONED.
|
||||
- [x] W2 `lib/contract-interview.md` — STEP 3 gagne CHECK/EXPECT/EVIDENCE
|
||||
optionnels par critère + les 4 règles de falsifiabilité; template mis à
|
||||
jour; ABANDON dans Lifecycle; ligne de poids par flow.
|
||||
- [x] W3 `agents/verifier.md` — EVIDENCE fail-closed (coché+pending = NOT-MET),
|
||||
bucket ABANDONED, verdict `CONFORME` impossible si abandon présent.
|
||||
- [x] W4 `lib/verify-secure-loop.md` — GATE 0 déterministe avant GATE 1
|
||||
(rouge ⇒ re-dispatch exécuteur sans brûler un verifier).
|
||||
- [x] W5 `agents/feater.md` + `agents/bugfixer.md` — discipline 4 passes.
|
||||
- [x] W6 `lib/tests/gates.test.sh` — comportemental sur gates.sh (fail-closed,
|
||||
exit≠0 avec marqueur = FAIL, pending, ABANDON, malformé, status
|
||||
n'exécute pas) + locks de structure sur W2/W3/W4/W5.
|
||||
- [x] W7 shellcheck + bash -n + `make test` complet.
|
||||
- [x] W8 CHANGELOG + registres (BDR + LRN + journal).
|
||||
- [ ] W9 PAS de merge — gate humain explicite.
|
||||
|
||||
Reference in New Issue
Block a user