chore(memory): EVAL-028 + LRN-143/144 + BDR-086 + journal — darwin run capitalized; TODO round-count corrected

This commit is contained in:
Bastien Chanot
2026-08-26 23:00:41 +02:00
parent 7f457f09fd
commit a15854aa87
5 changed files with 32 additions and 1 deletions
+8
View File
@@ -95,6 +95,7 @@ rules:
| BDR-083 | 2026-08-24 | Contract gates: deterministic floor (GATE 0) under the fresh verifier | accepted |
| BDR-084 | 2026-08-24 | /tour multi-project: parallel runners (LRN-083 derogation, bounded), runner inherits session model | accepted |
| BDR-085 | 2026-08-25 | User permanent rules: writing-style always-on in rules/, web build+security path-scoped | accepted |
| BDR-086 | 2026-08-26 | darwin: threshold gates full loops; verified defects fixed regardless of unit score (paired-validated, batched checkpoint) | accepted |
---
@@ -1105,3 +1106,10 @@ Conflict carve-outs, stated INSIDE the rules: registries keep caveman format (fr
Self-check rule scoped to DELIVERABLES (text, site, feature), not every conversational reply — literal "avant de me rendre quoi que ce soit" would append a compliance note to every chat answer, pure noise. User can re-widen.
Alternatives rejected: compress into CLAUDE.global.md (~11 lines to fit → loses the carve-outs, zero headroom left); path-scope writing-style (applies to conversation, not file reads → would never fire in chat-only sessions); one merged web file (two concerns, one-rule-one-file).
Branch feature/user-writing-web-rules, UNMERGED (human gate).
## BDR-086 — darwin bug-pass scope: verified defects fixed above threshold
- **Date**: 2026-08-26
- **Decision**: units < threshold get full weighted-gap optimization loops (per-unit checkpoint). Judge-VERIFIED defects (file:line, confirmed) in above-threshold units get targeted fixes in a grouped pass — same paired 3-judge validation, one batched checkpoint. User-gated at the scorecard.
- **Why**: leaving a verified destructive path (hotfix `git restore .` wiping tolerated user edits, file scored 85) unfixed = score-worship; rubric serves quality, not the inverse.
- **Alternatives rejected**: strict threshold (ships known bugs); optimize-everything (cost, HL-4 diminishing returns).
- **Reference**: run 2026-08-26, commits 6eceedb..6eac7fb, `.claude/audits/DARWIN-2026-08-26.md`.
+7
View File
@@ -38,6 +38,7 @@ rules:
| EVAL-019 | 2026-07-06 | job4 test-gap audit + execution: 11 specs + 5 fixes/seams, every mutation red-green verified, zero residual | keep |
| EVAL-025 | 2026-07-17 | opening seo/geo inventory (subagents): 7/7 verifiable claims false or overstated; real contact corrected all, 6 plan corrections + 4 features killed at measurement | keep |
| EVAL-027 | 2026-08-24 | contract-gates behavioral RED: 16/16 fresh unprimed runs followed new doctrine (GATE 0 order, vacuous oracle, ABANDONED routing, scope temptation resisted) | keep |
| EVAL-028 | 2026-08-26 | darwin v2.1 paired run 54 units: 60 paired verdicts 0 revert/tie; skeptics found 3 real residuals — engaged, not rubber-stamp | keep |
---
@@ -260,3 +261,9 @@ Dogfood: 3 blind lenses attacked the v1 plan for the plan-challenge feature itse
- **anomalies**: none against doctrine. Fixture flaw (mine): placeholder.txt trick used to fabricate a 2nd commit made v2/v3 diffs contain no feature work — every verifier CAUGHT it (out-of-scope + "implementation pre-exists base commit"), polluting v3's intended pure-ABANDONED measurement → v3b clean fixture added. Subjects sharper than the fixture: one flagged the abandon reason not covering the missing French doc.
- **limits**: N=1-3 per cell; subjects read short fresh docs in small sandboxes — long-context production noise not simulated; orchestrator subjects = general-purpose agents told to follow the doc, not the full /feat skill stack.
- **action**: keep — doctrine ships as written, no reinforcement wording needed. Artifacts: scratchpad/red/ (session-lived, not committed).
## EVAL-028 — darwin v2.1 paired run, 54 units
- **Date**: 2026-08-26. **Output**: 12 optimization rounds (13 sub-80 units) + 8 bug-fix commits, all kept.
- **Method**: paired same-judge 3-majority per round (v2.1); judges live-exec where artifact executable (5 units: skills-perso, profile, plugin-pair, status-reporter, gitflow). Absolute scores triage-only. Totals main-thread (LRN-018 applied).
- **Anomalies**: (1) 0 reverts/ties in 60 verdicts — homogeneous-better checked: skeptic lens found real residuals 3x (doctor.sh cost source, hotfix RULES leftover restore, FILE(S) new-marker) → judges engaged. (2) census lock RED on line-rewrap, make test caught → LRN-144. (3) head-pipe masked grep exit 2x → LRN-143.
- **Action**: v2.1 paired = standard. Post-run absolute rescore skipped by design (would be judge-noise theater).
+5
View File
@@ -443,3 +443,8 @@ rules:
## 2026-08-25
- User permanent rules integrated: rules/writing-style.md (always-on) + web-building.md + web-security.md (path-scoped). Security core already in §Security, not duplicated. Carve-outs protect caveman registries + skill templates + brand fonts. [[BDR-085]]. Branch feature/user-writing-web-rules UNMERGED (human gate).
## 2026-08-26 — darwin fresh baseline + threshold run (feature/darwin-optimize-20260825, UNMERGED)
- `/darwin-skill all skills and agents` (background). Fresh results.tsv (May file wiped). 7 blind judges, 54 rows (31 skill-systems + 23 agents), mean 83.4, 13 <80. find-docs excluded — machine-owned ctx7 (gitignored), 3rd exclusion ground after BDR-015/058.
- Phase 2: 12 rounds / 13 units, 0 reverts, all paired 3-0 ([[EVAL-028]]). Star: skills-perso detection 8/31 → 31/31 live-verified. Bug pass [[BDR-086]]: 8 commits in above-80 units kept 3-0 (hotfix git-restore data-loss path ★, onboarder contract bounce, plugin data-flow, plan-challenger grammar, handover stale §refs + gate order, tour report-only commit, harden severity, fixtures).
- make test green after census-rewrap fix ([[LRN-144]]); [[LRN-143]] head-pipe grep mask. 29 commits, report `.claude/audits/DARWIN-2026-08-26.md` + card PNG. Branch awaits human review + merge.
+11
View File
@@ -138,6 +138,7 @@ rules:
| LRN-133 | 2026-07-17 | an omission must stay LEGIBLE, never silent — tool that can't measure says so in its output | designing any audit/measure output; deciding what a cap/refusal/N-A emits |
| LRN-134 | 2026-07-17 | resolve-then-pin in stdlib http.client beats monkeypatching getaddrinfo — dual-stack, thread-safe, no requests; classify the OS-resolved IP not the URL text | closing SSRF/DNS-rebinding on any Python HTTP egress |
| LRN-135 | 2026-07-17 | a prefix-only scan for a dangerous construct is bypassable by padding — scan the WHOLE document | refusing any hostile construct (DTD/directive/marker) before parse |
| LRN-143 | 2026-08-26 | `cmd | head || fallback` — pipeline rc is head's (0), fallback dead; bounded output → drop head, else pipefail | any probe/fallback bash in skills before trusting `||` |
---
@@ -1378,3 +1379,13 @@ Future application: any skill/plugin adoption — skills-external/, /plugin-chec
## LRN-142 — structure locks are fixed-string: reflowing a doctrine paragraph reds them (2026-08-24)
Context: contract-gates ([[BDR-083]]). Editing lib/verify-secure-loop.md rewrapped 5 locked phrases across line breaks ("Max 3 conformity iterations", "Max 3 security iterations", "re-verify the REQUEST first", "always re-checked BEFORE security", "one verifier dispatch + one security dispatch") → loops-light.test.sh 30 pass / 5 fail, though ZERO doctrine was dropped. Locks did their job: they cannot distinguish "clause deleted" from "clause rewrapped", and that conservative bias is correct — the alternative (fuzzy matching) would miss real deletions.
Rule: when editing a doctrine file under structure locks, grep the test's lock strings FIRST, then re-flow AROUND them — each locked phrase stays on one unbroken line. Fix the DOC, not the lock, unless the doctrine genuinely changed. Under locks today: verify-secure-loop.md, contract-interview.md, verifier / security-auditor / plan-challenger agents, seo+geo (71 locks).
## LRN-143 — pipe to head masks grep exit; `|| fallback` never fires
- **Context**: darwin 2026-08-26 — plugin-probe FRAMEWORK-DEPS (`grep … | head || echo none`) emitted silent-empty on no-match; same bug in run's own probe test.
- **Pattern**: pipeline rc = LAST command's (head = 0 always). `|| fallback` after pipe = dead code. Bounded output → drop head; else `set -o pipefail` or capture + test.
- **Future**: any skill/agent bash probe with a `||` fallback: check what the pipeline rc actually is first.
## LRN-144 — census locks grep EXACT single-line phrases; prose rewrap breaks them
- **Context**: darwin 2026-08-26 — hotfix RULES rewrap split "No verifier is dispatched at hotfix weight"; loops-light.test.sh lock RED; make test caught post-edit.
- **Pattern**: lib/tests/*.test.sh lock sentences verbatim, single-line. Rewording/rewrapping skill+agent md near locked phrases silently breaks census.
- **Future**: before editing skill/agent prose, grep lib/tests/ for locks in the touched region; run make test BEFORE dispatching judges, not after.
+1 -1
View File
@@ -19,7 +19,7 @@ paired same-judge majority, absolute scores triage-only.
allowed-tools, skills-perso 8/32 detection...).
- [ ] T4 Phase 1 gate: scorecard checkpoint, user picks optimization set.
- [x] T5 Phase 2 DONE: 13/13 units, 21 rounds kept 3-0, 0 reverts +
- [x] T5 Phase 2 DONE: 13/13 units, 12 rounds kept 3-0, 0 reverts +
bug pass 8 commits kept 3-0 (2 skeptic residuals amended). make test
green.
- [x] T6 Phase 3 DONE: report .claude/audits/DARWIN-2026-08-26.md + card