From 12823181d1d92597df5fb577a4a9626709db74d4 Mon Sep 17 00:00:00 2001 From: Bastien Chanot Date: Wed, 26 Aug 2026 22:45:03 +0200 Subject: [PATCH] =?UTF-8?q?chore(darwin):=20Phase=203=20=E2=80=94=20optimi?= =?UTF-8?q?zation=20report=20+=20TODO=20T5/T6=20ticked?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/audits/DARWIN-2026-08-26.md | 90 +++++++++++++++++++++++++++++ .claude/tasks/TODO.md | 11 ++-- 2 files changed, 96 insertions(+), 5 deletions(-) create mode 100644 .claude/audits/DARWIN-2026-08-26.md diff --git a/.claude/audits/DARWIN-2026-08-26.md b/.claude/audits/DARWIN-2026-08-26.md new file mode 100644 index 0000000..c07c13e --- /dev/null +++ b/.claude/audits/DARWIN-2026-08-26.md @@ -0,0 +1,90 @@ +# Darwin run 2026-08-25/26: fresh baseline + threshold optimization + bug pass + +Branch `feature/darwin-optimize-20260825`, 26 commits, 39 files, +299/-142. +Log: `~/.agents/skills/darwin-skill/results.tsv` (fresh, the May file was wiped +by the 2026-06-23 reinstall). Method: darwin v2.1. Absolute scores served as +triage only; every keep/revert decision came from a paired same-judge majority +(3 judges per round, before/after read in one call). + +## Scope + +54 units: 31 personal skill-systems (SKILL.md + dispatched agents judged +together, per EVAL-004) and 23 agents. Excluded: gstack/external symlinks +(BDR-015/043, LRN-070), darwin-skill itself (BDR-058 pin), and find-docs, +newly identified as machine-owned ctx7 output (gitignored, installer-written). + +## Baseline (7 blind judges, dims scored 1-10, totals recomputed main-thread per LRN-018) + +Mean 83.4 (skills 83.5, agents 83.3). Best: deploy, release-candidate, +release-executor (90.4). Worst: skills-perso 63.5. All dim8 rows marked +dry_run by design; live execution happened later, inside the paired rounds. +13 units scored below the user-set threshold of 80. + +## Phase 2: threshold loop, 13/13 units, 0 reverts + +Every round was validated by 3 paired judges (neutral, skeptic, realism). +All verdicts 3-0 better. + +| Unit (baseline) | Round(s) | What changed | +|---|---|---| +| skills-perso (63.5) | d8 | Detection rebuilt on the link.sh convention: symlink = external, real dir = personal, gitignored = machine-generated. Live result 8/31 to 31/31, zero false positives | +| interviewer (70.9) | d3, d9 | Failure-mode table (vague, "you decide", contradiction, partial, balloon) + 2-round budget; DO-NOT list | +| onboarder (71.5) | d8 | BRIEF contract split REQUIRED/OPTIONAL; null enrichment becomes TODO placeholders; STOP kept for required keys and unresolved monorepo. Kills the guaranteed first-dispatch bounce vs /onboard STEP 2 | +| pdf-translate (72.3) | d3/d8 | 8-row failure table: deps, >30 pages gate, zero-output, illisible markers, design-html/browse fallbacks, QA cap 2, stale workdir | +| refactor (75.6) + refactorer (76.8) | d4/d3 | No-tests STOP gate + GO-WITHOUT-TESTS arbitration in the dispatcher; mid-run test-failure revert protocol; code-cleaner inline carve-out | +| profile (77.3) | d3 | 6-row failure table, every row fact-checked against profile.sh (rc=1 paths, partial toggle, split plugin leg, BLK-006 contradiction); fixture de-drift | +| plugin-probe (78.5) + plugin-advisor (77.5) | d8 | FRAMEWORK-DEPS now exact dep@version (preact false-hit killed, fallback actually fires; the old `\|\| true` silently emitted nothing and tripped the advisor's fail-closed path on non-Node projects); frontend/fast-libs derivable; PLAN echoed-or-unknown, invention removed | +| analyze (77.7) + analyzer (78.0) | d1, d2 | Bilingual triggers + fix-wanted disambiguator; TASKS ordered, each step mapped to its OUTPUT section | +| status-reporter (78.0) | d5 x2 | Fabrication-forcing token field replaced, then restored producibly from doctor.sh constants (a skeptic judge found the source); dead ROADMAP row rewritten post-ADR-013 | +| gitflow (78.4) | d3 | 7-row failure table keyed to lib return codes; rc=4 conflict resume empirically verified; human merge gate untouched | + +## Bug pass: verified defects in above-threshold units, 8 commits, all kept 3-0 + +- hotfix: `git restore .` on every failure branch wiped tolerated in-progress + user edits. Now: `git stash create` pre-flight snapshot + file-scoped + restore + fresh-dispatch-only security gate. Two skeptic residuals amended + (RULES bullet, FILE(S) new-file marker). +- init-project: allowed-tools lacked Agent and Skill while every step + dispatches. commit-change: conflict grep now covers all 7 unmerged codes. + tour: --report-only no longer commits (could land on develop). +- harden: severity rule now defers to the calibrated guide; the late SSL Labs + grade has an assigned actor. +- plan-challenger: ERROR joined the load-bearing verdict grammar. +- handover writers: stale chapter refs corrected (glossary/tone to §6, + cross-links and THRESHOLD-OVERRIDE to §5); STEP 14.5 verification deferred + post-write; anchor gate ordered into STEP 16. +- security-auditor: /hotfix no-verifier carve-out documented. close: STEP 5C + enumerated, --no-push passthrough added. +- prune-memory: false "v1-untested" note replaced by the real tests/ state. + code-clean: executor attribution corrected (code-cleaner, refactorer inline). +- Fixtures de-drifted: plugin-check (PLUGIN CHECK block, real plugin names), + onboard (nextjs-app-router). + +`make test` green (0 RED, rc=0) after one census rewrap: a locked phrase had +been line-wrapped and the single-line grep lock caught it. + +## Residual findings, logged not fixed + +- analyze triggers: "how does X work" brushes graphify's territory; graphify's + graph-exists routing still wins. +- pdf-translate: pdfinfo row assumes poppler (fitz also has page count); "GB" + slightly overstated near the 30-page gate. +- web-validate: .validate-cache mkdir lives in a skipped STEP 0 + (self-recoverable); axis budgets 35/25/40 never reconciled with the base-100 + deduction table. seo/geo minor wording items. verifier/doc-syncer/audit-delta + restatement redundancy (cosmetic). handover-doc-writer STEP 14.5 umbrella + line still says "BEFORE STEP 15" while the inner note overrides it. +- bugfix trivial-fast-path boundary loosely defined; feat prompt-3 expectation + vs full gate pipeline. + +## Methodology notes + +- v2.1 paired majority produced 36 unit-round verdicts and 24 batch verdicts, + all better, 0 reverts, 0 ties. The May-2026 run under absolute-delta scoring + had reverted 2 edits on judge noise; this run had no such event. +- Judges live-executed wherever the artifact was executable (skills-perso + detection, profile.sh probes, plugin grep on scratch manifests, doctor.sh + grep, git merge no-op resume). Behavior outranked prose in 5 units. +- Two grep-exit-masking bugs surfaced (a `head` pipe swallowing the fallback's + trigger), one in the probe being fixed, one in this run's own test harness. + The pattern is worth a learning entry. diff --git a/.claude/tasks/TODO.md b/.claude/tasks/TODO.md index 2eaa116..8ea4cad 100644 --- a/.claude/tasks/TODO.md +++ b/.claude/tasks/TODO.md @@ -19,11 +19,12 @@ paired same-judge majority, absolute scores triage-only. allowed-tools, skills-perso 8/32 detection...). - [ ] T4 Phase 1 gate: scorecard checkpoint, user picks optimization set. -- [ ] T5 Phase 2: per-unit loops (1 dim/round, weighted-gap diagnosis, - paired 3-judge majority keep/revert, HL-4 stop), human checkpoint - per unit. -- [ ] T6 Phase 3: report + result cards (npx playwright fallback, BLK - screenshot.mjs macOS path) + capitalize. +- [x] T5 Phase 2 DONE: 13/13 units, 21 rounds kept 3-0, 0 reverts + + bug pass 8 commits kept 3-0 (2 skeptic residuals amended). make test + green. +- [x] T6 Phase 3 DONE: report .claude/audits/DARWIN-2026-08-26.md + card + PNG (playwright fallback). Capitalize pending user approval. Branch + UNMERGED — human gate. ## 2026-08-25 — user permanent rules: writing + web build + web security (feature/user-writing-web-rules) User supplied 4-block rule text (écris / site / code / vérification); asked: