docs(plan): phase 2 results recorded

This commit is contained in:
bchanot committed 2026-11-02 20:00:22 +01:00
1 parent e11d1f8a62
commit b30cf2b7b3
1 file changed
+9
@@ -896,3 +896,12 @@ MINORs closed: corr 9 (T63 list = approved.txt), corr 10 (ids T62/T63; flip test
- **Bench gaps (conf 10).** Rows added: `M3x ask Bash(bash *gitflow.sh finish*) bash "$HOME/nonexistent/lib/gitflow.sh" finish MEASURE` (if ASKED, M3q is redundant but harmless); `A2b deny Bash(awk *system*) awk '{print}' /dev/null RAN`. G1 runs with `--settings '{"permissions":{"allow":["Bash(*)","Agent(*)"],"ask":["Bash(git merge *)"],"defaultMode":"auto"},"skipAutoPermissionPrompt":true}'`. Task 9 Step 1 adds a live row `bash "$HOME/nonexistent/lib/gitflow.sh" finish` (ASKED). The long brief is a file the implementer writes in Task 1: `.claude/tasks/plans/2026-11-02-phase2-bench/long-brief.txt`, about 3000 characters, a pasted multi-paragraph feature request for `doctor.sh` (show the rtk version, warn under 0.23, add a test), in French like the measured briefs; Task 9 Step 2 replays it.
- **Guard edges (conf 11).** `isFreePath` returns false for any path containing `/../` or ending in `/..`. The Skill-hook latch is set after `next(e)` resolves and only when the answer has no `deny`. The registerGuard handler is wrapped so every line stays ≤ 80 columns. Stated limit: an unrowed typed skill latches through the typed-fallback path only when no sub-agent is live (register.ts typedPath); the model's own Skill call latches it anyway.
- **Task 10 folds (conf 12).** Step 2 also requeues TODO "Phase 2 folds: narrow the pre-commit `.claude/**` exemption" as still open (not in this phase's scope).
## RESULTS (recorded 2026-11-02, controller)
- Task 1: bench `.claude/tasks/plans/2026-11-02-phase2-bench/` (results.tsv, every row `auto`, every control RAN). Approved 16 ask, 7 deny (approved.txt). Not approved: quoted finish rule (M3q RAN; M3x shows the plain rule asks on the quoted call-site form), `gitflow_finish*` (M5 RAN), `bash -s*` (S4 RAN). H0 RAN → flag-form denies added. rtk.tsv: no ask form rewritten (rc 1), M7 rc 3. G1: an ask inside a headless sub-agent raised 8 permission requests and never ran.
- Task 9 Step 1 (live settings, hooks on): L1 `git merge -h` ASKED, L2 `sh -c` ASKED, L3 awk system DENIED, L4 `git stash list` RAN, L5 `git push --dry-run …` ASKED (rtk passthrough), L6 quoted finish ASKED.
- Task 9 Step 2 (throwaway clone, opus headless): short prompt → `Skill:feat`, then `Skill:hotfix`, the doctor.sh Edit inside the run; long brief → `Skill:feat`, contract written; no main-loop code write outside a skill, no redirect needed. Live tree snapshot before/after: the only difference is the controller's own TODO.md edit made during the run.
- Guard off = today: `/route off` and `enabled:false` skip the guard (GUARD tests "/route off turns it off", "/route off then on re-arms it"; `st.off` is the router's existing switch).
- Task 9 Step 3: `make test` all suites green, shellcheck clean (after the fix wave too).
- Task 9 Step 4 (user): `/close` one-click finish — pending, asked at the merge gate.
- Final whole-branch review (opus): With fixes → fix wave c58b3e7..fd3a1bf (T13 shasum fallback, rtk comment widths + re-pin, code-limits symlink path, deny-answer GUARD test, release STEP 5 general failure stop, guard documented in README and USAGE) + hard_deny wording (controller).