Live failure (run 2): the checklist printed above AskUserQuestion never
reached the user. Fix is structural: the checklist is never written to a
file (throwaway — PENDING.json + live runbook regenerate it in any
session) and every hand-back/re-display ends the turn with the full
checklist as the FINAL text, no tool call after it. Cold resume without a
report regenerates + re-displays. Artifacts 5 -> 4 files; bootstrap
gitignore step drops NEXT.sh; mistakes/red-flags updated (no tool call
after the print, no file 'for reference').
2026-07-05 20:19:13 +02:00
6 changed files with 91 additions and 55 deletions
- /deploy first real run (bchanot-cv): bootstrap→mark full cycle, live-proven (full security-header stack live — tour→prod closed, tag deploy/2026-07-05). Skill patched post-run on user UX feedback: session-style NEXT.sh (one command per line) + hand-back prints the checklist inline ([[EVAL-016]]); template + generated runbook restyled. impeccable chain + Node 24 baseline shipped develop+RC, pushed. settings.json: +inputNeededNotifEnabled committed (layout unchanged).
- /deploy pass 2 (user feedback live): checklist DISPLAY-ONLY — NEXT.sh file eliminated (throwaway artifact, PENDING+runbook regenerate anywhere), hand-back ends the turn with the checklist as final text (a print above AskUserQuestion never reached the user, [[LRN-102]]). Skill+template+CHANGELOG patched; legacy NEXT.sh removed from bchanot-cv; deploy run 2 (residuals b24c58b) re-handed-back inline.
| LRN-097 | 2026-07-04 | community blog pattern ≠ official feature — "contexts dir" doesn't exist in Claude Code; verify feature against official docs (claude-code-guide) BEFORE building infra; the intent was already covered by real mechanisms (agents/skills/rules) | any "add support for X" request naming a Claude Code feature |
| LRN-099 | 2026-07-05 | auto-orchestrator autonomy boundary: git discipline transfers naturally (branch, no-merge), declared-state discipline does NOT — baseline silently rewrote target TODO + authored registries + scope-crept | designing any auto/headless flow — enumerate declared surfaces, mark each read-only or gated |
| LRN-100 | 2026-07-05 | tool gated on clean tree must clean its OWN scratch (else self-DoS next run); contract-changing auto-fix needs structural BREAKING flag in the reviewed artifact | any recurring tool w/ cleanliness precondition; any auto-fix touching an API contract |
| LRN-102 | 2026-07-05 | deliverable text placed BEFORE a tool call may never render — only the turn's FINAL text is guaranteed displayed; a checklist printed above AskUserQuestion was invisible to the user | any flow whose deliverable is conversational text (checklist, commands, report): end the turn with it, blocking questions come before, never after |
---
@@ -1041,3 +1042,11 @@ rules:
- **context**: 2026-07-04 /tour GREEN on fixture; both patched at REFACTOR (SKILL.md STEP 3). Additions template-structural, NOT re-run through 3rd full pass (cost) — re-test first real use.
- **future application**: any recurring tool gated on repo cleanliness → audit what IT leaves behind; any auto-applied fix changing a contract → structural BREAKING flag in the human-reviewed artifact.
- **cousin**: [[LRN-099]] same chantier; [[LRN-071]] swallowed-failure class (silent residue ≈ masked state).
## LRN-102 — Deliverable text before a tool call may never render: the turn's FINAL text is the only guaranteed display
- **pattern**: /deploy hand-back printed the full checklist in the assistant message, then called AskUserQuestion. The user saw ONLY the question UI — the checklist never reached them ("là on a rien, je dois ouvrir le fichier"). The harness renders reliably only the LAST text of a turn; text between/before tool calls can be swallowed by the tool UI.
- **why**: a skill whose deliverable is conversational (commands to copy-paste, a report) fails silently if any tool call follows the print — the user experiences "nothing displayed" while the transcript technically contains it. Structural fix: the deliverable IS the turn's final text; collect answers BEFORE printing, or let the reply arrive as the next user message.
- **context**: 2026-07-05 /deploy run 2 (bchanot-cv). Skill patched same turn: checklist display-only (no NEXT.sh file at all — user: throwaway once deployed) + hand-back ends the turn, no tool call after.
- **future application**: designing any skill/flow output meant to be read+used from the conversation — put it LAST; never sandwich a deliverable between tool calls; prefer plain-text report requests over blocking question tools after a deliverable.
- **cousin**: [[LRN-100]] same skill lineage; CLAUDE.md communication doctrine (final message carries everything).
@@ -8,7 +8,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/).
### Changed
- graphify skill dist refreshed 0.8.45 → 0.9.6 (out-of-band `make plugin`; SKILL.md + query/extraction references updated by the generator).
-`/deploy`NEXT.sh reshaped on first-real-run feedback: runbook steps are **one command per line, interactive-session style** (an early step opens the ssh session; later lines run on the box; local steps say "from your machine") instead of folded `ssh host "cd … && …"` one-liners, and the **hand-back prints the full checklist inline** in the conversation (also on every re-hand-back) so the user never has to open `NEXT.sh` to know what to run. Step = comment header + command lines up to the next blank line; a `@delta:` directive governs the whole block. Template `templates/deploy/PROCEDURE.md` restyled to match.
-`/deploy`checklist reshaped on first-real-run feedback, in two passes: runbook steps are **one command per line, interactive-session style** (an early step opens the ssh session; later lines run on the box; local steps say "from your machine") instead of folded `ssh host "cd … && …"` one-liners — step = comment header + command lines up to the next blank line, a `@delta:` directive governs the whole block; and the checklist is now **display-only** — `NEXT.sh` is no longer written at all (throwaway artifact; `PENDING.json` + the live runbook regenerate it in any session) and every hand-back**ends the turn with the full checklist as the final text, no tool call after it** (a checklist printed above a blocking question tool was observed never reaching the user). Template `templates/deploy/PROCEDURE.md` restyled to match.
comes after the print — none.** Do NOT wrap the report request in a blocking
question tool: text printed before a tool call may never reach the user
(observed live — a checklist printed above an AskUserQuestion was invisible;
the user had to open the file this rule exists to make unnecessary). The report
arrives as the user's next message; `PENDING.json` on disk marks the wait.
The same rule applies to every re-hand-back (STEP 4.3) and every cold-resume
re-display: regenerated checklist ⇒ full print as the turn's final text.
## STEP 3 — RESUME / REACT
@@ -357,13 +368,12 @@ fix (patch + incident committed atomically). Recover later via
Then:
1. Bump `PENDING.json.runbook_rev` to `git rev-parse HEAD` (full sha — not the helper's short-hash stdout); keep `step_reached` = `X`.
2. **Regenerate `NEXT.sh` from `step_reached` against the PATCHED runbook**
(steps X…end — X+1…end never ran). This is NOT replaying one step: the bumped
`runbook_rev` is exactly the staleness trigger — runbook changed ⇒ prior
`NEXT.sh` is stale ⇒ regenerate.
3. Re-present via **STEP 2's [GATE] + hand-back** (the regenerated `NEXT.sh`;
`PENDING.json` keeps `base/target/delta`, `step_reached` back to
`awaiting-user`).
2. **Regenerate the checklist from `step_reached` against the PATCHED runbook**
(steps X…end — X+1…end never ran). This is NOT replaying one step: the
runbook changed ⇒ the prior checklist is stale ⇒ regenerate.
3. Re-present via **STEP 2's [GATE] + hand-back** (the regenerated checklist,
full print as the turn's final text; `PENDING.json` keeps
`base/target/delta`, `step_reached` back to `awaiting-user`).
## STEP 5 — MARK (success)
@@ -388,8 +398,9 @@ The deploy succeeded. Lay the oracle and close out.
bash lib/deploy-commit.sh commit "chore(deploy): mark <date> @ <short>" \
.claude/deploy/STATE.json
```
6. **Delete `.claude/deploy/PENDING.json` and `.claude/deploy/NEXT.sh`** — the
deploy is no longer in flight; the bridge is consumed.
6. **Delete `.claude/deploy/PENDING.json`** — the deploy is no longer in
flight; the bridge is consumed. (Also remove any legacy `NEXT.sh` left by
an older skill version.)
7. Report: deployed SHA, tag (+ push result), state committed, any `DEP-NNN`
learned this deploy. Then offer to capitalize per CLAUDE.md (recurring failure
pattern → `learnings.md`; deploy verdict → `evals.md`), gated, never silent.
@@ -404,10 +415,13 @@ The deploy succeeded. Lay the oracle and close out.
- Delta is `git diff --name-only <base> HEAD` (two endpoints). No `rev-list`, no
three-dot, no date ranges.
- First-deploy / fresh detection is file existence only — never `git describe`.
- Claude never executes the deploy. `NEXT.sh` is hand-run; `# VERIFY:` gates stay.
- Claude never executes the deploy. The checklist is hand-run; `# VERIFY:`
gates stay.
- The checklist is displayed, never written to a file; every hand-back and
re-display ends the turn with it — no tool call after the print.
- Patch + incident commit **atomically**, one `deploy-commit.sh` call, both files.
- A learn bumps `runbook_rev` and **regenerates** `NEXT.sh` from `step_reached`;
it never replays a single step.
- A learn bumps `runbook_rev` and **regenerates** the checklist from
`step_reached`; it never replays a single step.
- Tag push is best-effort; `STATE.json` is the oracle.
- JSON is read natively (Read tool), never parsed with `jq`/shell.
- `STATE.json` written only on confirmed success (STEP 5). A failed/partial deploy
@@ -420,9 +434,11 @@ The deploy succeeded. Lay the oracle and close out.
| On resume, recomputing delta from current HEAD | HEAD moved during the gap. Use `PENDING.json.{base,target,delta}` verbatim. |
| `git describe` to detect first deploy | Errors with no tag. Detect by `STATE.json` / `PENDING.json` existence. |
| `git rev-list` or three-dot for the delta | Phantom/undercounted deltas. Two-dot `<base> HEAD` only. |
| `bash NEXT.sh` to "just run it" | Claude never deploys. Hand back; user runs by hand with `# VERIFY:` gates. |
| Executing the checklist yourself to "just run it" | Claude never deploys. Hand back; user runs by hand with `# VERIFY:` gates. |
| Committing the patch without the incident (or vice versa) | Coupling invariant. One atomic `deploy-commit.sh` call, both files. |
| Replaying only the failed step after a patch | Steps X…end never ran. Regenerate `NEXT.sh` from `step_reached`. |
| Replaying only the failed step after a patch | Steps X…end never ran. Regenerate the checklist from `step_reached`. |
| Ending a hand-back with a blocking question tool after the checklist | Text before a tool call may never render. The checklist is the turn's FINAL text; the report comes as the user's next message. |
| Writing the checklist to a file "for reference" | Throwaway artifact — display only; PENDING.json + the runbook regenerate it anywhere. |
| Writing `STATE.json` before the user confirms success | Oracle marks success only. Failed deploy leaves it untouched. |
| Setting `deployed_sha` to HEAD at MARK time | Use `PENDING.target_sha` — the SHA actually deployed. |
| Parsing the JSON bridges with `jq` | Read them natively. No jq dependency. |
@@ -432,7 +448,9 @@ The deploy succeeded. Lay the oracle and close out.
- About to recompute the delta or re-read HEAD while a `PENDING.json` exists.
- About to run `git describe`, `git rev-list`, or a three-dot diff for the delta.
- About to `bash NEXT.sh` or run any prod command yourself.
- About to execute the checklist or run any prod command yourself.
- About to call ANY tool after printing the checklist in a hand-back.
- About to write the checklist to a file.
- About to commit `PROCEDURE.md` without `INCIDENTS.md` in the same call.
- About to write `STATE.json` before the user reported "Deployed OK".
- About to replay one failed step instead of regenerating from `step_reached`.
@@ -446,7 +464,7 @@ from it without conversation memory — the `audit-delta` "state file is the onl
memory between runs" convention, extended to a *mid-flow* pause. The forms here
match the failure modes the design identified: **discipline** failures
(recompute-on-resume, run-the-deploy, advance-the-oracle-early) get the
rationalization table + red flags; the **shape** of `NEXT.sh` and the schemas get
rationalization table + red flags; the **shape** of the checklist and the schemas get
positive recipes; the patch↔incident **omission** is a structural atomic-commit
requirement. Pressure-scenario baseline testing per the writing-skills Iron Law
is a follow-up — the failure modes were taken from the design spec, not a fresh
# === deploy runbook (reference) — NOT run directly. Instantiated to NEXT.sh per delta. ===
# === deploy runbook (reference) — NOT run directly. Instantiated into the deploy checklist per delta. ===
# Fixed steps run every deploy; # @delta: steps re-instantiate from the delta.
# @config push_deploy_tags=false
# NOTE grammar: glob=<pat>:each repeats the command per matching file (e.g. psql -f <each>);
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Bastien Chanot