chore(memory): perl runner journal, LRN-290, phase 4b-4d closed
This commit is contained in:
1 parent
721a2ca2bc
commit
f7a6598518
5 files changed
+63
-4
No files matched your search
@@ -752,3 +752,4 @@ rules:
|
||||
|
||||
## 2026-11-14
|
||||
- /hotfix doctor-skilloverrides-residue (block H7): doctor residue check compares {enabledPlugins, skillOverrides} HEAD vs work (9c9a024); doc audit caught MIGRATION step 1 restoring enabledPlugins only → extended to both keys (8dcee03). Security PASS, 1 LOW (echo -e escapes) in BUGS-FOUND. H + I TODO lines closed; LRN-270 bisect files moved to plans/2026-11-14-lrn270-bisect/ for the user.
|
||||
- /bugfix perl-timeout-group: perl fallback runner leads own process group, kills group at deadline, TERMs leftovers (490ad83); challenge r1 FATAL (digest assertion impossible without tty, single quotes = SC2016, timeout(1) misread) → r3; verifier CONFORME (flip red), security PASS. Phase 4b-4d closed: digest-after-compaction proven by this run's auto-compact. Residue TODO: design-tool-gate.sh:70 same idiom.
|
||||
@@ -2098,3 +2098,8 @@ Rule: when editing a doctrine file under structure locks, grep the test's lock s
|
||||
- **Pattern**: (1) comment line starting `# shellcheck` (e.g. `# shellcheck + width`) parsed as malformed directive: SC1073/SC1072, shellcheck stops parsing, rest of file never linted; counting findings vs HEAD hides it (same count both sides). (2) `set -o pipefail` on `cmd | jq | grep -v` masked: grep rc 1 on empty result = legit, pipefail cannot tell it from jq failure; read `${PIPESTATUS[@]}`, fail on stage rc ≠ 0 and grep rc ≥ 2.
|
||||
- **Context**: /bugfix bugs-found-scripts (H6b) 2026-11-13: lib/tests/profile.test.sh:123 unlinted since written; push-guard deny_lost (9045a60).
|
||||
- **Apply**: never start a comment with `shellcheck` word; gate "shellcheck clean" = rc 0, not "count equal to HEAD". Pipe ending in grep-filter → PIPESTATUS check, not pipefail.
|
||||
|
||||
## LRN-290 — alarm+exec bounds one process; captured pipe needs a process-group kill (2026-11-14)
|
||||
- **Pattern**: `perl -e 'alarm N; exec cmd'` bounds ONE process: `$(cmd)` waits for EOF on the pipe, so any child holding stdout/stderr outlives the bound. timeout(1) (GNU, uutils) default = new process group, signals whole group (only `--foreground` leaves children). Portable bounded runner: perl `setpgrp(0,0)` on ITSELF before fork (child born in group, no race), ALRM → `kill KILL, -$$`, after waitpid ignore TERM + `kill TERM, -$$` for leftovers, child `alarm(t+1)` backstop survives exec. Keep the program double-quoted with `\$` (single quotes = SC2016).
|
||||
- **Context**: /bugfix perl-timeout-group 2026-11-14 (490ad83), hooks/session-start.sh stock-macOS fallback. Timing-only tests pass on a runner that fails fast: also assert the command's output arrived.
|
||||
- **Apply**: any alarm/exec timeout wrapper around a captured command; lib/design-tool-gate.sh:70 has the same limit (TODO).
|
||||
@@ -26,7 +26,7 @@ Shared rules for every phase:
|
||||
- Never: forced continuation / Stop block (BDR-083, BDR-087), per-turn nag (BDR-037), forced skill invocation, hiding a dispatchable agent via `agent.offer` (breaks dispatch, proved), a `hard_deny` that names a path, file or source line (the auto classifier would refuse all mod maintenance): hard_deny names off switches only (`"enabled": false`, `enabledPlugins`, `disableAllHooks`) and "never edit `permissions.ask/deny` to pass a gate".
|
||||
- Cleanup is part of each phase: delete what the phase makes obsolete (scripts, hooks, tests, doctrine lines, Makefile targets, doctor checks), grep every citer and patch in the same branch (doctrine-citers census in `make test`).
|
||||
|
||||
- [ ] **Phase 4b-4d, memory v2 capture, read, feedback** (shrunk 2026-10-25 to the measured channels)
|
||||
- [x] **Phase 4b-4d, memory v2 capture, read, feedback** (closed 2026-11-14: every sub-item done or justified) (shrunk 2026-10-25 to the measured channels)
|
||||
- Measured recall (5 real FR prompts, 22 relevant entries): BM25 full text 9/22, haiku rerank of lexical top-30 14/22 (~8.5k chars). Error 4-gram match on real errors: BLK-030/031/034 exact, no false positive; 5 of 7 real errors of 2026-10-23 had NO entry: capture is the weak link.
|
||||
- **4b capture (mod).** Failure ledger: tool results with isError, deny or non-zero exit carrying a distinctive error line → normalized + verbatim + command, REDACTED at capture (rtk-rewrite redaction + gitleaks patterns), bounded ring per session in `$.store` (4 MiB cap), matched against the registry error index; unmatched = new failure modes. Milestones: feat/fix commit, `gitflow finish`, a suite red then green. `memory_pending` tool returns unmatched failures + milestones; /capitalize, /close and every CAPITALIZE step call it first; one context note at the next prompt after a milestone, never per turn, never at Stop. "Proactive capitalization" paragraph KEPT. Writing always = model draft + user approval + lib/memory.sh.
|
||||
- Built 2026-11-04 on feature/memory-capture (f1d4f9d..HEAD): mod `mods/memory`, kit 100+ cases, consumers wired, noise measured (NOISE.md); merged 2026-11-04 into develop (dfadf8a): GATE 0 MET, verifier criteria 1-15 met (9 proven live), security PASS ×4, BDR-137
|
||||
@@ -41,10 +41,10 @@ Shared rules for every phase:
|
||||
- [x] BLK-043 hotfix merged 2026-11-06 (`bd5e661` main, `7e47bea` develop, back-merged into feature/memory-recall `3564e3e`, worktree removed).
|
||||
- [x] BLK-043 Part B: suite-wide `claude` PATH stub + final empty-log check in `lib/tests/session-digest.test.sh` (SD0/SD15, done 2026-11-06 via /hotfix; security PASS, MEDIUM folded into the shortcut below).
|
||||
- [x] shortcut (BLK-043 gate 2 LOWs): `lib/tests/detect-plan.test.sh` unchecked `mktemp -d` (empty WORK → real `claude probe`), `bash -c "source '$LIB'"` interpolation, no `GIT_CEILING_DIRECTORIES` on hook runs; and the memory, graphify-gate, push-guard suites run the hook with no `claude` stub. Upgrade = `|| exit 1` + `command -v claude` assert, positional args, ceiling dir, shared stub helper. Same gap in `lib/tests/session-digest.test.sh` SD0 (gate MEDIUM 2026-11-06): `claude probe` runs before any check that the stub is first on PATH (stub write failure or noexec TMPDIR → real CLI), a red SD0 does not stop the suite; LOW: log path interpolated into the generated stub unquoted. Done 2026-11-06 dc5f127 + 7ca6b18 (shared `lib/tests/claude-stub.sh`, 6 suites, BLK-045).
|
||||
- [ ] BLK-044: trace origin of `oom_score_adj=-1000` on claude processes; set `OOMScoreAdjust` on the `cc` scope so a runaway claude dies first.
|
||||
- [x] BLK-044 resolved (block G, 6c10d3a): sshd drop-in OOMScoreAdjust=-1000 inherited; lib/cc.sh raises to 500.
|
||||
- [x] safety-residuals + phase 2 remaining LOWs done 2026-11-13 (block H4b, bugfix/safety-lows 1f107c3); `rtk read -n server.key`/`id_rsa` root forms done in H4a (a43ba1b).
|
||||
- [ ] session-start perl fallback (no timeout/gtimeout, stock macOS): `alarm` kills only the exec'd bash; a hung child of memory.sh keeps the pipe open. Upgrade: fork + setpgrp + kill the group (GATE 2 LOW 2026-11-13).
|
||||
- Proof (closed 2026-11-12, block B): precision = EVAL-058 (prompt gate; the ≥10-error half folded into the tool gate, BDR-140); redaction, `/context` delta and the known-error recall proven live (journal 2026-11-12); digest after a real compaction = pending this run's first auto-compact.
|
||||
- [x] session-start perl fallback: perl leads its own process group, kills it at the deadline, TERMs leftovers (490ad83, SD19/SD19b, flip red on HEAD runner).
|
||||
- Proof (closed 2026-11-12, block B): precision = EVAL-058 (prompt gate; the ≥10-error half folded into the tool gate, BDR-140); redaction, `/context` delta and the known-error recall proven live (journal 2026-11-12); digest after a real compaction proven 2026-11-14 (this run's auto-compact: SessionStart:compact printed the memory digest, open BLK-017 +3 upstream, todo, journal).
|
||||
- Gate 1 result (2026-11-12, EVAL-058): prompt precision 57 % (n=75), under the 64 % target; fresh 78 %, idle 44 %, keyword 55 %; tool gate 0 records.
|
||||
- [x] 4d gate 1: idle tightened to ≥ 8 content words (BDR-146, 7e87d77)
|
||||
- [x] 4d gate 1: recall capped at 2 (BDR-146, 7e87d77)
|
||||
@@ -76,6 +76,7 @@ Open children of the shipped phases (History 2026-11-12), text verbatim.
|
||||
- [x] D3 add-ons (done 9b12b12) from G2 (plugin-advisor.md): plan-budget WARN :317 (unreachable: plugin-gate never sends PLAN, detect_plan gone), :42 "absolute COST ESTIMATE still reported", :105 `(budget: ~<N>t | n/a)`, :222 "passive cost estimate = highest-cost package profile", "0t passive" claims for impeccable / design-motion-principles (:226 :227 :268 :269); USAGE.md :248 "Budget Pro (~11k tokens/5h)", :494 `(~400t)`, :923 `(~200t)`; session-banner.test.sh SB10 `gone()` passes on a missing file (add an existence check)
|
||||
- [x] D6 project rule (done c69ce9c; lead paragraph fixed too, follow-ups in BUGS-FOUND): .claude/rules/graphify-vendored.md says `graphify install --platform claude` "touches nothing else", but graphifyy 0.9.67 appends a `# graphify` block to ~/.claude/CLAUDE.md (= CLAUDE.global.md); `graphify claude install` writes into the caller's cwd, so the install-plugins.sh trap protects $REPO only when run from $REPO
|
||||
- [ ] gates.sh: warn or reject a criterion row labelled `^[0-9]+[a-z]\.` (LRN-284: `1b.` absorbed into criterion 1, its CHECK/EXPECT overwritten, false MET)
|
||||
- [ ] lib/design-tool-gate.sh:70 `perl -e 'alarm shift; exec @ARGV' 15 21st whoami` kills one process only (same limit as the old session-start runner, fixed 490ad83): reuse the group-kill runner when this file is next touched.
|
||||
- [ ] parked (code-clean 2026-11-09): hooks/rtk-rewrite.sh:84 no-op `[ "$CMD" = "$REWRITTEN" ] && REWRITTEN="$CMD"`: remove with the next real rtk-hook edit, re-pin `hooks/.rtk-hook.sha256` + rtk-rewrite.test.sh T13 in the same step
|
||||
- [ ] deferred (BDR-123): remap inline-writing best-tier skills (design Build stack, writing-skills, deprecation-and-migration, /tour single-project, /audit-delta 3d, /harden STEP 3); stale citers skills/feat/SKILL.md:6, skills/code-clean/SKILL.md:36; + impeccable row in routing.json (design Build stack has none; frontend-design has `reflect`; census test pins the table)
|
||||
- [ ] parked (GATE 2 LOW, 5a98c2c): probe table names not checked with `IDENT.fullmatch` per table (a `/` walks a path on the approved Supabase host); `printable` lets Unicode bidi/format controls (U+202E, U+2066…) through to the tty
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
# CONTRACT — perl-timeout-group (phase 4b-4d residual, GATE 2 LOW 2026-11-13)
|
||||
- date: 2026-11-14 | flow: bugfix | branch: bugfix/perl-timeout-group (off develop)
|
||||
- status: active
|
||||
|
||||
## REQUEST (verbatim — IMMUTABLE)
|
||||
session-start perl fallback (TODO phase 4b-4d line, GATE 2 LOW 2026-11-13, H4b): hooks/session-start.sh:45-46 `_SS_TMO=(perl -e "alarm shift; exec { \$ARGV[0] } @ARGV" 2)` used when neither timeout nor gtimeout exists (stock macOS). alarm kills only the exec'd bash; a background or hung child of memory.sh that inherited stdout keeps the command-substitution pipe open, so the digest call at :320 (and :263, :425) still blocks past 2 s. Upgrade: fork + setpgrp + kill the process group on alarm, exit 124 like timeout(1).
|
||||
|
||||
## CLARIFICATIONS
|
||||
- [gated 2026-11-14, challenge r2/r3] perl setpgrp's ITSELF before fork, ALRM kills `-$$` (rc 137, not 124: callers test non-zero only); after a normal exit, TERM to leftovers (perl ignores TERM); child alarm backstop; double-quoted program with `\$` escapes (single quotes = SC2016). Tests timing + banner `registry v1: 1 file(s)`, `</dev/null` kept, no digest-line assertion (no digest without tty on the perl path). Plan: .claude/tasks/plans/2026-11-14-perl-timeout-group-0030.md § r2 + r3.
|
||||
|
||||
## ACCEPTANCE CRITERIA
|
||||
1. Runner forks into its own group and kills the group.
|
||||
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'setpgrp(0, 0)' hooks/session-start.sh)$(grep -c "kill 'KILL', -" hooks/session-start.sh)$(grep -c 'alarm shift; exec' hooks/session-start.sh)
|
||||
EXPECT: 110
|
||||
EVIDENCE: MET exit=0 marker-found :: 110
|
||||
2. session-digest suite green with SD19 (child holding the pipe) and SD19b (straggler).
|
||||
CHECK: cd /home/bchanot/Documents/claude && echo $(grep -c 'SD19b' lib/tests/session-digest.test.sh | grep -c '^[2-9]')$(PATH=$HOME/.bun/bin:$PATH make test suite=lib/tests/session-digest.test.sh 2>&1 | tail -1)
|
||||
EXPECT: 1all suites green
|
||||
EVIDENCE: MET exit=0 marker-found :: 1all suites green
|
||||
3. shellcheck -x clean on both files, no new directive.
|
||||
CHECK: cd /home/bchanot/Documents/claude && shellcheck -x hooks/session-start.sh lib/tests/session-digest.test.sh && echo SC-OK
|
||||
EXPECT: SC-OK
|
||||
EVIDENCE: MET exit=0 marker-found :: SC-OK
|
||||
4. Flip: with the old runner line restored on a copy, SD19 and SD19b go red (≥ 8 s); make test green (manual, bun on PATH).
|
||||
|
||||
## FILE SCOPE
|
||||
hooks/session-start.sh (:39-47), lib/tests/session-digest.test.sh (SD19 area). TODO line + design-tool-gate.sh residue → STEP 8.
|
||||
@@ -9,3 +9,28 @@ FIX PLAN:
|
||||
2. lib/tests/session-digest.test.sh: new SD19b after SD19: memory.sh stub `sleep 8 & exec sleep 8`… no: `( sleep 8 ) &` then `sleep 8` (the background child keeps stdout/stderr open after the main process dies); same no-timeout PATH; assert run < 6 s and the digest shows `- open: unknown (memory.sh index failed)`. Flip: old runner → ≥ 8 s, red.
|
||||
3. SD19 unchanged (direct hang still bounded).
|
||||
RISK: low. KILL (not TERM) on the group: the bound is the goal, memory.sh read-only calls hold no lock state worth a graceful stop. Linux/macOS perl both have setpgrp/fork/kill negative pid.
|
||||
|
||||
## r2 (correctness FATAL(3), robustness CONCERNS(3), simplicity CONCERNS(1); all folded) — r2 WINS over r1
|
||||
- DIAGNOSIS corrected: timeout(1) (GNU and uutils) by default runs the command in a new process group and signals the whole group (only `--foreground` leaves children untimed); the perl fallback signals one process. Fix = parity with timeout/gtimeout, whose paths stay unchanged. Hook-time victim on the perl path = :263 only (the digest :320/:425 runs on the perl path only from a manual tty run, :21/:35).
|
||||
- Runner (keeps the DOUBLE-quoted program with `\$` escapes, the form 1f107c3 shipped shellcheck-clean; single quotes = SC2016). perl moves ITSELF into a new group before forking, so the child is born in it (no race, one setpgrp), and the alarm lives in perl, outside the hook's foreground group (Ctrl-C or a harness group kill of the hook does not reach it; the alarm still fires and bounds the group):
|
||||
```
|
||||
_SS_TMO=(perl -e "
|
||||
my \$t = shift; \$SIG{CHLD} = 'DEFAULT'; setpgrp(0, 0);
|
||||
my \$pid = fork; defined \$pid or exit 125;
|
||||
if (!\$pid) { alarm(\$t + 1); exec { \$ARGV[0] } @ARGV; exit 127 }
|
||||
\$SIG{ALRM} = sub { kill 'KILL', -\$\$ };
|
||||
alarm \$t; my \$w = waitpid(\$pid, 0); my \$st = \$?; alarm 0;
|
||||
\$SIG{TERM} = 'IGNORE'; kill 'TERM', -\$\$;
|
||||
exit(\$w < 0 ? 125 : (\$st & 127) ? 128 + (\$st & 127) : \$st >> 8)" 2)
|
||||
```
|
||||
- timeout: KILL to the whole group incl. perl → rc 137 (callers test non-zero only: :263 discards, :320 `|| {…}`, :425 `|| true`).
|
||||
- normal end: stragglers still in the group (a background job left after the command exited) get TERM, perl ignores it; memory.sh and memory-feedback-remind.sh start no background job (read, both challengers), so TERM suffices.
|
||||
- child backstop `alarm(\$t + 1)` survives exec: if perl itself is SIGKILLed, the direct child still ends.
|
||||
- `-\$\$` never 0: a failed setpgrp leaves `-\$\$` naming no group (ESRCH, harmless); fork failure exit 125 before any kill; waitpid -1 (SIGCHLD ignored upstream, reset to DEFAULT anyway) → 125.
|
||||
- Comment :39-40: "timeout, else gtimeout, else perl: each runs the command in its own process group and kills the group at the deadline, so a child holding the pipe dies too. Trade-off (as timeout without --foreground): in a terminal run the command is a background job of the tty (no stdin read; under `stty tostop` a tty write stops it until the deadline)."
|
||||
- Tests (lib/tests/session-digest.test.sh, `</dev/null` kept, timing only, no digest-line assertion): SD19 stub becomes `( sleep 8 ) & exec sleep 8` (direct hang + a child holding stderr at :263); new SD19b stub `( sleep 8 ) & exit 0` (straggler after the command exits). Both assert `SECONDS < 6` and stay under the no-timeout PATH. Flip: the old runner → ≥ 8 s on both.
|
||||
- Residue (TODO line, not this fix): lib/design-tool-gate.sh:70 `perl -e 'alarm shift; exec @ARGV' 15 21st whoami` has the same single-process limit.
|
||||
|
||||
## r3 (confirmation CONCERNS(1), folded; FINAL)
|
||||
- Tests prove the command RAN, not only the bound: both stubs print `echo "memory: v1: x" >&2` before hanging/straggling; SD19/SD19b capture the hook output (not `>/dev/null`) and assert it holds `registry v1: 1 file(s)` (measured: r2 runner gives v1=1 for both) plus `SECONDS < 6`. A fast-failing runner (missed `\$` escape, exec failure) goes red.
|
||||
- Comment :39-40 names the asymmetry: timeout/gtimeout kill the group at the deadline; the perl runner also TERMs leftovers after a normal exit (timeout does not; a leftover that ignores TERM is unbounded, none of the three callers starts one).
|
||||
Reference in new issue
Block a user