docs(phase5): spike methods and commands in § Facts, one-session reload canary, policy warning attributed to the deny profiles

This commit is contained in:
bchanot committed 2026-11-02 10:20:50 +01:00
1 parent e99cc2d7ef
commit 3a589f9fa4
4 files changed
+28 -7

No files matched your search

@@ -0,0 +1,13 @@
# GATE 1 round 1 report
Reload canary script: /home/bchanot/.cache/claude-tmp/p5/t12/reload-one.sh (coproc, claude -p --model haiku --max-turns 1 --tools "" --input-format stream-json --output-format stream-json --verbose). Log: /home/bchanot/.cache/claude-tmp/p5/t12/reload-one-session.jsonl
Result lines (session_id, subtype, result):
- ee65507c-a958-411b-9580-8aca8105e326 success `/seo` isn't available in this session, so it didn't run. Your message had no other text, so I don't know what you want checked. Tell me the goal and I'll do it with the tools I have. For example: -
- ee65507c-a958-411b-9580-8aca8105e326 success Reloaded: 3 plugins · 2 skills · 32 agents · 7 hooks · 0 plugin LSP servers
- ee65507c-a958-411b-9580-8aca8105e326 success # /seo — parallel SEO + GEO dispatcher
- ee65507c-a958-411b-9580-8aca8105e326 success # /seo — parallel SEO + GEO dispatcher
Stderr: Warning: claude.ai MCP servers blocked by enterprise policy: claude.ai higgsfield, claude.ai Higgsfield AI (deny profile minimal).
--tools "" did not break /reload-plugins. Turn 1 gave no literal 'Unknown skill': the model answered that /seo isn't available.
Verify: doctrine-citers PASS=23, pack-census PASS=33, pack-census.sh PACKS OK 9 packs 55 skills.
+1 -1
View File
@@ -52,7 +52,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/) and this project
- `lib/tests/doctrine-citers.test.sh`: the file-citation census also reads nested `lib/<dir>/` paths, `~/.claude/` and `$HOME/.claude/` prefixes (mapped to this repo) and `templates/project-rules/`; a bare `.claude/rules/` citation resolves by citer (this repo's own rules from `.claude/rules/` or `rules/`, else `templates/project-rules/`); T6 also scans `templates/` and `.claude/rules/`, T3 keeps its roots.
- `link.sh` no longer links the design and scroll skills flat; impeccable moves to `skills-external/impeccable`. Docs and routing (`CLAUDE.global.md`, `/profile`, README, USAGE, ARCHITECTURE) speak packs.
- Known limit: `cc x web` on a project in `dev` keeps the higgsfield connectors that `dev` denies (`deniedMcpServers` is a union); `profile.sh set web` in the project lifts them.
- Measured headless on 2026-11-02: dev = `pack-gstack-dev`, 41 tools, 0 higgsfield; web = the six web packs, 199 tools, 158 higgsfield tools listed in init (the stderr policy warning means they may not be callable); qa = `pack-gstack-browser pack-web-validate`; seo = qa + `pack-gstack-dev pack-seo`; qa and seo 41 tools, 0 higgsfield. `profile.sh current --plain` median 30 ms (target 50).
- Measured headless on 2026-11-02: dev = `pack-gstack-dev`, 41 tools, 0 higgsfield; web = the six web packs, 199 tools, both connectors `connected` and 158 higgsfield tools listed in init (the stderr line "blocked by enterprise policy: claude.ai higgsfield, claude.ai Higgsfield AI" appeared on the deny profiles dev, qa, seo and minimal: that is `deniedMcpServers` working); qa = `pack-gstack-browser pack-web-validate`; seo = qa + `pack-gstack-dev pack-seo`; qa and seo 41 tools, 0 higgsfield; reload canary in one flagless session (id `ee65507c-a958-411b-9580-8aca8105e326`): `/seo` unavailable on minimal, `profile.sh enable seo`, `/reload-plugins` -> "Reloaded: 3 plugins · 2 skills · 32 agents · 7 hooks", then `/seo` and `/pack-seo:seo` both load the SEO skill. `profile.sh current --plain` median 30 ms (target 50).
- Listing budget with the packs: measured headless on 2026-11-02 with `--model haiku` (name-only skills, entries 124 on full, 90 on dev): full default 58, 40000 11, 50000 0, 60000 0; dev default 9, 40000 and above 0. Set `SLASH_COMMAND_TOOL_CHAR_BUDGET=50000` in `settings.json` `env`, the smallest tested value with 0 name-only on full. Cost, estimated: full (48.5k - 30.3k bytes) / 4, about 4.5k tokens; dev 30.0k to 36.8k bytes, about 1.7k tokens. `env` is global, so every profile pays. Descriptions are no longer truncated at 50000 (natural size 48.5k). The default budget's absolute value was not read; it may depend on the model's context window.
### Removed
+1 -1
View File
@@ -217,7 +217,7 @@ an old profile). Step 3 is the only one that removes a real directory
flat links; remove every `skills-disabled/` link, and in `skills/`
everything except model-router
(the tracked link to `mods/model-router`), darwin-skill,
the seven superpowers and the agent-skills trio). If
the seven superpowers and the agent-skills trio. If
`skills-disabled/impeccable` is a directory: `mv skills-disabled/impeccable
skills-external/impeccable`. Then `rmdir skills-disabled; rm -f
.active-profile`.
@@ -36,15 +36,23 @@ r2 2026-11-02 after three lenses (correctness FATAL(11), robustness FATAL(10), s
## Facts (measured 2026-11-01, CLI 2.1.316, headless)
- A dir `~/.claude/skills/<pack>/` with `.claude-plugin/plugin.json` loads as plugin `<name>@skills-dir`. Nested `skills/<s>/SKILL.md` are listed ONCE as `<pack>:<s>`; `Skill(<s>)`, `Skill(<pack>:<s>)`, typed `/<s>` and `/<pack>:<s>` all resolve.
- Absent from every `enabledPlugins`: ON unless the manifest says `"defaultEnabled": false`. `.claude/settings.local.json` true/false obeyed; `--settings` false beats local true.
- `/reload-plugins` works in `-p` and reloads a pack's skills + hooks after a local edit; a pack `.mcp.json` server needs a new session. In a `--settings` conversation the flag value is read once: no reload brings a pack back.
- Connectors load under `-p`. `deniedMcpServers:[{serverName:"claude.ai <Name>"}]` drops the named ones (199 → 192 tools); `disableClaudeAiConnectors:true` drops all nine (26 tools), any-source-true. Both work via `--settings`. `--strict-mcp-config` drops connectors and project `.mcp.json` servers.
**Common method.** Every spike ran in a scratch project under `$HOME/.cache/claude-tmp/p5/` (`git init -q`, a `CLAUDE.md`, canary skills that reply a fixed token), in FRESH headless sessions, one per probe. Command shape: `claude -p '<prompt>' --model haiku --output-format stream-json --verbose`, then read the first `type=system subtype=init` message: `plugins[]`, `skills[]`, `tools[]`, `mcp_servers[]`. Multi-turn probes add `--input-format stream-json` (one JSON user message per line on stdin). Settings variants: `--settings <json or file>` (per conversation) and edits of the scratch project's `.claude/settings.local.json` (per project). Spike 4 also used `claude mcp list`.
**CHANGED since 2026-10-25: claude.ai connectors now load under `-p` (they did not on 2026-10-25).** The earlier "interactive only" fact is void.
- **Spike 1, nested skills in a pack.** Method: a skills-dir pack `~/.claude/skills/<pack>/` with `.claude-plugin/plugin.json` and `skills/<s>/SKILL.md` canaries; fresh `-p` session, read init `plugins[]` and `skills[]`, then invoke `Skill(<s>)`, `Skill(<pack>:<s>)`, typed `/<s>` and `/<pack>:<s>`. Result: the dir loads as plugin `<name>@skills-dir`; nested skills are listed ONCE as `<pack>:<s>`; all four invocations resolve.
- **Spike 1b, default state.** Method: same pack, `enabledPlugins` absent everywhere, then with `"defaultEnabled": false` in the manifest; then `.claude/settings.local.json` true/false and `--settings` true/false, init `plugins[]` read each time. Result: absent from every `enabledPlugins` means ON unless the manifest says `defaultEnabled:false` (then OFF with nothing written; local or `--settings` true turns it on). Local true/false is obeyed; `--settings` false beats local true.
- **Spike 1c, typed short slash.** Method: `-p '/<s>'` on a pack skill with canary reply. Result: the typed short `/<skill>` on a pack skill expands like `/<pack>:<skill>`.
- **Spike 2, `/reload-plugins` mid-session.** Method: two-turn session (`--input-format stream-json`): turn 1 a canary prompt, a `.claude/settings.local.json` edit between turns, turn 2 `/reload-plugins` then the canary again. Result: `/reload-plugins` works in `-p` and reloads a pack's skills + hooks after a local edit; a pack `.mcp.json` server needs a new session.
- **Spike 2b, reload under `--settings`.** Method: same two-turn session launched with `--settings <flag file>` setting a pack false, local edit true, `/reload-plugins`. Result: the flag value is read once; `/reload-plugins` re-reads neither the flag file nor a local true over a flag false, so a pack OFF by flag only comes back through a relaunch.
- **Spike 3, claude.ai connectors per project (F0-F5).** Method: fresh `-p` sessions in the scratch project, init `mcp_servers[]` and `tools[]` counted per run. F0 baseline: connectors listed. F1 `.claude/settings.local.json` `deniedMcpServers:[{serverName:"claude.ai <Name>"}]` for Gmail + Higgsfield AI: 199 -> 192 tools. F2 `disableClaudeAiConnectors:true`: all nine dropped, 26 tools, model-router kept. F3 the same two keys via `--settings`: same result per conversation. F4 `deniedMcpServers` with a `serverUrl` wildcard: matches no connector (no URL exposed). F5 `--strict-mcp-config`: drops connectors and project `.mcp.json` servers per conversation. `disableClaudeAiConnectors` is any-source-true. The native `/mcp` toggle writes `disabledMcpServers` per project into `~/.claude.json` (untested, interactive only). Docs, not measured: `/reload-plugins` re-reads `enabledPlugins` from every settings file; `--plugin-dir` loads a pack as `<name>@inline` for one session; the skills-dir id is the manifest `name`.
- **Spike 4, higgsfield MCP in a pack `.mcp.json`.** Method: a canary pack `spike-pack` whose `.mcp.json` is `{"mcpServers":{"spike-hf":{"type":"http","url":"https://mcp.higgsfield.ai/mcp"}}}`; fresh `-p` session reading init `mcp_servers[]`, and `claude mcp list` in the scratch project. Result: the server is registered as `plugin:<pack>:<server>` and `claude mcp list` shows "Needs authentication"; while unauthenticated it is absent from the init `mcp_servers`; the OAuth login is interactive (`/mcp`). Today higgsfield = two claude.ai connectors (`claude.ai higgsfield`, `claude.ai Higgsfield AI`), so a pack server would duplicate their tools.
- Connectors, summary: `deniedMcpServers:[{serverName:"claude.ai <Name>"}]` drops the named ones (199 -> 192 tools); `disableClaudeAiConnectors:true` drops all nine (26 tools), any-source-true. Both work via `--settings`. `--strict-mcp-config` drops connectors and project `.mcp.json` servers.
- model-router (`mods/model-router/hooks/register.ts`): `skillRow` matches `e.skill` by EXACT key in `routing.json` `skills`; `typedSkill` takes the text after `/` up to a space, so a typed `/pack-seo:seo` yields the key `pack-seo:seo` whatever the Skill tool reports. `/route show` prints `main: skill <phase> …`, never the skill name. Consequence: long-name rows are added unconditionally (Task 10); no router canary.
- The Bash tool sets neither `CLAUDE_PROJECT_DIR` nor `CLAUDE_SCRATCHPAD`; its cwd persists across calls, so `profile.sh` resolves the project root by walking up.
- 34 bare `jq -n 1` spawns = 118-137 ms on this machine: `current` must be one or two jq calls.
- FACT-DANGLING = kept (Task 1, 2026-11-02, headless haiku, pack-t1 OFF by manifest + `--settings` true): init listed `pack-t1:t1-alpha` (real dir) and `pack-t1:t1-beta` (symlink) next to the dangling `t1-gamma`; a dangling entry hides nothing. The census keeps WARN for a dangling target in `make test`, RED only with `--installed` on an ON pack. Run by the main loop (the implementer's compound call was denied by the permission rules; cleanup by `mv skills/pack-t1 <scratch>` instead of the `rm -f` trap).
- Task 12 proof (2026-11-02): same project, `--settings` dev -> `pack-gstack-dev`, 41 tools, 0 higgsfield; web -> the six web packs, 199 tools, 158 higgsfield tools listed in init (the stderr policy warning means they may not be callable). Local profiles: qa -> `pack-gstack-browser pack-web-validate`; seo -> `pack-gstack-browser pack-gstack-dev pack-seo pack-web-validate`; both 41 tools, 0 higgsfield. minimal: `seo` unknown; after `enable seo` + `/reload-plugins`, `/seo` and `/pack-seo:seo` both load the SEO skill. `current --plain` median 30 ms over 5 runs (target 50).
- Task 12 proof (2026-11-02): same project, `--settings` dev -> `pack-gstack-dev`, 41 tools, 0 higgsfield; web -> the six web packs, 199 tools, both connectors `connected`, 158 higgsfield tools listed in init. The stderr line "claude.ai MCP servers blocked by enterprise policy: claude.ai higgsfield, claude.ai Higgsfield AI" appeared on the DENY profiles (dev, qa, seo, minimal): that is `deniedMcpServers` doing its job, not a fault of `web`. Local profiles: qa -> `pack-gstack-browser pack-web-validate`; seo -> `pack-gstack-browser pack-gstack-dev pack-seo pack-web-validate`; both 41 tools, 0 higgsfield. Reload canary, ONE flagless session (`claude -p --model haiku --max-turns 1 --tools "" --input-format stream-json --output-format stream-json --verbose`, stdin from a `coproc`, session id `ee65507c-a958-411b-9580-8aca8105e326` on all four result lines): turn 1 `/seo` on profile minimal -> not available (the model said `/seo` isn't available in this session; no literal "Unknown skill" string under `--max-turns 1`); `profile.sh enable seo` with the session open; turn 2 `/reload-plugins` -> "Reloaded: 3 plugins · 2 skills · 32 agents · 7 hooks · 0 plugin LSP servers"; turn 3 `/seo …` and turn 4 `/pack-seo:seo …` -> both "# /seo — parallel SEO + GEO dispatcher". `--tools ""` did not break `/reload-plugins`. Log: `$HOME/.cache/claude-tmp/p5/t12/reload-one-session.jsonl`. `current --plain` median 30 ms over 5 runs (target 50).
## File Structure