892de288fe
Move tasks/ into .claude/tasks/ and create .claude/memory/ with 5 persistent registries (decisions/learnings/blockers/journal/evals) plus .claude/audits/ for audit reports. Also seed templates/memory/ for onboarder to copy into user projects. - tasks/TODO.md, tasks/rtk-upstream-issue.md → .claude/tasks/ - tasks/LESSONS.md deleted (empty template; superseded by learnings.md) - .claude/memory/: 5 append-only registries with YAML schema + seed entries - templates/memory/: 5 empty-index templates for new projects Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
86 lines
3.6 KiB
Markdown
86 lines
3.6 KiB
Markdown
# `rtk curl` breaks downstream parsers by returning compressed schema instead of raw payload when stdout is piped
|
|
|
|
## Summary
|
|
|
|
`rtk curl` always returns a token-compressed, schema-like representation of the response body, regardless of whether stdout is a TTY or a pipe. This silently breaks any command that pipes `rtk curl` into a parser expecting the raw payload (`python -c "json.load(sys.stdin)"`, `jq`, `node -e`, `awk`, `sed`, etc.).
|
|
|
|
Because the Claude Code hook (`rtk-rewrite.sh`) auto-rewrites `curl` → `rtk curl`, this affects every such pipeline the LLM constructs — even though the LLM never sees the rtk-compressed output, only the downstream parser error.
|
|
|
|
## Reproduction
|
|
|
|
```bash
|
|
# Expected: prints "Hello World"
|
|
curl -s "https://api.mymemory.translated.net/get?q=Bonjour%20monde&langpair=fr%7Cen" \
|
|
| python3 -c "import json,sys; d=json.load(sys.stdin); print(d['responseData']['translatedText'])"
|
|
```
|
|
|
|
Under the Claude Code hook (or when the user explicitly writes `rtk curl`), this pipeline fails:
|
|
|
|
```
|
|
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 3 (char 4)
|
|
```
|
|
|
|
Because `rtk curl` returns:
|
|
|
|
```
|
|
{
|
|
exception_code: null,
|
|
matches:
|
|
[{
|
|
create-date: string,
|
|
created-by: string,
|
|
...
|
|
```
|
|
|
|
instead of the raw JSON body that `json.load` expects.
|
|
|
|
## Impact
|
|
|
|
- Any LLM-generated pipeline using `curl | <parser>` breaks silently.
|
|
- Root cause is opaque to the LLM: the tool sees a JSONDecodeError and blames its own parsing code, not rtk.
|
|
- The Claude Code hook compounds the issue: users running `curl ... | jq` from the assistant never typed `rtk`, yet get hit by it.
|
|
- Other rtk subcommands that transform output (`rtk aws`, `rtk psql`, `rtk json`, `rtk cat`) likely have the same class of bug.
|
|
|
|
## Root cause
|
|
|
|
`rtk curl` is not a passthrough — it post-processes the response body for LLM consumption. This is correct behavior when an LLM is the consumer. It is incorrect when a parser is the consumer.
|
|
|
|
The tool currently has no way to distinguish those two cases.
|
|
|
|
## Suggested fix
|
|
|
|
Detect `isatty(stdout)` at startup. If stdout is **not** a TTY (i.e., it is piped, redirected to a file, or captured), skip the compression layer and passthrough the raw response bytes from the underlying `curl`.
|
|
|
|
This matches the long-standing Unix convention followed by `ls`, `grep`, `diff`, etc., which disable colors and column formatting when piped.
|
|
|
|
```rust
|
|
// pseudo-code inside rtk curl
|
|
if !io::stdout().is_terminal() {
|
|
return run_native_curl_passthrough(args);
|
|
}
|
|
// else: existing compression path
|
|
```
|
|
|
|
The same fix should apply to every rtk subcommand that transforms output (`cat`, `read`, `json`, `aws`, `psql`, `git`, `gh`, etc.).
|
|
|
|
## Alternative / workaround
|
|
|
|
- Users can prefix with `rtk proxy <cmd>` to bypass rewriting. But the Claude Code hook rewrites `curl` unconditionally, so the LLM has to *remember* to write `rtk proxy curl` every time it pipes, which does not scale.
|
|
- Users can set `[hooks] exclude_commands = ["curl"]` in `~/.config/rtk/config.toml`, but this disables all rtk curl savings globally, even when the LLM IS the consumer.
|
|
|
|
Neither replaces a proper TTY-aware passthrough inside rtk itself.
|
|
|
|
## Environment
|
|
|
|
- rtk: 0.34.3
|
|
- OS: Linux 6.17
|
|
- Shell: bash
|
|
- Claude Code hook: `hooks/rtk-rewrite.sh` (rtk-hook-version: 3)
|
|
|
|
## Checklist
|
|
|
|
- [ ] `rtk curl` passes through raw bytes when stdout is not a TTY
|
|
- [ ] Same behavior for `rtk cat`, `rtk read`, `rtk json`, `rtk aws`, `rtk psql`, `rtk git`, `rtk gh`
|
|
- [ ] Add a test matrix: piped to `jq`, `python -c "json.load"`, file redirect, `/dev/null`
|
|
- [ ] Document the TTY-aware behavior in the README
|