toolshrink

Cut large agent tool output by what it means, not by where it was cut.

I use Claude Code every day, and I always wanted to intervene in how it manages
context. In the early days you could edit the session JSONL directly. Then that
door closed.

When DeepSeek open-sourced Harness, where everything is a plugin, I looked
inside. Tool output there is cut by size: keep the head, keep the tail, drop
the middle. I read Codex and pi, and they do the same. None of them look at
what the text contains.

That fails in a predictable way. Your test suite prints 5,000 passing lines and
3 failures in the middle. A size cut keeps the passes and throws away the
failures. The model reads it, believes the run, and answers wrong.

So I built the shrinker I always wished Claude Code had. It reads the output
first, recognizes its shape, and keeps the part that carries the information:

input: a vitest run, 31,958 chars, 805 lines, budget 2,000 chars

head+tail cut:   1,904 chars   the model learns: the summary
toolshrink:        255 chars   the model learns: which test failed,
                               why, at which line, and the summary

Everything removed is counted in a marker the model can read, and the complete
original is saved to disk with a locator. Nothing is lost silently.

The cuts

Each cut recognizes one shape of text. The first one that recognizes the input
runs. When none does, the size fallback runs, so the result always fits the
budget.

Cut	Recognizes	Keeps	Drops
diff	git diff, patches	changed lines, file and hunk headers, 1 context line each side	unchanged context
json	one JSON value	the structure, 3 samples per long array, 5 keys per wide object, counts	repeated records
tests	vitest, jest, pytest, cargo test, go test	failures with their explanation, the summary	passing tests
build	tsc, cargo, gcc, webpack, esbuild	errors and warnings with their code frame, the summary	build progress
stacktrace	node, Python, Java, Ruby traces	the message and frames in YOUR code	dependency frames, counted
log	timestamped logs	errors and warnings with the lines before them, the ending	routine lines
tree	find, ls -R, file listings	the structure, 8 entries per directory, counts	crowded directories
repeat	retry storms, progress spam	2 samples per run plus "2,998 similar lines omitted"	consecutive near-identical lines
lint	eslint, ruff, clippy	each rule with its count and example locations, worst files	repeated occurrences of the same rule
install	npm, pip, pnpm, cargo	the summary, versions, deprecations, vulnerabilities, errors	fetch and download progress
csv	CSV, TSV, pipe tables	header, 5 rows from the start, 2 from the end, row and column counts	the rows between
gitlog	git log, both formats	the 15 newest commits, the total, the authors with counts	older commits
size	everything (fallback)	bash: the end · grep/read: the start · unknown: both ends	the rest, counted

Thirteen cuts ship today. Each one is a plain file with a shared interface, so
adding your own is one file, not a fork.

Every cut follows four rules, taken from the three agents I read:

• never return a partial line (from pi)
• never split a UTF-16 surrogate pair (from DeepSeek Harness)
• say exactly how much was removed: ... 15,903 characters, 401 lines omitted ... (from Codex)
• a second pass changes nothing

Use it with DeepSeek Harness

One command:

dsh plugin --profile web add github:unclecode/toolshrink

That is the whole install. The package carries a dsh.bundle manifest, so
the plugin mounts with a 50,000-character default budget on the next start.
Change the budget from your own layer, ~/.dsh/cordis.patch.yml:

- id: toolshrink
  config:
    maxChars: 20000
    log: /tmp/toolshrink.log

Hacking on it instead? Clone, npm install && npm run build, and mount the
adapter file by path with an insert row (see Adapter config below).

Adapter config

- insert:
    - id: toolshrink
      name: /path/to/toolshrink/adapters/harness/toolshrink.mjs
      config:
        maxChars: 50000        # cut above this many characters (default 50000)
        maxLines: 2000         # or above this many lines (default 2000)
        maxLineChars: 0        # cap single long lines, 0 = off (default 0)
        disable: [json]        # skip named cuts (default none)
        spillDir: ~/.dsh-toolshrink   # where full originals go
        log: /tmp/toolshrink.log      # one line per cut, omit for silence

The log line format: bash 64151 -> 2942 via tree+size.

Use it as a library

import { shrink, FileSpillStore } from 'toolshrink'

const out = shrink(bigText, { tool: 'bash', command: 'npm test' }, {
  budget: { maxChars: 20_000 },
  spill: new FileSpillStore({ dir: '/tmp/spills' }),  // optional
})

out.content   // the text to give the model
out.reduced   // false when the input already fit
out.strategy  // "tests", "diff+size", "size:tail", "none", ...
out.note      // one human-readable line about what happened
out.stats     // inputChars, outputChars, keptLines, droppedLines, ...

The hint (second argument) is optional and improves routing: tool picks the
size direction, command helps detect test runs and diffs, path helps detect
JSON and logs.

Write your own cut

A cut is one file that default-exports three members. The file name is the cut
name.

// mycut.mjs
export default {
  name: 'mycut',
  // Cheap and certain. When unsure, return false: a wrong match is worse
  // than the size fallback.
  detect(text, hint) {
    return hint.command?.startsWith('kubectl') ?? false
  },
  // Return null to decline after a closer look; the next cut then tries.
  reduce(text, hint, budget) {
    const content = text.slice(0, budget.maxChars) // your real logic here
    return {
      content,
      reduced: true,
      strategy: 'mycut',
      note: 'kept the part I know matters',
      stats: {
        inputChars: text.length, inputLines: 0,
        outputChars: content.length, outputLines: 0,
      },
    }
  },
}

Use it:

import { shrink, loadReducers } from 'toolshrink'

const mine = await loadReducers('/path/to/my-cuts')   // reads the directory
shrink(text, hint, { extra: mine })                    // tried BEFORE built-ins

Or control the built-ins: { only: ['tests', 'diff'] } restricts and orders,
{ disable: ['json'] } skips.

Spill: nothing is lost

With a spill store, the complete original is saved before any cut, and the cut
text ends with:

[full output saved as spill:bash-d63d2aebb643: directories sampled to 8 entries each]

store.load('spill:bash-d63d2aebb643') returns the original, byte for byte.
Files are cleaned after 24 hours. The store is an interface; the default writes
files, a host can plug its own storage.

What I saw in live use

With a 3,000-character budget, the agent got a 60,000-character find result
cut down to its head. Its reply began: "The output was truncated. Let me get a
count by directory" - it saw the omission marker, re-queried with aggregation,
and answered correctly from 4,000 total characters instead of 60,000.

That is the design working: an honest marker turns a cut from silent data loss
into a signal the model acts on. This is the intervention I always wanted, and
now it is a YAML row.

TODO: cuts I want next

Each of these is one file with the same interface. Pick one and send a pull
request.

Cut	Recognizes	Would keep
semantic	anything, given the agent's current goal	the chunks most relevant to the goal. Two stages: lexical scoring (BM25, no model needed), then optional embedding scoring for meaning beyond shared words
sql	query results, EXPLAIN plans	the plan's expensive nodes, sampled result rows
docker	build and compose output	the failing layer, the final image, dropped build chatter

The semantic cut is the interesting one: every cut above decides by SHAPE,
this one would decide by RELEVANCE. It needs one extra input, a query for what
the agent is working on right now, which the host adapter can pass through the
hint.

Adapters for other agents

The library knows nothing about any agent. The Harness adapter is 70 lines:
catch the result event, call shrink, return the replacement.

• pi (earendil-works/pi) has an extension API with tool-result access.
• Codex (openai/codex) has a plugin system in codex-rs/core-plugins.

Both adapters are open work. If you write one, a pull request is welcome.

License

MIT. Use it, change it, no need to ask.

Built by @unclecode, author of
Crawl4AI
.
Follow me on X for what I build next: x.com/unclecode.