toolshrink

toolshrink

Cut large agent tool output by what it means, not by where it was cut.

I use Claude Code every day, and I always wanted to intervene in how it manages context. In the early days you could edit the session JSONL directly. Then that door closed.

When DeepSeek open-sourced Harness, where everything is a plugin, I looked inside. Tool output there is cut by size: keep the head, keep the tail, drop the middle. I read Codex and pi, and they do the same. None of them look at what the text contains.

That fails in a predictable way. Your test suite prints 5,000 passing lines and 3 failures in the middle. A size cut keeps the passes and throws away the failures. The model reads it, believes the run, and answers wrong.

So I built the shrinker I always wished Claude Code had. It reads the output first, recognizes its shape, and keeps the part that carries the information:

input: a vitest run, 31,958 chars, 805 lines, budget 2,000 chars

head+tail cut:   1,904 chars   the model learns: the summary
toolshrink:        255 chars   the model learns: which test failed,
                               why, at which line, and the summary

Everything removed is counted in a marker the model can read, and the complete original is saved to disk with a locator. Nothing is lost silently.

The cuts

Each cut recognizes one shape of text. The first one that recognizes the input runs. When none does, the size fallback runs, so the result always fits the budget.

Thirteen cuts ship today. Each one is a plain file with a shared interface, so adding your own is one file, not a fork.

Every cut follows four rules, taken from the three agents I read:

• never return a partial line (from pi)
• never split a UTF-16 surrogate pair (from DeepSeek Harness)
• say exactly how much was removed: ... 15,903 characters, 401 lines omitted ... (from Codex)
• a second pass changes nothing

Use it with DeepSeek Harness

One command:

dsh plugin --profile web add github:unclecode/toolshrink

That is the whole install. The package carries a dsh.bundle manifest, so the plugin mounts with a 50,000-character default budget on the next start. Change the budget from your own layer, ~/.dsh/cordis.patch.yml:

- id: toolshrink
  config:
    maxChars: 20000
    log: /tmp/toolshrink.log

Hacking on it instead? Clone, npm install && npm run build, and mount the adapter file by path with an insert row (see Adapter config below).

Adapter config

- insert:
    - id: toolshrink
      name: /path/to/toolshrink/adapters/harness/toolshrink.mjs
      config:
        maxChars: 50000        # cut above this many characters (default 50000)
        maxLines: 2000         # or above this many lines (default 2000)
        maxLineChars: 0        # cap single long lines, 0 = off (default 0)
        disable: [json]        # skip named cuts (default none)
        spillDir: ~/.dsh-toolshrink   # where full originals go
        log: /tmp/toolshrink.log      # one line per cut, omit for silence

The log line format: bash 64151 -> 2942 via tree+size.

Use it as a library

import { shrink, FileSpillStore } from 'toolshrink'

const out = shrink(bigText, { tool: 'bash', command: 'npm test' }, {
  budget: { maxChars: 20_000 },
  spill: new FileSpillStore({ dir: '/tmp/spills' }),  // optional
})

out.content   // the text to give the model
out.reduced   // false when the input already fit
out.strategy  // "tests", "diff+size", "size:tail", "none", ...
out.note      // one human-readable line about what happened
out.stats     // inputChars, outputChars, keptLines, droppedLines, ...

The hint (second argument) is optional and improves routing: tool picks the size direction, command helps detect test runs and diffs, path helps detect JSON and logs.

Write your own cut

A cut is one file that default-exports three members. The file name is the cut name.

// mycut.mjs
export default {
  name: 'mycut',
  // Cheap and certain. When unsure, return false: a wrong match is worse
  // than the size fallback.
  detect(text, hint) {
    return hint.command?.startsWith('kubectl') ?? false
  },
  // Return null to decline after a closer look; the next cut then tries.
  reduce(text, hint, budget) {
    const content = text.slice(0, budget.maxChars) // your real logic here
    return {
      content,
      reduced: true,
      strategy: 'mycut',
      note: 'kept the part I know matters',
      stats: {
        inputChars: text.length, inputLines: 0,
        outputChars: content.length, outputLines: 0,
      },
    }
  },
}

Use it:

import { shrink, loadReducers } from 'toolshrink'

const mine = await loadReducers('/path/to/my-cuts')   // reads the directory
shrink(text, hint, { extra: mine })                    // tried BEFORE built-ins

Or control the built-ins: { only: ['tests', 'diff'] } restricts and orders, { disable: ['json'] } skips.

Spill: nothing is lost

With a spill store, the complete original is saved before any cut, and the cut text ends with:

[full output saved as spill:bash-d63d2aebb643: directories sampled to 8 entries each]

store.load('spill:bash-d63d2aebb643') returns the original, byte for byte. Files are cleaned after 24 hours. The store is an interface; the default writes files, a host can plug its own storage.

What I saw in live use

With a 3,000-character budget, the agent got a 60,000-character find result cut down to its head. Its reply began: *"The output was truncated. Let me get a count by directory"* - it saw the omission marker, re-queried with aggregation, and answered correctly from 4,000 total characters instead of 60,000.

That is the design working: an honest marker turns a cut from silent data loss into a signal the model acts on. This is the intervention I always wanted, and now it is a YAML row.

TODO: cuts I want next

Each of these is one file with the same interface. Pick one and send a pull request.

The semantic cut is the interesting one: every cut above decides by SHAPE, this one would decide by RELEVANCE. It needs one extra input, a query for what the agent is working on right now, which the host adapter can pass through the hint.

Adapters for other agents

The library knows nothing about any agent. The Harness adapter is 70 lines: catch the result event, call shrink, return the replacement.

• pi (earendil-works/pi) has an extension API with tool-result access.
• Codex (openai/codex) has a plugin system in codex-rs/core-plugins.

Both adapters are open work. If you write one, a pull request is welcome.

License

MIT. Use it, change it, no need to ask.

Built by @unclecode, author of Crawl4AI . Follow me on X for what I build next: x.com/unclecode.