3 ms·

Early on in experimenting with local models, I found that hooking them up to Claude Code worked very well, but it was also really slow. I used mitmproxy (setup

by mft_ 3mo ago

Early on in experimenting with local models, I found that hooking them up to Claude Code worked very well, but it was also really slow. I used mitmproxy (setup assisted by Claude, natch) to capture Claude Code's entire initial system prompt and the whole thing was (I just double-checked) 162k of JSON. This led me to start experimenting with Pi, OpenCode, and Hermes...

mh- 3mo ago
This is interesting, because if I start a fresh session of Claude Code right now and run /context, I see the following: Opus 4.8 (1M context) claude-opus-4-8[1m] 23k/1m tokens (2%) Estimated usage by category System prompt: 3.9k tokens (0.4%) System tools: 13.9k tokens (1.4%) Custom agents: 235 tokens (0.0%) Memory files: 28 tokens (0.0%) Skills: 4.9k tokens (0.5%) Messages: 8 tokens (0.0%) Compact buffer: 3k tokens (0.3%) Free space: 974k (97.4%) 4k tokens is 15-20kB. I'd ask you to paste that into a gist, but it might have sensitive data in it, because I suspect what you're seeing is not just the system prompt.
- mft_ 3mo ago
  Apologies, you're right - I used imprecise terminology. The entire initial JSON structure that was sent from Claude Claude to the LLM at the start of a session was 162k. This included the system prompt together with a list of tools (some with very extensive explanations), MCP server details, etc. I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
  - iamflimflam1 3mo ago
    A lot of people will just add as many tools as they can think of. I don’t think it’s obvious that this costs money.
    - mh- 3mo ago
      A smarter approach (progressive disclosure) for tools has been implemented by (I presume all) the harnesses over recent months, but you're 100% right in any case. I enable tools specific to each project only in that project, and have very very few in my global config. Like <5k tokens worth.
  - ricardobeat 3mo ago
    That’s entirely dependent on how many plugins, MCP tools, agents you have, and if you have pre-filling of all available tools enabled. Best way to avoid unnecessary expense is to avoid it all and use CLI tools instead.
    - mft_ 3mo ago
      Agree. It's a fairly minimal list with very few extras added. Current /context on a fresh session (compare to that above) is: Opus 4.8 15.8k/1m tokens (2%) System prompt: 4.5k tokens (0.4%) System tools: 7.9k tokens (0.8%) Memory files: 441 tokens (0.0%) Skills: 3.1k tokens (0.3%) Messages: 8 tokens (0.0%) Free space: 984.2k (98.4%)
  - mh- 3mo ago
    Ah that makes sense, wasn't trying to be pedantic. Thanks for clarifying.

by mft_ 3mo ago

mh- 3mo ago

This is interesting, because if I start a fresh session of Claude Code right now and run /context, I see the following: Opus 4.8 (1M context) claude-opus-4-8[1m] 23k/1m tokens (2%) Estimated usage by category System prompt: 3.9k tokens (0.4%) System tools: 13.9k tokens (1.4%) Custom agents: 235 tokens (0.0%) Memory files: 28 tokens (0.0%) Skills: 4.9k tokens (0.5%) Messages: 8 tokens (0.0%) Compact buffer: 3k tokens (0.3%) Free space: 974k (97.4%) 4k tokens is 15-20kB. I'd ask you to paste that into a gist, but it might have sensitive data in it, because I suspect what you're seeing is not just the system prompt.

mft_ 3mo ago
Apologies, you're right - I used imprecise terminology. The entire initial JSON structure that was sent from Claude Claude to the LLM at the start of a session was 162k. This included the system prompt together with a list of tools (some with very extensive explanations), MCP server details, etc. I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
- iamflimflam1 3mo ago
  A lot of people will just add as many tools as they can think of. I don’t think it’s obvious that this costs money.
  - mh- 3mo ago
    A smarter approach (progressive disclosure) for tools has been implemented by (I presume all) the harnesses over recent months, but you're 100% right in any case. I enable tools specific to each project only in that project, and have very very few in my global config. Like <5k tokens worth.
- ricardobeat 3mo ago
  That’s entirely dependent on how many plugins, MCP tools, agents you have, and if you have pre-filling of all available tools enabled. Best way to avoid unnecessary expense is to avoid it all and use CLI tools instead.
  - mft_ 3mo ago
    Agree. It's a fairly minimal list with very few extras added. Current /context on a fresh session (compare to that above) is: Opus 4.8 15.8k/1m tokens (2%) System prompt: 4.5k tokens (0.4%) System tools: 7.9k tokens (0.8%) Memory files: 441 tokens (0.0%) Skills: 3.1k tokens (0.3%) Messages: 8 tokens (0.0%) Free space: 984.2k (98.4%)
- mh- 3mo ago
  Ah that makes sense, wasn't trying to be pedantic. Thanks for clarifying.