2 ms·
I don’t think I understand. Why would faster token generation burn more tokens? The LLM should not be generating anything in between tool calls so the only diff
by jurgenburgen 22d ago
I don’t think I understand. Why would faster token generation burn more tokens? The LLM should not be generating anything in between tool calls so the only difference should be that the human waits less between turns.