4 ms·
In most use cases the main cost is always input, not output. Agentic workflows, on the other hand, do eat up a ton of tokens on multiple calls. Which can usuall
by orbital-decay 1y ago
In most use cases the main cost is always input, not output. Agentic workflows, on the other hand, do eat up a ton of tokens on multiple calls. Which can usually be optimized but nobody cares.
- simonw 1y agoWith multiple calls an important factor to consider is token caching, where repeat inputs are discounted. This is particularly important if you constantly replay the previous conversation and grow it with each subsequent prompt. GPT-5 offers a 90% discount on these cached tokens! That's a huge saving for this kind of pattern.