4 ms·

yes but due to reasoning models the same query uses VASTLY more tokens today then a couple years ago

• simonw 1y ago
  Sure, if you enable reasoning for your prompt. A lot of prompts don't need that.
• orbital-decay 1y ago
  In most use cases the main cost is always input, not output. Agentic workflows, on the other hand, do eat up a ton of tokens on multiple calls. Which can usually be optimized but nobody cares.
  • simonw 1y ago
    With multiple calls an important factor to consider is token caching, where repeat inputs are discounted. This is particularly important if you constantly replay the previous conversation and grow it with each subsequent prompt. GPT-5 offers a 90% discount on these cached tokens! That's a huge saving for this kind of pattern.