3 ms·
With multiple calls an important factor to consider is token caching, where repeat inputs are discounted. This is particularly important if you constantly repl
by simonw 1y ago
With multiple calls an important factor to consider is token caching, where repeat inputs are discounted.
This is particularly important if you constantly replay the previous conversation and grow it with each subsequent prompt.
GPT-5 offers a 90% discount on these cached tokens! That's a huge saving for this kind of pattern.