3 ms·
How much are you saving once the more expensive model already has all the context loaded and ready to go?
by blfr 2mo ago
How much are you saving once the more expensive model already has all the context loaded and ready to go?
- csullivannet 2mo agoAPI calls get more expensive, not less, as you've loaded more context. This is exactly when you want to switch to cheaper models.
- camdenreslink 2mo agoThere is caching to consider. Switching models throws away the cached tokens.