2 ms·
If you use threads / chains of messages in any form, I strongly encourage you to checkout caching. The cost savings are crazy. ($0.05 / cache read 1M tokens ins
by jasonjmcghee 2y ago
If you use threads / chains of messages in any form, I strongly encourage you to checkout caching. The cost savings are crazy. ($0.05 / cache read 1M tokens instead of $3 / 1M input tokens)
- bhouston 2y agoOkay I have token caching annd token costing implemented in a PR. Will go live tomorrow. Thanks for the suggestion!