4 ms·
They should work towards making repetitive prompts be free or much cheaper because they don't have to generate it token by token and can be cached.
by visarga 3y ago
They should work towards making repetitive prompts be free or much cheaper because they don't have to generate it token by token and can be cached.
- amitport 3y agoAs a customer of course I would want that, but they really shouldn't from their perspective. The customer is getting the value it wants and is willing to pay for this is where a lot of the profit will come from.