3 ms·
Ask it the other way around, "Why is the 8K context only twice as cheap as the 32K context?" and your original answer is clearer: because they think the demand
by cscheid 4y ago
Ask it the other way around, "Why is the 8K context only twice as cheap as the 32K context?" and your original answer is clearer: because they think the demand curve supports it.
Price is not determined by cost, but by how much people are willing to pay.
- sebzim4500 4y agoNo I'm just an idiot. Since the price is per token the actual ratio is more like 8x more (assuming you fill both contexts) which is plausibly the ratio between the costs.
- Maxion 4y agoIn the end the cost for you is how many tokens consumed, if you can get more value out of one request with 32k context that multiple with lower, then it’s cheaper (and faster). Some problems also require a larger context to be able to solve.