2 ms·
No I'm just an idiot. Since the price is per token the actual ratio is more like 8x more (assuming you fill both contexts) which is plausibly the ratio between
by sebzim4500 4y ago
No I'm just an idiot. Since the price is per token the actual ratio is more like 8x more (assuming you fill both contexts) which is plausibly the ratio between the costs.
- Maxion 4y agoIn the end the cost for you is how many tokens consumed, if you can get more value out of one request with 32k context that multiple with lower, then it’s cheaper (and faster). Some problems also require a larger context to be able to solve.