4 ms·
In the end the cost for you is how many tokens consumed, if you can get more value out of one request with 32k context that multiple with lower, then it’s cheap
by Maxion 4y ago
In the end the cost for you is how many tokens consumed, if you can get more value out of one request with 32k context that multiple with lower, then it’s cheaper (and faster).
Some problems also require a larger context to be able to solve.