3 ms·
Generation time is more or less proportional to tokens * model size, so if you can get the same quality result with fewer tokens from the same size of model, th
by meatmanek 11mo ago
Generation time is more or less proportional to tokens * model size, so if you can get the same quality result with fewer tokens from the same size of model, then you save time and money.
- kergonath 10mo agoThanks. That was not obvious to me either.