39 ms·
I’ve said it once and j in I’ll say it again, cost per a token is still part of the equation. Why throw money away by stuffing the context with unneeded tokens?
by _boffin_ 3y ago
I’ve said it once and j in I’ll say it again, cost per a token is still part of the equation. Why throw money away by stuffing the context with unneeded tokens?
- Spivak 3y agoAnd even when it's your GPUs you probably still don't want to tie them up sending a textbook with every request.