3 ms·
And that’s what matters the most! To me, at small model sizes (1-8B), anyway. A few thousans tokens already bog my RAM down quite a lot and I’d love to have mor
by solarkraft 2y ago
And that’s what matters the most! To me, at small model sizes (1-8B), anyway. A few thousans tokens already bog my RAM down quite a lot and I’d love to have more - I’d go as far as saying that context greatly determines LLM capability at this point.
- danielbln 2y agoYes, pretraining and post-training is nice and important, but in-context learning turns LLMs from toys into tools.