4 ms·
Is the 2B measured like that as well? I did use it with llama.cpp and noticed it ran slower than I expected. That's the danger of too much abstraction, it's ea
by andy99 3y ago
Is the 2B measured like that as well? I did use it with llama.cpp and noticed it ran slower than I expected.
That's the danger of too much abstraction, it's easy to have big gaps in one's understanding of what's really going on.
- rasbt 3y agoYes, it's somewhat similar to the 2B model as it uses the same vocabulary size.