3 ms·
That's why I provided the demo link, see the tokens/second. They are running LLaMA 70B at about 260T/s without quantization. This is the fastest LLaMA2 model
by visarga 3y ago
That's why I provided the demo link, see the tokens/second. They are running LLaMA 70B at about 260T/s without quantization. This is the fastest LLaMA2 model