4 ms·

Relevant thread from 5 months ago: https://news.ycombinator.com/item?id=37469434 https://news.ycombinator.com/item?id=37469434 I'm achieving consistent 450+ to

Relevant thread from 5 months ago: https://news.ycombinator.com/item?id=37469434 https://news.ycombinator.com/item?id=37469434 I'm achieving consistent 450+ tokens/sec for Mixtral 8x7b 32k and ~200 tps for Llama 2 70B-4k. As an aside, seeing that this is built with flutter Web, perhaps a mobile app is coming soon?

tome 3y ago
There was also another discussion about Groq a couple of months ago https://news.ycombinator.com/item?id=38739199 https://news.ycombinator.com/item?id=38739199