3 ms·

I always wonder what the world is going to be like when these are 2 orders of magnitude faster. Something like https://chatjimmy.ai/ https://chatjimmy.ai/ will

by nl 5mo ago

I always wonder what the world is going to be like when these are 2 orders of magnitude faster. Something like https://chatjimmy.ai/ https://chatjimmy.ai/ will do 14,000 tokens per second, and it's a completely different experience to what we see now. It's more like a page load than a conversation.

jjbinx007 5mo ago
That's really impressive from a speed point of view but it hallucinates like it's on drugs. I'd much rather trade speed for accuracy.
- gunalx 5mo ago
  Its llama 3.1 8b. So not a particular good model, mostly as a tech demo of the speed possible.