2 ms·
Something like this? (Llama 3.1-8B etched into custom silicon delivering 16,000 tok/s, doesn't use much PCIe bandwidth): - https://taalas.com/the-path-to-ubiqu
by garethsprice 8mo ago
Something like this? (Llama 3.1-8B etched into custom silicon delivering 16,000 tok/s, doesn't use much PCIe bandwidth):
- https://taalas.com/the-path-to-ubiquitous-ai/ https://taalas.com/the-path-to-ubiquitous-ai/
- https://chatjimmy.ai/ https://chatjimmy.ai/
- exabrial 8mo agoWowsa that’s amazing! Exactly what I was imagining. To do that with 2500 watts is incredible.