3 ms·
I feel like we need an entirely new type of silicon for LLMs. Something completely focused on bandwidth and storage probably at the sacrifice of raw computation
by exabrial 8mo ago
I feel like we need an entirely new type of silicon for LLMs. Something completely focused on bandwidth and storage probably at the sacrifice of raw computation power.
- deleted 8mo ago[deleted]
- garethsprice 8mo agoSomething like this? (Llama 3.1-8B etched into custom silicon delivering 16,000 tok/s, doesn't use much PCIe bandwidth): - https://taalas.com/the-path-to-ubiquitous-ai/ https://taalas.com/the-path-to-ubiquitous-ai/ - https://chatjimmy.ai/ https://chatjimmy.ai/
- exabrial 8mo agoWowsa that’s amazing! Exactly what I was imagining. To do that with 2500 watts is incredible.