2 ms·
I would assume asic based llm would work really well. Why did it not go well?
by anukin 1mo ago
I would assume asic based llm would work really well. Why did it not go well?
- striking 1mo agohttps://chatjimmy.ai/ https://chatjimmy.ai/ runs Llama 3.1-8B on an ASIC as a demo by https://taalas.com/ https://taalas.com/ I believe. That's quite a few parameters shy of today's trillion-weight behemoths, but it is fast.
- mchusma 1mo agoYou are correct. I think this is the bull case. It seems like this would be useful right now for some things (eg moderation).