3 ms·
I have a question for those who closely follows Cerebras: do they have a future beyond being inference platform based on (an unusual) in-house silicon?
by reliabilityguy 8mo ago
I have a question for those who closely follows Cerebras: do they have a future beyond being inference platform based on (an unusual) in-house silicon?
- bob1029 8mo agoThey can also train models using this silicon. They're advertising 24T parameter models on their site right now.
- adgjlsfhk1 8mo agotldr is possibly. their packaging does offer inherent advantages in giving you maximal compute without external communication, and that seems likely to remain true unless 3d stacking advances a lot further.
- sorenbs 8mo agoIf chip manufacturing advances allow them to eventually run leading edge models at speeds much faster than competition, that seems a really bright future all on its own. Their current chip is reportedly 5nm already, and much too small for the real 5.3-codex: https://www.cerebras.ai/press-release/cerebras-announces-third-generation-wafer-scale-engine https://www.cerebras.ai/press-release/cerebras-announces-thi...
- dgacmu 8mo agoMy mental model of cerebras is that they have a way of giving you 44GB of SRAM (and then more compute than you'll probably need relative to that). So if you have applications where the memory access patterns would benefit massively from basically having 44GB of L3-ish-speed SRAM, and it's worth $1-3m to get that, then it's a win. Honestly not sure what else fits that bill. Maybe some crazy radar applications? The amount of memory is awfully small for traditional HPC.
- wmf 8mo agoDo they need any future beyond inference? It's going to be a huge market.
- reliabilityguy 8mo agoIn principle? No. In practice? I think others, eg TPUs and Trainiums of the world will cannibalize a lot of Cerebras’s share. I am not an expert though, that’s why I’m asking opinions of others.