4 ms·
Groq's ultrafast LPU could well be the first LLM-native processor
- LorenDB 3y agoI foresee this being the game changer that will see more companies selfhosting their own AI applications. If you can serve an entire office with just one or two LPUs, you're kinda stupid to not do it.
- zachbee 3y agoYou need an entire rack or two worth of Groq systems to get this performance, because the only memory the system has is on-chip SRAM (220MB). If you're trying to run on a small number of systems, you need local DRAM or HBM, which Groq doesn't have.
- jasonjmcghee 3y agoThis article has a strong tone of doubt that it's real- which is a bit odd. Am I missing something? This isn't a magic leap situation- they demonstrated it working, right?
- Archit3ch 3y agoThey demonstrated that throwing $12 million worth of silicon at the problem beats a benchmark, which, of course, it does. https://news.ycombinator.com/item?id=39432384 https://news.ycombinator.com/item?id=39432384
- deleted 3y ago[deleted]
- 3abiton 3y agoAren't TPU the first LLM-Native processors?
- tacitusarc 3y agoTPUs are for matrix multiplication and nn in general. They have a model compiler so stuff runs on their hardware. LLMs are just the new hotness so they’re the current focus.
- zachbee 3y agoThey get impressive performance, but it's not really the first "LLM-native processor". They built the thing a while ago before LLMs were that hot, and rebranded it as an LPU to pivot into Generative AI: https://www.eetimes.com/groq-demos-fast-llms-on-4-year-old-silicon/ https://www.eetimes.com/groq-demos-fast-llms-on-4-year-old-s...
- dikaio 3y agoHope this company’s IP security is like Fort Knox because you know China is already trying to get access to everything they got.
- seungwoolee518 3y agoIt feels like it's pre-Cryptocurrency mining boom. Most of the people tried to get the proprietary GPU, and somebody makes an FPGA & ASIC for Accelerated devices.