3 ms·
To help those who got a bit confused (like me) this Groq the company making accelerators designed specifically for LLM's that they call LPUs (Language Process
by Game_Ender 2y ago
To help those who got a bit confused (like me) this Groq the company making accelerators designed specifically for LLM's that they call LPUs (Language Process Units) [0]. So they want to sell you their custom machines that, while expensive, will be much more efficient at running LLMs for you. While there is also Grok [0] which is xAI's series of LLMs and competes with ChatGPT and other models like Claude and DeepSeek.
EDIT - Seems that Groq has stopped selling their chips and now will only partner to fund large build outs of their cloud [2].
0 - https://groq.com/the-groq-lpu-explained/ https://groq.com/the-groq-lpu-explained/
1 - https://grok.com/ https://grok.com/
2 - https://www.eetimes.com/groq-ceo-we-no-longer-sell-hardware https://www.eetimes.com/groq-ceo-we-no-longer-sell-hardware
- ronsor 2y agoGroq was suing Grok at some point, but Elon Musk is basically untouchable now.
- deleted 2y ago[deleted]
- iJohnDoe 2y agohttps://groq.com/hey-elon-its-time-to-cease-de-grok/ https://groq.com/hey-elon-its-time-to-cease-de-grok/
- deleted 2y ago[deleted]
- andreresende 2y ago[dead]
- IAmNotACellist 2y agoI deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant. Something like Groq or Digits but at the cost of a high-end gaming PC, like $3k. This has to be a massive market, considering that even ancient Pascal-series GPUs that were once $50 are going for $500.
- almostgotcaught 2y ago> This has to be a massive market It's not - it's absolutely a vanishingly small market.
- renewiltord 2y agoAt home people would rather use the cloud.
- deleted 2y ago[deleted]
- numa7numa7 2y agoNvidia's working on it. 200B at $3k https://www.nvidia.com/en-us/products/workstations/dgx-spark/ https://www.nvidia.com/en-us/products/workstations/dgx-spark...
- zozbot234 2y ago> I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant. So, an Apple Mac Studio?
- darksaints 2y agoI have that irresistible urge too, but I have to keep reminding myself that I could spend $2000 in credits over the course of a year, and get the performance and utility of a $40k server, with scalable capacity, and without any risk that that investment will be obsolete when Llama5 comes out.
- sofixa 2y agoThe Framework Desktop is one not absurdly expensive option. The memory speed isn't great (200 something GB/s), but any faster with those requirements at least doubles the price (e.g. a Mac Studio, only the highest tier M chips have faster memory).
- latchkey 2y ago> So they want to sell you there custom machines They stopped selling the hardware to the public, and it takes an extraordinary amount of it to run these larger models due to limited ram.
- ozenhati 2y agohi! i work @ groq and just made an account here to answer any questions for anyone who might be confused. groq has been around since 2016 and although we do offer hardware for enterprises in the form of dedicated instances, our goal is to make the models that we host easily accessible via groqcloud and groq api (openai compatible) so you can instantly get access to fast inference. :) we have a pretty generous free tier and a dev tier you can upgrade to for higher rate limits. also, we deeply value privacy and don't retain your data. you can read more about that here: https://groq.com/privacy-policy/ https://groq.com/privacy-policy/