5 ms·
Cerebras Trains Llama Models to Leap over GPUs
- asdf1145 2y agoclickbait title: inference is not training
- mentalically 2y agoThe value proposition of Cerebras is that they can compile existing graphs to their hardware and allow inference at lower costs and higher efficiencies. The title does not say anything about creating or optimizing new architectures from scratch.
- germanjoey 2y agothe title says "Cerebras Trains Llama Models"...
- mentalically 2y agoThat's correct and if you read the whole thing you will realize that it is followed by "... to leap over GPUs" which indicates that they're not literally referring to optimizing the weights of the graph on a new architecture or freshly initialized variables on existing ones.
- pama 2y agoThis is as clickbaity as it gets. Trains has no other sensible interpretation in the context of LLM models. My impression was that they trained the models to be better than the models trained by GPUs, presumably because they trained faster and managed to train for longer than Meta, but this interpretation was far from the content. Also interesting to see the ommission of deepinfra from the price table, presumably because it would be cheaper than Cerebras, though I didnt even bother to check at that point because I hate these cheap clickbaity pieces that attempt to enrich some player at the cost of everyone’s time or money. Good luck with their IPO. We need competition but we dont need confusion.
- mentalically 2y agoWhat are you confused about? Their value proposition is very simple and obvious, custom hardware with a compiler that transforms existing graphs into a format that can run at lower cost and higher efficiency because it utilizes a special instruction set only available on Cerebras silicon.
- fancyfredbot 2y agoThe title is clickbait but that's how marketing works whether we like it or not. The achievement is real - Cerberas improved their software and the inference is much faster as a result. I find it easy to forgive annoying marketing tactics when they're being used to promote something cool.
- pama 2y agoIt is textbook bait and switch. If the achievemt is important, use the correct title. An advance in actual training performance or a better model is very important and interests a different set of people with deeper pockets than those who care about inference.
- 7e 2y ago"It would be interesting to see what the delta in accuracy is for these benchmarks." ^ the entirety of it
- 7e 2y ago"So, the delta in price/performance between Cerebras and the Hoppers in the cloud when buying iron is 2.75X but for renting iron it is 5.2X, which seems to imply that Cerebras is taking a pretty big haircut when it rents out capacity. That kind of delta between renting out capacity and selling it is not a business model, it is a loss leader from a startup trying to make a point." As always, it is about TCO, not who can make the biggest monster chip.
- asdf1145 2y agodid they release MLPerf data yet or wouldn't help their IPO?
- latchkey 2y ago1x MI300x has 192GB HBM3. 1x MI325x has 256GB HBM3e. They cost less, you can fit more into a rack and you can buy/deploy at least the 300's today and 325's early next year. AMD and library software performance for AI is improving daily [0]. I'm still trying to wrap my head around how these companies think they are going to do well in this market without more memory. [0] https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html
- wmf 2y agoGroq and Cerebras only make sense at massive scale which is why I guess they pivoted to being API providers so they can amortize the hardware over many customers.
- latchkey 2y agoCorrect except that massive scale doesn't work cause it just uses up exponentially more power/space/resources. They also have a very limited use case... if things ever shift away from LLM's and into another form of engineering that their hardware does not support, what are they going to do? Just keep deploying hardware? Slippery slope.
- krasin 2y ago> I'm still trying to wrap my head around how these companies think they are going to do well in this market without more memory. Cerebras and Groq provide the fastest (by an order of magnitude) inference. This is very useful for certain workflows, which require low-latency feedback: audio chat with LLM, robotics, etc. Outside that narrow niche, AMD stuff seems to be the only contender to NVIDIA, at the moment.
- latchkey 2y ago> Cerebras and Groq provide the fastest (by an order of magnitude) inference. Only on smaller models, their numbers are all 70b in the article. Those numbers also need to be adjusted for the comparable amounts of capex+opex costs. If the costs are so high that they have to subsidize the usage/results, then they are just going to run out of money, fast.
- KTibow 2y agoWhy is nobody mentioning that there is no such thing as Llama 3.2 70B
- htrp 2y agoTitle is about training.... article about inference
- pk-protect-ai 2y agoWow, 44GB SRAM, not HBM3 or HBM3e, but actual SRAM ...
- deleted 2y ago[deleted]