13 ms·
I have come across quite few startups who are trying a similar idea: break the nvidia monopoly by utilizing AMD GPUs (for inference at least): Felafax, Lamini,
by shihab 2y ago
I have come across quite few startups who are trying a similar idea: break the nvidia monopoly by utilizing AMD GPUs (for inference at least): Felafax, Lamini, tensorwave (partially), SlashML. Even saw optimistic claims like CUDA moat is only 18 months deep from some of them [1]. Let's see.
[1] https://www.linkedin.com/feed/update/urn:li:activity:7275885292513906689/ https://www.linkedin.com/feed/update/urn:li:activity:7275885...
- ryukoposting 2y agoPeculiar business model, at a glance. It seems like they're doing work that AMD ought to be doing, and is probably doing behind the scenes. Who is the customer for a third-party GPU driver shim?
- tesch1 2y agoAMD. Just one more dot to connect ;)
- dpkirchner 2y agoCould be trying to make themselves a target for a big acquihire.
- dboreham 2y ago> Could be trying to make themselves a target for a big acquihire. Is this something anyone sets out to do?
- seeknotfind 2y agoYes.
- ryukoposting 2y agoIt definitely is, yes.
- to11mtm 2y agoCynical take: Try to get acquired by Intel for Arc.
- shiroiushi 2y agoMore cynical take: this would be a bad strategy, because Intel hasn't shown much competence in its leadership for a long time, especially in regards to GPUs.
- rockskon 2y agoThey've actually been making positive moves with GPUs lately along with a success story for the B580.
- schmidtleonard 2y agoYeah but MLID says they are losing money on every one and have been winding down the internal development resources. That doesn't bode well for the future. I want to believe he's wrong, but on the parts of his show where I am in a position to verify, he generally checks out. Whatever the opposite of Gell-Mann Amnesia is, he's got it going for him.
- derektank 2y agoWait, are they losing money on every one in the sense that they haven't broken even on research and development yet? Or in the sense that they cost more to manufacture than they're sold at? Because one is much worse than the other.
- rockskon 2y agoThey're trying to unseat Radeon as the budget card. That means making a more enticing offer than AMD for a temporary period of time.
- ryao 2y agoThe former is likely true, but the latter is not: https://news.ycombinator.com/item?id=42505496 https://news.ycombinator.com/item?id=42505496 That being said, the IP blocks are shared by their iGPUs, so the discrete GPUs do not need to recoup the costs of most of the R&D, as it would have been done anyway for the iGPUs.
- dylan604 2y agoIt would be interesting to find out AMD is funding these other companies to ensure the shim happens while they focus on not doing it.
- bushbaba 2y agoAMD is kind of doing that funding by pricing its GPUs low and/or giving them away at cost to these startups
- shmerl 2y agoIs this effort benefiting everyone? I.e. where is it going / is it open source?
- britannio 2y agoSome of the work from Tinycorp is: https://github.com/tinygrad/7900xtx https://github.com/tinygrad/7900xtx
- jsheard 2y agoTinygrad was another one, but they ended up getting frustrated with AMD and semi-pivoted to Nvidia.
- nomel 2y agoThis is discussed in the lex Friedman episode. AMD’s own demo would kernel panic when run in a loop [1]. [1] https://youtube.com/watch?v=dNrTrx42DGQ&t=3218 https://youtube.com/watch?v=dNrTrx42DGQ&t=3218
- kranke155 2y agoInteresting. I wonder if focusing on GPUs and CPUs is something that requires two companies instead of one, whether the concentration of resources just leads to one arm of your company being much better than the other.
- noch 2y ago> Tinygrad was another one, but they ended up getting frustrated with AMD and semi-pivoted to Nvidia. From their announcement on 20241219[^0]: "We are the only company to get AMD on MLPerf, and we have a completely custom driver that's 50x simpler than the stock one. A bit shocked by how little AMD cared, but we'll take the trillions instead of them." From 20241211[^1]: "We gave up and soon tinygrad will depend on 0 AMD code except what's required by code signing. We did this for the 7900XTX (tinybox red). If AMD was thinking strategically, they'd be begging us to take some free MI300s to add support for it." --- [^0]: https://x.com/__tinygrad__/status/1869620002015572023 https://x.com/__tinygrad__/status/1869620002015572023 [^1]: https://x.com/__tinygrad__/status/1866889544299319606 https://x.com/__tinygrad__/status/1866889544299319606
- pinsiang 2y agoAMD GPUs are becoming a serious contender for LLM inference. vLLM is already showing impressive performance on AMD [1], even with consumer-grade Radeon cards (even support GGUF) [2]. This could be a game-changer for folks who want to run LLMs without shelling out for expensive NVIDIA hardware. [1] https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html https://blog.vllm.ai/2024/10/23/vllm-serving-amd.html [2] https://embeddedllm.com/blog/vllm-now-supports-running-gguf-on-amd-radeon-gpu https://embeddedllm.com/blog/vllm-now-supports-running-gguf-...
- MrBuddyCasino 2y agoFun fact: Nvidia H200 are currently half the price/hr of H100 bc people can’t get vLLM to work on it. https://x.com/nisten/status/1871325538335486049 https://x.com/nisten/status/1871325538335486049
- treprinum 2y agoAMD decided not to release a high-end GPU this cycle so any investment into 7x00 or 6x00 is going to be wasted as Nvidia 5x00 is likely going to destroy any ROI from the older cards and AMD won't have an answer for at least two years, possibly never due to being non-existing in high-end consumer GPUs usable for compute.
- BearOso 2y agoNo high-end consumer RDNA4 GPU this cycle. And it's only missing the very high-end model. So we'll still get at least a 7800xt equivalent and whatever CDNA MI models they come out with. The market for the extreme high-end consumer is pretty small, so they're only missing out on clout.
- llama-mini 2y agoFrom Lamini, we have a private AMD GPU cluster, ready to serve any one who want to try MI300x or MI250 with inference and tuning. We just onboarded a customer to move from openai API to on-prem solution, currently evaluating MI300x for inference. Email me at my profile email.
- 3abiton 2y agoMy understanding is that once JAX takes off, the cuda advantage is gone for nvidia. That's a big if/when though.
- latchkey 2y agoHot Aisle (my company) has MI300x compute available for rent too! =)