4 ms·

AMD's AI software efforts are so trash WebGPU will be the first API you can use to run PyTorch on consumer AMD GPUs, not because AMD put some effort in, but bec

by tormeh 3y ago

AMD's AI software efforts are so trash WebGPU will be the first API you can use to run PyTorch on consumer AMD GPUs, not because AMD put some effort in, but because people want to run PyTorch in the browser. AMD could have ported PyTorch to OpenCL. Could have ported it to Vulkan. Instead they've made their own shitty version of CUDA that works on like 3 of their professional cards and nothing else. Maybe I don't understand something. That's the only sane explanation for what I think is completely baffling behavior.

paulmd 3y ago
> Instead they've made their own shitty version of CUDA that works on like 3 of their professional cards and nothing else. Maybe I don't understand something. That's the only sane explanation for what I think is completely baffling behavior. AMD is laser-focused on surgically tapping high-margin markets. There is money in HPC, let's support that. There is money in AI, let's support that. Nothing more nothing less. The way to think about ROCm isn't a platform like CUDA, it's as an embedded processor that gets engineered into some other product. It doesn't matter if there isn't a good general-purpose OS and ecosystem etc for Zilog Z80 - we aren't making a computer, we are making a microwave, it only ever needs to run one specific piece of software (or a small handful). And that's what AMD has become, a processor that goes into someone else's platform rather than a platform in itself. We are building a HPC supercomputer, we are building an AI training platform, and for that specific product AMD might offer the best value for performance. But you're fundamentally someone else's platform. The ironic thing is that's exactly what everyone is implying that NVIDIA might be doing now with AI, and nothing could be farther from the truth. Abandoning gaming/graphics and focusing AI would be a one-way road to the same situation AMD is in. Once you've abandoned that virtuous cycle it's tremendously hard to get back. NVIDIA has always been laser-focused on making sure that innovation happens on their platform, making sure that prosumers can write CUDA on their gaming hardware and grad students can write their thesis in CUDA and so on. It's not that NVIDIA keeps accidentally falling into success, they're deliberately putting themselves there, and they're not going to stop because of AI or anything else. If they stop and chase AI to the exclusion of graphics, that spigot will dry up, and they will no longer be in the position to catch the next fountain of money when it happens. Besides, all their other products center around graphics anyway - do you sign a big multiproduct partnership with Mediatek or Nintendo if you don't have a good gaming IP, and DLSS, and wide adoption of that software? Does Blender integrate the next OptiX if none of the userbase can run it? Do you just make quadros and not do the single last step (gaming drivers) and forego that revenue because it's not enough? No, that's crazy. Platform is hugely important to NVIDIA. It's their core product. Jensen told everyone 15 years ago that NVIDIA was a software company and people scoffed. They're not just a company that writes software, it's almost their primary product really. They write the software that sells the hardware. DLSS and AI and CUDA are their products, and they just sell you the fuel to run their product. Razor-and-blade model in action. "What if they just stopped selling razors" ok then in the long term you won't sell many blades, will you? They're just not going to do it at zero cost or a loss. And just like the Radeon 7850 has no real successor in the $150 segment, that is creeping higher and higher in the stack as fixed costs overwhelm the progress now that moore's law is dying. The true threat to NVIDIA's platform is the rising costs of low-end products. Fixed assembly/testing/shipping costs, fixed die area overhead for memory PHYs, increased VRAM needs (without drops in the per-GB cost of VRAM) - are gradually sapping the low-end segment. It is already not possible to make a $100 or $150 GPU that's very compelling, you can equivocate about whether a $200 or $300 GPU could be better for the price but nobody is making a good $100 or $150 GPU that's a worthy enthusiast-tier successor to the Radeon 7850/R7 270 or similar, because it's just not possible, and everyone agrees on that at least. The RX 6500XT and similar products are never going to compel anyone to upgrade, that segment has gone terminal. And that threshold is creeping higher every time they shrink because people want more memory and GDDR density hasn't kept up and PHYs don't shrink. There is de-facto a "minimum die size that is worth it" because of the fixed size of PHYs (to get the fixed amount of memory people want) and in a world of spiraling cost-per-mm2 that means there is a minimum cost that's worth it, and it's inching higher and higher ever time you shrink. Like try to even imagine what the $200 segment is going to look like with RDNA4 - are they going to launch a 8500XT 16GB on N5P or N3 at $200, just a tiny sliver of compute area sandwiched between PHYs? No, they can't do it either. Console-style or Apple-style APUs are the way out and that is a market that NVIDIA doesn't control, and that is the primary long-term threat to NVIDIA's platform. But for now - they are an incredibly powerful accelerator and everyone reaches for their software when they have a hard task. People get super upset when they can't upgrade the platform for NVIDIA at a compelling performance increase every 2 years. Why would you ever give that up willingly? Not even AMD wants to be where AMD is.
bavell 3y ago
Uhhh, I've been running stable diffusion on my 6750XT for a few months now... just started playing with chat and text models (oobabooga) too. Didn't even get the card thinking I could use it with ROCm for hobbyist AI stuff but turns out it works great and having 12GB of VRAM is super nice.

by tormeh 3y ago

paulmd 3y ago

> Instead they've made their own shitty version of CUDA that works on like 3 of their professional cards and nothing else. Maybe I don't understand something. That's the only sane explanation for what I think is completely baffling behavior. AMD is laser-focused on surgically tapping high-margin markets. There is money in HPC, let's support that. There is money in AI, let's support that. Nothing more nothing less. The way to think about ROCm isn't a platform like CUDA, it's as an embedded processor that gets engineered into some other product. It doesn't matter if there isn't a good general-purpose OS and ecosystem etc for Zilog Z80 - we aren't making a computer, we are making a microwave, it only ever needs to run one specific piece of software (or a small handful). And that's what AMD has become, a processor that goes into someone else's platform rather than a platform in itself. We are building a HPC supercomputer, we are building an AI training platform, and for that specific product AMD might offer the best value for performance. But you're fundamentally someone else's platform. The ironic thing is that's exactly what everyone is implying that NVIDIA might be doing now with AI, and nothing could be farther from the truth. Abandoning gaming/graphics and focusing AI would be a one-way road to the same situation AMD is in. Once you've abandoned that virtuous cycle it's tremendously hard to get back. NVIDIA has always been laser-focused on making sure that innovation happens on their platform, making sure that prosumers can write CUDA on their gaming hardware and grad students can write their thesis in CUDA and so on. It's not that NVIDIA keeps accidentally falling into success, they're deliberately putting themselves there, and they're not going to stop because of AI or anything else. If they stop and chase AI to the exclusion of graphics, that spigot will dry up, and they will no longer be in the position to catch the next fountain of money when it happens. Besides, all their other products center around graphics anyway - do you sign a big multiproduct partnership with Mediatek or Nintendo if you don't have a good gaming IP, and DLSS, and wide adoption of that software? Does Blender integrate the next OptiX if none of the userbase can run it? Do you just make quadros and not do the single last step (gaming drivers) and forego that revenue because it's not enough? No, that's crazy. Platform is hugely important to NVIDIA. It's their core product. Jensen told everyone 15 years ago that NVIDIA was a software company and people scoffed. They're not just a company that writes software, it's almost their primary product really. They write the software that sells the hardware. DLSS and AI and CUDA are their products, and they just sell you the fuel to run their product. Razor-and-blade model in action. "What if they just stopped selling razors" ok then in the long term you won't sell many blades, will you? They're just not going to do it at zero cost or a loss. And just like the Radeon 7850 has no real successor in the $150 segment, that is creeping higher and higher in the stack as fixed costs overwhelm the progress now that moore's law is dying. The true threat to NVIDIA's platform is the rising costs of low-end products. Fixed assembly/testing/shipping costs, fixed die area overhead for memory PHYs, increased VRAM needs (without drops in the per-GB cost of VRAM) - are gradually sapping the low-end segment. It is already not possible to make a $100 or $150 GPU that's very compelling, you can equivocate about whether a $200 or $300 GPU could be better for the price but nobody is making a good $100 or $150 GPU that's a worthy enthusiast-tier successor to the Radeon 7850/R7 270 or similar, because it's just not possible, and everyone agrees on that at least. The RX 6500XT and similar products are never going to compel anyone to upgrade, that segment has gone terminal. And that threshold is creeping higher every time they shrink because people want more memory and GDDR density hasn't kept up and PHYs don't shrink. There is de-facto a "minimum die size that is worth it" because of the fixed size of PHYs (to get the fixed amount of memory people want) and in a world of spiraling cost-per-mm2 that means there is a minimum cost that's worth it, and it's inching higher and higher ever time you shrink. Like try to even imagine what the $200 segment is going to look like with RDNA4 - are they going to launch a 8500XT 16GB on N5P or N3 at $200, just a tiny sliver of compute area sandwiched between PHYs? No, they can't do it either. Console-style or Apple-style APUs are the way out and that is a market that NVIDIA doesn't control, and that is the primary long-term threat to NVIDIA's platform. But for now - they are an incredibly powerful accelerator and everyone reaches for their software when they have a hard task. People get super upset when they can't upgrade the platform for NVIDIA at a compelling performance increase every 2 years. Why would you ever give that up willingly? Not even AMD wants to be where AMD is.

bavell 3y ago

Uhhh, I've been running stable diffusion on my 6750XT for a few months now... just started playing with chat and text models (oobabooga) too. Didn't even get the card thinking I could use it with ROCm for hobbyist AI stuff but turns out it works great and having 12GB of VRAM is super nice.