6 ms·

Now if only I could purchase one* *I realize enterprise “GPUs” are difficult to purchase as an individual whether they’re AMD or Nvidia, but AMD might be well-

Now if only I could purchase one* *I realize enterprise “GPUs” are difficult to purchase as an individual whether they’re AMD or Nvidia, but AMD might be well-served to mimic their RX 480 strategy (“make a competitive mid-range GPU, distribute it through several board partners, and sell it at rock-bottom prices to get it to as many customers as possible”). If there’s a compelling reason to use AMD hardware over Nvidia, the software support will come. As an extreme example, if one could purchase an MI300X for $50 a pop, surely FAANG and others would invest time and effort into the software side to leverage the cost savings vs Nvidia, who is unquestionably price-gouging due to their monopolistic hold on the enterprise GPU market.

mpreda 2y ago
It seems AMD has to choose between monetizing the GPUs right now by selling them at the maximum effective price, or forgoing that gain and selling the GPUs at cost and increasing GPU market share in the future. The choice between short-term gain and long-term gain.
- mpreda 2y ago
  And the splitting into CDNA and RDNA comes from the same direction: market segmentation, to allow much higher prices for the CDNA data-center GPUs, while keeping the gamer-focused RDNA GPUs affordable for mere mortals. Of coures this backfires by making the powerful GPUs not available for mostly anybody anymore to experiment on. For example this blog post, about how great MI300X is. Really, what do I care -- I'm not a billionaire.
  - tormeh 2y ago
    They’re unifying the architectures. AMD will move to UDNA for both gaming and data center. The next graphics cards after RDNA4 will be UDNA. Makes sense given how ML-heavy graphics has become.
    - kouteiheika 2y ago
      The point is they shouldn't have done it in the first place. It was obvious right from the start it's a bad idea, except maybe for temporarily boosting short term profits. The whole AMD AI/ML strategy feels like this - prioritize short term profits and completely shoot themselves in the foot in the long term.
      - Cumpiler69 2y ago
        >The whole AMD AI/ML strategy feels like this - prioritize short term profits and completely shoot themselves in the foot in the long term. That's what the stock market rewards.
      - dragontamer 2y ago
        ROCm was clearly designed with Wave64 in mind. It was going to take years for ROCm to be reworked for Wave32 of RDNA. DirectX shaders however were already ready for Wave32, and other architectural changes that RDNA had. In fact, RDNA was basically AMD changing their architecture to be more "NVidia-like" on many regards (32-wide execute being the most noticeable). CDNA existed because HPC has $Billion+ contracts with code written for Wave64 and still needing ROCm support. That means staying on the older GCN-like architecture and continuing to support say, DPP instructions or other obscure features of GCN. --------- Remember how long it took for RDNA to get ROCm support? Did you want to screw the HPC customers for that whole time? Splitting the two architectures, focusing ROCm on HPC (where the money was in 2018 for GPU Compute research dollars), and focusing on better video game performance for RDNA (where money is for video game / consumer cards) just makes sense.
  - dragontamer 2y ago
    > And the splitting into CDNA and RDNA comes from the same direction: market segmentation Not really. Wave64 on CDNA is provably more throughput. But with most video game code written for NVidia's Wave32, RDNA being reworked to be more NVidia-like and Wave32 is how you reach better practical video game performance. HPC will prefer the wider execute, 64-bit execution, and other benefits. Video Gamers will prefer massive amounts of 32MB+ of "Infinity cache", which is used in practice for all kinds of screen-space calculations. But this would NEVER be used for fluid dynamics.
    - markstock 2y ago
      Maybe never by the big players, but RDNA and even fp32 are perfectly fine for a number of CFD algorithms and uses; Stable Fluids-like algorithms and Lagrangian Vortex Particle Methods to name two.
      - dragontamer 2y ago
        I'm talking about Wave64. CDNA executes 64-threads per compute unit per clock tick. RDNA only executes 32-threads. CDNA is smaller, more efficient, more parallel and much higher compute than RDNA. Furthermore, all ROCm code from GCN (and older) was on Wave64, because historically AMD's architecture from 2010 through 2020 was Wave64. RDNA changed to Wave32 so that they can match NVidia and have slightly better latency characteristics (at the cost of bandwidth). CDNA has more compute bandwidth and parallelism. RDNA is narrower, faster latency and less parallelism. Building a GPU out of 2048-bit compute (aka: 64-lanes x 32-bit wide/CDNA) is always going to be more bandwidth than 1024-bit compute (aka: 32-lanes x 32-bit wide) like RDNA.
        markstock 2y ago
        I wasn't familiar with the "Wave32" term, but took "RDNA" to mean the smaller wavefront size. I've used both, and wave32 is still quite effective for CFD.
        dragontamer 2y ago
BoredPositron 2y ago
You mean hard in regards of pricing? Because otherwise it’s not really hard to buy enterprise gpus at all.
- latchkey 2y ago
  Depends on what country you are in. These are technically export controlled.
- gymbeaux 2y ago
  Can you provide a link to where I can purchase the MI300X? The only way I seem to be able to buy one is to contact AMD Sales who will surely laugh at my request for “less than 1000 units.”
ieidkeheb 2y ago
Nope ... IMO AMD cannot compete on software with nvidia. I bought an rx580 to test rocm/tensorflow ... Only for it to be a buggy mess that was discontinued for support in the next version. AMD needs to invest a Fsck load of money in software... Until then they can have the greatest compute cards in the world.but it will.mean nothing
- bavell 2y ago
  Isn't rx580 like 4 gens behind now? It was released in 2017. Probably not the right card for anything AI.
  - imtringued 2y ago
    Why would anyone buy a new AMD card if the expectation is that the new one won't work either after support runs out? I use AMD at home and Nvidia A100s at work. There is no need to upgrade an old GPU to a new GPU if all it does is act as a fancy iGPU.
    - KeplerBoy 2y ago
      Because only the new ones are officially supported by AMDs compute stack?
    - gymbeaux 2y ago
      To be fair Nvidia is the de-facto king of “planned obsolescence” and “remove features for no reason other than to force people to pay more $$$”. I think they’re even worse than Apple in this regard. My favorite one of many examples of this is when Nvidia said “okay, fine, we’ll allow customers to enable G-Sync (adaptive refresh) on any display, but only 1000 series and newer GPUs!” No hardware limitation for this, they just didn’t want to give any 900-series holdouts a reason not to upgrade. Then there’s the arbitrary lock on using consumer Nvidia GPUs in passthrough on a level 1 hypervisor. Sometimes there are unofficial workarounds but why do I have to go through that hassle at all? Ain’t nobody realistically going to buy consumer grade GPUs to throw in datacenter racks… but even if they were, so what? It’s their hardware, they bought it, let them use it. No reason whatsoever to block passthrough on consumer cards except $$$.
latchkey 2y ago
Disclosure, I have a business in this. This is what I saw as well. As a developer, I wanted access to enterprise HPC compute, but I'm also not going to do a PhD just to play around with these things. So, I got funding, started a business and bought 8 of them as a PoC test. We got customers, we got more funding, got a real datacenter, we bought 128 more. Crawl, walk, run. You can now rent them by the minute from us for a few bucks an hour. Currently limited to docker containers for individual GPUs, but you can get a full bare metal 8x box too (with BIOS too!). Support for VM's is coming. If you want multiple boxes, we have the full 8x400G NICs too. The boxes are fully loaded with tons of enterprise NVMe, RAM and top core/clock Intel CPUs (not AMD cause Dell didn't have that as a solution). Our model is to follow AMD's roadmap and buy/release their products as they come. We're currently debating the 325x and looking forward to / planning for the 355x. Despite your desire, it will be a long time before there is a consumer version of these things. Especially as they move to more and more complex deployments. Look at the NV72 and the requirements around that... we can all guess where AMD is going. DC rails in the racks, DLC cooling, massive power requirements. It is only getting more and more capex/opex intensive. Let's also not forget that AMD is really just a hardware manufacturer. When you buy a RX480 (I had 130,000 of these previously), it was from an OEM, like Sapphire, that could handle all the end user support. This is why the whole NeoCloud industry has sprung up. Large clouds can only handle this pace by selling thousands at a time in multi-year contracts. We are taking the long tail and built a business around that. Short of doing everything we are doing yourself (which trust me, is not easy), your best bet is to work with companies like mine to get you access to this gear. You can now rent them by the minute from us for a few bucks an hour. Currently limited to docker containers for individual GPUs, but you can get a full bare metal 8x box too. Support for VM's is coming. If you want multiple boxes, we have the full 8x400G NICs too. The boxes are fully loaded with tons of enterprise NVMe, RAM and top core/clock Intel CPUs (not AMD cause Dell didn't have that as a solution).
- LtdJorge 2y ago
  You're doing good work on your subreddit!
  - latchkey 2y ago
LtdJorge 2y ago
I'm pretty sure they're selling at capacity

mpreda 2y ago

It seems AMD has to choose between monetizing the GPUs right now by selling them at the maximum effective price, or forgoing that gain and selling the GPUs at cost and increasing GPU market share in the future. The choice between short-term gain and long-term gain.

mpreda 2y ago
And the splitting into CDNA and RDNA comes from the same direction: market segmentation, to allow much higher prices for the CDNA data-center GPUs, while keeping the gamer-focused RDNA GPUs affordable for mere mortals. Of coures this backfires by making the powerful GPUs not available for mostly anybody anymore to experiment on. For example this blog post, about how great MI300X is. Really, what do I care -- I'm not a billionaire.
- tormeh 2y ago
  They’re unifying the architectures. AMD will move to UDNA for both gaming and data center. The next graphics cards after RDNA4 will be UDNA. Makes sense given how ML-heavy graphics has become.
  - kouteiheika 2y ago
    The point is they shouldn't have done it in the first place. It was obvious right from the start it's a bad idea, except maybe for temporarily boosting short term profits. The whole AMD AI/ML strategy feels like this - prioritize short term profits and completely shoot themselves in the foot in the long term.
    - Cumpiler69 2y ago
      >The whole AMD AI/ML strategy feels like this - prioritize short term profits and completely shoot themselves in the foot in the long term. That's what the stock market rewards.
    - dragontamer 2y ago
      ROCm was clearly designed with Wave64 in mind. It was going to take years for ROCm to be reworked for Wave32 of RDNA. DirectX shaders however were already ready for Wave32, and other architectural changes that RDNA had. In fact, RDNA was basically AMD changing their architecture to be more "NVidia-like" on many regards (32-wide execute being the most noticeable). CDNA existed because HPC has $Billion+ contracts with code written for Wave64 and still needing ROCm support. That means staying on the older GCN-like architecture and continuing to support say, DPP instructions or other obscure features of GCN. --------- Remember how long it took for RDNA to get ROCm support? Did you want to screw the HPC customers for that whole time? Splitting the two architectures, focusing ROCm on HPC (where the money was in 2018 for GPU Compute research dollars), and focusing on better video game performance for RDNA (where money is for video game / consumer cards) just makes sense.
- dragontamer 2y ago
  > And the splitting into CDNA and RDNA comes from the same direction: market segmentation Not really. Wave64 on CDNA is provably more throughput. But with most video game code written for NVidia's Wave32, RDNA being reworked to be more NVidia-like and Wave32 is how you reach better practical video game performance. HPC will prefer the wider execute, 64-bit execution, and other benefits. Video Gamers will prefer massive amounts of 32MB+ of "Infinity cache", which is used in practice for all kinds of screen-space calculations. But this would NEVER be used for fluid dynamics.
  - markstock 2y ago
    Maybe never by the big players, but RDNA and even fp32 are perfectly fine for a number of CFD algorithms and uses; Stable Fluids-like algorithms and Lagrangian Vortex Particle Methods to name two.
    - dragontamer 2y ago
      I'm talking about Wave64. CDNA executes 64-threads per compute unit per clock tick. RDNA only executes 32-threads. CDNA is smaller, more efficient, more parallel and much higher compute than RDNA. Furthermore, all ROCm code from GCN (and older) was on Wave64, because historically AMD's architecture from 2010 through 2020 was Wave64. RDNA changed to Wave32 so that they can match NVidia and have slightly better latency characteristics (at the cost of bandwidth). CDNA has more compute bandwidth and parallelism. RDNA is narrower, faster latency and less parallelism. Building a GPU out of 2048-bit compute (aka: 64-lanes x 32-bit wide/CDNA) is always going to be more bandwidth than 1024-bit compute (aka: 32-lanes x 32-bit wide) like RDNA.
      - markstock 2y ago
        I wasn't familiar with the "Wave32" term, but took "RDNA" to mean the smaller wavefront size. I've used both, and wave32 is still quite effective for CFD.
        dragontamer 2y ago

BoredPositron 2y ago

You mean hard in regards of pricing? Because otherwise it’s not really hard to buy enterprise gpus at all.

latchkey 2y ago
Depends on what country you are in. These are technically export controlled.
gymbeaux 2y ago
Can you provide a link to where I can purchase the MI300X? The only way I seem to be able to buy one is to contact AMD Sales who will surely laugh at my request for “less than 1000 units.”

ieidkeheb 2y ago

Nope ... IMO AMD cannot compete on software with nvidia. I bought an rx580 to test rocm/tensorflow ... Only for it to be a buggy mess that was discontinued for support in the next version. AMD needs to invest a Fsck load of money in software... Until then they can have the greatest compute cards in the world.but it will.mean nothing

bavell 2y ago
Isn't rx580 like 4 gens behind now? It was released in 2017. Probably not the right card for anything AI.
- imtringued 2y ago
  Why would anyone buy a new AMD card if the expectation is that the new one won't work either after support runs out? I use AMD at home and Nvidia A100s at work. There is no need to upgrade an old GPU to a new GPU if all it does is act as a fancy iGPU.
  - KeplerBoy 2y ago
    Because only the new ones are officially supported by AMDs compute stack?
  - gymbeaux 2y ago
    To be fair Nvidia is the de-facto king of “planned obsolescence” and “remove features for no reason other than to force people to pay more $$$”. I think they’re even worse than Apple in this regard. My favorite one of many examples of this is when Nvidia said “okay, fine, we’ll allow customers to enable G-Sync (adaptive refresh) on any display, but only 1000 series and newer GPUs!” No hardware limitation for this, they just didn’t want to give any 900-series holdouts a reason not to upgrade. Then there’s the arbitrary lock on using consumer Nvidia GPUs in passthrough on a level 1 hypervisor. Sometimes there are unofficial workarounds but why do I have to go through that hassle at all? Ain’t nobody realistically going to buy consumer grade GPUs to throw in datacenter racks… but even if they were, so what? It’s their hardware, they bought it, let them use it. No reason whatsoever to block passthrough on consumer cards except $$$.

latchkey 2y ago

Disclosure, I have a business in this. This is what I saw as well. As a developer, I wanted access to enterprise HPC compute, but I'm also not going to do a PhD just to play around with these things. So, I got funding, started a business and bought 8 of them as a PoC test. We got customers, we got more funding, got a real datacenter, we bought 128 more. Crawl, walk, run. You can now rent them by the minute from us for a few bucks an hour. Currently limited to docker containers for individual GPUs, but you can get a full bare metal 8x box too (with BIOS too!). Support for VM's is coming. If you want multiple boxes, we have the full 8x400G NICs too. The boxes are fully loaded with tons of enterprise NVMe, RAM and top core/clock Intel CPUs (not AMD cause Dell didn't have that as a solution). Our model is to follow AMD's roadmap and buy/release their products as they come. We're currently debating the 325x and looking forward to / planning for the 355x. Despite your desire, it will be a long time before there is a consumer version of these things. Especially as they move to more and more complex deployments. Look at the NV72 and the requirements around that... we can all guess where AMD is going. DC rails in the racks, DLC cooling, massive power requirements. It is only getting more and more capex/opex intensive. Let's also not forget that AMD is really just a hardware manufacturer. When you buy a RX480 (I had 130,000 of these previously), it was from an OEM, like Sapphire, that could handle all the end user support. This is why the whole NeoCloud industry has sprung up. Large clouds can only handle this pace by selling thousands at a time in multi-year contracts. We are taking the long tail and built a business around that. Short of doing everything we are doing yourself (which trust me, is not easy), your best bet is to work with companies like mine to get you access to this gear. You can now rent them by the minute from us for a few bucks an hour. Currently limited to docker containers for individual GPUs, but you can get a full bare metal 8x box too. Support for VM's is coming. If you want multiple boxes, we have the full 8x400G NICs too. The boxes are fully loaded with tons of enterprise NVMe, RAM and top core/clock Intel CPUs (not AMD cause Dell didn't have that as a solution).

LtdJorge 2y ago
You're doing good work on your subreddit!
- latchkey 2y ago

LtdJorge 2y ago

I'm pretty sure they're selling at capacity