5 ms·

Yeah, but all of this is pointless when RAM is as expensive as two CPUs by itself - if it's even in stock. AMD/Intel should focus on that first if they want to

Yeah, but all of this is pointless when RAM is as expensive as two CPUs by itself - if it's even in stock. AMD/Intel should focus on that first if they want to save their DIY business at all - which I'm starting to doubt they don't

DiabloD3 8mo ago
They can't fix stupid. Let me describe this in the most simple terms possible: You have speculators speculating about AI products. The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution. So, these speculators are like "oh no, more GPUs requires more RAM!", and then just start speculating on all RAM. Which of these RAMs are the ones that they need to worry about? Exclusively HBM, which is a minority in production, DDR and GDDR dominate production. If you're into inference, and have older machines, you're buying Hxxx or Bxxx cards that use HBM, fit into dual slot x16 configurations, and you're jamming (optimally) 8 of them in. If you're into hardware that is newer, somewhere in the middle of the inference boom, you're using MXM cards. In either situation, the host machine has DDR, but if you're OpenAI, Anthropic, Microsoft, or Google, you're not building (more) inference machines like this. The first two are buying Nvidia's all in one SBC solution: unified HBM, onboard ARM CPU to babysit the dual GPUs, has its own dual QSFP network controller that can RDMA, etc. No DDR or GDDR involved. Any machines built before this platform are being phased out entirely. Microsoft is doing the same, but with AMD's products, the MI series that co-locates Epyc-grade Zen 4/5 CCDs with CDNA compute chiplets, running the entire thing off HBM, thus also unified and no DDR/GDDR needed. They, too, are phasing out machines older than this. Google has a mix: they offer Nvidia all in one SBCs as part of GCP for legacy inference tasks (so your stack that can't run on AMD yet still can run), but also offer the same MI products that Microsoft offers via Azure's inference product, but also has their own TPUs that some of Gemini runs on; the TPUs run on HBM afiact. No DDR or GDDR here. So, what does AMD or Intel do here? Lets say they waste fab time to make their own dies on the wrong process (TSMC and Intel-Foundry do not have for-RAM optimized processes)... they would be producing DDR and GDDR for a market that almost has its entire demand met. Intel lacks the die stacking technology required to build HBM, and TSMC I think can't do it for that many layers (HBM has 8 to 16 layers in current gen stuff iirc). Micron, for example, already is bringing two large factories online here in the US to meet the projected growth in demand for the next 20+ years. When these factories finally start producing, it will not change the minds of speculators: they still seem to think AI datacenters need RAM, of any kind, and refuse to understand even the most basics of nuance. Also, when they come online, HBM will be a minority product; the AI inference boom is still just a bump in the road for them. Nvidia kinda screwed their consumer partners, btw: they no longer bundle the GDDR required for the card with the purchase of the die. There is a slight short term bump in GDDR spot prices as partners are building up warchests to push series 60 GPUs into production, and once that is done, spot prices return to normal (outside of the wild speculation manipulation). One last thing: what about LPDDR, used by AMD Strix Halo and Apple stuff? Speculation seems to have not actually effected it. I consider it as a sub-category of DDR (and some dies seem to work as either DDR or LPDDR as of DDR5, due to the merger of the specs by JEDEC), but since it isn't something you find in datacenters, it seems to have avoided speculation. The Ryzen Max CPUs mentioned in the linked article? Uses LPDDR. Doubling down on the Ryzen Max product line might be a brilliant move.
- PunchyHamster 8mo ago
  > The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution. The commenter is also not very smart and does not realize companies making the RAM can trade capacity of one for another and any re-tooling at current price is still profitable. The commenter also does not realize that is also true for lines currently making SSDs
  - nopurpose 8mo ago
    Reminds me of "false sharing" effect: hidden common dependency and bottleneck for what looks like independent variables on the surface.
  - DiabloD3 8mo ago
    They can trade capacity, but they generally aren't. The huge storage-only fabs owned by Samsung and Micron do runs that go for 9 months to 12 months. Flash chips haven't been speculated on nearly as hard, and are suffering from the same sort of weird lack-of-nuance. Samsung, for example, isn't reassigning capacity to meet some sort of phantom datacenter demand that isn't already there, generically, across all datacenters, AI or not. A lot of SSD price skyrocketing is largely "SSDs have RAM on them for cache", not "SSDs have flash chips, and they're both made at the same fabs"... which oddly effects low end SSDs that don't have external cache. To make it worse, for the speculators who do understand this, because it isn't some universal homogeneous group, the flash chips that go into enterprise SSDs aren't the same that go into consumer SSDs. The Big Three still aren't doing some major re-tasking of capacity, as the actual global demand isn't outstripping supply any more than normal. There is no short term problem to fix, speculators are just gonna have to stop hoarding toilet paper like its the start of Covid. Edit: Oh, and if you want to ask how AMD/TSMC or Intel solve this? They can't, same reason why making their own in-house HBM isn't happening.
    - davrosthedalek 8mo ago
corimaith 8mo ago
I find the panic over RAM prices to be overestimated. 32GB DDR5 RAM is around $500 which is comparable to to the 9800x3D. Sure it sucks that it increases by around 4x, but when you factor in the overall price of a top end PC at around 1000-2000, especially for the lion's sum of the GPU, the increase is marginal. This only effects a very narrow slice of highly budget conscious consumers trying to build high end PCs at razor thin margins.
- FpUser 8mo ago
  >"overall price of a top end PC at around 1000-2000" All 4 of my "top end PCs" have 128GB RAM. Me server (I self host everything is 512GB). Lucky for me all were bought before that insanity.
- no_ja 8mo ago
  I disagree with you. The issue does not only affect a “very narrow slice” of consumers. https://www.techspot.com/news/111472-hp-warns-ram-now-makes-up-35-cost.html https://www.techspot.com/news/111472-hp-warns-ram-now-makes-... A major brand is now suggesting that this is a “new normal” and one solution is to just offer systems with less ram. This is an issue when lots of modern software seems to expect an unending supply.
- zozbot234 8mo ago
  $500 for 32GB is about $15/GB which is a high we haven't seen since the mid-2000s. This is a big deal, it turns RAM and to some extent storage (especially fast storage) into a massive economic bottleneck.
  - BoredomIsFun 8mo ago
    > since the mid-2000s. Did you adjust for inflation ?
    - zozbot234 8mo ago
      No, I didn't adjust for the huge inflation in average RAM requirements since the mid-2000s.
Ajedi32 8mo ago
The RAM shortage is... a shortage. It's temporary by nature. RAM didn't suddenly get 4x more expensive to produce, it's just in high demand right now. Supply will eventually catch up even if it takes a few years.
- andriy_koval 8mo ago
  > Supply will eventually catch up even if it takes a few years. I am wondering if this so true. What resources and time are needed to increase supply by N times to catch demand.
  - Ajedi32 8mo ago
    The resources are certainly there; the high prices are providing them. It's just a matter of time.
  - wmf 8mo ago
    Supply will catch up to demand when demand goes down.
  - littlestymaar 8mo ago
    Also supply will only increase if producers believe the current level of demand is going to be sustainable in the long term. Which I don't think anyone really believes. The same way nobody wanted to invest in mask supply in the US or Western Europe during covid, because producers knew the demand spike wouldn't last and they'll be left with useless equipment to pay after the crisis passed.
    - Ajedi32 8mo ago
      Correct, but if they're right in that belief then it's still just a matter of time before demand goes down and the shortage is solved. If they're wrong and this is actually a permanent spike in demand, then it'll take the industry a while to realize it but eventually they'll collectively figure it out and increase supply. The ones who figure it out soonest and increase supply fastest will profit the most. The ones who figure it out slowest will lose market share.

DiabloD3 8mo ago

They can't fix stupid. Let me describe this in the most simple terms possible: You have speculators speculating about AI products. The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution. So, these speculators are like "oh no, more GPUs requires more RAM!", and then just start speculating on all RAM. Which of these RAMs are the ones that they need to worry about? Exclusively HBM, which is a minority in production, DDR and GDDR dominate production. If you're into inference, and have older machines, you're buying Hxxx or Bxxx cards that use HBM, fit into dual slot x16 configurations, and you're jamming (optimally) 8 of them in. If you're into hardware that is newer, somewhere in the middle of the inference boom, you're using MXM cards. In either situation, the host machine has DDR, but if you're OpenAI, Anthropic, Microsoft, or Google, you're not building (more) inference machines like this. The first two are buying Nvidia's all in one SBC solution: unified HBM, onboard ARM CPU to babysit the dual GPUs, has its own dual QSFP network controller that can RDMA, etc. No DDR or GDDR involved. Any machines built before this platform are being phased out entirely. Microsoft is doing the same, but with AMD's products, the MI series that co-locates Epyc-grade Zen 4/5 CCDs with CDNA compute chiplets, running the entire thing off HBM, thus also unified and no DDR/GDDR needed. They, too, are phasing out machines older than this. Google has a mix: they offer Nvidia all in one SBCs as part of GCP for legacy inference tasks (so your stack that can't run on AMD yet still can run), but also offer the same MI products that Microsoft offers via Azure's inference product, but also has their own TPUs that some of Gemini runs on; the TPUs run on HBM afiact. No DDR or GDDR here. So, what does AMD or Intel do here? Lets say they waste fab time to make their own dies on the wrong process (TSMC and Intel-Foundry do not have for-RAM optimized processes)... they would be producing DDR and GDDR for a market that almost has its entire demand met. Intel lacks the die stacking technology required to build HBM, and TSMC I think can't do it for that many layers (HBM has 8 to 16 layers in current gen stuff iirc). Micron, for example, already is bringing two large factories online here in the US to meet the projected growth in demand for the next 20+ years. When these factories finally start producing, it will not change the minds of speculators: they still seem to think AI datacenters need RAM, of any kind, and refuse to understand even the most basics of nuance. Also, when they come online, HBM will be a minority product; the AI inference boom is still just a bump in the road for them. Nvidia kinda screwed their consumer partners, btw: they no longer bundle the GDDR required for the card with the purchase of the die. There is a slight short term bump in GDDR spot prices as partners are building up warchests to push series 60 GPUs into production, and once that is done, spot prices return to normal (outside of the wild speculation manipulation). One last thing: what about LPDDR, used by AMD Strix Halo and Apple stuff? Speculation seems to have not actually effected it. I consider it as a sub-category of DDR (and some dies seem to work as either DDR or LPDDR as of DDR5, due to the merger of the specs by JEDEC), but since it isn't something you find in datacenters, it seems to have avoided speculation. The Ryzen Max CPUs mentioned in the linked article? Uses LPDDR. Doubling down on the Ryzen Max product line might be a brilliant move.

PunchyHamster 8mo ago
> The speculators are not very smart when it comes to technology, and think RAM is RAM. There is at least three kinds of RAM that are important to this: DDR for system RAM, GDDR for GPUs, and HBM for high density enterprise products, and they are not interchangeable, there is no one-die-fits-all solution. The commenter is also not very smart and does not realize companies making the RAM can trade capacity of one for another and any re-tooling at current price is still profitable. The commenter also does not realize that is also true for lines currently making SSDs
- nopurpose 8mo ago
  Reminds me of "false sharing" effect: hidden common dependency and bottleneck for what looks like independent variables on the surface.
- DiabloD3 8mo ago
  They can trade capacity, but they generally aren't. The huge storage-only fabs owned by Samsung and Micron do runs that go for 9 months to 12 months. Flash chips haven't been speculated on nearly as hard, and are suffering from the same sort of weird lack-of-nuance. Samsung, for example, isn't reassigning capacity to meet some sort of phantom datacenter demand that isn't already there, generically, across all datacenters, AI or not. A lot of SSD price skyrocketing is largely "SSDs have RAM on them for cache", not "SSDs have flash chips, and they're both made at the same fabs"... which oddly effects low end SSDs that don't have external cache. To make it worse, for the speculators who do understand this, because it isn't some universal homogeneous group, the flash chips that go into enterprise SSDs aren't the same that go into consumer SSDs. The Big Three still aren't doing some major re-tasking of capacity, as the actual global demand isn't outstripping supply any more than normal. There is no short term problem to fix, speculators are just gonna have to stop hoarding toilet paper like its the start of Covid. Edit: Oh, and if you want to ask how AMD/TSMC or Intel solve this? They can't, same reason why making their own in-house HBM isn't happening.
  - davrosthedalek 8mo ago

corimaith 8mo ago

I find the panic over RAM prices to be overestimated. 32GB DDR5 RAM is around $500 which is comparable to to the 9800x3D. Sure it sucks that it increases by around 4x, but when you factor in the overall price of a top end PC at around 1000-2000, especially for the lion's sum of the GPU, the increase is marginal. This only effects a very narrow slice of highly budget conscious consumers trying to build high end PCs at razor thin margins.

FpUser 8mo ago
>"overall price of a top end PC at around 1000-2000" All 4 of my "top end PCs" have 128GB RAM. Me server (I self host everything is 512GB). Lucky for me all were bought before that insanity.
no_ja 8mo ago
I disagree with you. The issue does not only affect a “very narrow slice” of consumers. https://www.techspot.com/news/111472-hp-warns-ram-now-makes-up-35-cost.html https://www.techspot.com/news/111472-hp-warns-ram-now-makes-... A major brand is now suggesting that this is a “new normal” and one solution is to just offer systems with less ram. This is an issue when lots of modern software seems to expect an unending supply.
zozbot234 8mo ago
$500 for 32GB is about $15/GB which is a high we haven't seen since the mid-2000s. This is a big deal, it turns RAM and to some extent storage (especially fast storage) into a massive economic bottleneck.
- BoredomIsFun 8mo ago
  > since the mid-2000s. Did you adjust for inflation ?
  - zozbot234 8mo ago
    No, I didn't adjust for the huge inflation in average RAM requirements since the mid-2000s.

Ajedi32 8mo ago

The RAM shortage is... a shortage. It's temporary by nature. RAM didn't suddenly get 4x more expensive to produce, it's just in high demand right now. Supply will eventually catch up even if it takes a few years.

andriy_koval 8mo ago
> Supply will eventually catch up even if it takes a few years. I am wondering if this so true. What resources and time are needed to increase supply by N times to catch demand.
- Ajedi32 8mo ago
  The resources are certainly there; the high prices are providing them. It's just a matter of time.
- wmf 8mo ago
  Supply will catch up to demand when demand goes down.
- littlestymaar 8mo ago
  Also supply will only increase if producers believe the current level of demand is going to be sustainable in the long term. Which I don't think anyone really believes. The same way nobody wanted to invest in mask supply in the US or Western Europe during covid, because producers knew the demand spike wouldn't last and they'll be left with useless equipment to pay after the crisis passed.
  - Ajedi32 8mo ago
    Correct, but if they're right in that belief then it's still just a matter of time before demand goes down and the shortage is solved. If they're wrong and this is actually a permanent spike in demand, then it'll take the industry a while to realize it but eventually they'll collectively figure it out and increase supply. The ones who figure it out soonest and increase supply fastest will profit the most. The ones who figure it out slowest will lose market share.