6 ms·

One unit of HBM capacity consumes roughly the wafer capacity that could have produced three units of DDR5 capacity. HBM dies need to be larger than ordinary DRA

by bob1029 3mo ago

One unit of HBM capacity consumes roughly the wafer capacity that could have produced three units of DDR5 capacity. HBM dies need to be larger than ordinary DRAM dies because of how the final packaging works. > As discussed previously, the ramp of HBM production will constrain industry supply growth in non-HBM products. Industrywide, HBM3E consumes approximately three times the wafer supply as D5 to produce a given number of bits in the same technology node. With increased performance and packaging complexity, across the industry, we expect this trade ratio for HBM4 to be even higher than the trade ratio for HBM3E. We anticipate strong HBM demand due to AI, combined with increasing silicon intensity of the HBM roadmap, to contribute to tight supply conditions for DRAM across all end markets. As the memory industry is still recovering from the challenging environment in 2023, this tight supply environment will help drive the considerable improvements in profitability and ROI (return on investment) that are needed to enable the investments required to support future growth. https://investors.micron.com/static-files/4550f98c-1054-4847-a929-c17d520a0564 https://investors.micron.com/static-files/4550f98c-1054-4847...

vannevar 3mo ago
>As the memory industry is still recovering from the challenging environment in 2023, this tight supply environment will help drive the considerable improvements in profitability and ROI (return on investment) that are needed to enable the investments required to support future growth. The problem is that they need actual chips tomorrow. I fear we are reaching the point in the semiconductor industry where, in order to sustain the revenue growth propping up their valuations, they're going to have to start selling future chips that cannot possibly be physically produced.
- freeone3000 3mo ago
  The alternative is they could charge more. They could. And people would pay it.
  - vannevar 3mo ago
    The problem isn't the price. No matter how much you pay, you can't take delivery of a chip that doesn't exist. Now, you could raise prices to the point where you destroy demand. But that's a tricky window to maneuver through.
    - boredatoms 3mo ago
      For consumers they’re already destroying demand
    - fn-mote 3mo ago
      > The problem isn't the price. No matter how much you pay, you can't take delivery of a chip that doesn't exist I’m confused because this reads like a denial of basic economics. If the price is high enough, the chip will be produced for you. Do you mean because of the production lead time, higher prices won’t result in increased production? Commodities like corn have been managing this for a long time… what’s special about chips? What is your actual argument?
      - JoeAltmaier 3mo ago
        It takes a billion dollars and years to produce a new chip fab line. So no, it cannot just be produced for you on a dime.
        XorNot 3mo ago
        Also supply contracts exist which is what started all this: OpenAI signed huge orders for most of the market all at once. No one had any chance to properly include that information in their pricing or bidding.
      - juiceland 3mo ago
        [flagged]
      - DannyBee 3mo ago
        Corn is heavily subsidized,also has (in the US) federally prvided insurance programs against plunging prices, has paid uselessness(ethanol). Despite all of this corn has caused massive agriculture bankruptcy/takeover when it has failed anyway. So it's more "what's special about corn". It is also fairly hilarious to claim the parent is denying basic economics and then bring up corn as an example of having successfully managed economics. If the scales were not being thumbed, and "basic economics" were in play, corn would be in very very bad shape. In the case of DRAM, there is an incredibly long history of these gloom/glut cycles, and they have stayed roughly the same timeframes (~3 years) since the 1990's. Almost all the ones who have survived this long are either in the same kind of boat as corn - protected in various forms from the downside - or don't increase production and get caught out until they are absoultely forced. The very temporarily increased profit is not worth going bankrupt for - they make more money long term by being very cautious and know this. There are a near infinite number of economic studies you could look at (and several sibling comments cite some) - DRAM manufactuers don't chase the price and probably couldn't anymore if they want to. None of this denies basic economic theory, of course, since economic theory is not exactly "rigorous", even to the degree it could be (IE even the parts that are pure analysis of data rarely reproduce!).
- adastra22 3mo ago
  We saw this play out in Bitcoin mining years ago.
- re-thc 3mo ago
  > The problem is that they need actual chips tomorrow. They need it today. Best if it was NOW!
- pmontra 3mo ago
  But maybe no customer will be there to actually buy, because cash flows could crash their businesses. So options on chips? Call or put?
  - codingrightnow 3mo ago
    Call, then put.
  - Y_Y 3mo ago
    Silicon wafers and fab time are the new pork bellies and onion futures.
- egorfine 3mo ago
  > start selling future chips So, another futures commodity on CME.
- baby_souffle 3mo ago
  > they're going to have to start selling future chips that cannot possibly be physically produced. This is already happening with gpus, no? Tsmc and McDaniel capacity are sold out into the future. Data center that might not be needed in 1 to 3 years is where they’re headed.
  - i_never_listen 3mo ago
    Yes, everything so far points to about half of 2026 capacity being delayed or cancelled. How much of that is due to construction delays or the power requirements being brought online I don't know, but those are the main culprits (plus possibly hubris on the AI companies part regarding demand)
- justin66 3mo ago
  > they're going to have to start selling future chips that cannot possibly be physically produced. …but that would be unethical. cue the laugh track
  - syncsynchalt 3mo ago
    Contracting for more chips than you can make isn't a fun situation to find yourself in, nobody is doing it for profit (it's actually quite expensive in time, money, and reputation to litigate the fallout). It happens due to failed estimates (we thought we could start production in January but we couldn't figure out the process until March), upstream supply failures (our surface-mount supplier can't get us enough capacitors), or global market conditions (concrete and steel unexpectedly doubled in price and we can't afford to build the whole building this year). It happens all the time and isn't your fault (but it is your responsibility to handle).
    - vannevar 3mo ago
      >Contracting for more chips than you can make isn't a fun situation to find yourself in, nobody is doing it for profit (it's actually quite expensive in time, money, and reputation to litigate the fallout). This is true in the ordinary course of business. But we may be in an extraordinary situation where, if the chipmakers do not overcommit, their stock price collapses today. Given the choice between blowing up now or blowing up 2 years from now, these companies may very well be choosing to blow up in the future. A lot could happen in the meantime, including bankruptcy of their customers. Which would certainly solve their overcommitment problem.
- i_never_listen 3mo ago
  Do they really need them tomorrow? Or everyone is locking in supply to ensure that they have the chips if the data center actually gets built in the next few years?
  - vannevar 3mo ago
    This is a great question. Data center demand != AI demand. And chips sales at this point is driven by data center projects, not directly by AI demand.
- thaPintYo 3mo ago
  [dead]
tverbeure 3mo ago
> HBM3E consumes approximately three times the wafer supply as D5 to produce a given number of bits in the same technology node That's really interesting, and I wonder why? I believe HBM has redundant ECC bits by default, which would add a few %, but other than that, is it just that the yield is much lower due to die stacking? Of course, this is a 2024 document so things may have changed a bit since.
- benjiro29 3mo ago
  HBM requires stacking the chips. So they need to shave the layers, glue, stack more, shave again. They also require a substrate what is even more wafers. The issue is that a error in the stack means a lot of losses. In order to get high bandwidth, you want memory as close as possible to the GPU. The more trace lane length = signal loss, bandwidth loss.. HBM is compact, and so you can stack 24GB modules, 8 around a GPU die. If you tried to do that with normal memory, you need like 64 modules. So a a TON of traces more that all need to be equal length, and because so many = far away from the GPU = less bandwidth. The issue is like stated above, its a process that waste a ton of wafers. Wafers that can make easily 3x more normal memory. Intel with "Crescent Island" is trying to make a 160GB card using LPDDR5x memory but the bandwidth is only ~700GB/s.
  - wtallis 3mo ago
    None of what you said actually explains how HBM could require 3x the wafers. The base interface die has eight or 12 memory dies stacked on top of it. The wafer cost of that interface die is therefore small. If the process of die thinning and TSV stacking reduced yields by a factor of three, nobody would consider HBM mature enough to put into production, and especially not mature enough to be increasing stack height from one generation to the next. Trace length has approximately nothing to do with die size. If anything, designing for shorter traces means you can get away with smaller PHYs at either end. What might go some ways toward explaining such a huge difference in die size is that the TSVs themselves take up significant die area and must be fairly numerous to carry both a large number of signal wires and all the power and ground required by the stack. But it's wildly implausible that the die area consumed by the TSVs would be significantly larger than the die area consumed by the memory arrays themselves, or that anyone would build a memory die where the memory array was not a large majority of the total die area. If there's any truth to that ~3x higher wafer requirement for the same number of bits as compared to DDR5, it must be a combination of several factors and probably includes something non-obvious and dubious, like counting all the area of the passive interposers that go between HBM stacks and GPUs (those interposers aren't competing for the same fab space).
lofaszvanitt 2mo ago
Time to start the no llm usage in 4th quarter akin to the no nut november madness.

by bob1029 3mo ago

vannevar 3mo ago

>As the memory industry is still recovering from the challenging environment in 2023, this tight supply environment will help drive the considerable improvements in profitability and ROI (return on investment) that are needed to enable the investments required to support future growth. The problem is that they need actual chips tomorrow. I fear we are reaching the point in the semiconductor industry where, in order to sustain the revenue growth propping up their valuations, they're going to have to start selling future chips that cannot possibly be physically produced.

freeone3000 3mo ago
The alternative is they could charge more. They could. And people would pay it.
- vannevar 3mo ago
  The problem isn't the price. No matter how much you pay, you can't take delivery of a chip that doesn't exist. Now, you could raise prices to the point where you destroy demand. But that's a tricky window to maneuver through.
  - boredatoms 3mo ago
    For consumers they’re already destroying demand
  - fn-mote 3mo ago
    > The problem isn't the price. No matter how much you pay, you can't take delivery of a chip that doesn't exist I’m confused because this reads like a denial of basic economics. If the price is high enough, the chip will be produced for you. Do you mean because of the production lead time, higher prices won’t result in increased production? Commodities like corn have been managing this for a long time… what’s special about chips? What is your actual argument?
    - JoeAltmaier 3mo ago
      It takes a billion dollars and years to produce a new chip fab line. So no, it cannot just be produced for you on a dime.
      - XorNot 3mo ago
        Also supply contracts exist which is what started all this: OpenAI signed huge orders for most of the market all at once. No one had any chance to properly include that information in their pricing or bidding.
    - juiceland 3mo ago
      [flagged]
    - DannyBee 3mo ago
      Corn is heavily subsidized,also has (in the US) federally prvided insurance programs against plunging prices, has paid uselessness(ethanol). Despite all of this corn has caused massive agriculture bankruptcy/takeover when it has failed anyway. So it's more "what's special about corn". It is also fairly hilarious to claim the parent is denying basic economics and then bring up corn as an example of having successfully managed economics. If the scales were not being thumbed, and "basic economics" were in play, corn would be in very very bad shape. In the case of DRAM, there is an incredibly long history of these gloom/glut cycles, and they have stayed roughly the same timeframes (~3 years) since the 1990's. Almost all the ones who have survived this long are either in the same kind of boat as corn - protected in various forms from the downside - or don't increase production and get caught out until they are absoultely forced. The very temporarily increased profit is not worth going bankrupt for - they make more money long term by being very cautious and know this. There are a near infinite number of economic studies you could look at (and several sibling comments cite some) - DRAM manufactuers don't chase the price and probably couldn't anymore if they want to. None of this denies basic economic theory, of course, since economic theory is not exactly "rigorous", even to the degree it could be (IE even the parts that are pure analysis of data rarely reproduce!).
adastra22 3mo ago
We saw this play out in Bitcoin mining years ago.
re-thc 3mo ago
> The problem is that they need actual chips tomorrow. They need it today. Best if it was NOW!
pmontra 3mo ago
But maybe no customer will be there to actually buy, because cash flows could crash their businesses. So options on chips? Call or put?
- codingrightnow 3mo ago
  Call, then put.
- Y_Y 3mo ago
  Silicon wafers and fab time are the new pork bellies and onion futures.
egorfine 3mo ago
> start selling future chips So, another futures commodity on CME.
baby_souffle 3mo ago
> they're going to have to start selling future chips that cannot possibly be physically produced. This is already happening with gpus, no? Tsmc and McDaniel capacity are sold out into the future. Data center that might not be needed in 1 to 3 years is where they’re headed.
- i_never_listen 3mo ago
  Yes, everything so far points to about half of 2026 capacity being delayed or cancelled. How much of that is due to construction delays or the power requirements being brought online I don't know, but those are the main culprits (plus possibly hubris on the AI companies part regarding demand)
justin66 3mo ago
> they're going to have to start selling future chips that cannot possibly be physically produced. …but that would be unethical. cue the laugh track
- syncsynchalt 3mo ago
  Contracting for more chips than you can make isn't a fun situation to find yourself in, nobody is doing it for profit (it's actually quite expensive in time, money, and reputation to litigate the fallout). It happens due to failed estimates (we thought we could start production in January but we couldn't figure out the process until March), upstream supply failures (our surface-mount supplier can't get us enough capacitors), or global market conditions (concrete and steel unexpectedly doubled in price and we can't afford to build the whole building this year). It happens all the time and isn't your fault (but it is your responsibility to handle).
  - vannevar 3mo ago
    >Contracting for more chips than you can make isn't a fun situation to find yourself in, nobody is doing it for profit (it's actually quite expensive in time, money, and reputation to litigate the fallout). This is true in the ordinary course of business. But we may be in an extraordinary situation where, if the chipmakers do not overcommit, their stock price collapses today. Given the choice between blowing up now or blowing up 2 years from now, these companies may very well be choosing to blow up in the future. A lot could happen in the meantime, including bankruptcy of their customers. Which would certainly solve their overcommitment problem.
i_never_listen 3mo ago
Do they really need them tomorrow? Or everyone is locking in supply to ensure that they have the chips if the data center actually gets built in the next few years?
- vannevar 3mo ago
  This is a great question. Data center demand != AI demand. And chips sales at this point is driven by data center projects, not directly by AI demand.
thaPintYo 3mo ago
[dead]

tverbeure 3mo ago

> HBM3E consumes approximately three times the wafer supply as D5 to produce a given number of bits in the same technology node That's really interesting, and I wonder why? I believe HBM has redundant ECC bits by default, which would add a few %, but other than that, is it just that the yield is much lower due to die stacking? Of course, this is a 2024 document so things may have changed a bit since.

benjiro29 3mo ago
HBM requires stacking the chips. So they need to shave the layers, glue, stack more, shave again. They also require a substrate what is even more wafers. The issue is that a error in the stack means a lot of losses. In order to get high bandwidth, you want memory as close as possible to the GPU. The more trace lane length = signal loss, bandwidth loss.. HBM is compact, and so you can stack 24GB modules, 8 around a GPU die. If you tried to do that with normal memory, you need like 64 modules. So a a TON of traces more that all need to be equal length, and because so many = far away from the GPU = less bandwidth. The issue is like stated above, its a process that waste a ton of wafers. Wafers that can make easily 3x more normal memory. Intel with "Crescent Island" is trying to make a 160GB card using LPDDR5x memory but the bandwidth is only ~700GB/s.
- wtallis 3mo ago
  None of what you said actually explains how HBM could require 3x the wafers. The base interface die has eight or 12 memory dies stacked on top of it. The wafer cost of that interface die is therefore small. If the process of die thinning and TSV stacking reduced yields by a factor of three, nobody would consider HBM mature enough to put into production, and especially not mature enough to be increasing stack height from one generation to the next. Trace length has approximately nothing to do with die size. If anything, designing for shorter traces means you can get away with smaller PHYs at either end. What might go some ways toward explaining such a huge difference in die size is that the TSVs themselves take up significant die area and must be fairly numerous to carry both a large number of signal wires and all the power and ground required by the stack. But it's wildly implausible that the die area consumed by the TSVs would be significantly larger than the die area consumed by the memory arrays themselves, or that anyone would build a memory die where the memory array was not a large majority of the total die area. If there's any truth to that ~3x higher wafer requirement for the same number of bits as compared to DDR5, it must be a combination of several factors and probably includes something non-obvious and dubious, like counting all the area of the passive interposers that go between HBM stacks and GPUs (those interposers aren't competing for the same fab space).

lofaszvanitt 2mo ago

Time to start the no llm usage in 4th quarter akin to the no nut november madness.