6 ms·

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point,

> Krishna also referenced the depreciation of the AI chips inside data centers as another factor: "You've got to use it all in five years because at that point, you've got to throw it away and refill it," he said This doesn't seem correct to me, or at least is built on several shaky assumptions. One would have to 'refill' your hardware if: - AI accelerator cards all start dying around the 5 year mark, which is possible given the heat density/cooling needs, but doesn't seem all that likely. - Technology advances such that only the absolute newest cards can be used to run _any_ model profitably, which only seems likely if we see some pretty radical advances in efficiency. Otherwise, it seems like assuming your hardware is stable after 5 years of burn in, you could continue to run older models on that hardware at only the cost of the floorspace/power. Maybe you need new cards for new models for some reason (maybe a new fp format that only new cards support? some magic amount of ram? etc), but it seems like there may be room for revenue via older/less capable models at a discounted rate.

mcculley 10mo ago
But if your competitor is running newer chips that consume less power per operation, aren't you forced to upgrade as well and dispose of the old hardware?
- Octoth0rpe 10mo ago
  Sure, assuming the power cost reduction or capability increase justifies the expenditure. It's not clear that that will be the case. That's one of the shaky assumptions I'm referring to. It may be that the 2030 nvidia accelerators will save you $2000 in electricity per month per rack, and you can upgrade the whole rack for the low, low price of $800,000! That may not be worth it at all. If it saves you $200k/per rack or unlocks some additional capability that a 2025 accelerator is incapable of and customers are willing to pay for, then that's a different story. There are a ton of assumptions in these scenarios, and his logic doesn't seem to justify the confidence level.
  - trollbridge 10mo ago
    A typical data centre is $2,500 per year per kW load (including overhead, hvac and so on). If it costs $800,000 to replace the whole rack, then that would pay off in a year if it reduces 320 kW of consumption. Back when we ran servers, we wouldn't assume 100% utilisation but AI workloads do do that; normal server loads would be 10kW per rack and AI is closer to 100. So yeah, it's not hard to imagine power savings of 3.2 racks being worth it.
    - Octoth0rpe 10mo ago
      Thanks for the numbers! Isn't it more likely that the amount of power/heat generated per rack will stay constant over each upgrade cycle, and the upgrade simply unlocks a higher amount of service revenue per rack?
      - PunchyHamster 10mo ago
        Not in the last few years. CPUs went from ~200W TDP to 500W. And they went from zero to multiple GPUs per server. Tho we might hit "the chips can't be bigger and the cooling can't get much better" point there. The usage would be similar if it was say a rack filled with servers full of bulk storage (hard drives generally keep the power usage similar while growing storage). But CPU/GPU wise, it's just bigger chips/more chiplets, more power. I'd imagine any flattening might be purely because "we have DC now, re-building cooling for next gen doesn't make sense so we will just build servers with similar power usage as previously", but given how fast AI pushed the development it might not happen for a while.
      - toast0 10mo ago
        > Isn't it more likely that the amount of power/heat generated per rack will stay constant over each upgrade cycle, Power density seems to grow each cycle. But eventually your DC hits power capacity limits, and you have to leave racks empty because there's no power budget.
      - bleepblap 10mo ago
        I've been in university research computing for 15 years, so large enough (~900 nodes) we need a dedicated DC, but not at the same scale as others around here. Our racks are provisioned so that there are two independent rails, which each can support 7kW. Up until the last few years, this was more than enough power. As CPU TDPs increased, we started to need to do things like not connect some nodes to both redundant rails or mix disk servers into compute racks to keep under 7kW/rack. A single HGX B300 box has 6x6kW power supplies. Even before we get to paying the (high) power bills, it's going to cost a small fortune to just update the racks, power distribution units, UPS, etc... to even be able to support more than a handful of those things
- HWR_14 10mo ago
  It depends on how much profit you are making. As long as you can still be profitable on the old hardware you don't have to upgrade.
  - AstroBen 10mo ago
    That's the thing though: a competitor with better power efficiency can undercut you and take your customers
    - tzs 10mo ago
      Or they could charge the same as you and make more money per customer. If they already have as many customers as they can handle doing that may be better than buying hardware to support a larger number of customers.
beachy 10mo ago
It's just the same dynamic as old servers. They still work fine but power costs make them uneconomical compared to latest tech.
- acdha 10mo ago
  It’s far more extreme: old servers are still okay on I/O, and memory latency, etc. won’t change that dramatically so you can still find productive uses for them. AI workloads are hyper-focused on a single type of work and, unlike most regular servers, a limiting factor in direct competition with other companies.
  - matt-p 10mo ago
    I mean you could use training GPUs for inference right? That would be use case number 1 for a 8 * a100 box in a couple of years. It can also be used for non IO limited things like folding proteins or other 'scientific' use cases. Push comes to shove im sure an old A100 will run crysis.
    - oblio 10mo ago
      All those use cases would probably use up 1% of the current AI infrastructure, let alone ahat they're planning to build. Yeah, just like gas, possible uses will expand if AI crashes out, but: * will these uses cover, say, 60% of all this infra? * will these uses scale up to use that 60% within the next 5-7 years, while that hardware is still relevant and fully functional? Also, we still have railroad tracks from the 1800s rail mania that were never truly used to capacity and dot com boom dark fiber that's also never been used fully, even with the internet growing 100x since. And tracks and fiber don't degrade as quickly as server hardware and especially GPUs.
    - physicsguy 10mo ago
      > Push comes to shove im sure an old A100 will run crysis. They don’t have video out ports!
      - 10mo ago
dmoy 10mo ago
5 years is maybe referring to the accounting schedule for depreciation on computer hardware, not the actual useful lifetime of the hardware. It's a little weird to phrase it like that though because you're right it doesn't mean you have to throw it out. Idk if this is some reflection of how IBM handles finance stuff or what. Certainly not all companies throw out hardware the minute they can't claim depreciation on it. But I don't know the numbers. Anyways, 5 years is an infection point on numbers. Before 5 years you get depreciation to offset some cost of running. After 5 years, you do not, so the math does change.
- skeeter2020 10mo ago
  that is how the investments are costed though, so makes sense when we're talking return on investment, so you can compare with alternatives under the same evaluation criteria.
austin-cheney 10mo ago
It’s not about assumptions on the hardware. It’s about the current demands for computation and expected growth of business needs. Since we have a couple years to measure against it should be extremely straightforward to predict. As such I have no reason to doubt the stated projections.
- andix 10mo ago
  Failure rates also go up. For AI inference it’s probably not too bad in most cases, just take the node offline and re-schedule the jobs to other nodes.
- 9cb14c1ec0 10mo ago
  > Since we have a couple years to measure against Trillion pound baby fallacy.
- lumost 10mo ago
  Networking gear was famously overbought. Enterprise hardware is tricky as there isn’t much of a resale market for this gear once all is said and done. The only valid use case for all of this compute which could reasonably replace ai is btc mining. I’m uncertain if the increased mining capacity would harm the market or not.
  - piva00 10mo ago
    BTC mining on GPUs haven't been profitable for a long time, it's mostly ASICs, GPUs can be used for some other altcoins which makes the potential market for used previous generation GPUs even smaller.
    - blackenedgem 10mo ago
      That assumes you can add compute in a vacuum. If your altcoin receives 10x compute then it becomes 10x more expensive to mine. That only scales if the coin goes up in value due to the extra "interest". Which isn't impossible but there's a limit, and it's more often to happen to smaller coins.
rzerowan 10mo ago
I think its illustrative to consider the previous computation cycle ala Cryptomining. Which passed through a similar lifecycle with energy and GPU accelerators. The need for cheap wattage forced the operations to arbitrage the where location for the cheapest/reliable existing supply - there rarely was new buildout as the cost was to be reimbursed by the coins the miningpool recovered. For the chip situation caused the same apprecaition in GPU cards with periodic offloading of cards to the secondary market (after wear and tear) as newer/faster/more efficient cards came out until custom ASICs took over the heavy lifting, causing the GPU card market to pivot. Similarly in the short to moedium term the uptick of custo ASICs like with Google TPU will definately make a dent in bot cpex/opex and potentially also lead to a market with used GPUs as ASICs dominate. So for GPUs i can certainly see the 5 year horizon making a impact in investment decisions as ASICs proliferate.
coliveira 10mo ago
There is the opportunity cost of using a whole datacenter to house ancient chips, even if they're still running. You're thinking like a personal use chip which you can run as long as it is non-defective. But for datacenters it doesn't make sense to use the same chips for more than a few years and I think 5 years is already stretching their real shelf life.
lithos 10mo ago
It's worse than that in reality, AI chips are on a two year cadence for backwards compatibility (NVIDIA can basically guarantee it, and you probably won't be able to pay real AI devs enough to stick around to make hardware work arounds). So their accounting is optimistic.
- Patrick_Devine 10mo ago
  5 years is normal-ish depreciation time frame. I know they are gaming GPUs, but the RTX 3090 came out ~ 4.5 years before the RTX 5090. The 5090 has double the performance and 1/3 more memory. The 3090 is still a useful card even after 5 years.
  - cubefox 10mo ago
    RTX 3090 MSRP: 1500 USD RTX 5090 MSRP: 2000 USD
rlupi 10mo ago
Do not forget that we're talking about supercomputers. Their interconnect makes machines not easily fungible, so even a low reduction in availability can have dramatic effects. Also, after the end of the product life, replacement parts may no longer be available. You need to get pretty creative with repair & refurbishment processes to counter these risks.
slashdave 10mo ago
5 years is long, actually. This is not a GPU thing. It's standard for server hardware.
- bigwheels 10mo ago
  Because usually it's more efficient for companies to retire the hardware and put in new stuff. Meanwhile, my 10-15 year old server hardware keeps chugging along just fine in the rack in my garage.
  - XorNot 10mo ago
    Sample size of 1 though. It's like how I've had hard disks last a decade, but a 100 node Hadoop cluster had 3 die per week after a few years.
    - snuxoll 10mo ago
      Spinning rust and fans are the outliers when it comes to longevity in compute hardware. I’ve had to replace a disk or two in my rack at home, but at the end of the day the CPUs, RAM, NICs, etc. all continue to tick along just fine. When it comes to enterprise deployments, the lifecycle always revolves around price/performance. Why pay for old gear that sucks up power and runs 30% slower than the new hotness, after all! But, here we are, hitting limits of transistor density. There’s a reason I still can’t get 13th or 14th gen poweredge boxes for the price I paid for my 12th gen ones years ago.
  - AdrianB1 10mo ago
    I thought the same until I calculated that newer hardware consumes a few times less energy and for something running 24x7 that adds up quite a bit (I live in Europe, energy is quite expensive). So my homelab equipment is just 5 years old and it will get replaced in 2-3 years with something even more power efficient.
    - tharkun__ 10mo ago
      Where in Europe? Asking coz I just did a quick comparison and it seems to depend but for comparison I have a really old AMD Athlon "e" processor (like literally September 2009 is when it came out according to some quick Google search, tho I probably bought it a few months later than that but still ...) that runs at ~45W TDP. In idle conditions, it typically consumes around 10 to 15 watts (internet wisdom, not kill-a-watt-wisdom). Some napkin math says it would cost me about 40 years worth of amortization to replace this at my current power rates for this system. So why would I replace it? And even with some EU countries' power rates we seem to be at 5-10 years amortization upon replacement. I've been running this motherboard, CPU + RAM combo for ~15 years now it seems, replacing only the hard drives every ~3 years. And the tower it's in is about 25 years old. Oh I forgot, I think I had to buy two new CR2032 batteries during those years (CMOS battery). Now granted, this processor can basically do "nothing" in comparison to a current system I might buy. But I also don't need more for what it does.
loeg 10mo ago
It's option #2. But 5 year deprecation is optimistic; 2-3 years is more realistic.
- anticensor 10mo ago
  Isn't the depreciation timespan imposed by the particular accounting standard?
deleted 10mo ago
[deleted]
more_corn 10mo ago
When you operate big data centers it makes sense to refresh your hardware every 5 years or so because that’s the point at which the refreshed hardware is enough better to be worth the effort and expense. You don’t HAVE to, but its more cost effective if you do. (Source, used to operate big data centers)
darth_avocado 10mo ago
Isn’t that what Michael Burry is complaining about? That five years is actually too generous when it comes to depreciation of these assets and that companies are being too relaxed with that estimate. The real depreciation is more like 2-3 years for these GPUs that cost tens of thousands of dollars a piece. https://x.com/michaeljburry/status/1987918650104283372 https://x.com/michaeljburry/status/1987918650104283372
- duped 10mo ago
  How different is this from rental car companies changing over their fleets? I don't know, this is a genuine question. The cars cost 3-4x as much and last about 2x as far as I know, and the secondary market is still alive.
  - afavour 10mo ago
    Rental car companies aren’t offering rentals at deep discount to try to kickstart a market. It would be much less of a deal if these companies were profitable and could cover the costs of renewing hardware, like car rental companies can.
  - chii 10mo ago
    > the secondary market is still alive. this is the crux. Will these data center cards, if a newer model came out with better efficiency, have a secondary market to sell to? It could be that second hand ai hardware going into consumers' hands is how they offload it without huge losses.
    - vesrah 10mo ago
      The GPUs going into data centers aren't the kind that can just be reused by putting them into a consumer PC and playing some video games, most don't even have video output ports and put out FPS similar to cheap integrated GPUs.
      - geerlingguy 10mo ago
marcosdumay 10mo ago
Historically, GPUs have improved in efficiency fast enough that people retired their hardware in way less than 5 years. Also, historically the top of the line fabs were focused on CPUs, not GPUs. That has not been true for a generation, so it's not really clear if the depreciation speed will be maintained.
- chii 10mo ago
  > that people retired their hardware in way less than 5 years. those people are end-consumers (like gamers), and only recently, bitcoin miners. Gamers don't care for "profit and loss" - they want performance. Bitcoin miners do need to switch if they want to keep up. But will an AI data center do the same?
  - TingPing 10mo ago
    The rate of change is equal for all groups. The gaming market can be the most conservative since it’s just luxury.
  - thinkmassive 10mo ago
    Mining bitcoin with a GPU hasn't been profitable in over a decade.
- cubefox 10mo ago
  > Historically, GPUs have improved in efficiency fast enough that people retired their hardware in way less than 5 years. This was a time when chip transistor cost was decreasing rapidly. A few years earlier even RAM cost was decreasing quickly. But these times are over now. For example, the PlayStation 5 (where the GPU is the main cost), which launched five years ago, even increased in price! This is historically unprecedented. Most price/performance progress is nowadays made via better GPU architecture instead, but these architectures are already pretty mature, so the room for improvement is limited. Given that the price per transistor (which TSMC & Co are charging) has decreased ever more slowly in recent years, I assume it will eventually come almost to a halt. By the way, this is strictly speaking compatible with Moore's law, as it is only about transistors per chip area, not price. Of course the price per chip area was historically approximately constant, which meant exponentially increasing transistor density implied exponentially decreasing transistor price.
protocolture 10mo ago
Actually my biggest issue here is that, assuming it hasnt paid off, you dont just convert to regular data center usage. Honestly if we see a massive drop in DC costs because the AI bubble bursts I will be stoked.
m101 10mo ago
Given power and price constraints, it's not that you cannot run them in 5 years time it's that you don't want to run them in 5 years time and neither will anyone else that doesn't have free power.
ExoticPearTree 10mo ago
> AI accelerator cards all start dying around the 5 year mark, More likely the technology will be much better in 5 years in terms of hardware that it is (very) uneconomical to run anything on old hardware.
araes 10mo ago
General question to people who might actually know. Is there anywhere that does anything like Backblaze's Hard Drive Failure Rates [1] for GPU Failure Rates in environments like data centers, high-performance computing, super-computers, mainframes? [1] https://www.backblaze.com/blog/backblaze-drive-stats-for-q3-2025/ https://www.backblaze.com/blog/backblaze-drive-stats-for-q3-... The best that came back on a search was a semi-modern article from 2023 [2] that appears to be a one-off and mostly related to consumer facing GPU purchases, rather than bulk data center, constant usage conditions. It's just difficult to really believe some of these kinds of hardware deprecation numbers since there appears to be so little info other than guesstimates. [2] https://www.tweaktown.com/news/93052/heres-look-at-gpu-failure-rates-and-warranty-claim-times-for-all-major-brands/index.html https://www.tweaktown.com/news/93052/heres-look-at-gpu-failu... Found an Arxiv paper on continued checking that's from UIUC UrbanaIL about a 1,056 A100 and H100 GPU system. [3] However, the paper is primarily about memory issues and per/job downtime that causes task failures and work loss. GPU Resilience is discussed, it's just mostly from the perspective of short-term robustness in the face of propagating memory corruption issues and error correction, rather than multi-year, 100% usage GPU burnout rates. [3] https://arxiv.org/html/2503.11901v3 https://arxiv.org/html/2503.11901v3 Any info on the longer term burnout / failure rates for GPUs similar to Backblaze? Edit: This article [4] claims it's 0.1-2% failure rate per year (0.8% (estimated)) with no real info about where the data came from (cites "industry reports and data center statistics"), and then claims they often last 3-5 years on average. [4] https://cyfuture.cloud/kb/gpu/what-is-the-failure-rate-of-the-h100-gpu https://cyfuture.cloud/kb/gpu/what-is-the-failure-rate-of-th...

mcculley 10mo ago

But if your competitor is running newer chips that consume less power per operation, aren't you forced to upgrade as well and dispose of the old hardware?

Octoth0rpe 10mo ago
Sure, assuming the power cost reduction or capability increase justifies the expenditure. It's not clear that that will be the case. That's one of the shaky assumptions I'm referring to. It may be that the 2030 nvidia accelerators will save you $2000 in electricity per month per rack, and you can upgrade the whole rack for the low, low price of $800,000! That may not be worth it at all. If it saves you $200k/per rack or unlocks some additional capability that a 2025 accelerator is incapable of and customers are willing to pay for, then that's a different story. There are a ton of assumptions in these scenarios, and his logic doesn't seem to justify the confidence level.
- trollbridge 10mo ago
  A typical data centre is $2,500 per year per kW load (including overhead, hvac and so on). If it costs $800,000 to replace the whole rack, then that would pay off in a year if it reduces 320 kW of consumption. Back when we ran servers, we wouldn't assume 100% utilisation but AI workloads do do that; normal server loads would be 10kW per rack and AI is closer to 100. So yeah, it's not hard to imagine power savings of 3.2 racks being worth it.
  - Octoth0rpe 10mo ago
    Thanks for the numbers! Isn't it more likely that the amount of power/heat generated per rack will stay constant over each upgrade cycle, and the upgrade simply unlocks a higher amount of service revenue per rack?
    - PunchyHamster 10mo ago
      Not in the last few years. CPUs went from ~200W TDP to 500W. And they went from zero to multiple GPUs per server. Tho we might hit "the chips can't be bigger and the cooling can't get much better" point there. The usage would be similar if it was say a rack filled with servers full of bulk storage (hard drives generally keep the power usage similar while growing storage). But CPU/GPU wise, it's just bigger chips/more chiplets, more power. I'd imagine any flattening might be purely because "we have DC now, re-building cooling for next gen doesn't make sense so we will just build servers with similar power usage as previously", but given how fast AI pushed the development it might not happen for a while.
    - toast0 10mo ago
      > Isn't it more likely that the amount of power/heat generated per rack will stay constant over each upgrade cycle, Power density seems to grow each cycle. But eventually your DC hits power capacity limits, and you have to leave racks empty because there's no power budget.
    - bleepblap 10mo ago
      I've been in university research computing for 15 years, so large enough (~900 nodes) we need a dedicated DC, but not at the same scale as others around here. Our racks are provisioned so that there are two independent rails, which each can support 7kW. Up until the last few years, this was more than enough power. As CPU TDPs increased, we started to need to do things like not connect some nodes to both redundant rails or mix disk servers into compute racks to keep under 7kW/rack. A single HGX B300 box has 6x6kW power supplies. Even before we get to paying the (high) power bills, it's going to cost a small fortune to just update the racks, power distribution units, UPS, etc... to even be able to support more than a handful of those things
HWR_14 10mo ago
It depends on how much profit you are making. As long as you can still be profitable on the old hardware you don't have to upgrade.
- AstroBen 10mo ago
  That's the thing though: a competitor with better power efficiency can undercut you and take your customers
  - tzs 10mo ago
    Or they could charge the same as you and make more money per customer. If they already have as many customers as they can handle doing that may be better than buying hardware to support a larger number of customers.

beachy 10mo ago

It's just the same dynamic as old servers. They still work fine but power costs make them uneconomical compared to latest tech.

acdha 10mo ago
It’s far more extreme: old servers are still okay on I/O, and memory latency, etc. won’t change that dramatically so you can still find productive uses for them. AI workloads are hyper-focused on a single type of work and, unlike most regular servers, a limiting factor in direct competition with other companies.
- matt-p 10mo ago
  I mean you could use training GPUs for inference right? That would be use case number 1 for a 8 * a100 box in a couple of years. It can also be used for non IO limited things like folding proteins or other 'scientific' use cases. Push comes to shove im sure an old A100 will run crysis.
  - oblio 10mo ago
    All those use cases would probably use up 1% of the current AI infrastructure, let alone ahat they're planning to build. Yeah, just like gas, possible uses will expand if AI crashes out, but: * will these uses cover, say, 60% of all this infra? * will these uses scale up to use that 60% within the next 5-7 years, while that hardware is still relevant and fully functional? Also, we still have railroad tracks from the 1800s rail mania that were never truly used to capacity and dot com boom dark fiber that's also never been used fully, even with the internet growing 100x since. And tracks and fiber don't degrade as quickly as server hardware and especially GPUs.
  - physicsguy 10mo ago
    > Push comes to shove im sure an old A100 will run crysis. They don’t have video out ports!
    - 10mo ago

dmoy 10mo ago

5 years is maybe referring to the accounting schedule for depreciation on computer hardware, not the actual useful lifetime of the hardware. It's a little weird to phrase it like that though because you're right it doesn't mean you have to throw it out. Idk if this is some reflection of how IBM handles finance stuff or what. Certainly not all companies throw out hardware the minute they can't claim depreciation on it. But I don't know the numbers. Anyways, 5 years is an infection point on numbers. Before 5 years you get depreciation to offset some cost of running. After 5 years, you do not, so the math does change.

skeeter2020 10mo ago
that is how the investments are costed though, so makes sense when we're talking return on investment, so you can compare with alternatives under the same evaluation criteria.

austin-cheney 10mo ago

It’s not about assumptions on the hardware. It’s about the current demands for computation and expected growth of business needs. Since we have a couple years to measure against it should be extremely straightforward to predict. As such I have no reason to doubt the stated projections.

andix 10mo ago
Failure rates also go up. For AI inference it’s probably not too bad in most cases, just take the node offline and re-schedule the jobs to other nodes.
9cb14c1ec0 10mo ago
> Since we have a couple years to measure against Trillion pound baby fallacy.
lumost 10mo ago
Networking gear was famously overbought. Enterprise hardware is tricky as there isn’t much of a resale market for this gear once all is said and done. The only valid use case for all of this compute which could reasonably replace ai is btc mining. I’m uncertain if the increased mining capacity would harm the market or not.
- piva00 10mo ago
  BTC mining on GPUs haven't been profitable for a long time, it's mostly ASICs, GPUs can be used for some other altcoins which makes the potential market for used previous generation GPUs even smaller.
  - blackenedgem 10mo ago
    That assumes you can add compute in a vacuum. If your altcoin receives 10x compute then it becomes 10x more expensive to mine. That only scales if the coin goes up in value due to the extra "interest". Which isn't impossible but there's a limit, and it's more often to happen to smaller coins.

rzerowan 10mo ago

I think its illustrative to consider the previous computation cycle ala Cryptomining. Which passed through a similar lifecycle with energy and GPU accelerators. The need for cheap wattage forced the operations to arbitrage the where location for the cheapest/reliable existing supply - there rarely was new buildout as the cost was to be reimbursed by the coins the miningpool recovered. For the chip situation caused the same apprecaition in GPU cards with periodic offloading of cards to the secondary market (after wear and tear) as newer/faster/more efficient cards came out until custom ASICs took over the heavy lifting, causing the GPU card market to pivot. Similarly in the short to moedium term the uptick of custo ASICs like with Google TPU will definately make a dent in bot cpex/opex and potentially also lead to a market with used GPUs as ASICs dominate. So for GPUs i can certainly see the 5 year horizon making a impact in investment decisions as ASICs proliferate.

coliveira 10mo ago

There is the opportunity cost of using a whole datacenter to house ancient chips, even if they're still running. You're thinking like a personal use chip which you can run as long as it is non-defective. But for datacenters it doesn't make sense to use the same chips for more than a few years and I think 5 years is already stretching their real shelf life.

lithos 10mo ago

It's worse than that in reality, AI chips are on a two year cadence for backwards compatibility (NVIDIA can basically guarantee it, and you probably won't be able to pay real AI devs enough to stick around to make hardware work arounds). So their accounting is optimistic.

Patrick_Devine 10mo ago
5 years is normal-ish depreciation time frame. I know they are gaming GPUs, but the RTX 3090 came out ~ 4.5 years before the RTX 5090. The 5090 has double the performance and 1/3 more memory. The 3090 is still a useful card even after 5 years.
- cubefox 10mo ago
  RTX 3090 MSRP: 1500 USD RTX 5090 MSRP: 2000 USD

rlupi 10mo ago

Do not forget that we're talking about supercomputers. Their interconnect makes machines not easily fungible, so even a low reduction in availability can have dramatic effects. Also, after the end of the product life, replacement parts may no longer be available. You need to get pretty creative with repair & refurbishment processes to counter these risks.

slashdave 10mo ago

5 years is long, actually. This is not a GPU thing. It's standard for server hardware.

bigwheels 10mo ago
Because usually it's more efficient for companies to retire the hardware and put in new stuff. Meanwhile, my 10-15 year old server hardware keeps chugging along just fine in the rack in my garage.
- XorNot 10mo ago
  Sample size of 1 though. It's like how I've had hard disks last a decade, but a 100 node Hadoop cluster had 3 die per week after a few years.
  - snuxoll 10mo ago
    Spinning rust and fans are the outliers when it comes to longevity in compute hardware. I’ve had to replace a disk or two in my rack at home, but at the end of the day the CPUs, RAM, NICs, etc. all continue to tick along just fine. When it comes to enterprise deployments, the lifecycle always revolves around price/performance. Why pay for old gear that sucks up power and runs 30% slower than the new hotness, after all! But, here we are, hitting limits of transistor density. There’s a reason I still can’t get 13th or 14th gen poweredge boxes for the price I paid for my 12th gen ones years ago.
- AdrianB1 10mo ago
  I thought the same until I calculated that newer hardware consumes a few times less energy and for something running 24x7 that adds up quite a bit (I live in Europe, energy is quite expensive). So my homelab equipment is just 5 years old and it will get replaced in 2-3 years with something even more power efficient.
  - tharkun__ 10mo ago
    Where in Europe? Asking coz I just did a quick comparison and it seems to depend but for comparison I have a really old AMD Athlon "e" processor (like literally September 2009 is when it came out according to some quick Google search, tho I probably bought it a few months later than that but still ...) that runs at ~45W TDP. In idle conditions, it typically consumes around 10 to 15 watts (internet wisdom, not kill-a-watt-wisdom). Some napkin math says it would cost me about 40 years worth of amortization to replace this at my current power rates for this system. So why would I replace it? And even with some EU countries' power rates we seem to be at 5-10 years amortization upon replacement. I've been running this motherboard, CPU + RAM combo for ~15 years now it seems, replacing only the hard drives every ~3 years. And the tower it's in is about 25 years old. Oh I forgot, I think I had to buy two new CR2032 batteries during those years (CMOS battery). Now granted, this processor can basically do "nothing" in comparison to a current system I might buy. But I also don't need more for what it does.

loeg 10mo ago

It's option #2. But 5 year deprecation is optimistic; 2-3 years is more realistic.

anticensor 10mo ago
Isn't the depreciation timespan imposed by the particular accounting standard?

deleted 10mo ago

[deleted]

more_corn 10mo ago

When you operate big data centers it makes sense to refresh your hardware every 5 years or so because that’s the point at which the refreshed hardware is enough better to be worth the effort and expense. You don’t HAVE to, but its more cost effective if you do. (Source, used to operate big data centers)

darth_avocado 10mo ago

Isn’t that what Michael Burry is complaining about? That five years is actually too generous when it comes to depreciation of these assets and that companies are being too relaxed with that estimate. The real depreciation is more like 2-3 years for these GPUs that cost tens of thousands of dollars a piece. https://x.com/michaeljburry/status/1987918650104283372 https://x.com/michaeljburry/status/1987918650104283372

duped 10mo ago
How different is this from rental car companies changing over their fleets? I don't know, this is a genuine question. The cars cost 3-4x as much and last about 2x as far as I know, and the secondary market is still alive.
- afavour 10mo ago
  Rental car companies aren’t offering rentals at deep discount to try to kickstart a market. It would be much less of a deal if these companies were profitable and could cover the costs of renewing hardware, like car rental companies can.
- chii 10mo ago
  > the secondary market is still alive. this is the crux. Will these data center cards, if a newer model came out with better efficiency, have a secondary market to sell to? It could be that second hand ai hardware going into consumers' hands is how they offload it without huge losses.
  - vesrah 10mo ago
    The GPUs going into data centers aren't the kind that can just be reused by putting them into a consumer PC and playing some video games, most don't even have video output ports and put out FPS similar to cheap integrated GPUs.
    - geerlingguy 10mo ago

marcosdumay 10mo ago

Historically, GPUs have improved in efficiency fast enough that people retired their hardware in way less than 5 years. Also, historically the top of the line fabs were focused on CPUs, not GPUs. That has not been true for a generation, so it's not really clear if the depreciation speed will be maintained.

chii 10mo ago
> that people retired their hardware in way less than 5 years. those people are end-consumers (like gamers), and only recently, bitcoin miners. Gamers don't care for "profit and loss" - they want performance. Bitcoin miners do need to switch if they want to keep up. But will an AI data center do the same?
- TingPing 10mo ago
  The rate of change is equal for all groups. The gaming market can be the most conservative since it’s just luxury.
- thinkmassive 10mo ago
  Mining bitcoin with a GPU hasn't been profitable in over a decade.
cubefox 10mo ago
> Historically, GPUs have improved in efficiency fast enough that people retired their hardware in way less than 5 years. This was a time when chip transistor cost was decreasing rapidly. A few years earlier even RAM cost was decreasing quickly. But these times are over now. For example, the PlayStation 5 (where the GPU is the main cost), which launched five years ago, even increased in price! This is historically unprecedented. Most price/performance progress is nowadays made via better GPU architecture instead, but these architectures are already pretty mature, so the room for improvement is limited. Given that the price per transistor (which TSMC & Co are charging) has decreased ever more slowly in recent years, I assume it will eventually come almost to a halt. By the way, this is strictly speaking compatible with Moore's law, as it is only about transistors per chip area, not price. Of course the price per chip area was historically approximately constant, which meant exponentially increasing transistor density implied exponentially decreasing transistor price.

protocolture 10mo ago

Actually my biggest issue here is that, assuming it hasnt paid off, you dont just convert to regular data center usage. Honestly if we see a massive drop in DC costs because the AI bubble bursts I will be stoked.

m101 10mo ago

Given power and price constraints, it's not that you cannot run them in 5 years time it's that you don't want to run them in 5 years time and neither will anyone else that doesn't have free power.

ExoticPearTree 10mo ago

> AI accelerator cards all start dying around the 5 year mark, More likely the technology will be much better in 5 years in terms of hardware that it is (very) uneconomical to run anything on old hardware.

araes 10mo ago

General question to people who might actually know. Is there anywhere that does anything like Backblaze's Hard Drive Failure Rates [1] for GPU Failure Rates in environments like data centers, high-performance computing, super-computers, mainframes? [1] https://www.backblaze.com/blog/backblaze-drive-stats-for-q3-2025/ https://www.backblaze.com/blog/backblaze-drive-stats-for-q3-... The best that came back on a search was a semi-modern article from 2023 [2] that appears to be a one-off and mostly related to consumer facing GPU purchases, rather than bulk data center, constant usage conditions. It's just difficult to really believe some of these kinds of hardware deprecation numbers since there appears to be so little info other than guesstimates. [2] https://www.tweaktown.com/news/93052/heres-look-at-gpu-failure-rates-and-warranty-claim-times-for-all-major-brands/index.html https://www.tweaktown.com/news/93052/heres-look-at-gpu-failu... Found an Arxiv paper on continued checking that's from UIUC UrbanaIL about a 1,056 A100 and H100 GPU system. [3] However, the paper is primarily about memory issues and per/job downtime that causes task failures and work loss. GPU Resilience is discussed, it's just mostly from the perspective of short-term robustness in the face of propagating memory corruption issues and error correction, rather than multi-year, 100% usage GPU burnout rates. [3] https://arxiv.org/html/2503.11901v3 https://arxiv.org/html/2503.11901v3 Any info on the longer term burnout / failure rates for GPUs similar to Backblaze? Edit: This article [4] claims it's 0.1-2% failure rate per year (0.8% (estimated)) with no real info about where the data came from (cites "industry reports and data center statistics"), and then claims they often last 3-5 years on average. [4] https://cyfuture.cloud/kb/gpu/what-is-the-failure-rate-of-the-h100-gpu https://cyfuture.cloud/kb/gpu/what-is-the-failure-rate-of-th...