3 ms·

how long do we expect these things to hold value?

by jes5199 3y ago

wmf 3y ago
Four years, maybe more. People aren't throwing away their A100s yet.
- belval 3y ago
  Even the larger V100 instances are still heavily used at AWS/GCP/Azure. The reality is that demand does not disappear if there are no H100/A100, it just finds another way. There's also the fact that if you are not training LLM, you can get a better deal using some older hardware.
H8crilA 3y ago
Possibly longer than some stocks against which rich people take loans. Much stupider things have happened in the credit market. Also, the Reuters article says that they have a depreciation schedule built into the contact. Compute hardware can depreciate pretty quickly, if I remember correctly major tech companies have around 3 years planned for the depreciation.
- dmoy 3y ago
  > if I remember correctly major tech companies have around 3 years planned for the depreciation. Historically yea, though now I think it's being stretched to 5+ years as they see hardware last longer in production (but you're atiop correct, it's very quickly relative to a lot of other loan collateral, still a short term deal even if it's 7 years)
stefan_ 3y ago
Once they stop working for training they can still do inference for a long time.
TechBro8615 3y ago
I expect they'll lose value in the step-function shape that's typical of technology, where each new generation decreases the value of the previous one. But that's fine, as long as each chip generates more revenue over its lifetime than it cost. And the lifetime can extend through multiple generations, as long as each successive generation is only a marginal improvement over its predecessor. Personally I think the bigger risk is software innovations making CPU training (and/or inference) sufficiently viable that it's cheaper to train models on a commodity CPU cluster than on some proportionally expensive GPU cluster. I don't know enough about the space to say whether that's likely, but it seems like a low risk, since pretty much any parallel algorithm will always be faster on GPU than CPU - it's just a question of the marginal benefits and cost (e.g. maybe it takes more CPU to train same model in same time, but cost of CPU is so much lower that it's worth buying more of them).
- jes5199 3y ago
  if it follows the trajectory of cryptocurrency, then the opposite will happen - even more expensive custom chips will replace GPUs
Ekaros 3y ago
What value is good question. But some value as long as computing power can be sold above running costs(power, cooling, etc.). Good question is when the newer model is so much more efficient it makes sense to replace them.

how long do we expect these things to hold value?

wmf 3y ago

Four years, maybe more. People aren't throwing away their A100s yet.

belval 3y ago
Even the larger V100 instances are still heavily used at AWS/GCP/Azure. The reality is that demand does not disappear if there are no H100/A100, it just finds another way. There's also the fact that if you are not training LLM, you can get a better deal using some older hardware.

H8crilA 3y ago

Possibly longer than some stocks against which rich people take loans. Much stupider things have happened in the credit market. Also, the Reuters article says that they have a depreciation schedule built into the contact. Compute hardware can depreciate pretty quickly, if I remember correctly major tech companies have around 3 years planned for the depreciation.

dmoy 3y ago
> if I remember correctly major tech companies have around 3 years planned for the depreciation. Historically yea, though now I think it's being stretched to 5+ years as they see hardware last longer in production (but you're atiop correct, it's very quickly relative to a lot of other loan collateral, still a short term deal even if it's 7 years)

stefan_ 3y ago

Once they stop working for training they can still do inference for a long time.

TechBro8615 3y ago

I expect they'll lose value in the step-function shape that's typical of technology, where each new generation decreases the value of the previous one. But that's fine, as long as each chip generates more revenue over its lifetime than it cost. And the lifetime can extend through multiple generations, as long as each successive generation is only a marginal improvement over its predecessor. Personally I think the bigger risk is software innovations making CPU training (and/or inference) sufficiently viable that it's cheaper to train models on a commodity CPU cluster than on some proportionally expensive GPU cluster. I don't know enough about the space to say whether that's likely, but it seems like a low risk, since pretty much any parallel algorithm will always be faster on GPU than CPU - it's just a question of the marginal benefits and cost (e.g. maybe it takes more CPU to train same model in same time, but cost of CPU is so much lower that it's worth buying more of them).

jes5199 3y ago
if it follows the trajectory of cryptocurrency, then the opposite will happen - even more expensive custom chips will replace GPUs

Ekaros 3y ago

What value is good question. But some value as long as computing power can be sold above running costs(power, cooling, etc.). Good question is when the newer model is so much more efficient it makes sense to replace them.