3 ms·

True, they burn through GPUs. However, I wonder what the actual curve looks like... what fraction of total GPU capacity is getting maxed out, to the 3 year "bur

True, they burn through GPUs. However, I wonder what the actual curve looks like... what fraction of total GPU capacity is getting maxed out, to the 3 year "burned to crisp" threshold. Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition). Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"? Thank you for that GPU lifespan explanation... taught me a thing or two today.

throwawayffffas 10mo ago
> Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"? From what a hear it's a mix, of completely dead to degraded performance. > Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition). I have heard over 70% quoted used for training, and like 5% for general purpose inference and the rest for code generation. But don't quote me on these numbers, I don't recall the sources. One has to assume that some capacity is also used for traditional high performance computing.
- adityaathalye 10mo ago
  Educated guesstimates are worth a lot. Thank you for the stats!
- adityaathalye 10mo ago
  Still, I do wonder about the GPU manufacturing capacity upstream of datacenters, even though it grows relatively slowly. I suppose NVIDIA's order book is booked solid a few years out. However, capacity that they add can't just be repurposed / retooled for other use cases. What could substitute LLM demand, if the LLM/AI business contracts rapidly?
- adityaathalye 10mo ago
  Another factor... They built it, and we didn't come. Groq investor sounds alarm on data centers (axios.com) 32 points by giuliomagnifico 2 hours ago | 21 comments https://news.ycombinator.com/item?id=46432791 https://news.ycombinator.com/item?id=46432791 > Venture capitalist Alex Davis is "deeply concerned" that too many data centers are being built without guaranteed tenants, according to a letter being sent this morning to his investors. > [snip] > What he's saying: "The 'build it and they will come' strategy is a trap. If you are a hyperscaler, you will own your own data centers. We foresee a significant financing crisis in 2027–2028 for speculative landlords."

throwawayffffas 10mo ago

> Even after that, what does a "burned out" GPU look like. Is it a total bust, or is still usable at... say, 25% capacity for "consumer type applications"? From what a hear it's a mix, of completely dead to degraded performance. > Training is harsher than inference is harsher than speculative capacity-hoarding (because, competition). I have heard over 70% quoted used for training, and like 5% for general purpose inference and the rest for code generation. But don't quote me on these numbers, I don't recall the sources. One has to assume that some capacity is also used for traditional high performance computing.

adityaathalye 10mo ago
Educated guesstimates are worth a lot. Thank you for the stats!
adityaathalye 10mo ago
Still, I do wonder about the GPU manufacturing capacity upstream of datacenters, even though it grows relatively slowly. I suppose NVIDIA's order book is booked solid a few years out. However, capacity that they add can't just be repurposed / retooled for other use cases. What could substitute LLM demand, if the LLM/AI business contracts rapidly?
adityaathalye 10mo ago
Another factor... They built it, and we didn't come. Groq investor sounds alarm on data centers (axios.com) 32 points by giuliomagnifico 2 hours ago | 21 comments https://news.ycombinator.com/item?id=46432791 https://news.ycombinator.com/item?id=46432791 > Venture capitalist Alex Davis is "deeply concerned" that too many data centers are being built without guaranteed tenants, according to a letter being sent this morning to his investors. > [snip] > What he's saying: "The 'build it and they will come' strategy is a trap. If you are a hyperscaler, you will own your own data centers. We foresee a significant financing crisis in 2027–2028 for speculative landlords."