5 ms·
> GPUs that have a 1-3 year lifespan In 10 years GPUs will have a lifespan for 5-7 years. The rate of improvement on this front has been slowing down faster t
by bitmasher9 1y ago
> GPUs that have a 1-3 year lifespan
In 10 years GPUs will have a lifespan for 5-7 years. The rate of improvement on this front has been slowing down faster then CPU.
- enord 1y agoWait… are you betting on exponential or logarithmic returns?
- stanac 1y agoThere was a video on YT (gamers nexus I think) and an excel sheet comparing jumps in performance gains between each new nvidia generation. They are becoming smaller and smaller, probably now driven by AI boom where most of the silicon is used for data centers. Regardless of that, I have a feeling we are approaching the ceiling of chip performance. Just comparing PS3 and PS4 and then PS4 with PS5, performance jump is smaller and size of the hardware has become enormous and GPUs are more and more power hungry. If generational jumps were good enough we wouldn't need more power and cooling and big boxes for desktop PCs that can hold long graphic cards.
- bobthepanda 1y agoWe also have hit ceilings of performance demand. As an example, 8K TVs never really got off the ground because your average consumer could give a hoot. Vision Pro is a flop because AR/VR are super niche. Crypto is more gambling than asset class. Etc. What is interesting is that it seems like the ever larger sums of money sloshing around are resulting in bigger, faster hype cycles. We are already seeing some companies face issues after blowback from adopting AI too fast.
- jay_kyburz 1y agoHaving a local AI might kick start consumer interest in hardware again. I haven't thought about buying a beefy PC in 10 years, but I was wondering how much I would have to spend to run a cutting edge LLM here at home. (It might be too expensive to pay for LLM subscriptions when every device in your house is "thinking" all day long. A 3-5k Computer for a local llm might pay itself off after a year or two. )
- bobthepanda 1y agoI don’t think this is super likely. The crypto boom showed how quickly a GPU, power-intensive computing demand mostly meant putting the chips outside the accessible price range for the consumer, and the market never really quite recovered.
- bitmasher9 1y agoIt’s funny, but I’m actually very interested in a 8k display with the right parameters. I like to use large displays for monitors, and the pixel density of 4k at a large display is surprisingly low.
- imtringued 1y agoMost of the performance gains also came from smaller datatypes such as fp4. Going from bfloat16/fp16 to fp8 is easy. fp4 is challenging but possible. fp2 can't exist independently, because it is just the sign bit plus a single bit. The next frontier would be training directly with block floating point, where you have a shared exponent plus the two remaining bits. It's getting tight. Maybe it is possible to have mini LoRA blocks where an n times n block is approximated by the outer product of two n sized vectors. For n = 4 the savings would be 50% less FLOPs and for n=8 the savings would be 75% less FLOPs.
- nemomarx 1y agoAfter heavy use, though? I don't think they mean aging out of being cutting edge but actually starting to fail sooner after being used in DCs.
- trenchpilgrim 1y agoThey're mostly solid state parts. The parts that do wear out like fans are easily replaced by hobbyists.
- nemomarx 1y agoI swear during the earlier waves of bitcoin mining (before good ASICs came out) people ran them overclocked and did cause damage. Used GPUs were pretty unreliable for a while there.
- trenchpilgrim 1y ago1. Miners were clocking them beyond the factory approved speeds - something not needed for AI, where the bottleneck is usually VRAM, not clock speed. 2. While comprehensive studies were never done, some tech channels did some testing and found used GPUs to be generally reliable or easily repairable, when scamming was excluded. https://youtu.be/UFytB3bb1P8 https://youtu.be/UFytB3bb1P8
- pclmulqdq 1y agoThis is not correct. Solid-state parts wear out like mechanical parts, especially when you run them hot. The mechanism of this wear-out comes from things like electromigration of materials and high-energy electrons literally knocking atoms out of place.
- trenchpilgrim 1y agoThose parts take decades to wear out, excepting a manufacturing defect.
- bee_rider 1y agoThe full quote is: > Most of the money is being spent on incredibly expensive GPUs that have a 1-3 year lifespan due to becoming obsolete quickly and wearing out under constant, high-intensity use. So it isn’t entirely tied to the rate of obsolescence, these things apparently get worn down from the workloads. In terms of performance improvement, it is slightly complicated, right? It turns out that it was possible to do ML training on existing GPGPU. Then there was spurt of improvement as they go after the low-hanging fruit for that application… If we’re talking about what we might be left with after the bubble pops, the rate of obsolescence doesn’t seem that relevant anyway. The chips as they are after the pop will be usable for the next thing or not, it is hard to guess.
- bitmasher9 1y agoThe failure rate of GPUs in professional datacenter environments is over estimated by the general public because of the large number of overlocked and undercooled cards used for GPU mining that hit eBay.
- bradleyjg 1y agoWhat’s the strategy when one does die? It’s just left in place until it’s worth it to pull the entire rack?
- lossolo 1y agoI'm still waiting to get a used Nvidia A100 80 GB (released in 2020) for well under $10,000.
- laluser 1y agoThe reason for not keeping them too much longer than a few years is that at the end of that timespan you can purchase GPUs with > 2x performance, but for the same amount of power. At some point, even though the fleet has been depreciated, they become too expensive to operate vs. what is on the market.
- mike_hearn 1y agoThere's some telephone game being played here. The three year number was a surprisingly low figure sourced to some anonymous Google engineer. Most people were assuming at least 5 years and maybe more. BUT, Google then went on record to deny that the three year figure was accurate. They could have just ignored it, so it seems likely that three years is too low. Now I read 1-3 years? Where did one year come from? GPU lifespan is I suspect also affected by whether it's used for training or inference. Inference loads can be made very smooth and don't experience the kind of massive power drops and spikes that training can generate.
- trenchpilgrim 1y ago> Where did one year come from? Perhaps the author confused "new GPU comes out" with "old GPU is obsolete and needs replacement"
- dingaling 1y ago> Now I read 1-3 years? Where did one year come from? I believe that lifespan range came from cryptocurrency mining experience, running the GPU at 100% load constantly until components failed.