7 ms·
What happens to older datacenter GPUs? Do they have a second life somewhere outside of datacenters? I could see Nvidia adding terms of sale requiring disposal
by sowbug 7mo ago
What happens to older datacenter GPUs? Do they have a second life somewhere outside of datacenters?
I could see Nvidia adding terms of sale requiring disposal rather than resale.
- AlotOfReading 7mo agoYou send them back to Nvidia or a third party e-waste recycler at end of life. Sometimes they're resold and reused, but my understanding is that most are eventually processed for materials.
- zasz 7mo agoIt seems like GPUs with a high utilization rate (60%+) degrade after 1-3 years: https://www.tomshardware.com/pc-components/gpus/datacenter-gpu-service-life-can-be-surprisingly-short-only-one-to-three-years-is-expected-according-to-unnamed-google-architect https://www.tomshardware.com/pc-components/gpus/datacenter-g... Would be interested to know if others have takes on this.
- latchkey 7mo agoI previously ran 150,000 AMD gpus in all conditions at 100% utilization for years. I currently have a multi-million $ cluster of enterprise AMD GPUs. A couple real world points: 1. They generally don't just fail. More likely a repairable component on a board fails and you can send it out to be repaired. 2. For my current stuff, I have a 3 year pro support contract that can be extended. Anything happens, Dell goes and fixes it. We also haven't had someone in our cage at the DC in over 6 months now.
- rurban 7mo agoI have to maintain our GPU's. Generally the worst parts are the watercooling pressure, the HVAC, and the power. I can run it stable only at 300W per CPU, the normal max is 310W. Now with throttling to 300 it's stable for a year, before it burned two mainboards already, with lots of downtimes.
- latchkey 7mo agoMy experience is that power problems stem from not having good power and/or poor airflow. I'm convinced that this is why we haven't had any issues in our current location. Zero outside air, zero dust, insanely well built zero expense spared airflow and power supply / management.
- MisterTea 7mo agoIt's likely the GPU boards are designed for water cooled data center racks and might not fit in a regular PC case. It's also possible the PCB the GPU's are mounted to might not be standard PCIe cards that fit into an ATX case. I bought a used NEC SX Aurora TSUBASA (PCIe x16 board that looks like a GPU board) and realized it has no fans. The server case it is designed to fit into is pressurized by fans forcing air through eight cards on a special 4 + 4 slot motherboard. I have to stack and mount three 40mm fans on the back.
- fc417fc802 7mo agoHow is software compatibility with the tsubasa BTW? Is it worthwhile to play with?
- MisterTea 7mo agoHonestly, I bought it more for collection than actually using. If I ever do anything with it, it will be from Plan 9. However, searching around there is plenty of documentation on the architecture in addition to a github repo: https://github.com/veos-sxarr-NEC https://github.com/veos-sxarr-NEC
- u1hcw9nx 7mo agoThey are build to physically last 5-7 years in 24/7 datacenter use, but they have effective lifetime just 3-4 years, then their value has deprecated and electricity and infrastructure cost dominates. Meta did a benchmark where 9% of the chips failed every year, 'infant mortality' is much higher in the first 3 months of use.
- fc417fc802 7mo ago9% is an absurd failure rate for solid state electronics. Particularly considering the profit margins. I assume it's related to the power densities involved. Would you happen to recall the source?
- wmf 7mo agoIt's pretty bad: https://www.datacenterdynamics.com/en/news/meta-report-details-hundreds-of-gpu-and-hbm3-related-interruptions-to-llama-3-training-run/ https://www.datacenterdynamics.com/en/news/meta-report-detai... Jensen said they added a lot of RAS in Blackwell which kind of admits Hopper wasn't reliable enough.
- jdiez17 7mo agoI've written about this elsewhere but I predict there will be a significant secondary market for repurposing parts of datacenter GPUs (for example, RAM chips) by desoldering them and soldering them onto new PCBs that fit PC/consumer use cases.
- Avicebron 7mo agoI wish there to be an active market like what Gamer's Nexus covered in China: https://www.youtube.com/watch?v=1H3xQaf7BFI&t=1577s https://www.youtube.com/watch?v=1H3xQaf7BFI&t=1577s in the States.
- Gigachad 7mo agoIt's all about the cost of labor. In the US you could not find someone capable of soldering BGA chips for a price that makes sense doing.
- wtallis 7mo agoThat might work to some extent with the DDR5 DIMMs connected to the CPUs, but is thoroughly impossible for the HBM DRAM stacks packaged with the GPUs.
- paxys 7mo agoPlenty of enterprise server hardware (racks, servers, RAM, disks) does have an active secondhand market after 3-5 years of use, but I think GPUs are too specialized for it to be viable. I doubt anyone has the setup to run a H200 in their home rig. I also don't think companies are going to have mandatory replacement cycles for GPU hardware the same way they do for everything else, because: 1. It is an order or magnitude (or more) more expensive. 2. It isn't clear whether Moore's law will apply to the AI GPU space the same way it has for everything else. Unless Nvidia can launch a new chip every 2-3 years with massively improved performance-per-watt at a lower price no one is going to rush to recycle the old one.
- tryauuum 7mo agoyou can absolutely run e.g. datacenter-level A100 at home, there are adapters from the SXM to the PCIe socket. Haven't seen people running SXM versions of H100s this way but this could be due to the price factor only
- baby_souffle 7mo agoWell technically true, I would wager that the home lab is going to require increasingly distinct and unusual adaptations to retrofit the hardware to home use. New stuff is all liquid cooled by default and that's a paradigm shift for your average home lab. I'm less aware of exactly what's happening on the power side of things but I think some of the architectures are now moving to relatively high voltage DC throughout and then down converting it to low voltage right before it's used. So not exactly just plug-and-play with your average nema15 outlet.
- tryauuum 7mo agoyou can still buy air-cooled versions of b300, seems nice to have for a hobbyist
- thefounder 7mo agoWell by the time the become obsolete you can run that computing on a Mac with no special cooling so I really doubt they will be of any use. Maybe in some parts of the world where electricity is cheap. If someone wants to really find out perhaps watching the crypto ASICs stories could help.
- observationist 7mo agoDepending on the elemental composition, it could definitely be worthwhile to recycle wherever scale is practical. For giant datacenters and companies using hundreds of thousands or millions of gpus, that adds up to a lot of gold and other valuable elements. In order to take advantage of that, someone needs to be positioned to process all that material economically, and to make the logistics achievable by the big players. If it costs Facebook $10million to store and transport phased out gpus vs just sending them to a landfil, they're not going to do it. If they get $100k for recycling - probably not going to do it. If they pocket $5 million, they will definitely contract that out, especially if it costs $50 million to build out the infrastructure to handle it. Probably a good company idea - transport, disposal, refurbishment of out of cycle GPUs and datacenter assets, creating a massive recycling pipeline for recapturing all the valuable elements is a pretty good niche.
- fc417fc802 7mo agoWe already have that. It's called ebay.
- throwup238 7mo agoLast I checked AWS is still offering g4dn instances that run on NVIDIA T4 GPUs, which were first released in 2018. I think most people underestimate how long superscalers can keep these things running profitably after they depreciate, and you probably don’t want anything they throw away. My last employer is still running a bunch of otherwise discontinued g3 instances with 2015 era GPUs.
- deleted 7mo ago[deleted]
- h4kunamata 7mo agoBin!! Why would them sell it cheaper to the 2nd market?? It will hurt the sales of new ones. This is the way even with food, let alone technology. Don't expect to buy cheaper 2nd GPU any century soon.
- Gigachad 7mo agoThe data center owners aren't the ones selling new GPUs.
- latchkey 7mo agoThis article misses the point that people are still actively running older compute.
- exikyut 7mo agoNo affiliation (I wish), but: https://gptshop.ai https://gptshop.ai This site apparently sources ex-enterprise(-only) systems and puts them into desktop style enclosures.
- mbesto 7mo agoWe literally do not know because it hasn't been 5 years yet...
- raw_anon_1111 7mo agoThey are thrown away. The average lifespan of a data center GPU is 3 years. Meaning they fail.
- randomgermanguy 7mo agoYou can already buy A100/H100s on eBay. While it might not ever be economical to run these at home (cost of electricity), but it's plenty fun.