4 ms·
Why would they be shot? Unlike the consumer cards that are basically factory overclocked to look good on benchmarks, the datacenter GPUs are designed to run at
by christina97 2mo ago
Why would they be shot? Unlike the consumer cards that are basically factory overclocked to look good on benchmarks, the datacenter GPUs are designed to run at full tilt 24/7 and survive for years.
- Chaosvex 2mo agoGiven the decades of consumer and enterprise GPUs often being identical or near identical hardware, it'd be interesting to see if there's any evidence of this actually being true.
- eru 2mo agoBinning can take in a supposedly uniform stream of chips and produce different tiers on the output.
- bigbuppo 2mo agoAccording to the article these cards have a 9% annual failure rate.
- htrp 2mo ago>The number traces to Meta’s Llama 3 technical report, which documented 419 unforeseen disruptions across 16,384 H100s over 54 days of training, of which 148 were GPU failures and 72 were HBM3 memory failures. From an annualized number on the llama 3 training report. would be interesting to see if we have a better idea given that we're already on rubin.
- christina97 2mo agoIt depends entirely on the time-to-failure distribution though whether used cards are a good deal or not. Often this kind of hardware has a bathtub shaped hazard rate, actually getting burned in cards may mean you get the weeded out solid specimens, and forgo the lemons.
- lmm 2mo agoMost hardware has an exponential (memoryless) failure distribution in practice. The bathtub curve is a myth.
- fc417fc802 2mo agoA bathtub curve is also memoryless though, and the back portion can be exponential. Consider two sub-populations of the same hardware. The first is small, evaded QA, and will fail early. Failures follow an exponential fall off. The second is much larger and failures rise exponentially with age. Also consider mixing in "was dropped during shipping" or "was stored improperly".
- eru 2mo agoHow is a bathtub curve memoryless?
- fc417fc802 2mo agoMy mistake, it seems I misunderstood what OP meant by that. Ignore that semantic detail and my position still stands. You've got a mix of subpopulations and many of the failure modes are definitely time dependent in some manner at least in the real world.
- eru 2mo agoNo worries. I just thought I was misunderstanding something.
- Xalutiono 2mo agoWhere is this exponential thing coming from? Why would hardware failure raise exponential with linear scale? Doesn't make sense
- eru 2mo ago> Where is this exponential thing coming from? That's what you get from a constant failure rate per unit of time. It's saying that hardware fails in the same way that Carbon-14 dating works.
- onlyrealcuzzo 2mo agoA 9% (static) failure rate, implies an average lifespan of 11 years...