4 ms·

It's good but I have a feeling anything you find on aftermarket will be used and abused. These chips are designed to handle high thermal load, but if it's been

It's good but I have a feeling anything you find on aftermarket will be used and abused. These chips are designed to handle high thermal load, but if it's been in a DC or server room, it may impact its longevity.

wmf 3y ago
"Abused" chips are mostly a myth and Milan is not that old so these chips should have plenty of life left in them.
- londons_explore 3y ago
  Agreed - with CPU's, if it's working the day you buy it, it will most likely still be working in 10 years with typical desktop use cases, no matter the past life it had.
  - paulmd 3y ago
    that's not true at all, XMP is very much within the wheelhouse of "typical desktop use-cases" and can absolutely damage a CPU from electromigration within a matter of years. (or rather, the overclocked/out-of-spec memory controller usually requires the board to kick up the CPU memory controller (VCCSA/VSOC) voltages, and that's what does the damage.) https://youtu.be/HLNk0NNQQ8s?t=510 https://youtu.be/HLNk0NNQQ8s?t=510 https://www.youtube.com/watch?v=uMHUz16MuYA https://www.youtube.com/watch?v=uMHUz16MuYA People have generally convinced themselves that it's safe but, the rate of CPU failures is incredibly high among enthusiasts compared to the general enterprise fleet and the reason is XMP. This has been "out there" for a long time if you know to look for it. But, enthusiasts fall into that classic "can't make a man understand when his salary depends on not understanding it" thing - everyone has every reason to convince themselves it doesn't, because it would affect "their lifesyle". But electromigration exists. Electromigration affects parts on consumer-relevant timescales, if you overclock. Electromigration particularly affects memory controllers/system fabric nowadays. And yes, you can absolutely burn out a memory controller with just XMP (and the aggressive CPU voltages it applies) and this is not new or a secret. And the problem of electromigration/lifespan is accelerating as the operating range becomes narrower on newer nodes etc. https://semiengineering.com/aging-problems-at-5nm-and-below/ https://semiengineering.com/aging-problems-at-5nm-and-below/ https://semiengineering.com/3d-ic-reliability-degrades-with-increasing-temperature/ https://semiengineering.com/3d-ic-reliability-degrades-with-... https://semiengineering.com/on-chip-power-distribution-modeling-becomes-essential-below-7nm/ https://semiengineering.com/on-chip-power-distribution-model... Similarly: "24/7 safe" fabric overclocks are really not. Not on the order of years. Everyone is already incentivized to push the "official" limit as much as is safe/reliable - AMD/Intel know about the impact on benchmark scores too, they want their parts to look as good as they can. There is no "safe" increase above the official spec, not really. The unique thing about Asus wasn't that they killed a chip from XMP - it's that they put so much voltage into it that it went into immediate runaway and popped instantly, explosively, and visibly. And it's not surprising it was Asus (those giant memory QVLs come from just throwing voltage at the problem) but low-key everyone has been applying at least some additional voltage for a long time. Eventually it kills chips. It's overclocking/out-of-spec and very deliberately and specifically excluded from the warranty (AMD GD-106/GD-112). It's completely understandable why AMD wants to make some fuses/degradation canary cells to monitor whether the CPU has operated out-of-spec as far as warranty coverage. This is a serious failure mode that probably causes a large % of overall/total "CPU premature failure" warranty returns etc. And essentially it continues to get worse on every new node and with every new DDR standard, and with the increased thermals that currently are characteristic of stacked solutions etc. https://www.amd.com/en/legal/claims/gaming-details.html https://www.amd.com/en/legal/claims/gaming-details.html https://www.extremetech.com/computing/amds-new-threadripper-chips-have-a-hidden-fuse-that-blows-when-overclocking https://www.extremetech.com/computing/amds-new-threadripper-...
    - wmf 3y ago
      Fortunately servers don't have XMP.
      - paulmd 3y ago
        true, I am just pushing back on the idea that "abusing a chip is mostly a myth" and "if a CPU is working on the day you buy it, it's fine for desktop use-cases". For server parts that can't be OC'd - true, I guess. For regular CPUs? Absolutely not true, enthusiasts abuse the shit out of them and even if you do no further damage yourself, the degradation can continue over time etc as parts of the circuit just become critically unstable from small routine usage etc. (people treat their CPUs like gamer piss-jugs, big deferred cost tomorrow for a small benefit today.) But yes - ironically this means surplus server CPUs are actually way more reliable than used enthusiast CPUs. In some cases they are drop-in compatible in the consumer platform (although not so much in newer stuff), and the server stuff got the better bins in the first place, and it's cheaper (because they sold a lot more units), and also hasn't been abused by an enthusiast for 5 years etc. If you are on a platform like Z97 or X99 that supports the Xeon chips, the server chips are a complete no-brainer. And some xeons are even multiplier unlocked etc - used to be a thing, back in the day. ("server bins are binned for leakage and don't compete with gaming cpus" is another myth that is not really true except for XOC binning - server CPUs are better binned than enthusiast ones for ambient use-cases.)
    - londons_explore 3y ago
      But are there many cases of a CPU being overclocked (& overheated & overvolted), then later not being overclocked (and working fine), but then failing shortly afterwards? Yes, I understand it is theoretically possible. But I think it is just super rare - I've never heard of a single case.
      - paulmd 3y ago
        Was looking through semiengineering for some more sources and some of them address it. Aging (I should probably say "aging" instead of electromigration, I'm not referring to just one effect here) is such a problem below 10nm that literally even just idling the chip wears it noticeably... and of course that leads to uneven wear on the cores too, etc. It's not just electromigration etc anymore. The reason you don't notice this is that the chip is engineered so you don't notice it. The boost clocks will slow down over time, the voltage applied will increase over time (dynamically controlled by measuring the degradation of the canary cells). Unless the chip catastrophically fails, you probably won't notice the slowdown etc, in scenarios that would have resulted in chip failure 20 years ago. The chips are simply designed to tolerate that - because they have to be, even during normal operation (!). The lifespan of a 5nm chip is not "infinite if treated properly" anymore. It is actually finite in terms of even idle power-on hours etc let alone load hours. A large number of power-on hours, and deliberately engineered to be large and to tolerate the damage gracefully, but people's mental models of "power-on hours doesn't hurt the chip" is fundamentally not correct anymore. Miners running lots of hours on that 7nm GPU etc is not "just fine" etc. Also, once you get it beyond the "damage point", especially in analog stuff you have simply changed the characteristics of the circuit. If the amplifier's bias circuit leads some other part of the circuit to be hit with a higher gain, that can continue damaging it even if you stop further damaging the bias circuit etc. Memory and PCIe are analog circuits here. https://semiengineering.com/design-for-reliability-2/ https://semiengineering.com/design-for-reliability-2/ > Digital and analog will be affected differently, as will devices subject to frequent change — and in some cases, infrequent change. “Any place where there’s a lot of activity will be more sensitive to device aging,” says Art Schaldenbrand, senior product manager at Cadence. “For devices, you can look at the clock tree and look at what is happening. Digital designs are sensitive to delay changes. The other place where this becomes a challenge is within analog designs. An example would be in a bias tree. With the bias transistors moving and aging, it can potentially accelerate the aging of other devices in the bias network. There’s always going to be some different elements in the design, and you have to look at them a little bit differently to be able to analyze the reliability.” [ ...] > But you have to be careful to consider all of the important areas. “There is a phenomenon called non-conductive stress,” says Cadence’s Schaldenbrand. “Consider a device such as a watch dog or monitor. It will be sitting idle, potentially for years, and you want it to spring into action if there’s some sort of condition that occurs. Even those circuits, that you think are you’re just sitting there doing nothing, are being stressed. They can age and potentially fail due to the aging that occurs while they’re sitting idle.” https://semiengineering.com/aging-not-always-a-bad-thing/ https://semiengineering.com/aging-not-always-a-bad-thing/ > This impacts the gate because of the natural behavior of the transistors, Elhak explained. “In the transistor you have a gate, which has an electric field that is supposed to control the current that is flowing between the drain and the source but there are random events. This electric field causes some of those carriers, instead of flowing between the gate and the source, to go and get injected into the gate. As more carriers get injected over time, the electrical properties of the gate start to differ because it’s not supposed to have those carriers in it. That changes the properties of the whole device because now the gate is supposed to control that electric field it is now made of a different material.” > The second mechanism that causes aging is called the bias temperature instability (BTI), which happens when there is a constant bias on the device meaning there is current flowing. Here, instead of being driven by electric field, it is driven here by bias and temperature. Also, charges start to get trapped into the gate and as this happens, the properties of the gate change and again it impacts the threshold voltage and the carrier mobility in that channel. “If you change the threshold voltage and if you change the mobility, then you have a different transistor,” he asserted. https://semiengineering.com/adding-aging-to-variability/ https://semiengineering.com/adding-aging-to-variability/ https://semiengineering.com/uneven-circuit-aging-becoming-a-bigger-problem/ https://semiengineering.com/uneven-circuit-aging-becoming-a-... https://semiengineering.com/transistor-aging-intensifies-10nm/ https://semiengineering.com/transistor-aging-intensifies-10n... https://semiengineering.com/chip-aging-becomes-design-problem/ https://semiengineering.com/chip-aging-becomes-design-proble... https://semiengineering.com/minimizing-chip-aging-effects/ https://semiengineering.com/minimizing-chip-aging-effects/ https://semiengineering.com/dealing-with-device-aging-at-advanced-nodes/ https://semiengineering.com/dealing-with-device-aging-at-adv... https://semiengineering.com/24142954-2/ https://semiengineering.com/24142954-2/ https://semiengineering.com/chip-aging-accelerates/ https://semiengineering.com/chip-aging-accelerates/
- irusensei 3y ago
  It kinda feels like the mining GPU being bad myth when in fact miners nursed those GPUs like babies because their income depended on those devices.
  - namibj 3y ago
    Tbf the fans may be broken on them, or at least not far from being broken. I.e., plan to waterblock it.
  - epolanski 3y ago
    People forget there's decade old servers out there working 24/7. Anyway there's a Microsoft research paper on silicon which essentially says that failure rates of CPUs increase by mostly two factors: - cycles. The more calculations the higher the rate of failure - temperature/power. I will let you guess it by yourself. Even minor slips overvoltages and overclocks can enhance failure rates by magnitude of orders. Getting back to your comment: I would've chosen a GPU used for mining (if properly cleaned during its life span, far from a given) over years rather than one used by some kid benchmarking and overclocking any day. Because years of crunching calculations did very little damage in comparison to a kid trying to find the overclock limits for few days. Most mining GPUs were used undervolted and underclocked (especially as Ethereum mining was memory rather than core intensive).
    - always2slow 3y ago
      Is this the paper you're referring to? https://www.microsoft.com/en-us/research/uploads/prod/2021/04/Zissou-Overclocking-ISCA21.pdf https://www.microsoft.com/en-us/research/uploads/prod/2021/0...
      - epolanski 3y ago
        Not exactly this one. Maybe it's referenced at the end.
  - coffeebeqn 3y ago
    When I ran a ETH miner for a little bit it was putting an insane amount of pressure on the video memory making it overheat since memory chips aren’t always properly cooled. It would crash my machine until I changed some settings. Never had such issues with any other workloads
usefulcat 3y ago
I'd much rather have something that came from a server room. Lots of cool, dust-free air--far better than a machine that's been sitting under someone's desk, clogged with dust and exposed to who know what temperatures.

wmf 3y ago

"Abused" chips are mostly a myth and Milan is not that old so these chips should have plenty of life left in them.

londons_explore 3y ago
Agreed - with CPU's, if it's working the day you buy it, it will most likely still be working in 10 years with typical desktop use cases, no matter the past life it had.
- paulmd 3y ago
  that's not true at all, XMP is very much within the wheelhouse of "typical desktop use-cases" and can absolutely damage a CPU from electromigration within a matter of years. (or rather, the overclocked/out-of-spec memory controller usually requires the board to kick up the CPU memory controller (VCCSA/VSOC) voltages, and that's what does the damage.) https://youtu.be/HLNk0NNQQ8s?t=510 https://youtu.be/HLNk0NNQQ8s?t=510 https://www.youtube.com/watch?v=uMHUz16MuYA https://www.youtube.com/watch?v=uMHUz16MuYA People have generally convinced themselves that it's safe but, the rate of CPU failures is incredibly high among enthusiasts compared to the general enterprise fleet and the reason is XMP. This has been "out there" for a long time if you know to look for it. But, enthusiasts fall into that classic "can't make a man understand when his salary depends on not understanding it" thing - everyone has every reason to convince themselves it doesn't, because it would affect "their lifesyle". But electromigration exists. Electromigration affects parts on consumer-relevant timescales, if you overclock. Electromigration particularly affects memory controllers/system fabric nowadays. And yes, you can absolutely burn out a memory controller with just XMP (and the aggressive CPU voltages it applies) and this is not new or a secret. And the problem of electromigration/lifespan is accelerating as the operating range becomes narrower on newer nodes etc. https://semiengineering.com/aging-problems-at-5nm-and-below/ https://semiengineering.com/aging-problems-at-5nm-and-below/ https://semiengineering.com/3d-ic-reliability-degrades-with-increasing-temperature/ https://semiengineering.com/3d-ic-reliability-degrades-with-... https://semiengineering.com/on-chip-power-distribution-modeling-becomes-essential-below-7nm/ https://semiengineering.com/on-chip-power-distribution-model... Similarly: "24/7 safe" fabric overclocks are really not. Not on the order of years. Everyone is already incentivized to push the "official" limit as much as is safe/reliable - AMD/Intel know about the impact on benchmark scores too, they want their parts to look as good as they can. There is no "safe" increase above the official spec, not really. The unique thing about Asus wasn't that they killed a chip from XMP - it's that they put so much voltage into it that it went into immediate runaway and popped instantly, explosively, and visibly. And it's not surprising it was Asus (those giant memory QVLs come from just throwing voltage at the problem) but low-key everyone has been applying at least some additional voltage for a long time. Eventually it kills chips. It's overclocking/out-of-spec and very deliberately and specifically excluded from the warranty (AMD GD-106/GD-112). It's completely understandable why AMD wants to make some fuses/degradation canary cells to monitor whether the CPU has operated out-of-spec as far as warranty coverage. This is a serious failure mode that probably causes a large % of overall/total "CPU premature failure" warranty returns etc. And essentially it continues to get worse on every new node and with every new DDR standard, and with the increased thermals that currently are characteristic of stacked solutions etc. https://www.amd.com/en/legal/claims/gaming-details.html https://www.amd.com/en/legal/claims/gaming-details.html https://www.extremetech.com/computing/amds-new-threadripper-chips-have-a-hidden-fuse-that-blows-when-overclocking https://www.extremetech.com/computing/amds-new-threadripper-...
  - wmf 3y ago
    Fortunately servers don't have XMP.
    - paulmd 3y ago
      true, I am just pushing back on the idea that "abusing a chip is mostly a myth" and "if a CPU is working on the day you buy it, it's fine for desktop use-cases". For server parts that can't be OC'd - true, I guess. For regular CPUs? Absolutely not true, enthusiasts abuse the shit out of them and even if you do no further damage yourself, the degradation can continue over time etc as parts of the circuit just become critically unstable from small routine usage etc. (people treat their CPUs like gamer piss-jugs, big deferred cost tomorrow for a small benefit today.) But yes - ironically this means surplus server CPUs are actually way more reliable than used enthusiast CPUs. In some cases they are drop-in compatible in the consumer platform (although not so much in newer stuff), and the server stuff got the better bins in the first place, and it's cheaper (because they sold a lot more units), and also hasn't been abused by an enthusiast for 5 years etc. If you are on a platform like Z97 or X99 that supports the Xeon chips, the server chips are a complete no-brainer. And some xeons are even multiplier unlocked etc - used to be a thing, back in the day. ("server bins are binned for leakage and don't compete with gaming cpus" is another myth that is not really true except for XOC binning - server CPUs are better binned than enthusiast ones for ambient use-cases.)
  - londons_explore 3y ago
    But are there many cases of a CPU being overclocked (& overheated & overvolted), then later not being overclocked (and working fine), but then failing shortly afterwards? Yes, I understand it is theoretically possible. But I think it is just super rare - I've never heard of a single case.
    - paulmd 3y ago
      Was looking through semiengineering for some more sources and some of them address it. Aging (I should probably say "aging" instead of electromigration, I'm not referring to just one effect here) is such a problem below 10nm that literally even just idling the chip wears it noticeably... and of course that leads to uneven wear on the cores too, etc. It's not just electromigration etc anymore. The reason you don't notice this is that the chip is engineered so you don't notice it. The boost clocks will slow down over time, the voltage applied will increase over time (dynamically controlled by measuring the degradation of the canary cells). Unless the chip catastrophically fails, you probably won't notice the slowdown etc, in scenarios that would have resulted in chip failure 20 years ago. The chips are simply designed to tolerate that - because they have to be, even during normal operation (!). The lifespan of a 5nm chip is not "infinite if treated properly" anymore. It is actually finite in terms of even idle power-on hours etc let alone load hours. A large number of power-on hours, and deliberately engineered to be large and to tolerate the damage gracefully, but people's mental models of "power-on hours doesn't hurt the chip" is fundamentally not correct anymore. Miners running lots of hours on that 7nm GPU etc is not "just fine" etc. Also, once you get it beyond the "damage point", especially in analog stuff you have simply changed the characteristics of the circuit. If the amplifier's bias circuit leads some other part of the circuit to be hit with a higher gain, that can continue damaging it even if you stop further damaging the bias circuit etc. Memory and PCIe are analog circuits here. https://semiengineering.com/design-for-reliability-2/ https://semiengineering.com/design-for-reliability-2/ > Digital and analog will be affected differently, as will devices subject to frequent change — and in some cases, infrequent change. “Any place where there’s a lot of activity will be more sensitive to device aging,” says Art Schaldenbrand, senior product manager at Cadence. “For devices, you can look at the clock tree and look at what is happening. Digital designs are sensitive to delay changes. The other place where this becomes a challenge is within analog designs. An example would be in a bias tree. With the bias transistors moving and aging, it can potentially accelerate the aging of other devices in the bias network. There’s always going to be some different elements in the design, and you have to look at them a little bit differently to be able to analyze the reliability.” [ ...] > But you have to be careful to consider all of the important areas. “There is a phenomenon called non-conductive stress,” says Cadence’s Schaldenbrand. “Consider a device such as a watch dog or monitor. It will be sitting idle, potentially for years, and you want it to spring into action if there’s some sort of condition that occurs. Even those circuits, that you think are you’re just sitting there doing nothing, are being stressed. They can age and potentially fail due to the aging that occurs while they’re sitting idle.” https://semiengineering.com/aging-not-always-a-bad-thing/ https://semiengineering.com/aging-not-always-a-bad-thing/ > This impacts the gate because of the natural behavior of the transistors, Elhak explained. “In the transistor you have a gate, which has an electric field that is supposed to control the current that is flowing between the drain and the source but there are random events. This electric field causes some of those carriers, instead of flowing between the gate and the source, to go and get injected into the gate. As more carriers get injected over time, the electrical properties of the gate start to differ because it’s not supposed to have those carriers in it. That changes the properties of the whole device because now the gate is supposed to control that electric field it is now made of a different material.” > The second mechanism that causes aging is called the bias temperature instability (BTI), which happens when there is a constant bias on the device meaning there is current flowing. Here, instead of being driven by electric field, it is driven here by bias and temperature. Also, charges start to get trapped into the gate and as this happens, the properties of the gate change and again it impacts the threshold voltage and the carrier mobility in that channel. “If you change the threshold voltage and if you change the mobility, then you have a different transistor,” he asserted. https://semiengineering.com/adding-aging-to-variability/ https://semiengineering.com/adding-aging-to-variability/ https://semiengineering.com/uneven-circuit-aging-becoming-a-bigger-problem/ https://semiengineering.com/uneven-circuit-aging-becoming-a-... https://semiengineering.com/transistor-aging-intensifies-10nm/ https://semiengineering.com/transistor-aging-intensifies-10n... https://semiengineering.com/chip-aging-becomes-design-problem/ https://semiengineering.com/chip-aging-becomes-design-proble... https://semiengineering.com/minimizing-chip-aging-effects/ https://semiengineering.com/minimizing-chip-aging-effects/ https://semiengineering.com/dealing-with-device-aging-at-advanced-nodes/ https://semiengineering.com/dealing-with-device-aging-at-adv... https://semiengineering.com/24142954-2/ https://semiengineering.com/24142954-2/ https://semiengineering.com/chip-aging-accelerates/ https://semiengineering.com/chip-aging-accelerates/
irusensei 3y ago
It kinda feels like the mining GPU being bad myth when in fact miners nursed those GPUs like babies because their income depended on those devices.
- namibj 3y ago
  Tbf the fans may be broken on them, or at least not far from being broken. I.e., plan to waterblock it.
- epolanski 3y ago
  People forget there's decade old servers out there working 24/7. Anyway there's a Microsoft research paper on silicon which essentially says that failure rates of CPUs increase by mostly two factors: - cycles. The more calculations the higher the rate of failure - temperature/power. I will let you guess it by yourself. Even minor slips overvoltages and overclocks can enhance failure rates by magnitude of orders. Getting back to your comment: I would've chosen a GPU used for mining (if properly cleaned during its life span, far from a given) over years rather than one used by some kid benchmarking and overclocking any day. Because years of crunching calculations did very little damage in comparison to a kid trying to find the overclock limits for few days. Most mining GPUs were used undervolted and underclocked (especially as Ethereum mining was memory rather than core intensive).
  - always2slow 3y ago
    Is this the paper you're referring to? https://www.microsoft.com/en-us/research/uploads/prod/2021/04/Zissou-Overclocking-ISCA21.pdf https://www.microsoft.com/en-us/research/uploads/prod/2021/0...
    - epolanski 3y ago
      Not exactly this one. Maybe it's referenced at the end.
- coffeebeqn 3y ago
  When I ran a ETH miner for a little bit it was putting an insane amount of pressure on the video memory making it overheat since memory chips aren’t always properly cooled. It would crash my machine until I changed some settings. Never had such issues with any other workloads

usefulcat 3y ago

I'd much rather have something that came from a server room. Lots of cool, dust-free air--far better than a machine that's been sitting under someone's desk, clogged with dust and exposed to who know what temperatures.