4 ms·

Your supercomputer won’t last more than a couple years, 5 at max. Then it will simply be unjustifiably expensive to run when newer, more power efficient hardwar

by eecc 6y ago

Your supercomputer won’t last more than a couple years, 5 at max. Then it will simply be unjustifiably expensive to run when newer, more power efficient hardware becomes available. What matters is the software platform and compatibility: you don’t want to retool and port all your existing programs. That’s where having a platform - CUDA - becomes a deal-maker. What I don’t understand is how consumers - big, institutional consumers- don’t insist on viable vendor neutral platforms like OpenCL. Obviously they’re the only ones that have an interest in cross-vendor compatibility.

dannyw 6y ago
CUDA is valuable and seen as a positive, even if it's vendor locked. It's battle tested. Engineers are familiar with it. Support is provided because it's all under one roof. Performance targets, roadmap timeframes, etc can be assured. The same reason a big institutional consumer will happily pay Microsoft for Windows Enterprise, instead of demanding Microsoft rebuild Windows using open source technologies. A supercomputer or DC cluster has a fixed lifespan and amortization period, if you can get something in a contract, with everything you want for a price you are happy with, why would you introduce risk to it by demanding open standards for no benefit?
- eecc 6y ago
  I’m not saying CUDA is not valuable. Heck, proprietary has a ton of value... see how Intel owes much of its position to its compiler (the FORTRAN one in particular) suite. Of course you don’t get to dominate a field to such an extent without some redeeming positive quality. What I’m arguing - or more appropriately, what I’m wondering - is why on Earth aren’t consumer consortiums not insisting on open-standard platforms. I get how you want to eek every drop of performance out of your tools without making it your only mission (that would be the research you’re using the tool for), but tying yourself to a single proprietary vendor defeats that purpose imho
lorenzhs 6y ago
Not sure why you're downvoted, HPC systems don't last more than ~6 years before they are decommissioned because they've become too power-hungry to justify their continued use. I just checked some of the systems I'm familiar with and they were all decommissioned after around 6 years of use (e.g. SuperMUC, 2012-2018; two smaller cluster at the university where I work: 08/2012-04/2017, 01/2014-03/2020). I also attended a presentation at IPDPS 2018 by someone from one of the big US national labs where they talked about how absolutely huge of an undertaking it was to port their codes to make efficient use of GPUs. You don't just re-do all of that if the payoff isn't enormous.
- saganus 6y ago
  Out of curiosity, do you know what happens to the decomissioned hardware? is it scrapped for useful parts like perhaps the PSUs? or is gold and other metals extracted from the chips and the metal sold for recycling? Is there anything useful someone could do with it or it's just too much of a problem to set it up and repurpose it?
  - lo_stronzo 6y ago
    > Out of curiosity, do you know what happens to the decomissioned hardware? is it scrapped for useful parts like perhaps the PSUs? or is gold and other metals extracted from the chips and the metal sold for recycling? I cannot state with 100% clarity what happens after the systems are acquired by surplus vendors, but I can say that we have received certificates of what was "recycled". Once the surplus vendors take possession of the hardware, it's theirs and they can do what they want with it. In fact, we recently had to purchase EOL'ed, refurbished Infiniband switches to continue to support an interconnect fabric still in use (~2016). Interestingly enough, some of the switches still had "core" and "edge" labels on them. > Is there anything useful someone could do with it or it's just too much of a problem to set it up and repurpose it? In my opinion, it really depends on the node. If the chassis supports hot swappable & redundant hardware (HDD, PSU, etc.), then we'll typically cannibalize several chassis to create an administrative and/or infrastructure node. Case in point, a large portion of our older 12 core nodes have been put aside to serve as administrative nodes, all fully redundant (RAID1 HDD's, dual PSU's, ECC memory, etc.). Since we have a stack of these, we're fairly confident that these will serve us for the next few years, worry free, given the abundance of parts lying around.
    - saganus 6y ago
      Interesting! thanks for the insight.

by eecc 6y ago

dannyw 6y ago

CUDA is valuable and seen as a positive, even if it's vendor locked. It's battle tested. Engineers are familiar with it. Support is provided because it's all under one roof. Performance targets, roadmap timeframes, etc can be assured. The same reason a big institutional consumer will happily pay Microsoft for Windows Enterprise, instead of demanding Microsoft rebuild Windows using open source technologies. A supercomputer or DC cluster has a fixed lifespan and amortization period, if you can get something in a contract, with everything you want for a price you are happy with, why would you introduce risk to it by demanding open standards for no benefit?

eecc 6y ago
I’m not saying CUDA is not valuable. Heck, proprietary has a ton of value... see how Intel owes much of its position to its compiler (the FORTRAN one in particular) suite. Of course you don’t get to dominate a field to such an extent without some redeeming positive quality. What I’m arguing - or more appropriately, what I’m wondering - is why on Earth aren’t consumer consortiums not insisting on open-standard platforms. I get how you want to eek every drop of performance out of your tools without making it your only mission (that would be the research you’re using the tool for), but tying yourself to a single proprietary vendor defeats that purpose imho

lorenzhs 6y ago

Not sure why you're downvoted, HPC systems don't last more than ~6 years before they are decommissioned because they've become too power-hungry to justify their continued use. I just checked some of the systems I'm familiar with and they were all decommissioned after around 6 years of use (e.g. SuperMUC, 2012-2018; two smaller cluster at the university where I work: 08/2012-04/2017, 01/2014-03/2020). I also attended a presentation at IPDPS 2018 by someone from one of the big US national labs where they talked about how absolutely huge of an undertaking it was to port their codes to make efficient use of GPUs. You don't just re-do all of that if the payoff isn't enormous.

saganus 6y ago
Out of curiosity, do you know what happens to the decomissioned hardware? is it scrapped for useful parts like perhaps the PSUs? or is gold and other metals extracted from the chips and the metal sold for recycling? Is there anything useful someone could do with it or it's just too much of a problem to set it up and repurpose it?
- lo_stronzo 6y ago
  > Out of curiosity, do you know what happens to the decomissioned hardware? is it scrapped for useful parts like perhaps the PSUs? or is gold and other metals extracted from the chips and the metal sold for recycling? I cannot state with 100% clarity what happens after the systems are acquired by surplus vendors, but I can say that we have received certificates of what was "recycled". Once the surplus vendors take possession of the hardware, it's theirs and they can do what they want with it. In fact, we recently had to purchase EOL'ed, refurbished Infiniband switches to continue to support an interconnect fabric still in use (~2016). Interestingly enough, some of the switches still had "core" and "edge" labels on them. > Is there anything useful someone could do with it or it's just too much of a problem to set it up and repurpose it? In my opinion, it really depends on the node. If the chassis supports hot swappable & redundant hardware (HDD, PSU, etc.), then we'll typically cannibalize several chassis to create an administrative and/or infrastructure node. Case in point, a large portion of our older 12 core nodes have been put aside to serve as administrative nodes, all fully redundant (RAID1 HDD's, dual PSU's, ECC memory, etc.). Since we have a stack of these, we're fairly confident that these will serve us for the next few years, worry free, given the abundance of parts lying around.
  - saganus 6y ago
    Interesting! thanks for the insight.