4 ms·

What's the track record on "super computers"? It seems risky to pour so much money into a platform that will be superseded in a few years time. Then again, it's

What's the track record on "super computers"? It seems risky to pour so much money into a platform that will be superseded in a few years time. Then again, it's not clear that there are really good alternatives.

halfeatenpie 5y ago
Well it's partially the cost/benefit analysis of: 1. How much benefit will we get? 2. Will this benefit be higher than the cost of purchasing this right now? 3. What other alternatives will satisfy our needs? For most of these solutions, the answers are: 1. A lot. We need computing power to perform these analysis faster and to be competitive. 2. Yes, as continuing to innovate in this space will keep us competitive and give our scientists the resources to remain productive. 3. AWS/GCP/Azure are alternatives sure, but then (the rate at which Meta probably uses these resources) it probably cost them less to build this out than to pay AWS/GCP/Azure for access to these hardware.
tedivm 5y ago
Major GPU architecture changes only happen every few years. * k80 - 2014, and this really was not that great of a chip. * v100 in 2017, and I'd consider this the first "built from the ground up" ML chip. * A100 in late 2020, with the first major cloud general availability being in 2021. Even when new chips come out the old ones are still usable- you can rent k80s for fairly cheap on all cloud providers, and have kept a surprising amount of their resale value. The v100s are also very much still in demand. The A100 is also an amazing system- the new nvswitch archetecture means the A100s work together far far better than their V100 counterparts. I was part of an upgrade project setting up an A100 cluster with infiniband and it really is amazing how well these chips work together. That communication barrier was a pretty obvious next step though (the k80s had crap intergpu communication, the v100s introduced the nvlink, and the nvswitch was the obvious way to go). There isn't an obvious next step, and I imagine the A100s to be the standard for at least the next four years (with lots of continued use after that).
- boulos 5y ago
  You skipped the T4 which was in between Volta and Ampere, as well as the P100 between K80 and V100. So I'd say "meaningful chip changes" is closer to every 18 months. The T4 though isn't a "big part", but for people who fit within its envelope, it's a huge win (since it's cost is so much lower). A lot of deep learning folks had built out Turing-based workstations in that time period, and I think they're still reasonable value for money.
  - tedivm 5y ago
    The T4 isn't the same- you'd never do training on it. It's a cute little inference chip.

halfeatenpie 5y ago

Well it's partially the cost/benefit analysis of: 1. How much benefit will we get? 2. Will this benefit be higher than the cost of purchasing this right now? 3. What other alternatives will satisfy our needs? For most of these solutions, the answers are: 1. A lot. We need computing power to perform these analysis faster and to be competitive. 2. Yes, as continuing to innovate in this space will keep us competitive and give our scientists the resources to remain productive. 3. AWS/GCP/Azure are alternatives sure, but then (the rate at which Meta probably uses these resources) it probably cost them less to build this out than to pay AWS/GCP/Azure for access to these hardware.

tedivm 5y ago

Major GPU architecture changes only happen every few years. * k80 - 2014, and this really was not that great of a chip. * v100 in 2017, and I'd consider this the first "built from the ground up" ML chip. * A100 in late 2020, with the first major cloud general availability being in 2021. Even when new chips come out the old ones are still usable- you can rent k80s for fairly cheap on all cloud providers, and have kept a surprising amount of their resale value. The v100s are also very much still in demand. The A100 is also an amazing system- the new nvswitch archetecture means the A100s work together far far better than their V100 counterparts. I was part of an upgrade project setting up an A100 cluster with infiniband and it really is amazing how well these chips work together. That communication barrier was a pretty obvious next step though (the k80s had crap intergpu communication, the v100s introduced the nvlink, and the nvswitch was the obvious way to go). There isn't an obvious next step, and I imagine the A100s to be the standard for at least the next four years (with lots of continued use after that).

boulos 5y ago
You skipped the T4 which was in between Volta and Ampere, as well as the P100 between K80 and V100. So I'd say "meaningful chip changes" is closer to every 18 months. The T4 though isn't a "big part", but for people who fit within its envelope, it's a huge win (since it's cost is so much lower). A lot of deep learning folks had built out Turing-based workstations in that time period, and I think they're still reasonable value for money.
- tedivm 5y ago
  The T4 isn't the same- you'd never do training on it. It's a cute little inference chip.