4 ms·

I think what you’re missing here is the NVLink part. The fact that you can get a small cluster of these linked up like that for 400k, all wrapped in a box, make

I think what you’re missing here is the NVLink part. The fact that you can get a small cluster of these linked up like that for 400k, all wrapped in a box, makes HPC quite a bit more accessible. Even 5 years ago, if you wanted to run a regional sized weather model at reasonable resolution, you needed to have some serious funding (say, nation states or oil / insurance companies). Nowadays you could do it with some angel investment and get one of these Nvidia boxes and just program them like they’re one GPU.

kllrnohj 6y ago
Critically it's CPU to GPU NVLink here, not the "boring" GPU to GPU NVLink that's common on Quadros. 500GB/s bandwidth between CPU & GPU massively changes when & how you can GPU accelerate things, that's a 10X difference over the status quo.
- kimixa 6y ago
  Also "cpu->cpu" NVLink is interesting. Though it was my understanding that NVLink is point-to-point, and would require some massive switching system to be able to access any node in the cluster anywhere near that rate without some locality bias (IE nodes on the "first" downstream switch are faster to access and less contention)
  - wmf 6y ago
    Fortunately NVSwitch already exists so they should be able to reuse that for CPUs.
lmeyerov 6y ago
Yep, we generally care about growing a few bandwidth #'s over current: - GPU<>CPU/RAM - GPU<>storage - GPU<>network (- GPU<>GPU bandwidth is already insane, as is GPU compute speed) In the above, they're about cases like logs where there is ~infinite off-GPU data (S3, storage, ...), yet current PCI etc CPU stuff is like a tiny straw clogging it all. It's now ~easy to do stuff like regex search on GPU, so systems being redesigned to quickly shove 1TB through a python 1-liner is awesome. To get a feel for where all this is in practice, I did a fun talk w/ the pavilion team for this year's GTC on building graphistry UIs & interactive dashboards on top of this: https://pavilion.io/nvidia/ https://pavilion.io/nvidia/ Edit: A good search term here is 'GPU Direct Storage', which is explicitly about skipping the CPU bandwidth indirection & performance handcuffs. Tapping directly into the network or storage is super exciting for matching what the compute tier can do!
Dylan16807 6y ago
I'm not missing it, I'm saying that a part other then nvlink isn't the important part!

kllrnohj 6y ago

Critically it's CPU to GPU NVLink here, not the "boring" GPU to GPU NVLink that's common on Quadros. 500GB/s bandwidth between CPU & GPU massively changes when & how you can GPU accelerate things, that's a 10X difference over the status quo.

kimixa 6y ago
Also "cpu->cpu" NVLink is interesting. Though it was my understanding that NVLink is point-to-point, and would require some massive switching system to be able to access any node in the cluster anywhere near that rate without some locality bias (IE nodes on the "first" downstream switch are faster to access and less contention)
- wmf 6y ago
  Fortunately NVSwitch already exists so they should be able to reuse that for CPUs.

lmeyerov 6y ago

Yep, we generally care about growing a few bandwidth #'s over current: - GPU<>CPU/RAM - GPU<>storage - GPU<>network (- GPU<>GPU bandwidth is already insane, as is GPU compute speed) In the above, they're about cases like logs where there is ~infinite off-GPU data (S3, storage, ...), yet current PCI etc CPU stuff is like a tiny straw clogging it all. It's now ~easy to do stuff like regex search on GPU, so systems being redesigned to quickly shove 1TB through a python 1-liner is awesome. To get a feel for where all this is in practice, I did a fun talk w/ the pavilion team for this year's GTC on building graphistry UIs & interactive dashboards on top of this: https://pavilion.io/nvidia/ https://pavilion.io/nvidia/ Edit: A good search term here is 'GPU Direct Storage', which is explicitly about skipping the CPU bandwidth indirection & performance handcuffs. Tapping directly into the network or storage is super exciting for matching what the compute tier can do!

Dylan16807 6y ago

I'm not missing it, I'm saying that a part other then nvlink isn't the important part!