3 ms·
Based on Future ARM Neoverse, so basically nothing much to see here from CPU perspective, What really stands out, are those ridiculous number from its Memory sy
by ksec 6y ago
Based on Future ARM Neoverse, so basically nothing much to see here from CPU perspective, What really stands out, are those ridiculous number from its Memory system and Interconnect.
CPU: LPDDR5X with ECC Memory at 500+GB/s Memory Bandwidth. ( Something Apple may dip into. R.I.P for Mac with upgradable Memory )
GPU: HBM2e at 2000 GB/s. Yes, three zeros, this is not a typo.
NVLink: 500GB/s
This will surely further solidify CUDA dominance. Not entirely sure how Intel's XE with OneAPI and AMD's ROCm is going to compete.
- Dylan16807 6y ago> GPU: HBM2e at 2000 GB/s. Yes, three zeros, this is not a typo. It's a good step forward but your average consumer GPU is already around a quarter to a third of that and a Radeon VII had 1000 GB/s two years ago.
- m_mueller 6y agoI think what you’re missing here is the NVLink part. The fact that you can get a small cluster of these linked up like that for 400k, all wrapped in a box, makes HPC quite a bit more accessible. Even 5 years ago, if you wanted to run a regional sized weather model at reasonable resolution, you needed to have some serious funding (say, nation states or oil / insurance companies). Nowadays you could do it with some angel investment and get one of these Nvidia boxes and just program them like they’re one GPU.
- kllrnohj 6y agoCritically it's CPU to GPU NVLink here, not the "boring" GPU to GPU NVLink that's common on Quadros. 500GB/s bandwidth between CPU & GPU massively changes when & how you can GPU accelerate things, that's a 10X difference over the status quo.
- kimixa 6y agoAlso "cpu->cpu" NVLink is interesting. Though it was my understanding that NVLink is point-to-point, and would require some massive switching system to be able to access any node in the cluster anywhere near that rate without some locality bias (IE nodes on the "first" downstream switch are faster to access and less contention)
- wmf 6y agoFortunately NVSwitch already exists so they should be able to reuse that for CPUs.
- lmeyerov 6y agoYep, we generally care about growing a few bandwidth #'s over current: - GPU<>CPU/RAM - GPU<>storage - GPU<>network (- GPU<>GPU bandwidth is already insane, as is GPU compute speed) In the above, they're about cases like logs where there is ~infinite off-GPU data (S3, storage, ...), yet current PCI etc CPU stuff is like a tiny straw clogging it all. It's now ~easy to do stuff like regex search on GPU, so systems being redesigned to quickly shove 1TB through a python 1-liner is awesome. To get a feel for where all this is in practice, I did a fun talk w/ the pavilion team for this year's GTC on building graphistry UIs & interactive dashboards on top of this: https://pavilion.io/nvidia/ https://pavilion.io/nvidia/ Edit: A good search term here is 'GPU Direct Storage', which is explicitly about skipping the CPU bandwidth indirection & performance handcuffs. Tapping directly into the network or storage is super exciting for matching what the compute tier can do!
- Dylan16807 6y agoI'm not missing it, I'm saying that a part other then nvlink isn't the important part!
- jabl 6y agoThe Nvidia A100 80GB already provides 2 TB/s mem BW today. Also using HBM2e.
- nickflood 6y agoRTX 3090 also does 936 GB/s which is very close to Radeon 7 but with conventional GDDR