3 ms·
Maybe I am not understanding something, but I thought that total memory bandwidth is critical for Deep Learning applications. This is where HBM on-die would shi
by g42gregory 5y ago
Maybe I am not understanding something, but I thought that total memory bandwidth is critical for Deep Learning applications. This is where HBM on-die would shine, no? I am deferring the purchase of a new desktop/server until processors with HBM come to market. I think AMD is shipping EPYC engineering samples with some version of memory and Intel is slated the release by the end of the year. Am I wrong about this?
- wmf 5y agoThe only CPU with HBM is Sapphire Rapids and it may cost $20K; for that money you're probably better off buying an H100.
- eyegor 5y agoLooks like the h100 doesn't have a release date let alone a price. There are also some applications that don't scale well on a gpu (lots of interdependence) which could still benefit greatly from hbm on a cpu. Although deep learning typically scales very well.
- sliken 5y agoWhy not just get a GPU? They seem much closer to the needs of deep learning than trying to shoe horn a HBM memory interface onto a chip that wasn't particularly designed for it.
- rajnathani 5y agoFuture deep learning hardware systems such as Nvidia's Grace system will use CPU memory in a hierarchy [0] (check the illustrations here). While HBM memory is fast, it is limited in size (eg: 48GB-96GB), whereas DDRx memory can be in the TBs easily and that too inexpensively. You can think of the HBM-DDRx hierarchy analogously to the L1-L2-L(n) cache hierarchy for CPU processors. IIRC, even the CPU HBM implementations will use this HBM-DDRx hierarchy. [0] https://www.nextplatform.com/2021/04/12/nvidia-enters-the-arms-race-with-homegrown-grace-cpus/ https://www.nextplatform.com/2021/04/12/nvidia-enters-the-ar...