3 ms·
I’m not sure of all the details of the architecture, but my understanding is that there’s all sorts of “GPU Direct Storage” and massive on-chip CPU/GPU shared m
by tomoyoirl 3y ago
I’m not sure of all the details of the architecture, but my understanding is that there’s all sorts of “GPU Direct Storage” and massive on-chip CPU/GPU shared memory spaces on the new GH100s, plus all the Mellanox stuff for interconnects. At least in the datacenter space, anyway.
- nostrebored 3y agoThanks! The last time I cared about this I was doing SGD on GPU clusters and memory bandwidth was the limiting factor. Almost the entirety of the work was focused on chunking the problem so that a maximal number of subproblems was on the GPU. If storage really is becoming integrated at similar speed (EG not on an order of magnitude difference like L2 vs RAM) then I think I'm wrong