4 ms·
Impressive. I've lately been wondering though, how does one load data fast enough into such rigs to come close to saturating this kind of compute power? I felt
by maldeh 8y ago
Impressive. I've lately been wondering though, how does one load data fast enough into such rigs to come close to saturating this kind of compute power? I felt this was an issue even with TPUs and V100s for any application that isn't thoroughly bottlenecked by compute (ex:- one could spend hours loading up a petabyte, and process it all in tens of minutes)
- faitswulff 8y agoI don't think it's supposed to be saturated by a single user. Looks like it's meant for cloud providers to farm out to their clients: > Nvidia is distributing them strictly to resellers, who will likely package these babies up and sell them to hyperscale data centers and cloud providers.
- dogma1138 8y agoConsidering that it’s main advantage is 16 GPUs which are addressable as a single GPU through NVLINK which is a memory coherent fabric while cloud providers are buying it because most users can’t afford it it’s not designed to be a multi user system but quite the contrary it’s designed to run a single application very very quickly.
- shaklee3 8y agoIt depends on the task. Some tasks require a bunch of GPU processing where it churns for a while before getting new data, while others process data streaming in. The DGX-2 has 8 100Gbps NICs, so you can load data very fast.
- liuliu 8y agoIt is challenging. Most likely your data at rest are compressed. Go with the highest-end SSD money can buy, RAID0 them, and layout your compressed data as sequentially as possible. You still need to trip to the main memory and decode though, and then the bandwidth between device / main memory becomes bottleneck. Alternatively you can write a decompressor to do that on device memory directly, haven't heard anyone doing that yet.