3 ms·

To/from what sort of devices could the GPU read or write at 1TB/sec, besides main memory? The fastest consumer SSDs top out at several GB/sec (I guess with mas

To/from what sort of devices could the GPU read or write at 1TB/sec, besides main memory? The fastest consumer SSDs top out at several GB/sec (I guess with massive hardware RAID they could be faster, but not sure if they’d be 1TB/sec fast). Even a network adapter that does 10 Gbit/sec is only recently becoming slightly more common for the average consumer. Not sure if any consumer adapters in the 10 or 1Tbit/sec range exist at all.

chasd00 2y ago
> Not sure if any consumer adapters in the 10 or 1Tbit/sec range exist at all. Further, what exactly is a consumer going to plug a 1Tbit/sec adapter into? Your little home ATT fiber connection isn’t going to come close to using that available bandwidth.
- simoncion 2y ago
  > Further, what exactly is a consumer going to plug a 1Tbit/sec adapter into? Another similarly-equipped machine on the LAN, and the switch(es) between them.
semi-extrinsic 2y ago
Consumer? Even in the "if-you-need-to-ask-what-it-costs-you-can't-afford-it" world of frontier HPC systems, were only getting teasers of 0.8 Tbit/sec NICs this year. As you say, only the GPU, maybe RAM, and the NIC will be able to churn data at these speeds. There is a reason why Mellanox (Nvidia) has developed GPUDirect RDMA, so the GPU and NIC can talk directly to each other. https://www.servethehome.com/this-is-the-next-gen-nvidia-connectx-8-supernic-for-800gbps-networking/ https://www.servethehome.com/this-is-the-next-gen-nvidia-con...
simoncion 2y ago
> ...besides main memory? Main memory is an important thing to have fast, though. The faster (and lower-wallclock-latency) it is, the less time your system spends waiting around when it needs to swap things in and out of it. It's my understanding that programs that need to be fast (like many video games) take pains to preemptively load data into RAM from disk, and (when appropriate for the program) from main RAM into VRAM. If main RAM's transfer speed was equal to or greater than VRAM's, and it access latency was a small fraction of a frame render time, (presumably) some of that preloading complexity could go away. > I guess with massive hardware RAID they could be faster... This section of the comment is for folks who haven't been paying attention to how fast storage has gotten: It's nowhere near 1TB per second, but... I have four 4TB SATA-attached Crucial MX500s set up in LVM2 RAID 0. This array is a bit faster than a 10gbit link. (That is, I get 1.5GByte/s transfer rate off of the thing.) Even a single non-garbage U.2-attached (or (barf) M.2-attached) device can saturate a 10Gbit link.
Cthulhu_ 2y ago
I'm reading the latest SSDs do 14 GB/s of sequential reading and/or 15 million IOPS; transfer speed wise that's close to the highest end DDR3 (2007) and the lowest end DDR4 (2014). SSDs are definitely not memory speed fast yet (definitely not for random access) but definitely getting closer.
- wkat4242 2y ago
  Yeah that was intel's optane idea. To make SSDs so fast they could act as memory. Didn't quite work out though.
Aurornis 2y ago
> To/from what sort of devices could the GPU read or write at 1TB/sec, besides main memory? The 1TB/sec is between the GPU and the GPU memory, which is the bottleneck. You don’t need that much bandwidth for loading inputs and outputting results, just for random access during compute.

chasd00 2y ago

> Not sure if any consumer adapters in the 10 or 1Tbit/sec range exist at all. Further, what exactly is a consumer going to plug a 1Tbit/sec adapter into? Your little home ATT fiber connection isn’t going to come close to using that available bandwidth.

simoncion 2y ago
> Further, what exactly is a consumer going to plug a 1Tbit/sec adapter into? Another similarly-equipped machine on the LAN, and the switch(es) between them.

semi-extrinsic 2y ago

Consumer? Even in the "if-you-need-to-ask-what-it-costs-you-can't-afford-it" world of frontier HPC systems, were only getting teasers of 0.8 Tbit/sec NICs this year. As you say, only the GPU, maybe RAM, and the NIC will be able to churn data at these speeds. There is a reason why Mellanox (Nvidia) has developed GPUDirect RDMA, so the GPU and NIC can talk directly to each other. https://www.servethehome.com/this-is-the-next-gen-nvidia-connectx-8-supernic-for-800gbps-networking/ https://www.servethehome.com/this-is-the-next-gen-nvidia-con...

simoncion 2y ago

> ...besides main memory? Main memory is an important thing to have fast, though. The faster (and lower-wallclock-latency) it is, the less time your system spends waiting around when it needs to swap things in and out of it. It's my understanding that programs that need to be fast (like many video games) take pains to preemptively load data into RAM from disk, and (when appropriate for the program) from main RAM into VRAM. If main RAM's transfer speed was equal to or greater than VRAM's, and it access latency was a small fraction of a frame render time, (presumably) some of that preloading complexity could go away. > I guess with massive hardware RAID they could be faster... This section of the comment is for folks who haven't been paying attention to how fast storage has gotten: It's nowhere near 1TB per second, but... I have four 4TB SATA-attached Crucial MX500s set up in LVM2 RAID 0. This array is a bit faster than a 10gbit link. (That is, I get 1.5GByte/s transfer rate off of the thing.) Even a single non-garbage U.2-attached (or (barf) M.2-attached) device can saturate a 10Gbit link.

Cthulhu_ 2y ago

I'm reading the latest SSDs do 14 GB/s of sequential reading and/or 15 million IOPS; transfer speed wise that's close to the highest end DDR3 (2007) and the lowest end DDR4 (2014). SSDs are definitely not memory speed fast yet (definitely not for random access) but definitely getting closer.

wkat4242 2y ago
Yeah that was intel's optane idea. To make SSDs so fast they could act as memory. Didn't quite work out though.

Aurornis 2y ago

> To/from what sort of devices could the GPU read or write at 1TB/sec, besides main memory? The 1TB/sec is between the GPU and the GPU memory, which is the bottleneck. You don’t need that much bandwidth for loading inputs and outputting results, just for random access during compute.