2 ms·

From my read of that Volta picture (here's a close-up glamour shot of the V100 module: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2017/05/SX

by dgacmu 9y ago

From my read of that Volta picture (here's a close-up glamour shot of the V100 module: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2017/05/SXM2-VoltaChipDetails.png https://devblogs.nvidia.com/parallelforall/wp-content/upload... ), that's just the chip module itself. Note how none of the interconnect hardware is present? The V100 supports NVLink just like the P100. When you actually assemble that chip module onto a board, it looks more like this: http://hothardware.com/ContentImages/NewsItem/37034/content/small_QuantPlex-Deep-Learning-Server.jpg http://hothardware.com/ContentImages/NewsItem/37034/content/... (That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)

asdf_ 9y ago
The interconnect hardware is present on the dgx-1 board. The point of my previous comment is that it doesn't make sense to compare an entire TPU module (containing 4x TPUs) to a V100 board. The closest comparison to a TPU module (with 4x TPUs) would be a dgx-1 board, which contains the nvlink bus that you mention, but also contains 8x V100 boards, hence why in my previous post I said you should compare the compute performance of a Volta DGX-1 (8x V100s) to 2 TPU modules (8x TPUs). At the end of the day, it is simple, in a given area of space, you can get more compute performance from provisioning that area with V100s (in the form of using dgx-1s) versus provisioning it with TPUs (in the form of using TPU modules).