7 ms·

180 = 12 x 15 Where are you getting the extra 3 zeros from? (Also 'up to 180 TFLOPs' is a bit misleading. As I read in some benchmarking posts by Google as we

180 = 12 x 15 Where are you getting the extra 3 zeros from? (Also 'up to 180 TFLOPs' is a bit misleading. As I read in some benchmarking posts by Google as well as by nVidida, a TPU is much faster than a GPU for doing inference, but they haven't released any data on training performance, the real bottleneck IMO).

superfx 9y ago
Also, I suspect the more relevant number is the "Tensor" ops that Nvidia recently reported for Volta at 120 TFLOPS [1], which would be relative to Google's TPU a "mere" 1.5x speed up. Of course, this is pure speculation on my part. 1. https://devblogs.nvidia.com/parallelforall/inside-volta/ https://devblogs.nvidia.com/parallelforall/inside-volta/
- slizard 9y ago
  Might well be at 10x less power -- or more. Might also be TPU gen2 given the "64 GB of ultra-high-bandwidth memory" was not something IIRC the recent TPU paper was talking about. And this is here now, while Volta is paper-launched and will be very scarce until late 2017/2018.
- asdf_ 9y ago
  Google's 180 teraflop number is for a module, which contains 4 TPU chips. Meaning each TPU is 45 teraflops. See: https://arstechnica.com/information-technology/2017/05/google-brings-45-teraflops-tensor-flow-processors-to-its-compute-cloud/ https://arstechnica.com/information-technology/2017/05/googl...
  - jlebar 9y ago
    I replied to this comment elsewhere in the thread [1], but to reply here as well: A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric. [1] https://news.ycombinator.com/item?id=14364169 https://news.ycombinator.com/item?id=14364169
    - asdf_ 9y ago
      A single V100 board is vastly smaller than a TPU module with 4 TPU chips. The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips). Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/dgx-1/data-center-products-dgx-1-components-843-u1.jpg https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/images/tpu-V2-hero.width-1000.png https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446705711_76ab786243_o.jpg https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446... You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.
      - dgacmu 9y ago
        From my read of that Volta picture (here's a close-up glamour shot of the V100 module: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2017/05/SXM2-VoltaChipDetails.png https://devblogs.nvidia.com/parallelforall/wp-content/upload... ), that's just the chip module itself. Note how none of the interconnect hardware is present? The V100 supports NVLink just like the P100. When you actually assemble that chip module onto a board, it looks more like this: http://hothardware.com/ContentImages/NewsItem/37034/content/small_QuantPlex-Deep-Learning-Server.jpg http://hothardware.com/ContentImages/NewsItem/37034/content/... (That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)
        asdf_ 9y ago
        The interconnect hardware is present on the dgx-1 board. The point of my previous comment is that it doesn't make sense to compare an entire TPU module (containing 4x TPUs) to a V100 board. The closest comparison to a TPU module (with 4x TPUs) would be a dgx-1 board, which contains the nvlink bus that you mention, but also contains 8x V100 boards, hence why in my previous post I said you should compare the compute performance of a Volta DGX-1 (8x V100s) to 2 TPU modules (8x TPUs). At the end of the day, it is simple, in a given area of space, you can get more compute performance from provisioning that area with V100s (in the form of using dgx-1s) versus provisioning it with TPUs (in the form of using TPU modules).
aub3bhat 9y ago
The cluster has 1000 TPUs
- fizixer 9y ago
  Well, then MS Azure, Amazon AWS have their own clusters offering way more than $18 million worth of compute power (I'm sure they also provide it for research purposes worth in the millions). But not to a single client. Similarly, Google is not offering a 1000 TPU cluster to each client. Your metric is strange to say the least.
  - aub3bhat 9y ago
    They are offering a dedicated 1000 TPU cluster for researchers, at least that's how I read it.
    - vazamb 9y ago
      AFAIK in the original TPU announcement they said the cluster would be shared between all participants of the program.
dgacmu 9y ago
The benchmarking posts were about the previous-generation TPUs, which are detailed in an ISCA paper this year: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk... The previous TPU was aimed at inference. The Cloud TPUs do training. But you're right - I haven't seen any publications about training performance yet. But they'll be available in cloud (currently in alpha), and once they go GA, I'd expect to see a raft of benchmarks by third parties. I can't wait. :)

superfx 9y ago

Also, I suspect the more relevant number is the "Tensor" ops that Nvidia recently reported for Volta at 120 TFLOPS [1], which would be relative to Google's TPU a "mere" 1.5x speed up. Of course, this is pure speculation on my part. 1. https://devblogs.nvidia.com/parallelforall/inside-volta/ https://devblogs.nvidia.com/parallelforall/inside-volta/

slizard 9y ago
Might well be at 10x less power -- or more. Might also be TPU gen2 given the "64 GB of ultra-high-bandwidth memory" was not something IIRC the recent TPU paper was talking about. And this is here now, while Volta is paper-launched and will be very scarce until late 2017/2018.
asdf_ 9y ago
Google's 180 teraflop number is for a module, which contains 4 TPU chips. Meaning each TPU is 45 teraflops. See: https://arstechnica.com/information-technology/2017/05/google-brings-45-teraflops-tensor-flow-processors-to-its-compute-cloud/ https://arstechnica.com/information-technology/2017/05/googl...
- jlebar 9y ago
  I replied to this comment elsewhere in the thread [1], but to reply here as well: A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric. [1] https://news.ycombinator.com/item?id=14364169 https://news.ycombinator.com/item?id=14364169
  - asdf_ 9y ago
    A single V100 board is vastly smaller than a TPU module with 4 TPU chips. The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips). Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/dgx-1/data-center-products-dgx-1-components-843-u1.jpg https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/images/tpu-V2-hero.width-1000.png https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446705711_76ab786243_o.jpg https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446... You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.
    - dgacmu 9y ago
      From my read of that Volta picture (here's a close-up glamour shot of the V100 module: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2017/05/SXM2-VoltaChipDetails.png https://devblogs.nvidia.com/parallelforall/wp-content/upload... ), that's just the chip module itself. Note how none of the interconnect hardware is present? The V100 supports NVLink just like the P100. When you actually assemble that chip module onto a board, it looks more like this: http://hothardware.com/ContentImages/NewsItem/37034/content/small_QuantPlex-Deep-Learning-Server.jpg http://hothardware.com/ContentImages/NewsItem/37034/content/... (That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)
      - asdf_ 9y ago
        The interconnect hardware is present on the dgx-1 board. The point of my previous comment is that it doesn't make sense to compare an entire TPU module (containing 4x TPUs) to a V100 board. The closest comparison to a TPU module (with 4x TPUs) would be a dgx-1 board, which contains the nvlink bus that you mention, but also contains 8x V100 boards, hence why in my previous post I said you should compare the compute performance of a Volta DGX-1 (8x V100s) to 2 TPU modules (8x TPUs). At the end of the day, it is simple, in a given area of space, you can get more compute performance from provisioning that area with V100s (in the form of using dgx-1s) versus provisioning it with TPUs (in the form of using TPU modules).

aub3bhat 9y ago

The cluster has 1000 TPUs

fizixer 9y ago
Well, then MS Azure, Amazon AWS have their own clusters offering way more than $18 million worth of compute power (I'm sure they also provide it for research purposes worth in the millions). But not to a single client. Similarly, Google is not offering a 1000 TPU cluster to each client. Your metric is strange to say the least.
- aub3bhat 9y ago
  They are offering a dedicated 1000 TPU cluster for researchers, at least that's how I read it.
  - vazamb 9y ago
    AFAIK in the original TPU announcement they said the cluster would be shared between all participants of the program.

dgacmu 9y ago

The benchmarking posts were about the previous-generation TPUs, which are detailed in an ISCA paper this year: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk... The previous TPU was aimed at inference. The Cloud TPUs do training. But you're right - I haven't seen any publications about training performance yet. But they'll be available in cloud (currently in alpha), and once they go GA, I'd expect to see a raft of benchmarks by third parties. I can't wait. :)