4 ms·

For comparison, at 180 teraflops per TPU (assumption: at fp32) this is equivalent to offering a cluster with 15,000 Titax Xp GPUs (1200$ per GPU = 18 Million $

For comparison, at 180 teraflops per TPU (assumption: at fp32) this is equivalent to offering a cluster with 15,000 Titax Xp GPUs (1200$ per GPU = 18 Million $ worth of compute power). If TPU teraflops are reported at fp16, then the number would be half. Titan X offers 12 TFLOPs per GPU https://blogs.nvidia.com/blog/2017/04/06/titan-xp/ https://blogs.nvidia.com/blog/2017/04/06/titan-xp/

qeternity 9y ago
Not including power, motherboard, processor, ram, etc...
fizixer 9y ago
180 = 12 x 15 Where are you getting the extra 3 zeros from? (Also 'up to 180 TFLOPs' is a bit misleading. As I read in some benchmarking posts by Google as well as by nVidida, a TPU is much faster than a GPU for doing inference, but they haven't released any data on training performance, the real bottleneck IMO).
- superfx 9y ago
  Also, I suspect the more relevant number is the "Tensor" ops that Nvidia recently reported for Volta at 120 TFLOPS [1], which would be relative to Google's TPU a "mere" 1.5x speed up. Of course, this is pure speculation on my part. 1. https://devblogs.nvidia.com/parallelforall/inside-volta/ https://devblogs.nvidia.com/parallelforall/inside-volta/
  - slizard 9y ago
    Might well be at 10x less power -- or more. Might also be TPU gen2 given the "64 GB of ultra-high-bandwidth memory" was not something IIRC the recent TPU paper was talking about. And this is here now, while Volta is paper-launched and will be very scarce until late 2017/2018.
  - asdf_ 9y ago
    Google's 180 teraflop number is for a module, which contains 4 TPU chips. Meaning each TPU is 45 teraflops. See: https://arstechnica.com/information-technology/2017/05/google-brings-45-teraflops-tensor-flow-processors-to-its-compute-cloud/ https://arstechnica.com/information-technology/2017/05/googl...
    - jlebar 9y ago
      I replied to this comment elsewhere in the thread [1], but to reply here as well: A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric. [1] https://news.ycombinator.com/item?id=14364169 https://news.ycombinator.com/item?id=14364169
      - asdf_ 9y ago
        A single V100 board is vastly smaller than a TPU module with 4 TPU chips. The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips). Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/dgx-1/data-center-products-dgx-1-components-843-u1.jpg https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/images/tpu-V2-hero.width-1000.png https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446705711_76ab786243_o.jpg https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446... You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.
        dgacmu 9y ago
        From my read of that Volta picture (here's a close-up glamour shot of the V100 module: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2017/05/SXM2-VoltaChipDetails.png https://devblogs.nvidia.com/parallelforall/wp-content/upload... ), that's just the chip module itself. Note how none of the interconnect hardware is present? The V100 supports NVLink just like the P100. When you actually assemble that chip module onto a board, it looks more like this: http://hothardware.com/ContentImages/NewsItem/37034/content/small_QuantPlex-Deep-Learning-Server.jpg http://hothardware.com/ContentImages/NewsItem/37034/content/... (That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)
- aub3bhat 9y ago
  The cluster has 1000 TPUs
  - fizixer 9y ago
    Well, then MS Azure, Amazon AWS have their own clusters offering way more than $18 million worth of compute power (I'm sure they also provide it for research purposes worth in the millions). But not to a single client. Similarly, Google is not offering a 1000 TPU cluster to each client. Your metric is strange to say the least.
    - aub3bhat 9y ago
      They are offering a dedicated 1000 TPU cluster for researchers, at least that's how I read it.
      - vazamb 9y ago
        AFAIK in the original TPU announcement they said the cluster would be shared between all participants of the program.
- dgacmu 9y ago
  The benchmarking posts were about the previous-generation TPUs, which are detailed in an ISCA paper this year: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk... The previous TPU was aimed at inference. The Cloud TPUs do training. But you're right - I haven't seen any publications about training performance yet. But they'll be available in cloud (currently in alpha), and once they go GA, I'd expect to see a raft of benchmarks by third parties. I can't wait. :)
arcanus 9y ago
There is virtually no way this would be fp32. If it was, they could sell these devices for many more tasks than just machine learning.
asdf_ 9y ago
It is actually 45 teraflops per TPU2 chip and 180 teraflops for a module (4x chips): https://arstechnica.com/information-technology/2017/05/google-brings-45-teraflops-tensor-flow-processors-to-its-compute-cloud/ https://arstechnica.com/information-technology/2017/05/googl... V100 is 120 teraflops of tensor ops per chip: https://arstechnica.com/gadgets/2017/05/nvidia-tesla-v100-gpu-details/ https://arstechnica.com/gadgets/2017/05/nvidia-tesla-v100-gp...
- jlebar 9y ago
  A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric.
  - asdf_ 9y ago
    A single V100 board is vastly smaller than a TPU module with 4 TPU chips. The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips). Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/dgx-1/data-center-products-dgx-1-components-843-u1.jpg https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/images/tpu-V2-hero.width-1000.png https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446705711_76ab786243_o.jpg https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446... You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.

qeternity 9y ago

Not including power, motherboard, processor, ram, etc...

fizixer 9y ago

180 = 12 x 15 Where are you getting the extra 3 zeros from? (Also 'up to 180 TFLOPs' is a bit misleading. As I read in some benchmarking posts by Google as well as by nVidida, a TPU is much faster than a GPU for doing inference, but they haven't released any data on training performance, the real bottleneck IMO).

superfx 9y ago
Also, I suspect the more relevant number is the "Tensor" ops that Nvidia recently reported for Volta at 120 TFLOPS [1], which would be relative to Google's TPU a "mere" 1.5x speed up. Of course, this is pure speculation on my part. 1. https://devblogs.nvidia.com/parallelforall/inside-volta/ https://devblogs.nvidia.com/parallelforall/inside-volta/
- slizard 9y ago
  Might well be at 10x less power -- or more. Might also be TPU gen2 given the "64 GB of ultra-high-bandwidth memory" was not something IIRC the recent TPU paper was talking about. And this is here now, while Volta is paper-launched and will be very scarce until late 2017/2018.
- asdf_ 9y ago
  Google's 180 teraflop number is for a module, which contains 4 TPU chips. Meaning each TPU is 45 teraflops. See: https://arstechnica.com/information-technology/2017/05/google-brings-45-teraflops-tensor-flow-processors-to-its-compute-cloud/ https://arstechnica.com/information-technology/2017/05/googl...
  - jlebar 9y ago
    I replied to this comment elsewhere in the thread [1], but to reply here as well: A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric. [1] https://news.ycombinator.com/item?id=14364169 https://news.ycombinator.com/item?id=14364169
    - asdf_ 9y ago
      A single V100 board is vastly smaller than a TPU module with 4 TPU chips. The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips). Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/dgx-1/data-center-products-dgx-1-components-843-u1.jpg https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/images/tpu-V2-hero.width-1000.png https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446705711_76ab786243_o.jpg https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446... You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.
      - dgacmu 9y ago
        From my read of that Volta picture (here's a close-up glamour shot of the V100 module: https://devblogs.nvidia.com/parallelforall/wp-content/uploads/2017/05/SXM2-VoltaChipDetails.png https://devblogs.nvidia.com/parallelforall/wp-content/upload... ), that's just the chip module itself. Note how none of the interconnect hardware is present? The V100 supports NVLink just like the P100. When you actually assemble that chip module onto a board, it looks more like this: http://hothardware.com/ContentImages/NewsItem/37034/content/small_QuantPlex-Deep-Learning-Server.jpg http://hothardware.com/ContentImages/NewsItem/37034/content/... (That's a P100 server, obviously, but it's the same kind of design. The P100 module looks a lot like the V100 module.)
aub3bhat 9y ago
The cluster has 1000 TPUs
- fizixer 9y ago
  Well, then MS Azure, Amazon AWS have their own clusters offering way more than $18 million worth of compute power (I'm sure they also provide it for research purposes worth in the millions). But not to a single client. Similarly, Google is not offering a 1000 TPU cluster to each client. Your metric is strange to say the least.
  - aub3bhat 9y ago
    They are offering a dedicated 1000 TPU cluster for researchers, at least that's how I read it.
    - vazamb 9y ago
      AFAIK in the original TPU announcement they said the cluster would be shared between all participants of the program.
dgacmu 9y ago
The benchmarking posts were about the previous-generation TPUs, which are detailed in an ISCA paper this year: https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk/view https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk... The previous TPU was aimed at inference. The Cloud TPUs do training. But you're right - I haven't seen any publications about training performance yet. But they'll be available in cloud (currently in alpha), and once they go GA, I'd expect to see a raft of benchmarks by third parties. I can't wait. :)

arcanus 9y ago

There is virtually no way this would be fp32. If it was, they could sell these devices for many more tasks than just machine learning.

asdf_ 9y ago

It is actually 45 teraflops per TPU2 chip and 180 teraflops for a module (4x chips): https://arstechnica.com/information-technology/2017/05/google-brings-45-teraflops-tensor-flow-processors-to-its-compute-cloud/ https://arstechnica.com/information-technology/2017/05/googl... V100 is 120 teraflops of tensor ops per chip: https://arstechnica.com/gadgets/2017/05/nvidia-tesla-v100-gpu-details/ https://arstechnica.com/gadgets/2017/05/nvidia-tesla-v100-gp...

jlebar 9y ago
A GPU is also comprised of multiple chips (RAM, etc). I don't think "performance per discrete piece of silicon" is an interesting metric.
- asdf_ 9y ago
  A single V100 board is vastly smaller than a TPU module with 4 TPU chips. The closest comparison in terms of size would be 1 Volta DGX-1 (8x V100s) compared to 2 TPU modules (8x TPU2 chips). Volta DGX-1: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/dgx-1/data-center-products-dgx-1-components-843-u1.jpg https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... TPU Module: https://storage.googleapis.com/gweb-uniblog-publish-prod/images/tpu-V2-hero.width-1000.png https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... And for completeness, this is the size of a single V100: https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446705711_76ab786243_o.jpg https://cdn.arstechnica.net/wp-content/uploads/2017/05/34446... You can see that 8x V100s are still more computationally dense than 8x TPU2 chips. Density is a very important factor in datacenter design.