4 ms·

Half the cost? Where are you reading that? Yeah on demand rental in AWS is expensive, but both long term and buying V100 yourself is significantly cheaper. Clou

by nojvek 9y ago

Half the cost? Where are you reading that? Yeah on demand rental in AWS is expensive, but both long term and buying V100 yourself is significantly cheaper. Cloud companies have pretty fat margins on on demand rentals. You can’t buy a TPU, it’s a cloud only thing. They also show it’s not a huge difference in both perf and time to converge (albeit only one architecture) I would say kudos to V100 and this benchmark that breaks the TPU hype.

jacksmith21006 9y ago
The chart has 6.7 per hour for 3186 images Google and 12.2 per hour for 3128 AWS. Or maybe reading it wrong? That is close to half has much to use Google is it not? BTW, The TPUs are also about twice as fast also. Sounds like Google is pretty far ahead of Nvidia. Which really just makes sense as Google does the entire stack and just going to have the data to optimize the silicon. About half the cost is hype? I want in the cloud and not have to deal with updating, etc. Would think most are the same for anything of any scale. Could not imagine any longer building up rigs and dealing with all the issues. Plus much harder to scale.
- nightski 9y ago
  It's more a comparison of AWS vs. Google Cloud pricing than Nvidia vs. TPUv2.
  - mtgx 9y ago
    If anything, the pricing likely benefits Google. As in Google may be more profitable with the TPU usage, even at 1/2 the cost of Amazon's V100 usage.
  - jacksmith21006 9y ago
    Strongly disagree. If Google is able to offer at about 1/2 the cost using their own silicon versus AWS using Nvidia that is all about the silicon difference. But we also have the V1 TPU paper and can see the TPUs are able to use less joules per inference compared to an older Nvidia architecture. Was not that close. Just makes sense Google V2 TPUs would do the same. Hope Google does a V3 TPU and then will share a V2 TPU paper like they did on V1 of the TPUs. What is far more impressive of the TPUs is https://cloudplatform.googleblog.com/2018/03/introducing-Cloud-Text-to-Speech-powered-by-Deepmind-WaveNet-technology.html https://cloudplatform.googleblog.com/2018/03/introducing-Clo... If really doing 16k a second through a NN and at a price you can offer generally now that is incredible. I want this paper even more so.
    - shaklee3 9y ago
      Maybe, maybe not. They have the advantage that they make the hardware, so they're not paying as much retail as nvidia is charging them for their cards. I don't think there's any way you can say the TPU is cheaper compared to buying your own system. If Google decides to release it to the public, that's a different story. Also, keep in mind that Google allows you to mix and match the CPU core count to GPU, whereas AWS doesn't. It's possible that the Google cloud price with fewer CPU cores will be much cheaper than the AWS instance.
      - jacksmith21006 9y ago
        That is true. But the cost of running is where all the cost is at really not so much in making the chips. Yes I can say it is a lot cheaper. That is what this article is all about. You can do about twice the images per dollar using the TPUs with GCP versus using Nvidia with AWS. Or what am I missing? BTW, Google has released to the general public. What are you talking about? "Google’s AI chips are now open for public use" https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-open-for-public-use/ https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-...
        shaklee3 9y ago
        You misunderstood. They released them to the public on GCP only. Nvidia's cards are released to the public as a hardware device that you can customize around. Big difference.
        jacksmith21006 9y ago
        Yes in the cloud as you would expect in 2018. Available to the general public.
        shaklee3 9y ago
        They announced in 2016 they had TPUs. So no, I would not expect that 2 full years later they're just now being available in the public cloud. These are not new products to them; they likely just don't want to deal with supporting them in different configurations.
- deepnotderp 9y ago
  fwiw, the "TPU instance " has more than one tpu chip on it.
  - ndesaulniers 9y ago
    The architectures are so radically different that I don't think it makes sense to try to compare anything but the whole system performance. Trying to do a 1 to 1 comparison for a core or a chip becomes pretty nebulous because the architectures are radically different.
  - puzzle 9y ago
    It has more than the chips, too, since the TPUs can't run a TCP/IP stack, gRPC server, etc.
deleted 9y ago
[deleted]
ndesaulniers 9y ago
See the chart titled: Performance in images per second per $. TPUv2 is has 1.27x-1.86x the images/s/$. And the other chart titled: Cost to reach 75.7% top-1 accuracy. Where TPUv2 costs 62.5% the reserved GPU instance and 42.6% the unreserved GPU cost. Key takeaway from the article: > While the V100s perform similarly fast, the higher price and slower convergence of the implementation results in a considerably higher cost-to-solution.

by nojvek 9y ago

jacksmith21006 9y ago

The chart has 6.7 per hour for 3186 images Google and 12.2 per hour for 3128 AWS. Or maybe reading it wrong? That is close to half has much to use Google is it not? BTW, The TPUs are also about twice as fast also. Sounds like Google is pretty far ahead of Nvidia. Which really just makes sense as Google does the entire stack and just going to have the data to optimize the silicon. About half the cost is hype? I want in the cloud and not have to deal with updating, etc. Would think most are the same for anything of any scale. Could not imagine any longer building up rigs and dealing with all the issues. Plus much harder to scale.

nightski 9y ago
It's more a comparison of AWS vs. Google Cloud pricing than Nvidia vs. TPUv2.
- mtgx 9y ago
  If anything, the pricing likely benefits Google. As in Google may be more profitable with the TPU usage, even at 1/2 the cost of Amazon's V100 usage.
- jacksmith21006 9y ago
  Strongly disagree. If Google is able to offer at about 1/2 the cost using their own silicon versus AWS using Nvidia that is all about the silicon difference. But we also have the V1 TPU paper and can see the TPUs are able to use less joules per inference compared to an older Nvidia architecture. Was not that close. Just makes sense Google V2 TPUs would do the same. Hope Google does a V3 TPU and then will share a V2 TPU paper like they did on V1 of the TPUs. What is far more impressive of the TPUs is https://cloudplatform.googleblog.com/2018/03/introducing-Cloud-Text-to-Speech-powered-by-Deepmind-WaveNet-technology.html https://cloudplatform.googleblog.com/2018/03/introducing-Clo... If really doing 16k a second through a NN and at a price you can offer generally now that is incredible. I want this paper even more so.
  - shaklee3 9y ago
    Maybe, maybe not. They have the advantage that they make the hardware, so they're not paying as much retail as nvidia is charging them for their cards. I don't think there's any way you can say the TPU is cheaper compared to buying your own system. If Google decides to release it to the public, that's a different story. Also, keep in mind that Google allows you to mix and match the CPU core count to GPU, whereas AWS doesn't. It's possible that the Google cloud price with fewer CPU cores will be much cheaper than the AWS instance.
    - jacksmith21006 9y ago
      That is true. But the cost of running is where all the cost is at really not so much in making the chips. Yes I can say it is a lot cheaper. That is what this article is all about. You can do about twice the images per dollar using the TPUs with GCP versus using Nvidia with AWS. Or what am I missing? BTW, Google has released to the general public. What are you talking about? "Google’s AI chips are now open for public use" https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-open-for-public-use/ https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-...
      - shaklee3 9y ago
        You misunderstood. They released them to the public on GCP only. Nvidia's cards are released to the public as a hardware device that you can customize around. Big difference.
        jacksmith21006 9y ago
        Yes in the cloud as you would expect in 2018. Available to the general public.
        shaklee3 9y ago
        They announced in 2016 they had TPUs. So no, I would not expect that 2 full years later they're just now being available in the public cloud. These are not new products to them; they likely just don't want to deal with supporting them in different configurations.
deepnotderp 9y ago
fwiw, the "TPU instance " has more than one tpu chip on it.
- ndesaulniers 9y ago
  The architectures are so radically different that I don't think it makes sense to try to compare anything but the whole system performance. Trying to do a 1 to 1 comparison for a core or a chip becomes pretty nebulous because the architectures are radically different.
- puzzle 9y ago
  It has more than the chips, too, since the TPUs can't run a TCP/IP stack, gRPC server, etc.

deleted 9y ago

[deleted]

ndesaulniers 9y ago

See the chart titled: Performance in images per second per $. TPUv2 is has 1.27x-1.86x the images/s/$. And the other chart titled: Cost to reach 75.7% top-1 accuracy. Where TPUv2 costs 62.5% the reserved GPU instance and 42.6% the unreserved GPU cost. Key takeaway from the article: > While the V100s perform similarly fast, the higher price and slower convergence of the implementation results in a considerably higher cost-to-solution.