3 ms·

Strongly disagree. If Google is able to offer at about 1/2 the cost using their own silicon versus AWS using Nvidia that is all about the silicon difference.

Strongly disagree. If Google is able to offer at about 1/2 the cost using their own silicon versus AWS using Nvidia that is all about the silicon difference. But we also have the V1 TPU paper and can see the TPUs are able to use less joules per inference compared to an older Nvidia architecture. Was not that close. Just makes sense Google V2 TPUs would do the same. Hope Google does a V3 TPU and then will share a V2 TPU paper like they did on V1 of the TPUs. What is far more impressive of the TPUs is https://cloudplatform.googleblog.com/2018/03/introducing-Cloud-Text-to-Speech-powered-by-Deepmind-WaveNet-technology.html https://cloudplatform.googleblog.com/2018/03/introducing-Clo... If really doing 16k a second through a NN and at a price you can offer generally now that is incredible. I want this paper even more so.

shaklee3 8y ago
Maybe, maybe not. They have the advantage that they make the hardware, so they're not paying as much retail as nvidia is charging them for their cards. I don't think there's any way you can say the TPU is cheaper compared to buying your own system. If Google decides to release it to the public, that's a different story. Also, keep in mind that Google allows you to mix and match the CPU core count to GPU, whereas AWS doesn't. It's possible that the Google cloud price with fewer CPU cores will be much cheaper than the AWS instance.
- jacksmith21006 8y ago
  That is true. But the cost of running is where all the cost is at really not so much in making the chips. Yes I can say it is a lot cheaper. That is what this article is all about. You can do about twice the images per dollar using the TPUs with GCP versus using Nvidia with AWS. Or what am I missing? BTW, Google has released to the general public. What are you talking about? "Google’s AI chips are now open for public use" https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-open-for-public-use/ https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-...
  - shaklee3 8y ago
    You misunderstood. They released them to the public on GCP only. Nvidia's cards are released to the public as a hardware device that you can customize around. Big difference.
    - jacksmith21006 8y ago
      Yes in the cloud as you would expect in 2018. Available to the general public.
      - shaklee3 8y ago
        They announced in 2016 they had TPUs. So no, I would not expect that 2 full years later they're just now being available in the public cloud. These are not new products to them; they likely just don't want to deal with supporting them in different configurations.
        ndesaulniers 8y ago
        > they likely just don't want to deal with supporting them in different configurations. It is a lot of work. But mainly that TPUv1 only did inference while TPUv2 does training+inference.
        shaklee3 8y ago
        Exactly my point. It's a lot of work. That's the reason why Nvidia has such a large team doing it, and also why they spent 3 billion dollars to build the V100 ASIC.
        jacksmith21006 8y ago
        Big difference is Google does the entire stack and also has scale to conceptually be able to create a better solution. But that is theoretical. Here we can see the results where the Google TPU gets almost twice result per dollar over Nvidia. But then Google should be able to iterate more quickly. Take the move from using CNN to using capsule networks. The idea for Capsules came from Hinton and Google is going to be there first to optimize in hardware. This is the benefit of playing in all layers of the stack. Or the using NN for text to speech and offering at scale. Google just has inherent advantages over Nvidia and now we get to see a little more concrete results. But hope we get a lot more similar and see if the Google advantage holds up.
nightski 8y ago
What makes you so sure it is all the silicon difference and not just AWS pricing their product at a more profitable price point? These costs also ignore transferring and storing massive data sets in the cloud. In general the cloud is a huge pain and I'd avoid it like the plague unless I was caught and really, really needed the scalability. But even then that only works if you have a scalable implementation of the algorithm you are working on.

shaklee3 8y ago

Maybe, maybe not. They have the advantage that they make the hardware, so they're not paying as much retail as nvidia is charging them for their cards. I don't think there's any way you can say the TPU is cheaper compared to buying your own system. If Google decides to release it to the public, that's a different story. Also, keep in mind that Google allows you to mix and match the CPU core count to GPU, whereas AWS doesn't. It's possible that the Google cloud price with fewer CPU cores will be much cheaper than the AWS instance.

jacksmith21006 8y ago
That is true. But the cost of running is where all the cost is at really not so much in making the chips. Yes I can say it is a lot cheaper. That is what this article is all about. You can do about twice the images per dollar using the TPUs with GCP versus using Nvidia with AWS. Or what am I missing? BTW, Google has released to the general public. What are you talking about? "Google’s AI chips are now open for public use" https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-open-for-public-use/ https://venturebeat.com/2018/02/12/googles-ai-chips-are-now-...
- shaklee3 8y ago
  You misunderstood. They released them to the public on GCP only. Nvidia's cards are released to the public as a hardware device that you can customize around. Big difference.
  - jacksmith21006 8y ago
    Yes in the cloud as you would expect in 2018. Available to the general public.
    - shaklee3 8y ago
      They announced in 2016 they had TPUs. So no, I would not expect that 2 full years later they're just now being available in the public cloud. These are not new products to them; they likely just don't want to deal with supporting them in different configurations.
      - ndesaulniers 8y ago
        > they likely just don't want to deal with supporting them in different configurations. It is a lot of work. But mainly that TPUv1 only did inference while TPUv2 does training+inference.
        shaklee3 8y ago
        Exactly my point. It's a lot of work. That's the reason why Nvidia has such a large team doing it, and also why they spent 3 billion dollars to build the V100 ASIC.
        jacksmith21006 8y ago
        Big difference is Google does the entire stack and also has scale to conceptually be able to create a better solution. But that is theoretical. Here we can see the results where the Google TPU gets almost twice result per dollar over Nvidia. But then Google should be able to iterate more quickly. Take the move from using CNN to using capsule networks. The idea for Capsules came from Hinton and Google is going to be there first to optimize in hardware. This is the benefit of playing in all layers of the stack. Or the using NN for text to speech and offering at scale. Google just has inherent advantages over Nvidia and now we get to see a little more concrete results. But hope we get a lot more similar and see if the Google advantage holds up.

nightski 8y ago

What makes you so sure it is all the silicon difference and not just AWS pricing their product at a more profitable price point? These costs also ignore transferring and storing massive data sets in the cloud. In general the cloud is a huge pain and I'd avoid it like the plague unless I was caught and really, really needed the scalability. But even then that only works if you have a scalable implementation of the algorithm you are working on.