All comes down to the cost. We can see that the TPUs were about 1/2 as much as using Nvidia before the TPU 3.0. This is based on doing the same task with the

All comes down to the cost. We can see that the TPUs were about 1/2 as much as using Nvidia before the TPU 3.0. This is based on doing the same task with the TPUs versus Nvidia with AWS. But it might be a bigger difference and Google taking margins. Could have been the other way but think WaveNet proves not that direction. Would hope we get an even bigger price difference with this generation. BTW, you are mixing up graphic generations and ML generations. Google never did or will graphics. For ML Google is 2 generations ahead. Hard to imagine Nvidia ever catching up as so much of the AI breakthroughs are coming from Google. A perfect example is WaveNet. Google is using a NN for audio in real time at 16k cycles a second. That is competing with using the old way of doing this. The Google approach gets a better result but people are only willing to pay too much more for a better result. So Google had to price WaveNet competitively to the old approach which they have. I suspect that would have been impossible using Nvidia. The cost in running would have been just too high. The computation required for WaveNet is huge and the old way it is minimal. Also we are going to need to really separate training with inference. They have different requirements and the costs in running are different. Has Nvidia done an inference only solution? Or do they still always mix them? "Google would have made a CPU to compete with Intel. " I do a lot of surfing and have to say one of the more silly things to read. There is little gains today with CPUs. Why on earth would you invest in doing your own? Google has ported all their stuff to Power so they have a backup and not reliant on Intel. But CPUs are NOT the future and would be crazy to invest into them at this point. Processing is moving to TPU type processors and we will see more and more traditional things. A fantastic paper on this from Jeff Dean at Google I suggest you read. https://arxiv.org/abs/1712.01208 https://arxiv.org/abs/1712.01208 One thing I love about Google is they do NOT reinvent the wheel. If there is something that can work like the Linux kernel they use. Versus having an ego of having to do yourself. They did all their own network silicon by hiring the Lanai team several years ago because there was nothing that could work for their needs. They then created a determinate network stack so they could do Spanner. There was nothing off the shelve. Heck they laid their own fiber under the ocean to make possible. Spanner is the first and only horizontally scalable RDBMS solution to exist. Google beat the speed of light by using their custom silicon, custom stack that is determinate, their own fiber. Then using atomic clocks with GPS to remove the latency from speed of light out of the equation. Highly recommend the papers including the Spanner paper. https://ai.google/research/pubs/pub39966 https://ai.google/research/pubs/pub39966 The network uses custom silicon inside and then they put the intelligence on out edge. Then they control exactly the traffic so they get a determinate result. This also allows them to use far cheaper hardware as do not need to over provision as they never drop packets on the ground or need much memory for buffers. Google would have used Nvidia in a second if they could do what they need. But what is being done in AI (ML) today requires unique silicon. Just not possible without as the power footprint would be prohibitive. I do expect Google more and more to leverage on client ML where needed with their PVC. So it is NOT just the TPUs but for somethings you are going to need on client. Take the new voices as an example. They are doing 6 voices as each has a different model. The cost in switching models is prohibitive in the cloud. So you will see them eventually move to some things being done on the client. But they are so far ahead they have tons of time. I would not be surprised if we do not seen anyone else do Wavenet for a couple of years. What Nvidia should be working on is finding a way to do WaveNet at a reasonable cost for Amazon and others. But the long term is the big cloud providers will do their own silicon. Google set the path with the TPUs and now the others will have to do the same to keep up. But just makes sense as the cloud providers have the data to iterate. MS is using a FPGA which was foolish, IMO. Never going to get there with that route. BTW, you need to realize how this works for inference. You keep the model in memory. Google invented Capsule networks that use dynamic routing and are going to cause different patterns in memory access and would be curious if the TPU 3.0 were optimized for such? They have a huge advantage of inventing the algorithms and then get to make their own silicon optimized. Something Nvidia really needs to be able to better compete, IMO. BTW2, Google using Power did include the excellent Nvidia memory interconnect as part of the spec. Not sure did Google license or does Nvidia give away? The spec is called Open Power if not aware? https://www.theregister.co.uk/2016/04/07/open_power_summit_power9/ https://www.theregister.co.uk/2016/04/07/open_power_summit_p...