4 ms·
My assumption is based on this: For a long time, Microsoft's core business was Windows and (offline) Office. You don't need a super cluster for that. Google on
by albertzeyer 4y ago
My assumption is based on this:
For a long time, Microsoft's core business was Windows and (offline) Office. You don't need a super cluster for that. Google on the other side always had search as a service, available to the public, which needs more compute. So historically, Google needed more compute than Microsoft.
Then, from experience, as I do research on neural networks, read papers in the field, Google/DeepMind were always more active in this field than Microsoft, so again, historically, they needed more computing power because of this. Microsoft has catched up, but still, I think Google/DeepMind is more active in the field.
Then, Google has TPUs. I think this gives them computing power much more cost effectively than when you need to buy Nvidia GPUs. This is maybe the biggest argument.
But in any case, as you say, both companies are big enough to easily train sth like ChatGPT, or also sth like GPT4, however big it might be. Their computing power is almost infinite (for today's standards), so it doesn't really matter who has more.