4 ms·

Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have

Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive advantage, then why not sell it to other hosting providers, or hell, even directly to ML scientists and enthusiasts? They'll finally get economies of scale, and take business (and mind share) away from NVidia's monopoly.

totoglazer 3y ago
There’s demand in GCP for H100s so they offer them. I doubt Google itself is a big user.
jsnell 3y ago
There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better. And isn't your second paragraph the obvious reason for why this product (A3) exists? It's something they expect to sell to cloud customers who have an existing GPU-based workflow, and just want to run it as-is as fast/cheap/scalable as possible, without worrying about compatibility, and making sure they can always move the workload to some other cloud provider or on-prem if needed. It's like suggesting Sony releasing some of their games on the PC means they're deprecating Playstation. (Maybe there would be more details in the IO talk. Does anyone know which one this announcement is from?)
- joseph_grobbles 3y ago
  >There's nothing in the article suggesting they've changed their strategy with TPUs in any way Google owns and designs their own TPUs. They offer these TPUs in the cloud. I've seen many comments in here about how next-level TPUs are (despite zero evidence indicating that). Google even disclaims their TPU by saying that you shouldn't compare it with the H100 given node levels et al. Their premiere offering is an nvidia H100 offering. Yes, of course this is a pretty telling indication. If Google was all in on TPUs they'd be building mega TPU systems and pushing those. Instead they're pushing nvidia AI offerings.
- onion2k 3y ago
  The word TPU isn't even mentioned here. A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either.
  - Xenomto 3y ago
    That's not true. Google is huge. Just a few H100 doesn't represent anything huge in Google scale. I also tried to find your analogy in that article and google announcement and it's not there.
  - PragmaticPulp 3y ago
    > A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either Where are you reading that Google is going “all in” on nVidia GPUs? I don’t see that in the linked article at all. These are clearly targeted at their cloud customers who have workloads tailored to GPUs. They’re supplying demand, as cloud providers do. Companies can do more than thing at a time.
  - amf12 3y ago
    > A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. TBF, there is no mention of anything remotely similar to "I am going to eat nothing but vegetables from now on".
  - jsnell 3y ago
    Sure. That's why I mentioned multiple ways in which the article could have been relevant to TPUs, which you chose not to quote. But it didn't have any of those either. The sentence you're offering up as a demonstration is just something you made up that does not appear in the article. If anything, this just reinforces the point I was making. There is nothing at all in the article supporting this narrative. So, where is this coming from? Why are you so intent on this idea that you're reduced to fabricating support for it?
- foobiekr 3y ago
  Companies never announce change of direction like you seem to think. There is no positive outcome in doing so. Instead they announce the new thing and promise to continue to support the old thing and then just don’t.
deleted 3y ago
[deleted]
impulser_ 3y ago
This is for GCP. Google themselves probably still trains on custom hardware but they don't offer their latest and greatest hardware on GCP. Offering more options to customers is always better especially when Nvidia has great market share in this area. This is probably the reason why Microsoft is trying to help AMD catch up so their is more competition. AI GPU prices are insane compared to standard GPU because of the lack of competition.
- jocaal 3y ago
  I haven't heard anything about microsoft helping AMD, it sounds interesting. Do you mind linking an article?
  - avrionov 3y ago
    It doesn't seems to be true. There were articles that Microsoft was helping AMD, but the denied it. https://arstechnica.com/gadgets/2023/05/microsoft-and-amd-are-allegedly-teaming-up-to-combat-nvidias-ai-dominance/ https://arstechnica.com/gadgets/2023/05/microsoft-and-amd-ar...
m3kw9 3y ago
If they are selling gpu compute, nobody wants to use a Google TPU, they want cuda
- cma 3y ago
  And they want to support people migrating from other cloud providers where they are already using nvidia/Cuda. Though it also helps support the opposite migration, they are the smaller cloud player trying to get customers, not the big one trying to constrain them as much yet.
mathisfun123 3y ago
>Does this mean Google is giving up on TPUs? https://arxiv.org/abs/2304.01433 https://arxiv.org/abs/2304.01433 from April 4 of this year. > I wonder why they didn't try to polish the rough edges with Pytorch, et al. It's always funny to me when people have this blindspot - because TPUs aren't for you, they're for the ads org. Neither are PyTorch nor TF for that matter. They're more than happy to get external bug fixers but trust me those individual teams dgaf about external customers. They're not in the least bit community driven projects.
jeffbee 3y ago
This is thinking about the issue all wrong. Google's internal infrastructure is terrifyingly large. They won't "get scale" by selling TPUs. That would expand the scale of TPUs only slightly.
joshlk 3y ago
> subtle incompatibility with TPUs that requires modifying the training/eval scripts Do you have any more details or links to articles expanding on this?

totoglazer 3y ago

There’s demand in GCP for H100s so they offer them. I doubt Google itself is a big user.

jsnell 3y ago

There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better. And isn't your second paragraph the obvious reason for why this product (A3) exists? It's something they expect to sell to cloud customers who have an existing GPU-based workflow, and just want to run it as-is as fast/cheap/scalable as possible, without worrying about compatibility, and making sure they can always move the workload to some other cloud provider or on-prem if needed. It's like suggesting Sony releasing some of their games on the PC means they're deprecating Playstation. (Maybe there would be more details in the IO talk. Does anyone know which one this announcement is from?)

joseph_grobbles 3y ago
>There's nothing in the article suggesting they've changed their strategy with TPUs in any way Google owns and designs their own TPUs. They offer these TPUs in the cloud. I've seen many comments in here about how next-level TPUs are (despite zero evidence indicating that). Google even disclaims their TPU by saying that you shouldn't compare it with the H100 given node levels et al. Their premiere offering is an nvidia H100 offering. Yes, of course this is a pretty telling indication. If Google was all in on TPUs they'd be building mega TPU systems and pushing those. Instead they're pushing nvidia AI offerings.
onion2k 3y ago
The word TPU isn't even mentioned here. A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either.
- Xenomto 3y ago
  That's not true. Google is huge. Just a few H100 doesn't represent anything huge in Google scale. I also tried to find your analogy in that article and google announcement and it's not there.
- PragmaticPulp 3y ago
  > A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either Where are you reading that Google is going “all in” on nVidia GPUs? I don’t see that in the linked article at all. These are clearly targeted at their cloud customers who have workloads tailored to GPUs. They’re supplying demand, as cloud providers do. Companies can do more than thing at a time.
- amf12 3y ago
  > A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. TBF, there is no mention of anything remotely similar to "I am going to eat nothing but vegetables from now on".
- jsnell 3y ago
  Sure. That's why I mentioned multiple ways in which the article could have been relevant to TPUs, which you chose not to quote. But it didn't have any of those either. The sentence you're offering up as a demonstration is just something you made up that does not appear in the article. If anything, this just reinforces the point I was making. There is nothing at all in the article supporting this narrative. So, where is this coming from? Why are you so intent on this idea that you're reduced to fabricating support for it?
foobiekr 3y ago
Companies never announce change of direction like you seem to think. There is no positive outcome in doing so. Instead they announce the new thing and promise to continue to support the old thing and then just don’t.

deleted 3y ago

[deleted]

impulser_ 3y ago

This is for GCP. Google themselves probably still trains on custom hardware but they don't offer their latest and greatest hardware on GCP. Offering more options to customers is always better especially when Nvidia has great market share in this area. This is probably the reason why Microsoft is trying to help AMD catch up so their is more competition. AI GPU prices are insane compared to standard GPU because of the lack of competition.

jocaal 3y ago
I haven't heard anything about microsoft helping AMD, it sounds interesting. Do you mind linking an article?
- avrionov 3y ago
  It doesn't seems to be true. There were articles that Microsoft was helping AMD, but the denied it. https://arstechnica.com/gadgets/2023/05/microsoft-and-amd-are-allegedly-teaming-up-to-combat-nvidias-ai-dominance/ https://arstechnica.com/gadgets/2023/05/microsoft-and-amd-ar...

m3kw9 3y ago

If they are selling gpu compute, nobody wants to use a Google TPU, they want cuda

cma 3y ago
And they want to support people migrating from other cloud providers where they are already using nvidia/Cuda. Though it also helps support the opposite migration, they are the smaller cloud player trying to get customers, not the big one trying to constrain them as much yet.

mathisfun123 3y ago

>Does this mean Google is giving up on TPUs? https://arxiv.org/abs/2304.01433 https://arxiv.org/abs/2304.01433 from April 4 of this year. > I wonder why they didn't try to polish the rough edges with Pytorch, et al. It's always funny to me when people have this blindspot - because TPUs aren't for you, they're for the ads org. Neither are PyTorch nor TF for that matter. They're more than happy to get external bug fixers but trust me those individual teams dgaf about external customers. They're not in the least bit community driven projects.

jeffbee 3y ago

This is thinking about the issue all wrong. Google's internal infrastructure is terrifyingly large. They won't "get scale" by selling TPUs. That would expand the scale of TPUs only slightly.

joshlk 3y ago

> subtle incompatibility with TPUs that requires modifying the training/eval scripts Do you have any more details or links to articles expanding on this?