4 ms·

It's crazy that Google doesn't spin-out their TPU work as a separate company. TPUs are the second most widely used environment for training after Nvidia. It's

by nl 2y ago

It's crazy that Google doesn't spin-out their TPU work as a separate company. TPUs are the second most widely used environment for training after Nvidia. It's the only environment that people build optimized kernels for outside CUDA. If it was separate to Google then there a bunch of companies who would happily spend some money on a real, working NVidia alternative. It might be profitable from day one, and it surely would gain substantial market capitalization - Alphabet shareholders should be agitating for this!

qwertox 2y ago
There would probably a huge demand, but would Google be able to satisfy it? Is it currently able to satisfy its own demand?
- credit_guy 2y ago
  That would be the point of spinning it out. They could have an IPO, raise as much capital as there is in the observable Universe, and build enough fabs to satisfy all the demand.
  - mike_hearn 2y ago
    That wouldn't work. Even TPUv4 was on a 7nm node and you don't just build a 7nm fab just like that. If it were that easy NVIDIA would already be building their own fabs, as they have basically raised as much capital as there is in the known universe (bigger market cap than the entire London stock exchange), but they seem to prefer to let the fab experts get on with it rather than compete with them. LLM AI is largely HBM bottlenecked anyway i.e. Samsung, SK Hynix and Micron are where the supply chain limits enter the picture.
    - throwaway48476 2y ago
      Fabless companies that are large enough such as apple front the capital for fab companies like TSMC to build fabs dedicated to their use. They do, in effect, build their own fabs. If the Google TPU group had the inclination they could have done the same. The memory industry just got busted from the covid bubble and are not too keen to jump into the AI bubble.
      - ragebol 2y ago
        They might front the money, but don't own them. Apple gladly lets someone else own and operate the fabs and take the risk (which is smaller with Apple as a client)
        nsteel 2y ago
        Let's not forget that a 7nm fab has a very limited period of usefulness for the likes of Apple etc. That leading edge is always moving forward and while it might be financially viable for some aspects of the process to be upgraded to the next node, that's not always the case and that's where TSMC's hundreds of other customers join in and the (now old) equipment can be still used for many more years. Edit: But perhaps with the exclusivity deals, the likes of TSMC are less reliant on spreading the cost over 15+ years than they used to be. To be clear, I was talking about long-term use.
        throwaway48476 2y ago
        The leading edge has slowed down a lot. Apple is still selling M1 chips and AMD is just now releasing new models of zen3 AM4 chips.
        throwaway48476 2y ago
        There is more co development and risk sharing than you think. TSMC has nodes only apple uses.
        nsteel 2y ago
        They do, but as far as I know those nodes are just early/late verisons and tweaks to the main, popular process.
        michaelt 2y ago
        But if Apple pays TSMC $$$$$$$$ in advance to build a 2nm node production line especially for Apple, and it turns out the 2nm node doesn't deliver the hoped-for improvements in power efficiency? The money's already spent.
    - rpeden 2y ago
      Unless they've been issuing a ton of new shares recently and then selling them into the market at something resembling the current share price, the amount of capital they've raised is nowhere near their current market cap. But it looks like they've actually been buying back some shares - they've got fewer shares outstanding than they did a year or two ago. Not that it matters much - they've still got plenty of cash and other capital available.
  - bearjaws 2y ago
    There seems to be this idea that the people who design and operate fabs are infinite, when it's actually a technically demanding job. We don't even have enough McDonald's employees, how the hell are we going to just suddenly have multiple companies creating fabs left and right? TSMC cannot even build their Arizona plant without a shortage of workers.
    - Arainach 2y ago
      Every time someone says "we don't have enough employees", remember to add "....at the (almost certainly too low) wage being offered".
      - bluecalm 2y ago
        Maybe but then where are those CPU fab experts working right now that offer them higher wage?
        michaelt 2y ago
        Writing computer software, mostly.
        ahefner 2y ago
        China, supposedly.
  - Workaccount2 2y ago
    Intel has been trying to make cutting edge fabs...and we all know how that is going. There is good reason nobody wants to be in the fab business.
jankeymeulen 2y ago
The TPUs are highly integrated with the rest of the internal Google ecosystem, both hardware and software. Untangling that would be ... interesting.
- hengheng 2y ago
  Knowing what I know about big corporations, the biggest entanglement is going to be IP ownership, political constraints and promises to shareholders.
- michaelt 2y ago
  We have a perfectly reasonable blueprint for an ML accelerator that isn't tied into the google ecosystem: nvidia's entire product line. Between that and the fact Google already sells "Coral Edge TPUs" [1] I'd think they could manage to untangle things. Whether the employees would want to be spun off or not is a different matter, of course... [1] https://coral.ai/products/ https://coral.ai/products/
  - nine_k 2y ago
    Do you think that NVidia is happy to not have an online ecosystem to tie to its GPUs, for added (sales) value? They are more than happy to entangle the GPUs with their proprietary CUDA language. For a large, established, quasi-monopoly company it's always more attractive to keep things inside their walled gardens. Suggesting that Google should start supporting TPUs outside Google Cloud is like suggesting that Apple should start supporting iOS on non-Apple hardware.
    - michaelt 2y ago
      > Do you think that NVidia is happy to not have an online ecosystem to tie to its GPUs, for added (sales) value? I think nvidia is ecstatic about having commoditised their complement, and having the only ML acceleration option that's available from every cloud provider and on-prem. Why have Amazon, Google and Microsoft as competitors when you can have them as customers instead?
monkeydust 2y ago
Any activist investors lurking in here?
theptip 2y ago
Why would you spin out a competitive moat?
bushbaba 2y ago
> It's crazy that Google doesn't spin-out their TPU work as a separate company. Not really. Google TPUs require google's specific infrastructure, and cannot be deployed out side the Google Datacenter. The software is google specific, the monetization model is google specific. We also have no idea how profitable TPUs would actually be if a separate company. The only customer of TPUs is Google and Google Cloud.
aseipp 2y ago
People constantly bring this point up every 2 weeks here, the cost competitiveness of TPUs for Google comes exactly from the fact they make them in house and don't sell them. They don't need sales channels, support, leads, any of that stuff. They can design for exactly one software stack, one hardware stack, and one set of staff. You cannot just magically spin up a billion-dollar hardware company overnight with software, customers, sales channels and support, etc. Nvidia has spent 20 years on this which is why they're good at it. > If it was separate to Google then there a bunch of companies who would happily spend some money on a real, working NVidia alternative. Unfortunately, most people really don't care about Nvidia alternatives, actually -- they care about price, above all else. People will say they want Nvidia alternatives and support them, then go back to buying Nvidia the moment the price goes down. Which is fine, to be clear, but this is not the outcome people often allude to.
- authorfly 2y ago
  You can or at least historically could buy access to TPUs and request it for non-profit projects too through the TPU research programme. Certainly you have been able to pay for pro membership on Notebook to get TPU access, which is how many of the AI generation before ChatGPT learned to run AI. TPUs however were kind of always for training, never geared for inference.
  - aseipp 2y ago
    That is correct, and I should have been more, clear: when I say "Buy them" I mean direct sales of the hardware from seller to buyer. I am not referring to cloud compute style sales. Yes, they have been offering TPUs through Google Cloud for a long while now, but this still falls under all the stuff I said earlier: they don't need to have sales pipelines or channels (outside GCloud's existing ones), they don't need to design the hardware/software for arbitrary environments, they have one set of staff and machines to train and support, etc. All of that stuff costs money and ultimately it results in an entirely different sales and financial model. Google could spin the TPU division out of Google, but 99% of the time people refer to moves like that they omit the implied follow up sentence which is "I can then buy a TPU with my credit card off the shelf from a website that uses Stripe." It is just not that simple or easy.

by nl 2y ago

qwertox 2y ago

There would probably a huge demand, but would Google be able to satisfy it? Is it currently able to satisfy its own demand?

credit_guy 2y ago
That would be the point of spinning it out. They could have an IPO, raise as much capital as there is in the observable Universe, and build enough fabs to satisfy all the demand.
- mike_hearn 2y ago
  That wouldn't work. Even TPUv4 was on a 7nm node and you don't just build a 7nm fab just like that. If it were that easy NVIDIA would already be building their own fabs, as they have basically raised as much capital as there is in the known universe (bigger market cap than the entire London stock exchange), but they seem to prefer to let the fab experts get on with it rather than compete with them. LLM AI is largely HBM bottlenecked anyway i.e. Samsung, SK Hynix and Micron are where the supply chain limits enter the picture.
  - throwaway48476 2y ago
    Fabless companies that are large enough such as apple front the capital for fab companies like TSMC to build fabs dedicated to their use. They do, in effect, build their own fabs. If the Google TPU group had the inclination they could have done the same. The memory industry just got busted from the covid bubble and are not too keen to jump into the AI bubble.
    - ragebol 2y ago
      They might front the money, but don't own them. Apple gladly lets someone else own and operate the fabs and take the risk (which is smaller with Apple as a client)
      - nsteel 2y ago
        Let's not forget that a 7nm fab has a very limited period of usefulness for the likes of Apple etc. That leading edge is always moving forward and while it might be financially viable for some aspects of the process to be upgraded to the next node, that's not always the case and that's where TSMC's hundreds of other customers join in and the (now old) equipment can be still used for many more years. Edit: But perhaps with the exclusivity deals, the likes of TSMC are less reliant on spreading the cost over 15+ years than they used to be. To be clear, I was talking about long-term use.
        throwaway48476 2y ago
        The leading edge has slowed down a lot. Apple is still selling M1 chips and AMD is just now releasing new models of zen3 AM4 chips.
      - throwaway48476 2y ago
        There is more co development and risk sharing than you think. TSMC has nodes only apple uses.
        nsteel 2y ago
        They do, but as far as I know those nodes are just early/late verisons and tweaks to the main, popular process.
        michaelt 2y ago
        But if Apple pays TSMC $$$$$$$$ in advance to build a 2nm node production line especially for Apple, and it turns out the 2nm node doesn't deliver the hoped-for improvements in power efficiency? The money's already spent.
  - rpeden 2y ago
    Unless they've been issuing a ton of new shares recently and then selling them into the market at something resembling the current share price, the amount of capital they've raised is nowhere near their current market cap. But it looks like they've actually been buying back some shares - they've got fewer shares outstanding than they did a year or two ago. Not that it matters much - they've still got plenty of cash and other capital available.
- bearjaws 2y ago
  There seems to be this idea that the people who design and operate fabs are infinite, when it's actually a technically demanding job. We don't even have enough McDonald's employees, how the hell are we going to just suddenly have multiple companies creating fabs left and right? TSMC cannot even build their Arizona plant without a shortage of workers.
  - Arainach 2y ago
    Every time someone says "we don't have enough employees", remember to add "....at the (almost certainly too low) wage being offered".
    - bluecalm 2y ago
      Maybe but then where are those CPU fab experts working right now that offer them higher wage?
      - michaelt 2y ago
        Writing computer software, mostly.
      - ahefner 2y ago
        China, supposedly.
- Workaccount2 2y ago
  Intel has been trying to make cutting edge fabs...and we all know how that is going. There is good reason nobody wants to be in the fab business.

jankeymeulen 2y ago

The TPUs are highly integrated with the rest of the internal Google ecosystem, both hardware and software. Untangling that would be ... interesting.

hengheng 2y ago
Knowing what I know about big corporations, the biggest entanglement is going to be IP ownership, political constraints and promises to shareholders.
michaelt 2y ago
We have a perfectly reasonable blueprint for an ML accelerator that isn't tied into the google ecosystem: nvidia's entire product line. Between that and the fact Google already sells "Coral Edge TPUs" [1] I'd think they could manage to untangle things. Whether the employees would want to be spun off or not is a different matter, of course... [1] https://coral.ai/products/ https://coral.ai/products/
- nine_k 2y ago
  Do you think that NVidia is happy to not have an online ecosystem to tie to its GPUs, for added (sales) value? They are more than happy to entangle the GPUs with their proprietary CUDA language. For a large, established, quasi-monopoly company it's always more attractive to keep things inside their walled gardens. Suggesting that Google should start supporting TPUs outside Google Cloud is like suggesting that Apple should start supporting iOS on non-Apple hardware.
  - michaelt 2y ago
    > Do you think that NVidia is happy to not have an online ecosystem to tie to its GPUs, for added (sales) value? I think nvidia is ecstatic about having commoditised their complement, and having the only ML acceleration option that's available from every cloud provider and on-prem. Why have Amazon, Google and Microsoft as competitors when you can have them as customers instead?

monkeydust 2y ago

Any activist investors lurking in here?

theptip 2y ago

Why would you spin out a competitive moat?

bushbaba 2y ago

> It's crazy that Google doesn't spin-out their TPU work as a separate company. Not really. Google TPUs require google's specific infrastructure, and cannot be deployed out side the Google Datacenter. The software is google specific, the monetization model is google specific. We also have no idea how profitable TPUs would actually be if a separate company. The only customer of TPUs is Google and Google Cloud.

aseipp 2y ago

People constantly bring this point up every 2 weeks here, the cost competitiveness of TPUs for Google comes exactly from the fact they make them in house and don't sell them. They don't need sales channels, support, leads, any of that stuff. They can design for exactly one software stack, one hardware stack, and one set of staff. You cannot just magically spin up a billion-dollar hardware company overnight with software, customers, sales channels and support, etc. Nvidia has spent 20 years on this which is why they're good at it. > If it was separate to Google then there a bunch of companies who would happily spend some money on a real, working NVidia alternative. Unfortunately, most people really don't care about Nvidia alternatives, actually -- they care about price, above all else. People will say they want Nvidia alternatives and support them, then go back to buying Nvidia the moment the price goes down. Which is fine, to be clear, but this is not the outcome people often allude to.

authorfly 2y ago
You can or at least historically could buy access to TPUs and request it for non-profit projects too through the TPU research programme. Certainly you have been able to pay for pro membership on Notebook to get TPU access, which is how many of the AI generation before ChatGPT learned to run AI. TPUs however were kind of always for training, never geared for inference.
- aseipp 2y ago
  That is correct, and I should have been more, clear: when I say "Buy them" I mean direct sales of the hardware from seller to buyer. I am not referring to cloud compute style sales. Yes, they have been offering TPUs through Google Cloud for a long while now, but this still falls under all the stuff I said earlier: they don't need to have sales pipelines or channels (outside GCloud's existing ones), they don't need to design the hardware/software for arbitrary environments, they have one set of staff and machines to train and support, etc. All of that stuff costs money and ultimately it results in an entirely different sales and financial model. Google could spin the TPU division out of Google, but 99% of the time people refer to moves like that they omit the implied follow up sentence which is "I can then buy a TPU with my credit card off the shelf from a website that uses Stripe." It is just not that simple or easy.