11 ms·

I wonder when Meta, Microsoft and OpenAI will partner on an open chip design to compete with NVIDIA. They’re all blowing billions of dollars on NVIDIA hardware

by m_ke 2y ago

I wonder when Meta, Microsoft and OpenAI will partner on an open chip design to compete with NVIDIA. They’re all blowing billions of dollars on NVIDIA hardware with like 70% margin and with triton backing PyTorch it shouldn’t be that hard to move off of CUDA stack.

deleted 2y ago
[deleted]
throwaway48476 2y ago
Each of them is designing their own hardware. The goal isn't really to compete with nvidia though, whose market is general purpose GPU compute. Instead they're customizing hardware for inference to drive down product cost.
infecto 2y ago
What is a large mount of money to you, is not that significant to some of these companies. I suspect for the vast majority of these companies, it still represents a small expense. General sentiment is that there is probably overspending in the area but its better to spend it and not risk being left behind.
- m_ke 2y ago
  Half of NVIDIA's 2nd quarter revenue (30 billion) came from 4 customers, with Microsoft and Meta already having spent 40-60 billion each on GPU data centers (of which most goes to NVIDIA). "Open"AI just raised a few billion and is supposedly planning on building their own training clusters soon. For a small fraction of that they could poach a ton of people from NVIDIA and publish a new open chip spec that anyone could manufacture. https://www.fool.com/investing/2024/09/12/46-nvidias-30-billion-revenue-4-mystery-customers/ https://www.fool.com/investing/2024/09/12/46-nvidias-30-bill...
  - infecto 2y ago
    That again underestimates the challenges in that undertaking. After all these costs are still drops on the bucket. Why distract yourself from your business to go and build chips. They all use SFDC, should they go and create and open source sales platform?
    - m_ke 2y ago
      https://developers.facebook.com/blog/post/2021/09/07/eli5-open-compute-project-advancing-hardware-technologies-through-open-source/ https://developers.facebook.com/blog/post/2021/09/07/eli5-op... That's exactly what they id with their server design. I'm saying come up with an open standard for tensor processing chips, with open drivers and core compute libraries, then let hardware vendors innovate and compete to drive down the price. Meta spent like 10% of their revenue on ML hardware, it's not a drop in a bucket and with model scaling and large scale deployment these costs are not going down. https://www.datacenterdynamics.com/en/news/meta-to-operate-600000-gpus-by-year-end/ https://www.datacenterdynamics.com/en/news/meta-to-operate-6...
insane_dreamer 2y ago
I can't see Meta and MSFT getting into the chip design business. Maybe OpenAI. Apple's already not using nVidia chips to train its models.
- mhh__ 2y ago
  I think Microsoft have dabbled in the space already.
- mhandley 2y ago
  Meta is already in the chip design business, albeit in collaboration with Broadcom who build it for them: https://ai.meta.com/blog/next-generation-meta-training-inference-accelerator-AI-MTIA/ https://ai.meta.com/blog/next-generation-meta-training-infer...
mhh__ 2y ago
It would require a fairly big bet on AI models not changing structure all that much, I guess I think if the tooling and supply chain falls into place it would surprise me if meta and friends didn't make their own chips, assuming it was a good fit of course. Note: AMD's missed opportunity here is so bad people jump "make their own chip" rather than "buy AMD". Although watch that space.
- ClassyJacket 2y ago
  Aren't they basically safe as long as future AI models are still basically multiplying tensors of floats?
  - cma 2y ago
    Training uses other operations for normalizing, etc.
wmf 2y ago
https://ai.meta.com/blog/next-generation-meta-training-inference-accelerator-AI-MTIA/ https://ai.meta.com/blog/next-generation-meta-training-infer... https://azure.microsoft.com/en-us/blog/azure-maia-for-the-era-of-ai-from-silicon-to-software-to-systems/ https://azure.microsoft.com/en-us/blog/azure-maia-for-the-er... They're not teaming up and neither design is open, but they definitely have their own designs.

by m_ke 2y ago

deleted 2y ago

[deleted]

throwaway48476 2y ago

Each of them is designing their own hardware. The goal isn't really to compete with nvidia though, whose market is general purpose GPU compute. Instead they're customizing hardware for inference to drive down product cost.

infecto 2y ago

What is a large mount of money to you, is not that significant to some of these companies. I suspect for the vast majority of these companies, it still represents a small expense. General sentiment is that there is probably overspending in the area but its better to spend it and not risk being left behind.

m_ke 2y ago
Half of NVIDIA's 2nd quarter revenue (30 billion) came from 4 customers, with Microsoft and Meta already having spent 40-60 billion each on GPU data centers (of which most goes to NVIDIA). "Open"AI just raised a few billion and is supposedly planning on building their own training clusters soon. For a small fraction of that they could poach a ton of people from NVIDIA and publish a new open chip spec that anyone could manufacture. https://www.fool.com/investing/2024/09/12/46-nvidias-30-billion-revenue-4-mystery-customers/ https://www.fool.com/investing/2024/09/12/46-nvidias-30-bill...
- infecto 2y ago
  That again underestimates the challenges in that undertaking. After all these costs are still drops on the bucket. Why distract yourself from your business to go and build chips. They all use SFDC, should they go and create and open source sales platform?
  - m_ke 2y ago
    https://developers.facebook.com/blog/post/2021/09/07/eli5-open-compute-project-advancing-hardware-technologies-through-open-source/ https://developers.facebook.com/blog/post/2021/09/07/eli5-op... That's exactly what they id with their server design. I'm saying come up with an open standard for tensor processing chips, with open drivers and core compute libraries, then let hardware vendors innovate and compete to drive down the price. Meta spent like 10% of their revenue on ML hardware, it's not a drop in a bucket and with model scaling and large scale deployment these costs are not going down. https://www.datacenterdynamics.com/en/news/meta-to-operate-600000-gpus-by-year-end/ https://www.datacenterdynamics.com/en/news/meta-to-operate-6...

insane_dreamer 2y ago

I can't see Meta and MSFT getting into the chip design business. Maybe OpenAI. Apple's already not using nVidia chips to train its models.

mhh__ 2y ago
I think Microsoft have dabbled in the space already.
mhandley 2y ago
Meta is already in the chip design business, albeit in collaboration with Broadcom who build it for them: https://ai.meta.com/blog/next-generation-meta-training-inference-accelerator-AI-MTIA/ https://ai.meta.com/blog/next-generation-meta-training-infer...

mhh__ 2y ago

It would require a fairly big bet on AI models not changing structure all that much, I guess I think if the tooling and supply chain falls into place it would surprise me if meta and friends didn't make their own chips, assuming it was a good fit of course. Note: AMD's missed opportunity here is so bad people jump "make their own chip" rather than "buy AMD". Although watch that space.

ClassyJacket 2y ago
Aren't they basically safe as long as future AI models are still basically multiplying tensors of floats?
- cma 2y ago
  Training uses other operations for normalizing, etc.

wmf 2y ago

https://ai.meta.com/blog/next-generation-meta-training-inference-accelerator-AI-MTIA/ https://ai.meta.com/blog/next-generation-meta-training-infer... https://azure.microsoft.com/en-us/blog/azure-maia-for-the-era-of-ai-from-silicon-to-software-to-systems/ https://azure.microsoft.com/en-us/blog/azure-maia-for-the-er... They're not teaming up and neither design is open, but they definitely have their own designs.