4 ms·

I'd be great if they could invest in an alternative to nvidia -- then, in one fell swoop, destroy the moats of everyone in the industry.

math_dandy 3y ago
A company moving away from Nvidia/CUDA while the field is developing so rapidly would result in that company falling behind. When (if) the rate of progress in the AI space slows, then perhaps the big players will have the breathing room to consider rethinking foundational components of their infrastructure. But even at that point, their massive investment in Nvidia will likely render this impractical. Nvidia decisively won the AI hardware lottery, and that's why it's worth trillions.
- mjburgess 3y ago
  I'm more concerned to avoid nvidia (et al.) market domination, than chasing the top-edge of the genAI benefits sigmoid. This will prevent much broad-based innovation.
  - hx8 3y ago
    This space is so compeitive, even if Nvidia is asleep at the wheel a competitor will come and push them before too long. AMD has a history of noticing when their competitors are going soft and rapidly being compeitive.
- whiplash451 3y ago
  People said the same thing when tensorflow was all the rage and pytorch was a side project. Granted, HW is much harder than SW, but I would not discount Meta's ability to displace NVIDIA entirely.
  - Cthulhu_ 3y ago
    I don't think they could; nvidia has tons of talent, Meta would have to steal that. Meta doesn't do anything in either consumer or datacenter hardware that isn't for themselves either. Meta is a services company, their hardware is secondary and for their own usage.
    - Wazako 3y ago
      meta has the Quest. It's not so bad that they're looking to create an LPU for their headset to offer local play.
John23832 3y ago
https://engineering.fb.com/2023/10/18/ml-applications/meta-ai-custom-silicon-olivia-wu/ https://engineering.fb.com/2023/10/18/ml-applications/meta-a...
paxys 3y ago
Except that "one fell swoop" would realistically be 20+ years of research and development from the top minds in the semiconductor industry.
- logicchains 3y ago
  It's not the hardware keeping NVidia ahead, it's the software. Hardware-wise AMD is competitive with NVidia, but their lack of a competitive CUDA alternative is hurting adoption.
aeyes 3y ago
Isn't Google trying to do this with their TPUs?
- crakenzak 3y ago
  I still, for the life of me, can't understand why Google doesn't just start selling their TPUs to everyone. Nvidia wouldn't be anywhere near their size if they only made H100s available through their DGX cloud, which is what Google is doing only making TPUs available through Google Cloud. Good hardware, good software support, and market is starving for performant competitors to the H100s (and soon B100s). Would sell like hotcakes.
  - qiine 3y ago
    Maybe selling hardware to customers worldwide + support like Nvidia does is actually not trivial ?
  - ajcp 3y ago
    And undercut what they'd like to use as a huge motivator in people moving to GCP? Not likely. Even if they wanted to they can't keep up with their own internal demand. Beyond that they might not be as stable or resilient outside of the closely curated confines of their own data-centers. In that case selling them would be more of an embarrassment.
    - htrp 3y ago
      >Beyond that they might not be as stable or resilient outside of the closely curated confines of their own data-centers. In that case selling them would be more of an embarrassment. Once you go out of your heavily curated hardware stack, the headaches multiply exponentially.
  - aseipp 3y ago
    It is an absolutely massive amount of work to turn something designed for your custom software stack and data centers (custom rack designs, water cooling, etc) into a COTS product that is plug-and-play; not just technically but also things like sales, support, etc. You are introducing a massive amount of new problems to solve and pay for. And the in-house designs like TPUs (or Meta's accelerators) are cost effective in part because they don't do that stuff at all. They would not be as cheap per unit of work if they had to also pay off all that other stuff. They also have had a very strong demand for TPUs internally which takes priority over GCP.
brucethemoose2 3y ago
Facebook very specifically bought and customized Intel SKUs tailored for AI workloads for some time.

math_dandy 3y ago

A company moving away from Nvidia/CUDA while the field is developing so rapidly would result in that company falling behind. When (if) the rate of progress in the AI space slows, then perhaps the big players will have the breathing room to consider rethinking foundational components of their infrastructure. But even at that point, their massive investment in Nvidia will likely render this impractical. Nvidia decisively won the AI hardware lottery, and that's why it's worth trillions.

mjburgess 3y ago
I'm more concerned to avoid nvidia (et al.) market domination, than chasing the top-edge of the genAI benefits sigmoid. This will prevent much broad-based innovation.
- hx8 3y ago
  This space is so compeitive, even if Nvidia is asleep at the wheel a competitor will come and push them before too long. AMD has a history of noticing when their competitors are going soft and rapidly being compeitive.
whiplash451 3y ago
People said the same thing when tensorflow was all the rage and pytorch was a side project. Granted, HW is much harder than SW, but I would not discount Meta's ability to displace NVIDIA entirely.
- Cthulhu_ 3y ago
  I don't think they could; nvidia has tons of talent, Meta would have to steal that. Meta doesn't do anything in either consumer or datacenter hardware that isn't for themselves either. Meta is a services company, their hardware is secondary and for their own usage.
  - Wazako 3y ago
    meta has the Quest. It's not so bad that they're looking to create an LPU for their headset to offer local play.

John23832 3y ago

https://engineering.fb.com/2023/10/18/ml-applications/meta-ai-custom-silicon-olivia-wu/ https://engineering.fb.com/2023/10/18/ml-applications/meta-a...

paxys 3y ago

Except that "one fell swoop" would realistically be 20+ years of research and development from the top minds in the semiconductor industry.

logicchains 3y ago
It's not the hardware keeping NVidia ahead, it's the software. Hardware-wise AMD is competitive with NVidia, but their lack of a competitive CUDA alternative is hurting adoption.

aeyes 3y ago

Isn't Google trying to do this with their TPUs?

crakenzak 3y ago
I still, for the life of me, can't understand why Google doesn't just start selling their TPUs to everyone. Nvidia wouldn't be anywhere near their size if they only made H100s available through their DGX cloud, which is what Google is doing only making TPUs available through Google Cloud. Good hardware, good software support, and market is starving for performant competitors to the H100s (and soon B100s). Would sell like hotcakes.
- qiine 3y ago
  Maybe selling hardware to customers worldwide + support like Nvidia does is actually not trivial ?
- ajcp 3y ago
  And undercut what they'd like to use as a huge motivator in people moving to GCP? Not likely. Even if they wanted to they can't keep up with their own internal demand. Beyond that they might not be as stable or resilient outside of the closely curated confines of their own data-centers. In that case selling them would be more of an embarrassment.
  - htrp 3y ago
    >Beyond that they might not be as stable or resilient outside of the closely curated confines of their own data-centers. In that case selling them would be more of an embarrassment. Once you go out of your heavily curated hardware stack, the headaches multiply exponentially.
- aseipp 3y ago
  It is an absolutely massive amount of work to turn something designed for your custom software stack and data centers (custom rack designs, water cooling, etc) into a COTS product that is plug-and-play; not just technically but also things like sales, support, etc. You are introducing a massive amount of new problems to solve and pay for. And the in-house designs like TPUs (or Meta's accelerators) are cost effective in part because they don't do that stuff at all. They would not be as cheap per unit of work if they had to also pay off all that other stuff. They also have had a very strong demand for TPUs internally which takes priority over GCP.

brucethemoose2 3y ago

Facebook very specifically bought and customized Intel SKUs tailored for AI workloads for some time.