4 ms·

> OpenAI, Meta, AWS, AMD, and others have long attempted to eliminate the Nvidia tax, yet failed. Gemini / Google runs and trains on TPUs. You have no incenti

> OpenAI, Meta, AWS, AMD, and others have long attempted to eliminate the Nvidia tax, yet failed. Gemini / Google runs and trains on TPUs. You have no incentive to infer on AMD if you need to buy a massive Nvidia cluster to train.

panabee 2y ago
Google was omitted because they own the hardware and the models, but in retrospect, they represent a proof point nearly as compelling as OpenAI. Thanks for the comment. Google has leading models operating on leading hardware, backed by sophisticated tech talent who could facilitate migrations, yet Google still cannot leap over the CUDA moat and capture meaningful inference market share. Yes, training plays a crucial role. This is where companies get shoehorned into the CUDA ecosystem, but if CUDA were not so intertwined with performance and reliability, customers could theoretically switch after training.
- onlyrealcuzzo 2y ago
  > yet Google still cannot leap over the CUDA moat and capture meaningful inference market share. It's almost as if being a first-mover is more important than whether or not you use CUDA.
  - talldayo 2y ago
    Both matter quite a bit. The first-mover advantage obviously rewards OEMs in a first-come, first-serve order, but CUDA itself isn't some light switch that OEMs can flick and get working overnight. Everyone would do it if it was easy, and even Google is struggling to find buy-in for their TPU pods and frameworks. Short-term value has been dependent on how well Nvidia has responded to burgeoning demands. Long-term value is going to be predicated on the number of Nvidia alternatives that exist, and right now the number is still zero.
- baq 2y ago
  Google has a self inflicted wound in the time to get an api key.
boroboro4 2y ago
Meta trains on Nvidia and infers on AMD. There is incentive if your inference costs are high.
- vidarh 2y ago
  Meta also has a second generation of their own AI accelerator chips designed.

panabee 2y ago

Google was omitted because they own the hardware and the models, but in retrospect, they represent a proof point nearly as compelling as OpenAI. Thanks for the comment. Google has leading models operating on leading hardware, backed by sophisticated tech talent who could facilitate migrations, yet Google still cannot leap over the CUDA moat and capture meaningful inference market share. Yes, training plays a crucial role. This is where companies get shoehorned into the CUDA ecosystem, but if CUDA were not so intertwined with performance and reliability, customers could theoretically switch after training.

onlyrealcuzzo 2y ago
> yet Google still cannot leap over the CUDA moat and capture meaningful inference market share. It's almost as if being a first-mover is more important than whether or not you use CUDA.
- talldayo 2y ago
  Both matter quite a bit. The first-mover advantage obviously rewards OEMs in a first-come, first-serve order, but CUDA itself isn't some light switch that OEMs can flick and get working overnight. Everyone would do it if it was easy, and even Google is struggling to find buy-in for their TPU pods and frameworks. Short-term value has been dependent on how well Nvidia has responded to burgeoning demands. Long-term value is going to be predicated on the number of Nvidia alternatives that exist, and right now the number is still zero.
baq 2y ago
Google has a self inflicted wound in the time to get an api key.

boroboro4 2y ago

Meta trains on Nvidia and infers on AMD. There is incentive if your inference costs are high.

vidarh 2y ago
Meta also has a second generation of their own AI accelerator chips designed.