4 ms·

I can't see the cloud providers being happy about this, it whitelabels away their branding and customer experience flows. It puts nvidia on both the vendor and

I can't see the cloud providers being happy about this, it whitelabels away their branding and customer experience flows. It puts nvidia on both the vendor and customer side of the relationship, which seems odd

seydor 1y ago
google make their own chips too
- londons_explore 1y ago
  TPU's have serious compatibility problems with a good chunk of the ML ecosystem. That alone means many users will want to use Nvidia hardware even at a decent price premium when the alternative is an extra few months of engineering time in a very fast moving market.
  - jszymborski 1y ago
    I haven't worked with TPUs, but my understanding is that they are pretty plug-n-play for Google frameworks (JAX, TF) but is also pretty simple to use with PyTorch [0]. That covers nearly all of the marketshare [0] https://docs.pytorch.org/xla/release/r2.7/learn/xla-overview.html https://docs.pytorch.org/xla/release/r2.7/learn/xla-overview...
    - shakna 1y ago
      If only things were simple [0]. [0] https://github.com/pytorch/xla/issues/9189 https://github.com/pytorch/xla/issues/9189
      - jszymborski 1y ago
        Certainly not plug-and-play, and perhaps this is just one of many bugs, but if it isn't I can't imagine a scenario where LazyLinear layers are essential.
    - que-encrypt 1y ago
      pytorch xla is barely supported in the pytorch ecosystem (for instance, pytorch lightning still doesn't easily support tpu pods, with only a singular short page about google colab v2-8 tpus that is out of date. Then there are the various libraries/implementations with pytorch that have a .cuda(), etc. More limitations at: https://lightning.ai/docs/pytorch/stable/accelerators/tpu_faq.html https://lightning.ai/docs/pytorch/stable/accelerators/tpu_fa...). I haven't worked with tensorflow, but I've heard it's a pain even when using gpus. JAX is the real deal, and does make my code transferrable between GPUs/TPUs relatively easily (excluding any custom pallas kernels for flash vs splash attention, but this is usually not a massive code change). However, with JAX, there are often not a bunch of pre-existing implementations due to network effects, etc.
ketzo 1y ago
Well, what are they gonna do about it? Nvidia has the most desirable chips in the world, and their insane prices reflect that. Every hyperscaler is already massively incentivized to build their own chips, find some way to take Nvidia down a peg in the value chain. Everyone in the world who can is already coming for Nvidia’s turf. No reason they can’t repay the favor. And beyond just margin-taking, Nvidia’s true moat is the CUDA ecosystem. Given that, it’s hugely beneficial to them to make it as easy as possible for every developer in the world to build stuff on top of Nvidia chips — so they never even think about looking elsewhere.
- Xevion 1y ago
  While I don't dispute that they're objectively the most desirable at the current moment - I do think your comment implies that they deserve it, or that people WANT Nvidia to be the best. It almost sounds like you're cheering on Nvidia, framing it as "everyone else trying to reduce the value of Nvidia", meanwhile they have a long, long history of closed-source drivers, proprietary & patented cost-inflated technology that would be identical if not inferior to alternatives - if it weren't for their market share and vendor lock-in strategies. "Well, what are they gonna do about it?" When dealing with a bully, you go find friends. They're going to fund other chip manufacturers and push for diversity, fund better drivers and compatibility. That's the best possible future anyone could hope for.
  - almostgotcaught 1y ago
    > that would be identical if not inferior to alternatives - if it weren't for their market share and vendor lock-in strategies. 1. "Identical if not for market share" is a complete contradiction when what we're talking about is the network effect of CUDA 2. What vendor lock in? What are you talking about? They have a software and compiler stack that works with their chips. How is that lock in, that's literally just their product offering. In fact the truth is you can compile CUDA for AMD (using hipify) and guess what - the result sucks because AMD isn't a comparable alternative!
    - Ygg2 1y ago
alexgartrell 1y ago
The cloud business model is to use scale and customer ownership to crush hardware margins to dust. They’re also building their own accelerators to try to cut Nvidia out altogether.
- shrubble 1y ago
  Cloud is propped up by the tax laws. Cloud bills can be written off in the month in which they are paid; while buying hardware has to be depreciated over years.
  - sokoloff 1y ago
    Section 179 allows immediate expensing of equipment including computers, but is limited to $1.25M/yr. That’s enough for many small and medium businesses.
- cbg0 1y ago
  I've always felt that the business model is nickel & diming for things like storage/bandwidth and locking in customers with value-add black box services that you can't easily replace with open source solutions. Just took a random server: https://instances.vantage.sh/aws/ec2/m5d.8xlarge?duration=monthly https://instances.vantage.sh/aws/ec2/m5d.8xlarge?duration=mo... - to get a decent price on it you need to commit to three years at $570 per month(no storage or bandwidth included). Over the course of 3 years that's $20520 for a server that's ~10K to buy outright, and even with colo costs over the same time frame you'll spend a lot less, so not exactly crushing those margins to dust.
xbmcuser 1y ago
This is what Nvidia has always done creeping into the margins of its partners and taking over. All it's gpu board partners will tell the same story.
neximo64 1y ago
Why? If the GPUs are used and they want more it makes it easy, also its opt-in.
deleted 1y ago
[deleted]
Hilift 1y ago
> DGX Cloud Lepton, is designed to link artificial intelligence developers with Nvidia’s network of cloud providers, which provide access to its graphics processing units, or GPUs. Some of Nvidia’s cloud provider partners include CoreWeave, Lambda and Crusoe. > "Nvidia DGX Cloud Lepton connects our network of global GPU cloud providers with AI developers," said Jensen Huang, chief executive of Nvidia in a statement. The news was announced at the Computex conference in Taiwan. Sounds like a preferred developer resource. The target audience isn't the usual cro-mag that wants to run LLM's for food.
moralestapia 1y ago
Cool, they're also free to start making their own GPUs ...

seydor 1y ago

google make their own chips too

londons_explore 1y ago
TPU's have serious compatibility problems with a good chunk of the ML ecosystem. That alone means many users will want to use Nvidia hardware even at a decent price premium when the alternative is an extra few months of engineering time in a very fast moving market.
- jszymborski 1y ago
  I haven't worked with TPUs, but my understanding is that they are pretty plug-n-play for Google frameworks (JAX, TF) but is also pretty simple to use with PyTorch [0]. That covers nearly all of the marketshare [0] https://docs.pytorch.org/xla/release/r2.7/learn/xla-overview.html https://docs.pytorch.org/xla/release/r2.7/learn/xla-overview...
  - shakna 1y ago
    If only things were simple [0]. [0] https://github.com/pytorch/xla/issues/9189 https://github.com/pytorch/xla/issues/9189
    - jszymborski 1y ago
      Certainly not plug-and-play, and perhaps this is just one of many bugs, but if it isn't I can't imagine a scenario where LazyLinear layers are essential.
  - que-encrypt 1y ago
    pytorch xla is barely supported in the pytorch ecosystem (for instance, pytorch lightning still doesn't easily support tpu pods, with only a singular short page about google colab v2-8 tpus that is out of date. Then there are the various libraries/implementations with pytorch that have a .cuda(), etc. More limitations at: https://lightning.ai/docs/pytorch/stable/accelerators/tpu_faq.html https://lightning.ai/docs/pytorch/stable/accelerators/tpu_fa...). I haven't worked with tensorflow, but I've heard it's a pain even when using gpus. JAX is the real deal, and does make my code transferrable between GPUs/TPUs relatively easily (excluding any custom pallas kernels for flash vs splash attention, but this is usually not a massive code change). However, with JAX, there are often not a bunch of pre-existing implementations due to network effects, etc.

ketzo 1y ago

Well, what are they gonna do about it? Nvidia has the most desirable chips in the world, and their insane prices reflect that. Every hyperscaler is already massively incentivized to build their own chips, find some way to take Nvidia down a peg in the value chain. Everyone in the world who can is already coming for Nvidia’s turf. No reason they can’t repay the favor. And beyond just margin-taking, Nvidia’s true moat is the CUDA ecosystem. Given that, it’s hugely beneficial to them to make it as easy as possible for every developer in the world to build stuff on top of Nvidia chips — so they never even think about looking elsewhere.

Xevion 1y ago
While I don't dispute that they're objectively the most desirable at the current moment - I do think your comment implies that they deserve it, or that people WANT Nvidia to be the best. It almost sounds like you're cheering on Nvidia, framing it as "everyone else trying to reduce the value of Nvidia", meanwhile they have a long, long history of closed-source drivers, proprietary & patented cost-inflated technology that would be identical if not inferior to alternatives - if it weren't for their market share and vendor lock-in strategies. "Well, what are they gonna do about it?" When dealing with a bully, you go find friends. They're going to fund other chip manufacturers and push for diversity, fund better drivers and compatibility. That's the best possible future anyone could hope for.
- almostgotcaught 1y ago
  > that would be identical if not inferior to alternatives - if it weren't for their market share and vendor lock-in strategies. 1. "Identical if not for market share" is a complete contradiction when what we're talking about is the network effect of CUDA 2. What vendor lock in? What are you talking about? They have a software and compiler stack that works with their chips. How is that lock in, that's literally just their product offering. In fact the truth is you can compile CUDA for AMD (using hipify) and guess what - the result sucks because AMD isn't a comparable alternative!
  - Ygg2 1y ago

alexgartrell 1y ago

The cloud business model is to use scale and customer ownership to crush hardware margins to dust. They’re also building their own accelerators to try to cut Nvidia out altogether.

shrubble 1y ago
Cloud is propped up by the tax laws. Cloud bills can be written off in the month in which they are paid; while buying hardware has to be depreciated over years.
- sokoloff 1y ago
  Section 179 allows immediate expensing of equipment including computers, but is limited to $1.25M/yr. That’s enough for many small and medium businesses.
cbg0 1y ago
I've always felt that the business model is nickel & diming for things like storage/bandwidth and locking in customers with value-add black box services that you can't easily replace with open source solutions. Just took a random server: https://instances.vantage.sh/aws/ec2/m5d.8xlarge?duration=monthly https://instances.vantage.sh/aws/ec2/m5d.8xlarge?duration=mo... - to get a decent price on it you need to commit to three years at $570 per month(no storage or bandwidth included). Over the course of 3 years that's $20520 for a server that's ~10K to buy outright, and even with colo costs over the same time frame you'll spend a lot less, so not exactly crushing those margins to dust.

xbmcuser 1y ago

This is what Nvidia has always done creeping into the margins of its partners and taking over. All it's gpu board partners will tell the same story.

neximo64 1y ago

Why? If the GPUs are used and they want more it makes it easy, also its opt-in.

deleted 1y ago

[deleted]

Hilift 1y ago

> DGX Cloud Lepton, is designed to link artificial intelligence developers with Nvidia’s network of cloud providers, which provide access to its graphics processing units, or GPUs. Some of Nvidia’s cloud provider partners include CoreWeave, Lambda and Crusoe. > "Nvidia DGX Cloud Lepton connects our network of global GPU cloud providers with AI developers," said Jensen Huang, chief executive of Nvidia in a statement. The news was announced at the Computex conference in Taiwan. Sounds like a preferred developer resource. The target audience isn't the usual cro-mag that wants to run LLM's for food.

moralestapia 1y ago

Cool, they're also free to start making their own GPUs ...