4 ms·

I wonder how this will impact consumer gpus? If nvidia can print money making h100's you have to wonder if they will continue to invest on the retail side or i

I wonder how this will impact consumer gpus? If nvidia can print money making h100's you have to wonder if they will continue to invest on the retail side or if they'll funnel most of their efforts to server and let desktop parts languish.

654wak654 3y ago
They're selling consumer GPUs with huge margins too. The markup on the 40xx series is insane but they're still shipping a ton of units.
- pandemicsyn 3y ago
  is that the case? They sold a bunch early on but theres been lots of press about the lousy 40 series sales volume this year and their struggle to get people buy 4080/70's. In Feb they artifically constrained 4090 volume to try and push 4080s no one wanted. The 4070 was dead on arrival and had production cut by April. They've near stopped production now - theoretically because the AI stuff is so lucrative, but if they were really "shipping a ton of units" of the 40 series and didn't have excess inventory i doubt they'd have done that. - https://www.dexerto.com/tech/nvidia-rtx-4070-production-reportedly-cut-2120600/ https://www.dexerto.com/tech/nvidia-rtx-4070-production-repo... - https://www.dexerto.com/tech/nvidia-rtx-40-series-production-ai-gpu-boom-2235345/ https://www.dexerto.com/tech/nvidia-rtx-40-series-production...
  - Miraste 3y ago
    I blame it on Nvidia's product segmentation falling apart. Only a few years ago, the 1060, 1070, 1080, and Titan Xp had price differences that matched their performance differences. In their current lineup, buying anything that isn't a 4090 is getting ripped off. The 4060 and 4070 are basically e-waste. The 4090 had a 100% performance improvement in a lot of tasks over its predecessor, and the 4060 managed 10%. Sometimes. I don't know why they bothered making them--it's no wonder they cut production.
    - Ekaros 3y ago
      Have to fill that line-up so they have something to sell to OEMs and retail customers have something to pick up.
- babypuncher 3y ago
  Aren't consumer GPU shipments the lowest they've been in over a decade?
  - ls612 3y ago
    Yes but it’s hard to disentangle the effect of the crypto crisis ending last year, the pandemic boom in consumer hardware sales coming to a close, and the pricing of Ada from one another. So many GPUs were sold to miners and made their way to the secondary market that gamers have lots and lots of options that aren’t new sales.
    - ghaff 3y ago
      Maybe. But the traditional gaming market doesn't have a lot of growth and I have to believe the high-end may be a dedicated niche but it's a niche.
      - ls612 3y ago
        Looking at Steam concurrent player count it is continuing to ride a steady escalator upwards to I don't think PC gaming as a hobby is gonna shrink anytime soon. My guess is that in a year or two you'll see a big rebound in sales as all of the pandemic hardware starts to get old.
- giobox 3y ago
  The consumer GPU margins aren't even close to what Nvidia rakes in on an H100 sale. Rarely does Nvidia sell a GPU for 2000 dollars or more, and when they do its relatively low volume. An H100 sells for tens of thousands of dollars and is sold out to 2024 - companies are literally fighting over them - I can absolutely see conditions might arise Nvidia might want to use some fab capacity for their much more profitable H100 parts vs consumer GPUs.
wongarsu 3y ago
That would be foolish. A large part of their moat is that lots of people already have Nvidia cards in their computers, making development really smooth and allowing hobbyists to easily get started and contribute to the ecosystem. If they were to give up their consumer gpu lead and let AMD overtake them that will lead to more AMD support in ML, enabling AMD to eventually compete in the server gpu space too.
- nomel 3y ago
  > allowing hobbyists to easily get started and contribute to the ecosystem I think many, maybe even most, would claim this is now best done on cloud services, like huggingface.co.
  - Aerroon 3y ago
    I think that's still a minority. Everyone has heard of cloud platforms giving people huge surprise bills and other such limitations. Another factor is availability and potential T&S concerns when you're doing something they don't want you to do. Stable Diffusion blew up when people are able to run it at home.
    - deleted 3y ago
      [deleted]
    - wing-_-nuts 3y ago
      Nah, when you ask around on AI/ML subs, the default advice is to use google collab. Even shelling out for their premium offerings, you could pay for all the compute you would need through a PHD for the price of a 4090.
- light_hue_1 3y ago
  That's not their moat at all. Most students don't have a capable gpu in their laptop. And these days everyone who is learning doesn't care about running on Nvidia, AMD, a cpu, etc. It's easy to move from one to the other (nominally). The moat is their high quality libraries for speeding up common operations on Nvidia gpus. AMD refuses to invest in ML and build the same libraries. Their stack is horribly buggy to the point of being unusable. It takes forever for their ML stack to get support for their new GPUs, even the crappy version they always produce.
siva7 3y ago
I expect nvidia to sell or close down their consumer division. Their whole business reality has changed in a few short months. Investors want undivided attention from the ceo to the new b2b cash cow.
- sangnoir 3y ago
  This would be extremely foolish, considering the rate of ML research. There is no evidence that LLMs & supersized models are the end-state of ML research. In 3 years time, there might be completely new SoA technique that doesn't demand high memory bandwidth, which will completely flip the economics of pricey high-end cards. It's good they are selling shovels while there's a gold rush, but discarding the rest of your business to focus on a specific shovel that's currently popular may not end well.
  - ethbr1 3y ago
    > ... but discarding the rest of your business to focus on a specific shovel that's currently popular may not end well. I mean, we can just go ahead say "Intel" instead of beating around the bush.
  - trashtester 3y ago
    You may be right. However, Nvidia seems to develop their stack at a pace that it's difficult for the competition to keep up with. They may end up in a position similar to US Steel or Standard Oil at the start of the 20th century if they continue to meet market demand as they do now, and AI is goes on to have the economic impact that steel and oil had in the last century.

654wak654 3y ago

They're selling consumer GPUs with huge margins too. The markup on the 40xx series is insane but they're still shipping a ton of units.

pandemicsyn 3y ago
is that the case? They sold a bunch early on but theres been lots of press about the lousy 40 series sales volume this year and their struggle to get people buy 4080/70's. In Feb they artifically constrained 4090 volume to try and push 4080s no one wanted. The 4070 was dead on arrival and had production cut by April. They've near stopped production now - theoretically because the AI stuff is so lucrative, but if they were really "shipping a ton of units" of the 40 series and didn't have excess inventory i doubt they'd have done that. - https://www.dexerto.com/tech/nvidia-rtx-4070-production-reportedly-cut-2120600/ https://www.dexerto.com/tech/nvidia-rtx-4070-production-repo... - https://www.dexerto.com/tech/nvidia-rtx-40-series-production-ai-gpu-boom-2235345/ https://www.dexerto.com/tech/nvidia-rtx-40-series-production...
- Miraste 3y ago
  I blame it on Nvidia's product segmentation falling apart. Only a few years ago, the 1060, 1070, 1080, and Titan Xp had price differences that matched their performance differences. In their current lineup, buying anything that isn't a 4090 is getting ripped off. The 4060 and 4070 are basically e-waste. The 4090 had a 100% performance improvement in a lot of tasks over its predecessor, and the 4060 managed 10%. Sometimes. I don't know why they bothered making them--it's no wonder they cut production.
  - Ekaros 3y ago
    Have to fill that line-up so they have something to sell to OEMs and retail customers have something to pick up.
babypuncher 3y ago
Aren't consumer GPU shipments the lowest they've been in over a decade?
- ls612 3y ago
  Yes but it’s hard to disentangle the effect of the crypto crisis ending last year, the pandemic boom in consumer hardware sales coming to a close, and the pricing of Ada from one another. So many GPUs were sold to miners and made their way to the secondary market that gamers have lots and lots of options that aren’t new sales.
  - ghaff 3y ago
    Maybe. But the traditional gaming market doesn't have a lot of growth and I have to believe the high-end may be a dedicated niche but it's a niche.
    - ls612 3y ago
      Looking at Steam concurrent player count it is continuing to ride a steady escalator upwards to I don't think PC gaming as a hobby is gonna shrink anytime soon. My guess is that in a year or two you'll see a big rebound in sales as all of the pandemic hardware starts to get old.
giobox 3y ago
The consumer GPU margins aren't even close to what Nvidia rakes in on an H100 sale. Rarely does Nvidia sell a GPU for 2000 dollars or more, and when they do its relatively low volume. An H100 sells for tens of thousands of dollars and is sold out to 2024 - companies are literally fighting over them - I can absolutely see conditions might arise Nvidia might want to use some fab capacity for their much more profitable H100 parts vs consumer GPUs.

wongarsu 3y ago

That would be foolish. A large part of their moat is that lots of people already have Nvidia cards in their computers, making development really smooth and allowing hobbyists to easily get started and contribute to the ecosystem. If they were to give up their consumer gpu lead and let AMD overtake them that will lead to more AMD support in ML, enabling AMD to eventually compete in the server gpu space too.

nomel 3y ago
> allowing hobbyists to easily get started and contribute to the ecosystem I think many, maybe even most, would claim this is now best done on cloud services, like huggingface.co.
- Aerroon 3y ago
  I think that's still a minority. Everyone has heard of cloud platforms giving people huge surprise bills and other such limitations. Another factor is availability and potential T&S concerns when you're doing something they don't want you to do. Stable Diffusion blew up when people are able to run it at home.
  - deleted 3y ago
    [deleted]
  - wing-_-nuts 3y ago
    Nah, when you ask around on AI/ML subs, the default advice is to use google collab. Even shelling out for their premium offerings, you could pay for all the compute you would need through a PHD for the price of a 4090.
light_hue_1 3y ago
That's not their moat at all. Most students don't have a capable gpu in their laptop. And these days everyone who is learning doesn't care about running on Nvidia, AMD, a cpu, etc. It's easy to move from one to the other (nominally). The moat is their high quality libraries for speeding up common operations on Nvidia gpus. AMD refuses to invest in ML and build the same libraries. Their stack is horribly buggy to the point of being unusable. It takes forever for their ML stack to get support for their new GPUs, even the crappy version they always produce.

siva7 3y ago

I expect nvidia to sell or close down their consumer division. Their whole business reality has changed in a few short months. Investors want undivided attention from the ceo to the new b2b cash cow.

sangnoir 3y ago
This would be extremely foolish, considering the rate of ML research. There is no evidence that LLMs & supersized models are the end-state of ML research. In 3 years time, there might be completely new SoA technique that doesn't demand high memory bandwidth, which will completely flip the economics of pricey high-end cards. It's good they are selling shovels while there's a gold rush, but discarding the rest of your business to focus on a specific shovel that's currently popular may not end well.
- ethbr1 3y ago
  > ... but discarding the rest of your business to focus on a specific shovel that's currently popular may not end well. I mean, we can just go ahead say "Intel" instead of beating around the bush.
- trashtester 3y ago
  You may be right. However, Nvidia seems to develop their stack at a pace that it's difficult for the competition to keep up with. They may end up in a position similar to US Steel or Standard Oil at the start of the 20th century if they continue to meet market demand as they do now, and AI is goes on to have the economic impact that steel and oil had in the last century.