4 ms·

I understand Nvidia is in a very dominant position. But $4T market cap still seems absolutely insane to me. I've only read about the Cisco boom and bust during

by ra7 1y ago

I understand Nvidia is in a very dominant position. But $4T market cap still seems absolutely insane to me. I've only read about the Cisco boom and bust during the Internet era, and this feels eerily similar (people who actually experienced it might feel differently though). What could actually drag Nvidia down and make them spend decades in the dark like Cisco still does? So far the two things I've come up with are: (a) general disillusionment in AI and companies not being able to monetize enough to justify spending on GPUs. (b) Big companies designing their own chips in-house lowering demand for Nvidia GPUs. I don't think Nvidia can counter (a), but can they overcome (b) by also offering custom chip design services instead of insisting on selling a proprietary AI stack?

fckgw 1y ago
Also just LLMs getting more efficient in general can lead to the end of the GPU buying frenzy
- bigyabai 1y ago
  I don't really buy it. If you can get GPT-3 performance out of a 4B parameter model, then people are going to use the GPUs for even higher-quality remote inference.
blitzo 1y ago
If China finally figured out how to make whatever machines ASML does, that the sell signal for NVDA.
- oersted 1y ago
  I do broadly agree, but want to note that the ASML machines are not necessarily the bottleneck, or at least not the final bottleneck, and their valuation reflects that. There’s a reason why only TSMC and to a lesser extent now Samsung and Intel are the only serious players in top-end semiconductors. You can’t just buy the machine and print chips, the amount of iterative tuning and know-how required to get good yields is immense. Weirdly, the actual bottleneck seems to be the availability of what can almost be described as “master craftsmanship”. But it’s not enough either to hire a couple of masters, it’s the collective institutional knowledge built up over >50 years. And, of course, TSMC is not worth nearly as much as NVidia either even if they manufacture all their hardware.
  - vel0city 1y ago
    We have a class of shawmans who knows how to speak the right incantations over the magic crystals during their formation which causes our machines to think and create value in the world.
  - FirmwareBurner 1y ago
    >You can’t just buy the machine and print chips Exactly this, as otherwise there would be nothing stopping ASML opening up its own competing fabs next door with their cutting edge machines 12 months before they sell them to their customers for maximum profits, the same way Bitcoin ASMIC mining companies did with their chips. Or at least some Dutch/EU company in the area doing it, but nobody else can do what TSMC does at the cutting edge. For context, EU's most cutting edge fabs will be the Dresden TSMC one at 12nm.
  - maxglute 1y ago
    Bottleneck for PRC likely equipment now. Shortly after US export controls, SMIC poached Jian Shanyi TSMC R&D Chief, he got SMIC to 7nm in like 2 years from mediocre 14nm - full node leap, i.e. 2 ~generations, before Intel and when Samsung 7nm was barely competent. This was before heavier equipment controls. In terms of producing "master craftsmanship" PRC seems fully capable, a lot of their fab technicians trained by TSMC/Samsung have taken over by now, and doing reasonable job of clawing their way to 5nm with DUV, i.e. they have the technical chops and east asian work ethic. Ultimately since raising semi to first-level dicipline, PRC currently the only major semi power without projected 100,000s talent shortfall in coming years. They have all the capable people... lots of state capacity to buy talent and espionage processes. They just need the machines.
- moralestapia 1y ago
  Also, the world is not going to "hyperscale" forever. But also also, it most likely will for the next 10 years.
Ologn 1y ago
Cisco stock (which I thought about buying in 1992 and didn't, unfortunately) doubled in 1990, tripled in 1991, doubled in 1992, and kept going up every year - in 1995 it doubled, in 1998 it doubled, in 1999 it doubled. So it had a long run (and is also still worth over $250 billion). The monetary push is very LLM based. One thing being pushed that I am familiar with is LLM assisted programming. LLMs are being pushed to do other things as well. If LLMs don't improve more, or if companies don't see the monetary benefits of using them in the short/medium term, that would drag Nvidia down. Nvidia has a lot of network effects. Probably only Google has some immunity to that (with its TPUs). I doubt Nvidia will have competition in training LLMs for a while. It is possible a competitor could start taking market share on the low end for inference, but even that would take a while. People have been talking about AMD competition for over two years, and I haven't seen anything that even seems like it might have potential yet, especially on the high end.
- ra7 1y ago
  There's a lot of push for inference hardware now (e.g. Ironwood TPUs). How does Nvidia maintain an edge there? Also, I think the market has to expand beyond LLMs to areas like robotics and self driving cars (and they need to have real success) for Nvidia to maintain this valuation. I don't think only LLMs are enough because I don't see code assist/image generation/chatbots as a massive market.
  - Jlagreen 1y ago
    That's because Nvidia is offering a full ecosystem stack with HW, SW and networking clusters. And that gives customers the most flexibility. Nvidia dominates training and is highly competitive in inferencing. At the same time, SW improvements speed up single node and networking performance. H100 released 3 years ago is today several times faster than it was on release with constant SW updates. Customers who buy Nvidia for training today can use the older GPUs from Nvidia for inferencing later. And Nvidia supports even V100 still in SW updates and speed improvements. And since all is based on the same SW ecosystem, it allows for more seamless operations for customers. You can mix different Nvidia GPU clusters but you can't easily mix Nvidia solutions with other solutions. That is also why Nvidia has always been dominant and that's flexibility. NVFP4 is a good example of what they do to stay ahead. And it is even supported by Hopper so any old customer can use Nvidia's new format to further improve model training performance. Suddenly old Hopper clusters become more valuable with some SW releases by Nvidia. Nvidia has a track record which no competitor can match. Going with Nvidia is no mistake today while going with any competitor is a risky bet. If you spend billions, you think twice about making bets.
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]
Night_Thastus 1y ago
(b) isn't likely because CUDA. Nvidia spent a LOT of time developing CUDA into the de-facto standard for working with GPUs. No one has anything close to comparable. They poured a ton of time and effort into it and pushed it like crazy in the education space as well. (a) can happen, but Nvidia has some buffer. All these companies promised their shareholders huge massive gains if only they embrace the AI revolution. They will be extremely resistant to admit they can't make any money out of it and will keep the charade going for as long as they can. It won't be like a cliff, so Nvidia has time to adjust and handle it. The market cap is nonsense though, it's just hype. I never put any real weight on them.
svnt 1y ago
In what way would the successful implementation of chip design services overcome (b)? They would need to make more offering design services, giving away their design secrets in the process, than selling a product protected by a massive moat. This would be slaughtering the goose. If you can present an example of someone executing this strategy as a pivot from an extremely high margin proprietary product and increasing their market cap as a result I would be very interested to read about it.
- ra7 1y ago
  It isn’t a pivot. Rather it’s gaining business that would otherwise go to Broadcom/Marvell. They continue to keep their high margin proprietary product. They don’t have to give away all their design secrets, just enough to make them a valuable chip design partner. Admittedly, I haven’t thought this out, but I can’t find compelling reasons why they shouldn’t do this.
mcv 1y ago
Not so long ago, the most valuable company in the world was $200B. It feels like only a few years ago that the $1T barrier was broken. Where will this stop? Are these companies already more valuable than the VOC at its height, when it owned entire countries? Is that where we're headed?
- lucaspauker 1y ago
  Apple was first company to reach $1T and it happened on August 2nd, 2018
- strbean 1y ago
  Is it possible that a higher concentration of wealth means inflation is reflected more strongly in asset markets than it is in consumer goods? Billionaires aren't exactly buying more eggs when they have more cash, they're cash is competing for ownership of assets.
  - Nifty3929 1y ago
    I agree with you, but I think you have cause/effect reversed. It's not that a high concentration of wealth results in high assets prices. It's that the money has nowhere else to go except into tech/AI, so those are the only assets that appreciate/inflate, and that leads to a concentration of wealth.
  - pfannkuchen 1y ago
    Their cash isn’t actually tied up in the asset though, per se. Whomever they buy the asset from gets the cash. A good chunk of it likely does eventually filter out into consumer goods, since people selling assets are sometimes selling to fund living expenses and not just to shift into another asset.
- deleted 1y ago
  [deleted]
- chung8123 1y ago
  I think this is a mixure of inflation and fewer places to put money
bluecalm 1y ago
One thing that can hurt them is a shift from current neural networks to something that is more sequential in nature and thus better suited for a CPU. One scenario is that maybe a smaller NN is enough for most tasks but you have to train it in a smart way (search, creating feedback, reasoning). It's a long shot but maybe GPUs won't be the best hardware for the job. It's a pure speculation though.
yibg 1y ago
(c) a technological breakthrough that makes what Nvidia makes obsolete.
- falcor84 1y ago
  My favorite sci-fi fever dream is of biological computation. With the rapid advances in computational biology, I think we might have full biological ALUs in my lifetime, which would offer incredible power efficiency.
  - kjellsbells 1y ago
    I've been thinking about this a lot lately too. There are biological systems that exist in vast quantity, like, say, ants or blades of grass (or, simply cells). I think there is a way to make them computing machines (there was a paper from Adleman, of RSA crypto fame, years ago about it, for example). The challenge is getting the problem broken down across billions of individual compute elements, and of getting the data in and out quickly.
    - danudey 1y ago
      Probably makes more sense to build a biological computer out of neurons than to try to build a beowulf cluster of ants or whatever.
  - M95D 1y ago
    Biology is slow.
Mars008 1y ago
In other words we hate them and want them to fail. Bill Gates, Elon Musk, the same story repeats again. (c) NVidia isn't all about LLM, it has robotics, embedded, vision.. This is going to be huge as generic robotics hits.
- Jensson 1y ago
  > (c) NVidia isn't all about LLM, it has robotics, embedded, vision.. This is going to be huge as generic robotics hits. But its the LLM frenzy that made their market cap 20x in a few years, the other things didn't.
  - Mars008 1y ago
    > But its the LLM frenzy that made their market cap 20x in a few years, the other things didn't. Yes, and they have the second big wave coming. This time they have no close competitors. BTW, without LLMs Russian drones are using latest NVidia for navigation and targeting.
  - Jlagreen 1y ago
    The worlds spends several trillions per year on public and private R&D. The AI frenzy could go on for a decade without making any money simple by R&D spend world wide. That's what people don't get. We're still primarily in AI research mode. The race in LLM training isn't about making money, it's about a R&D race and whoever gets a better product faster than competition. Therefore Nvidia won't fail, because we're far far away from any AI production mode since the computing for that would need decades to install. Imagine how much compute power you would need for 24/7 assistant inferencing in real time for every person on earth. We're are just scratching the surface. For Nvidia to fail at this time would be the same as that the world would stop on AI research lol. Imagine, you have an industry where 80% of all companies' / private investors R&D money becomes revenue of a single company. That's basically Nvidia's position in a nutshell. Basically, Nvidia could do $1 trillion revenue on world wide R&D budgets alone, no need for their customers to make money yet.
    - Mars008 1y ago
NomDePlum 1y ago
Dollar value is decreasing fairly rapidly. Will be interesting to see what this means for financial markets, particularly overvalued elements.
alecco 1y ago
Intel spins off GPUs and smart people make a decent competitor. AMD gets its act together, maybe acquires one of the tensor-based companies. Some other competitor like Samsung releases something unexpected they've been building in secret.

by ra7 1y ago

fckgw 1y ago

Also just LLMs getting more efficient in general can lead to the end of the GPU buying frenzy

bigyabai 1y ago
I don't really buy it. If you can get GPT-3 performance out of a 4B parameter model, then people are going to use the GPUs for even higher-quality remote inference.

blitzo 1y ago

If China finally figured out how to make whatever machines ASML does, that the sell signal for NVDA.

oersted 1y ago
I do broadly agree, but want to note that the ASML machines are not necessarily the bottleneck, or at least not the final bottleneck, and their valuation reflects that. There’s a reason why only TSMC and to a lesser extent now Samsung and Intel are the only serious players in top-end semiconductors. You can’t just buy the machine and print chips, the amount of iterative tuning and know-how required to get good yields is immense. Weirdly, the actual bottleneck seems to be the availability of what can almost be described as “master craftsmanship”. But it’s not enough either to hire a couple of masters, it’s the collective institutional knowledge built up over >50 years. And, of course, TSMC is not worth nearly as much as NVidia either even if they manufacture all their hardware.
- vel0city 1y ago
  We have a class of shawmans who knows how to speak the right incantations over the magic crystals during their formation which causes our machines to think and create value in the world.
- FirmwareBurner 1y ago
  >You can’t just buy the machine and print chips Exactly this, as otherwise there would be nothing stopping ASML opening up its own competing fabs next door with their cutting edge machines 12 months before they sell them to their customers for maximum profits, the same way Bitcoin ASMIC mining companies did with their chips. Or at least some Dutch/EU company in the area doing it, but nobody else can do what TSMC does at the cutting edge. For context, EU's most cutting edge fabs will be the Dresden TSMC one at 12nm.
- maxglute 1y ago
  Bottleneck for PRC likely equipment now. Shortly after US export controls, SMIC poached Jian Shanyi TSMC R&D Chief, he got SMIC to 7nm in like 2 years from mediocre 14nm - full node leap, i.e. 2 ~generations, before Intel and when Samsung 7nm was barely competent. This was before heavier equipment controls. In terms of producing "master craftsmanship" PRC seems fully capable, a lot of their fab technicians trained by TSMC/Samsung have taken over by now, and doing reasonable job of clawing their way to 5nm with DUV, i.e. they have the technical chops and east asian work ethic. Ultimately since raising semi to first-level dicipline, PRC currently the only major semi power without projected 100,000s talent shortfall in coming years. They have all the capable people... lots of state capacity to buy talent and espionage processes. They just need the machines.
moralestapia 1y ago
Also, the world is not going to "hyperscale" forever. But also also, it most likely will for the next 10 years.

Ologn 1y ago

Cisco stock (which I thought about buying in 1992 and didn't, unfortunately) doubled in 1990, tripled in 1991, doubled in 1992, and kept going up every year - in 1995 it doubled, in 1998 it doubled, in 1999 it doubled. So it had a long run (and is also still worth over $250 billion). The monetary push is very LLM based. One thing being pushed that I am familiar with is LLM assisted programming. LLMs are being pushed to do other things as well. If LLMs don't improve more, or if companies don't see the monetary benefits of using them in the short/medium term, that would drag Nvidia down. Nvidia has a lot of network effects. Probably only Google has some immunity to that (with its TPUs). I doubt Nvidia will have competition in training LLMs for a while. It is possible a competitor could start taking market share on the low end for inference, but even that would take a while. People have been talking about AMD competition for over two years, and I haven't seen anything that even seems like it might have potential yet, especially on the high end.

ra7 1y ago
There's a lot of push for inference hardware now (e.g. Ironwood TPUs). How does Nvidia maintain an edge there? Also, I think the market has to expand beyond LLMs to areas like robotics and self driving cars (and they need to have real success) for Nvidia to maintain this valuation. I don't think only LLMs are enough because I don't see code assist/image generation/chatbots as a massive market.
- Jlagreen 1y ago
  That's because Nvidia is offering a full ecosystem stack with HW, SW and networking clusters. And that gives customers the most flexibility. Nvidia dominates training and is highly competitive in inferencing. At the same time, SW improvements speed up single node and networking performance. H100 released 3 years ago is today several times faster than it was on release with constant SW updates. Customers who buy Nvidia for training today can use the older GPUs from Nvidia for inferencing later. And Nvidia supports even V100 still in SW updates and speed improvements. And since all is based on the same SW ecosystem, it allows for more seamless operations for customers. You can mix different Nvidia GPU clusters but you can't easily mix Nvidia solutions with other solutions. That is also why Nvidia has always been dominant and that's flexibility. NVFP4 is a good example of what they do to stay ahead. And it is even supported by Hopper so any old customer can use Nvidia's new format to further improve model training performance. Suddenly old Hopper clusters become more valuable with some SW releases by Nvidia. Nvidia has a track record which no competitor can match. Going with Nvidia is no mistake today while going with any competitor is a risky bet. If you spend billions, you think twice about making bets.

deleted 1y ago

[deleted]

deleted 1y ago

[deleted]

Night_Thastus 1y ago

(b) isn't likely because CUDA. Nvidia spent a LOT of time developing CUDA into the de-facto standard for working with GPUs. No one has anything close to comparable. They poured a ton of time and effort into it and pushed it like crazy in the education space as well. (a) can happen, but Nvidia has some buffer. All these companies promised their shareholders huge massive gains if only they embrace the AI revolution. They will be extremely resistant to admit they can't make any money out of it and will keep the charade going for as long as they can. It won't be like a cliff, so Nvidia has time to adjust and handle it. The market cap is nonsense though, it's just hype. I never put any real weight on them.

svnt 1y ago

In what way would the successful implementation of chip design services overcome (b)? They would need to make more offering design services, giving away their design secrets in the process, than selling a product protected by a massive moat. This would be slaughtering the goose. If you can present an example of someone executing this strategy as a pivot from an extremely high margin proprietary product and increasing their market cap as a result I would be very interested to read about it.

ra7 1y ago
It isn’t a pivot. Rather it’s gaining business that would otherwise go to Broadcom/Marvell. They continue to keep their high margin proprietary product. They don’t have to give away all their design secrets, just enough to make them a valuable chip design partner. Admittedly, I haven’t thought this out, but I can’t find compelling reasons why they shouldn’t do this.

mcv 1y ago

Not so long ago, the most valuable company in the world was $200B. It feels like only a few years ago that the $1T barrier was broken. Where will this stop? Are these companies already more valuable than the VOC at its height, when it owned entire countries? Is that where we're headed?

lucaspauker 1y ago
Apple was first company to reach $1T and it happened on August 2nd, 2018
strbean 1y ago
Is it possible that a higher concentration of wealth means inflation is reflected more strongly in asset markets than it is in consumer goods? Billionaires aren't exactly buying more eggs when they have more cash, they're cash is competing for ownership of assets.
- Nifty3929 1y ago
  I agree with you, but I think you have cause/effect reversed. It's not that a high concentration of wealth results in high assets prices. It's that the money has nowhere else to go except into tech/AI, so those are the only assets that appreciate/inflate, and that leads to a concentration of wealth.
- pfannkuchen 1y ago
  Their cash isn’t actually tied up in the asset though, per se. Whomever they buy the asset from gets the cash. A good chunk of it likely does eventually filter out into consumer goods, since people selling assets are sometimes selling to fund living expenses and not just to shift into another asset.
deleted 1y ago
[deleted]
chung8123 1y ago
I think this is a mixure of inflation and fewer places to put money

bluecalm 1y ago

One thing that can hurt them is a shift from current neural networks to something that is more sequential in nature and thus better suited for a CPU. One scenario is that maybe a smaller NN is enough for most tasks but you have to train it in a smart way (search, creating feedback, reasoning). It's a long shot but maybe GPUs won't be the best hardware for the job. It's a pure speculation though.

yibg 1y ago

falcor84 1y ago
My favorite sci-fi fever dream is of biological computation. With the rapid advances in computational biology, I think we might have full biological ALUs in my lifetime, which would offer incredible power efficiency.
- kjellsbells 1y ago
  I've been thinking about this a lot lately too. There are biological systems that exist in vast quantity, like, say, ants or blades of grass (or, simply cells). I think there is a way to make them computing machines (there was a paper from Adleman, of RSA crypto fame, years ago about it, for example). The challenge is getting the problem broken down across billions of individual compute elements, and of getting the data in and out quickly.
  - danudey 1y ago
    Probably makes more sense to build a biological computer out of neurons than to try to build a beowulf cluster of ants or whatever.
- M95D 1y ago
  Biology is slow.

Mars008 1y ago

In other words we hate them and want them to fail. Bill Gates, Elon Musk, the same story repeats again. (c) NVidia isn't all about LLM, it has robotics, embedded, vision.. This is going to be huge as generic robotics hits.

Jensson 1y ago
> (c) NVidia isn't all about LLM, it has robotics, embedded, vision.. This is going to be huge as generic robotics hits. But its the LLM frenzy that made their market cap 20x in a few years, the other things didn't.
- Mars008 1y ago
  > But its the LLM frenzy that made their market cap 20x in a few years, the other things didn't. Yes, and they have the second big wave coming. This time they have no close competitors. BTW, without LLMs Russian drones are using latest NVidia for navigation and targeting.
- Jlagreen 1y ago
  The worlds spends several trillions per year on public and private R&D. The AI frenzy could go on for a decade without making any money simple by R&D spend world wide. That's what people don't get. We're still primarily in AI research mode. The race in LLM training isn't about making money, it's about a R&D race and whoever gets a better product faster than competition. Therefore Nvidia won't fail, because we're far far away from any AI production mode since the computing for that would need decades to install. Imagine how much compute power you would need for 24/7 assistant inferencing in real time for every person on earth. We're are just scratching the surface. For Nvidia to fail at this time would be the same as that the world would stop on AI research lol. Imagine, you have an industry where 80% of all companies' / private investors R&D money becomes revenue of a single company. That's basically Nvidia's position in a nutshell. Basically, Nvidia could do $1 trillion revenue on world wide R&D budgets alone, no need for their customers to make money yet.
  - Mars008 1y ago

NomDePlum 1y ago

Dollar value is decreasing fairly rapidly. Will be interesting to see what this means for financial markets, particularly overvalued elements.

alecco 1y ago

Intel spins off GPUs and smart people make a decent competitor. AMD gets its act together, maybe acquires one of the tensor-based companies. Some other competitor like Samsung releases something unexpected they've been building in secret.