4 ms·

So I was pondering, NVidia quarterly datacenter revenue is around 18.4 billion. Meaning that the raw cost input to the AI industry is somewhere around 14-24 bil

by lumost 2y ago

So I was pondering, NVidia quarterly datacenter revenue is around 18.4 billion. Meaning that the raw cost input to the AI industry is somewhere around 14-24 billion dollars per year post depreciation. This is against known revenues of ~3.8 Billion at OpenAI and ~800 Million at Anthropic. Based on reported revenue at cohere of 20 MM - I think it's a fair assumption that the only other material revenue in the industry is in the applications side either at megacaps or smaller startups targeting various back office tasks. One could make a bearish claim on NVidia, that their revenue/valuation is unsustainable unless the AI industry grows 100x over the next few years.

zitterbewegung 2y ago
Random commenter on HN just tried to do a bearish analysis that Morningstar or any other analysis group you can obtain. News at 11. How do we know that AI is going to be the only thing that GPGPUs are the end game?
bamboozled 2y ago
and that it grows using current day approaches and technologies...
m3kw9 2y ago
Regardless NVDA gonna pop Monday on this news
- kortilla 2y ago
  What news?
candiddevmike 2y ago
It is unsustainable, but no one wants to see the music stop. AI hype is floating a lot of tech stocks right now, and NVIDIA is at the center of it.
- mlinhares 2y ago
  If they can keep the narrative, like Elon has kept the Tesla narrative, they'll have made so much money that it doesn't matter if it is sustainable or not.
  - omgwtfbyobbq 2y ago
    It matters if it's sustainable, but having products in the pipeline adds to capital and that can help things out in the short term. Tesla's done very expensive EVs -> home/utility PV/storage -> moderately expensive EVs -> FSD (really ADAS) -> Semi/cyber truck so far. Besides decreasing costs/ramping/improving what they have so far, my guess is they're they're moving to the FSD taxi/model 2/Optimus next. I suspect Nvidia will try a similar approach to AI hardware/software.
    - ein0p 2y ago
      As of the latest update FSD is not yet FSD, but it’s _way_ more than ADAS. Underestimate it at your own peril. It behaves like an experienced, if too timid, driver.
      - omgwtfbyobbq 2y ago
        Yeah, 12.3+ is pretty good. It handles 80-90% of my mileage, and the majority of the time I take over because it's too slow. The only place where it's worse than earlier versions is yellow lights. I'm going to dl/try 12.4 tomorrow. Being able to move from FSD to optimus to whatever comes next is where Tesla can really shine.
      - omgwtfbyobbq 2y ago
        It's also wild to me that it appears to understand driving mostly in the context of other vehicles/people. FSD on city streets is as good as if not better than EAP was on highways in 2019. Doing all this at +/-200wh/mile is pretty good too. I remember when (2000-2008?) people claimed it wouldn't be viable to get an EV below 500wh/mile.
hectormalot 2y ago
Interesting way to look at it! At least looks like companies are investing with some expected future return that is not yet there today. Other thoughts: 1. Current revenues might be a bit higher than you calculated. E.g. I’m not sure if copilot and azure OpenAI service are fully included in those revenues, and those might be relevant figures at this scale. 2. The 10-100x growth might actually materialize. Corporates are much slower to adopt and scale a technology than people might expect. As a result, many big potential users are only at the very beginning of adopting AI. (I am assuming there are valuable applications for them to use AI for)
- chasd00 2y ago
  Outside of your usual customer service chat bots I haven’t seen very many real genai applications in enterprises. I led a team that put one in prod that was pretty basic RAG over a knowledge base to answer questions for about 100 specialists internal to my employer. The knowledge base had to be so refined and the system prompt so tuned that any additional content added would blow it up. There was another bot that popped up in our firm meant to answer questions about corporate policy. I’m guessing that team did something similar but over our policy docs. It vanished about 3 months of being online. It probably gave a bad answer and someone called it out to legal. I’m curious if anyone else has seen a real application in an enterprise worthy of the hype.
  - hectormalot 2y ago
    We’re automatically logging and summarizing calls. I’m in a regulated industry so it’s not an optional activity and there are quite some requirements to how and what we log. Interesting thing we learned is that agents tend to log during the calls, not afterwards. We now see (qualitative feedback) that they are less busy logging and therefore have more attention for the client, and (quantitatively) we see the calls are getting shorter and people with the tool are doing more calls per day. We do many millions of calls a year, so it sums out to a good number. Similarly, we have processes with 100s analysts with very high standards to their outputs. The traditional way is to have QA teams review and provide feedback for a few rounds. We’re introducing AI for the first round(s) of feedback to shorten the cycle time reduces context switches) and have the QA teams focus on final reviews. But I get your point. RAG knowledge bases for experts are in a hard spot. After a few months of employment, the experts tend to know the general knowledge well. As a result, the RAG-bot mostly gets questions about exceptions and niches, where it doesn’t perform very well, and mistakes might be expensive.
ipaddr 2y ago
You are only calculating some sources. I'm sure other cloud venders like aws are part of the global datacenter spend. As well as many smaller private or government entities. OpenAI will continue to spend more taking a bigger percentage if supply remains flat.
tikkun 2y ago
Yes. Also, how much can Nvidia revenue go up from here? There are only a small handful of companies that want more than 100k H100s - if there are ten, that’s $30b of GPUs, which Nvidia would sell in less than 6 months, and then who keeps buying? Or are we expecting a few companies to be buying 1 million plus H100s each next year
- soloist11 2y ago
  What's the failure rate of GPUs over a 10 year period?
mgraczyk 2y ago
One huge oversight in this analysis is the assumption that LLMs will not get better. Imagine there is a 10% chance that LLM capabilities increase so significantly in the next 5 years that a significant portion (say 1%) of the economy will be automated by them. That's enough to make Nvidia a multi trillion dollar company.
- __loam 2y ago
  That's a pretty big assumption. We can also assume they do not get much better and extrapolate from that, and I think that it's responsible to analyze both cases and try to ascertain which is more likely. 56% of the American economy is also based on intellectual property so it's also a big claim that the existing status quo will have nothing to say about large tech firms trying to displace that, if it is even good enough to do that.
  - mgraczyk 2y ago
    It's not an assumption, it's an expected value computation. You have to choose a distribution of expected outcomes in terms of future capabilities. Nearly all outcomes that are plausible to me come out with Nvidia worth at least 50% of what's it's worth today.
    - __loam 2y ago
      Raw improvements to computation aren't a guarantee that LLMs improve.
- ganeshkrishnan 2y ago
  >That's enough to make Nvidia a multi trillion dollar company. There are tons of chip companies breathing down it's neck. AMD dropped the ball but Huawei Ascend chips are already making a dent and NVIDIA had to drop prices in China. Rest of the industry will start eating into NVIDIA as the market cap grows.
therobots927 2y ago
Seeing this sentiment more and more. And I share it. All bubbles pop sometime.
mrb 2y ago
AI generating material new revenue at megacaps is likely. If AI currently generates only 5% of Alphabet's revenue, and if 100% of the cost of this revenue is $ spent on Nvidia's hardware, then this represents 15% of Nvidia's revenue. Seen another way, in this example, Nvidia would only need ~7 Alphabets to sustain current revenue.
modeless 2y ago
People are going to want to use GPT-4o style real time conversational voice AI a lot. I can see people having hour-long conversations on a regular basis within a few years. That application alone is going to need mountains of GPUs, more than are needed for training. And it will be extended to real time video input and output within a few years with another increase in GPU requirements.
- nmfisher 2y ago
  If it does play out like this, I feel that inferencing will be taken over by lower-power ASICs (e.g. Groq) or even commodity CPU hardware. GPUs are an expensive solution (both power consumption and capex) to a straightforward problem. There are already real-time TTS models running on ARM, and while the fidelity isn't on par with large GPU models yet, a year or two of hardware improvements and software optimizations will probably close that gap. The same may not apply to audio2audio models, though, and NVIDIA will probably keep a firm grip on training, too.

by lumost 2y ago

zitterbewegung 2y ago

Random commenter on HN just tried to do a bearish analysis that Morningstar or any other analysis group you can obtain. News at 11. How do we know that AI is going to be the only thing that GPGPUs are the end game?

bamboozled 2y ago

and that it grows using current day approaches and technologies...

m3kw9 2y ago

Regardless NVDA gonna pop Monday on this news

kortilla 2y ago
What news?

candiddevmike 2y ago

It is unsustainable, but no one wants to see the music stop. AI hype is floating a lot of tech stocks right now, and NVIDIA is at the center of it.

mlinhares 2y ago
If they can keep the narrative, like Elon has kept the Tesla narrative, they'll have made so much money that it doesn't matter if it is sustainable or not.
- omgwtfbyobbq 2y ago
  It matters if it's sustainable, but having products in the pipeline adds to capital and that can help things out in the short term. Tesla's done very expensive EVs -> home/utility PV/storage -> moderately expensive EVs -> FSD (really ADAS) -> Semi/cyber truck so far. Besides decreasing costs/ramping/improving what they have so far, my guess is they're they're moving to the FSD taxi/model 2/Optimus next. I suspect Nvidia will try a similar approach to AI hardware/software.
  - ein0p 2y ago
    As of the latest update FSD is not yet FSD, but it’s _way_ more than ADAS. Underestimate it at your own peril. It behaves like an experienced, if too timid, driver.
    - omgwtfbyobbq 2y ago
      Yeah, 12.3+ is pretty good. It handles 80-90% of my mileage, and the majority of the time I take over because it's too slow. The only place where it's worse than earlier versions is yellow lights. I'm going to dl/try 12.4 tomorrow. Being able to move from FSD to optimus to whatever comes next is where Tesla can really shine.
    - omgwtfbyobbq 2y ago
      It's also wild to me that it appears to understand driving mostly in the context of other vehicles/people. FSD on city streets is as good as if not better than EAP was on highways in 2019. Doing all this at +/-200wh/mile is pretty good too. I remember when (2000-2008?) people claimed it wouldn't be viable to get an EV below 500wh/mile.

hectormalot 2y ago

Interesting way to look at it! At least looks like companies are investing with some expected future return that is not yet there today. Other thoughts: 1. Current revenues might be a bit higher than you calculated. E.g. I’m not sure if copilot and azure OpenAI service are fully included in those revenues, and those might be relevant figures at this scale. 2. The 10-100x growth might actually materialize. Corporates are much slower to adopt and scale a technology than people might expect. As a result, many big potential users are only at the very beginning of adopting AI. (I am assuming there are valuable applications for them to use AI for)

chasd00 2y ago
Outside of your usual customer service chat bots I haven’t seen very many real genai applications in enterprises. I led a team that put one in prod that was pretty basic RAG over a knowledge base to answer questions for about 100 specialists internal to my employer. The knowledge base had to be so refined and the system prompt so tuned that any additional content added would blow it up. There was another bot that popped up in our firm meant to answer questions about corporate policy. I’m guessing that team did something similar but over our policy docs. It vanished about 3 months of being online. It probably gave a bad answer and someone called it out to legal. I’m curious if anyone else has seen a real application in an enterprise worthy of the hype.
- hectormalot 2y ago
  We’re automatically logging and summarizing calls. I’m in a regulated industry so it’s not an optional activity and there are quite some requirements to how and what we log. Interesting thing we learned is that agents tend to log during the calls, not afterwards. We now see (qualitative feedback) that they are less busy logging and therefore have more attention for the client, and (quantitatively) we see the calls are getting shorter and people with the tool are doing more calls per day. We do many millions of calls a year, so it sums out to a good number. Similarly, we have processes with 100s analysts with very high standards to their outputs. The traditional way is to have QA teams review and provide feedback for a few rounds. We’re introducing AI for the first round(s) of feedback to shorten the cycle time reduces context switches) and have the QA teams focus on final reviews. But I get your point. RAG knowledge bases for experts are in a hard spot. After a few months of employment, the experts tend to know the general knowledge well. As a result, the RAG-bot mostly gets questions about exceptions and niches, where it doesn’t perform very well, and mistakes might be expensive.

ipaddr 2y ago

You are only calculating some sources. I'm sure other cloud venders like aws are part of the global datacenter spend. As well as many smaller private or government entities. OpenAI will continue to spend more taking a bigger percentage if supply remains flat.

tikkun 2y ago

Yes. Also, how much can Nvidia revenue go up from here? There are only a small handful of companies that want more than 100k H100s - if there are ten, that’s $30b of GPUs, which Nvidia would sell in less than 6 months, and then who keeps buying? Or are we expecting a few companies to be buying 1 million plus H100s each next year

soloist11 2y ago
What's the failure rate of GPUs over a 10 year period?

mgraczyk 2y ago

One huge oversight in this analysis is the assumption that LLMs will not get better. Imagine there is a 10% chance that LLM capabilities increase so significantly in the next 5 years that a significant portion (say 1%) of the economy will be automated by them. That's enough to make Nvidia a multi trillion dollar company.

__loam 2y ago
That's a pretty big assumption. We can also assume they do not get much better and extrapolate from that, and I think that it's responsible to analyze both cases and try to ascertain which is more likely. 56% of the American economy is also based on intellectual property so it's also a big claim that the existing status quo will have nothing to say about large tech firms trying to displace that, if it is even good enough to do that.
- mgraczyk 2y ago
  It's not an assumption, it's an expected value computation. You have to choose a distribution of expected outcomes in terms of future capabilities. Nearly all outcomes that are plausible to me come out with Nvidia worth at least 50% of what's it's worth today.
  - __loam 2y ago
    Raw improvements to computation aren't a guarantee that LLMs improve.
ganeshkrishnan 2y ago
>That's enough to make Nvidia a multi trillion dollar company. There are tons of chip companies breathing down it's neck. AMD dropped the ball but Huawei Ascend chips are already making a dent and NVIDIA had to drop prices in China. Rest of the industry will start eating into NVIDIA as the market cap grows.

therobots927 2y ago

Seeing this sentiment more and more. And I share it. All bubbles pop sometime.

mrb 2y ago

AI generating material new revenue at megacaps is likely. If AI currently generates only 5% of Alphabet's revenue, and if 100% of the cost of this revenue is $ spent on Nvidia's hardware, then this represents 15% of Nvidia's revenue. Seen another way, in this example, Nvidia would only need ~7 Alphabets to sustain current revenue.

modeless 2y ago

People are going to want to use GPT-4o style real time conversational voice AI a lot. I can see people having hour-long conversations on a regular basis within a few years. That application alone is going to need mountains of GPUs, more than are needed for training. And it will be extended to real time video input and output within a few years with another increase in GPU requirements.

nmfisher 2y ago
If it does play out like this, I feel that inferencing will be taken over by lower-power ASICs (e.g. Groq) or even commodity CPU hardware. GPUs are an expensive solution (both power consumption and capex) to a straightforward problem. There are already real-time TTS models running on ARM, and while the fidelity isn't on par with large GPU models yet, a year or two of hardware improvements and software optimizations will probably close that gap. The same may not apply to audio2audio models, though, and NVIDIA will probably keep a firm grip on training, too.