3 ms·

Still seems like they were in the right place at the right time. When they first developed Cuda it was a hammer in search of a nail. Then they had the huge wind

Still seems like they were in the right place at the right time. When they first developed Cuda it was a hammer in search of a nail. Then they had the huge windfall from crypto-mining. That probably led to a lot of discussions on looking for GPGPU opportunities. Then AI came along and Cuda was just sitting there. Then they put two and two together and started investing heavily as they saw momentum build.

sevagh 3y ago
>Then they put two and two together and started investing heavily as they saw momentum build. There's been a decade or more of deep learning models breaking records in almost every single research field, powered (indirectly) through CUDA, cuDNN and other NVIDIA software. AI didn't "come along" when OpenAI released ChatGPT. DNNs that have been 99% NVIDIA-focused have been beating the state-of-the-art for years and years. Also for the record the ADA architecture (very dominant AI accelerator) was released when the stock price was like $100 (compared to the $500 now).
- velcrovan 3y ago
  You could say that, for commercial purposes, AI actually did land with ChatGPT. It existed long before that, but that was unquestionably the event that started diverting billions of investment dollars into AI applications.
  - KaoruAoiShiho 3y ago
    For everyone except Nvidia yes, who had been spending billions on AI before then. That's why they're in the lead.
  - ghaff 3y ago
    You could say that "AI" has had potential for decades. It was also a great money pit for a very long time.
- chongli 3y ago
  I don’t think what you’re saying disagrees with my point. NVidia saw the momentum in AI research building long before ChatGPT arrived. AlexNet came out in 2012 and drew a ton of attention back then.
  - sevagh 3y ago
    Your implied chronology of "crypto died, _then_ NVIDIA lucked out with AI" is off, in that case. Peak of ETH mining was 2021.
    - chongli 3y ago
      Sorry, my points are rather disorganized since I just woke up this morning, but I think Nvidia CEO's claim of "betting the farm on AI" is questionable. The chronology is this: 2007: CUDA released 2012: AlexNet (which runs on CUDA) turns heads and everyone (in the research world, not the public) starts getting really excited about AI 2012-2021: NVidia invests in CUDA while getting carried by gaming and crypto 2022-2023: AI explodes into the public consciousness and NVidia valuation balloons to absurd levels Now, I don't think anyone can argue that NVidia was not smart to build CUDA. What no one would have predicted was how important it would turn out to be back when they started. It was more like "hey, these gaming devices are pretty good at matrix/vector math, let's build an API for people who want to do that" and then it turning out that AI is all matrix/vector math. The point is that all of NVidia's investment in CUDA and AI was subsidized by the gaming and crypto cash cows. They didn't have to "bet the farm", they just had to keep it going as a side project and increase their investment as they saw the momentum build.
    - 15155 3y ago
      I learned CUDA because of ETH mining, this was back in 2017.
johngossman 3y ago
As the article implies, they were lucky and good. I was surprised how quickly they were able to implement Tensor cores and head off alternate architectures.
blackoil 3y ago
> they were in the right place at the right time Yes, and it takes lot of effort to be in the right place.
whywhywhywhy 3y ago
Lot of things were happening before you started paying attention to it. GPGPU definitely wasn't just there being useless before crypto and AI didn't just "Come along" after crypto. CUDA has been heavily utilized for AI for many many years now, whole reason Nvidia is so entrenched in it is because they were they only ones taking GPGPU seriously like OpenCL (1) rose and was abandoned before we even get to your interpretation of the timeline. (1): Easy to forget now that AMD and Apple had an common standard competitor to CUDA and completely fumbled it.
einpoklum 3y ago
When they first developed CUDA, they did not yet have that huge of a windfall from crypto-mining. It was first released in June 2007. I would say it was more of the result of seeing initiatives of ham-fisting general-purpose computation onto graphics-specific API like OpenGL and such.

sevagh 3y ago

>Then they put two and two together and started investing heavily as they saw momentum build. There's been a decade or more of deep learning models breaking records in almost every single research field, powered (indirectly) through CUDA, cuDNN and other NVIDIA software. AI didn't "come along" when OpenAI released ChatGPT. DNNs that have been 99% NVIDIA-focused have been beating the state-of-the-art for years and years. Also for the record the ADA architecture (very dominant AI accelerator) was released when the stock price was like $100 (compared to the $500 now).

velcrovan 3y ago
You could say that, for commercial purposes, AI actually did land with ChatGPT. It existed long before that, but that was unquestionably the event that started diverting billions of investment dollars into AI applications.
- KaoruAoiShiho 3y ago
  For everyone except Nvidia yes, who had been spending billions on AI before then. That's why they're in the lead.
- ghaff 3y ago
  You could say that "AI" has had potential for decades. It was also a great money pit for a very long time.
chongli 3y ago
I don’t think what you’re saying disagrees with my point. NVidia saw the momentum in AI research building long before ChatGPT arrived. AlexNet came out in 2012 and drew a ton of attention back then.
- sevagh 3y ago
  Your implied chronology of "crypto died, _then_ NVIDIA lucked out with AI" is off, in that case. Peak of ETH mining was 2021.
  - chongli 3y ago
    Sorry, my points are rather disorganized since I just woke up this morning, but I think Nvidia CEO's claim of "betting the farm on AI" is questionable. The chronology is this: 2007: CUDA released 2012: AlexNet (which runs on CUDA) turns heads and everyone (in the research world, not the public) starts getting really excited about AI 2012-2021: NVidia invests in CUDA while getting carried by gaming and crypto 2022-2023: AI explodes into the public consciousness and NVidia valuation balloons to absurd levels Now, I don't think anyone can argue that NVidia was not smart to build CUDA. What no one would have predicted was how important it would turn out to be back when they started. It was more like "hey, these gaming devices are pretty good at matrix/vector math, let's build an API for people who want to do that" and then it turning out that AI is all matrix/vector math. The point is that all of NVidia's investment in CUDA and AI was subsidized by the gaming and crypto cash cows. They didn't have to "bet the farm", they just had to keep it going as a side project and increase their investment as they saw the momentum build.
  - 15155 3y ago
    I learned CUDA because of ETH mining, this was back in 2017.

johngossman 3y ago

As the article implies, they were lucky and good. I was surprised how quickly they were able to implement Tensor cores and head off alternate architectures.

blackoil 3y ago

> they were in the right place at the right time Yes, and it takes lot of effort to be in the right place.

whywhywhywhy 3y ago

Lot of things were happening before you started paying attention to it. GPGPU definitely wasn't just there being useless before crypto and AI didn't just "Come along" after crypto. CUDA has been heavily utilized for AI for many many years now, whole reason Nvidia is so entrenched in it is because they were they only ones taking GPGPU seriously like OpenCL (1) rose and was abandoned before we even get to your interpretation of the timeline. (1): Easy to forget now that AMD and Apple had an common standard competitor to CUDA and completely fumbled it.

einpoklum 3y ago

When they first developed CUDA, they did not yet have that huge of a windfall from crypto-mining. It was first released in June 2007. I would say it was more of the result of seeing initiatives of ham-fisting general-purpose computation onto graphics-specific API like OpenGL and such.