5 ms·

No, Nvidia just found themselves in a lucky situation. They were already building GPUs mainly for gaming. Crypto then came along and swiped up a bunch of GPUs.

by 1-6 3y ago

No, Nvidia just found themselves in a lucky situation. They were already building GPUs mainly for gaming. Crypto then came along and swiped up a bunch of GPUs. When the self mining craze somewhat waned the AI craze started as the boundaries on its ethicality were broken down in an uncertain economic environment where corporations started racing to see who will get to the top.

IG_Semmelweiss 3y ago
You are right. Even more interesting, is the timing itself: The crypto craze waned just as LLMs were picking up. Any more delay with LLMs, and perhaps nvidia would have been overextended with no demand to take their inventory. There may in fact , be no nvidia in that future depending on the level of the bet. Or take it 1 step further . if COVID hadnt happened, the crypto craze would never have occurred, and nvidia would have only been a bit-level player in the LLM craze we are now in. I still remember nvidia ads in pc game magazines. That and 3dfx. Who knew?
- qwytw 3y ago
  > and nvidia would have only been a bit-level player in the LLM craze we are now in Why? They were the only ones taking GPGPU seriously? AMD completely ignored it for years and Intel only became serious about GPUs very recently.
- dylan604 3y ago
  How are you pinning the crypto craze to COVID? People were mining long before 2020, and that demand was so high putting pressure on GPU inventory that the manufacturers were artificially crippling their performance when used as miners.
  - IG_Semmelweiss 3y ago
    Crypto prices really took off when speculation went thru the roof due to govt "stimmies". Look at the increase in $ nominal pricing of non-standard crypto (everything not BTC, ETH, xmr), the largest upward move recorded in history will be the period 20-22 in the aggregate NFTs at 100K , etc. Thats the "craze" Before that it was still a speculative play, but hardly pervasive and available to so many.
mrkstu 3y ago
As Seneca is said to have quipped- “Fortuna est quae fit cum praeparatio in occasionem incidit." or "Luck is what happens when preparation meets opportunity.” Nvidia has been doing the hard work in preparing to succeed in this market. CUDA has been meticulously developed and maintained, creating an adhesion to their hardware that would not otherwise exist in the AI market. It also has been willing and capable of creating lines of business hardware aimed at maximizing utility for their customers. They also have hired and maintained a roster of the best engineers in their specialties, including the software part of the equation. There is no part of their success that they weren't prepared to take advantage of when the opportunity presented itself. They didn't control the size of the opportunity itself, but no greatly successful company does.
- dkasper 3y ago
  I thought you were gonna go with “Audentes Fortuna Iuvat” for a minute.
- dhruvdh 3y ago
  Nvidia in the past made only GPUs, CUDA is used to program GPUs and get use out of them. It doesn't take any magical foresight to want to invest in CUDA, it is the most obvious way to expand your TAM.
  - throw0101c 3y ago
    > It doesn't take any magical foresight to want to invest in CUDA, it is the most obvious way to expand your TAM. Tell that to ATI/AMD, and all the now-defunct and also-ran (Matrox, 3dfx) graphics companies.
    - mratsim 3y ago
      What if I told you ... Matrox is still alive. Iirc they found a niche in medical device imaging.
      - throw0101c 3y ago
qwytw 3y ago
> No, Nvidia just found themselves in a lucky situation. Well yes and no. They were certainly lucky to be at right place in the right time. But they were also consistently investing into CUDA and the AI/ML ecosystem while their competitor(s) ignored it to such an extent that NVDIA became the only real option (and deservedly so). This is why they can effectively behave like a monopoly these days and just almost inconceivably high margins.
DSingularity 3y ago
Uh no. IIRC were on CUDA 5 or something when imagenet came out and changed the world. They might not have imagined LLMs when they decided to invest in making their GPUs programmable but I guarantee you they extrapolated the future compute potential of vector programmable machines and decided it was not a huge risk to enable it as it is simply betting that some important application would be around to tap into it.
- m_mueller 3y ago
  Didn't even need to extrapolate future potential. The world of supercomputing was massively sunsetting the old vector processors (Cray, Sparc etc.) and switched to commodity x86 hardware back in that timeframe when CUDA came out. Perfect opening for a new vector processor on steroids, for which the chip development was already payed by gamers...
  - Gh0stRAT 3y ago
    IIRC people were already converting their data into texture maps and writing their SIMD instructions as shaders operating on those textures before Nvidia released CUDA.
    - mratsim 3y ago
      Ah yeah there was an article on HN about that a couple years ago. GPGPU before Cuda, now onto a treasure hunt ...
    - hnlmorg 3y ago
      I remember some DAWs supporting pluggins to run PCM over GPU back in the early 00s. It was hugely experimental but does demonstrate that the idea of a GPUs being used for compute isn’t a recent one.
    - PH95VuimJjqBqy 3y ago
      ^ exactly. What people find contentious is this claim that they specifically did it for ML/AI. That's not why they invested the way they did.
mensetmanusman 3y ago
He said it was lucky.
m_mueller 3y ago
you're missing the software part, which is essential to this story. Nvidia was the only GPU maker betting the farm on positioning for GPGPU/HPC all the way back in ~2007. What is happening now and since the last couple of years is just payoff for that massive R&D and software maintenance cost they've been fronting. They bet big and won big.
dataking 3y ago
I think that's oversimplifying things. The acquired podcast has covered Nvidia's different growth periods in depth. I highly recommend anyone interested to give them a listen: https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993-2006 https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993... https://www.acquired.fm/episodes/nvidia-the-machine-learning-company-2006-2022 https://www.acquired.fm/episodes/nvidia-the-machine-learning... https://www.acquired.fm/episodes/nvidia-the-dawn-of-the-ai-era https://www.acquired.fm/episodes/nvidia-the-dawn-of-the-ai-e...
xnx 3y ago
Luck is a big part of it, but doesn't explain why other GPU makers haven't benefited in the same way.
VerminOctopus1 3y ago
Chalking it up to dumb luck is kind of ridiculous. They invested massively in developing an ecosystem for GPU driven computing, which was a well reasoned gamble, and it paid off.
whywhywhywhy 3y ago
If it was pure luck why did they build CUDA over a decade ago? Think you're just looking at this from the angle of a gamer and not someone who's been paying attention to GPGPU compute earlier than the past 6 months.
PheonixPharts 3y ago
> just found themselves in a lucky situation. Then this luck should have equally found AMD, who even today are struggling to pick up the ball they've been dropping for a decade now. My last PC had a Radeon, and I waited the life time of that PC assuming AMD support was just around the corner, all the while renting Nvidia cards in the cloud for any serious projects. I've been in the ML space long enough to remember when people were just speculating about doing ML/computation on GPUs. Nvidia made that much easier and has continued to improve support and features for the past decade+ There insane success is certainly part luck, but I wouldn't be so quick to dismiss all of it as merely happenstance.
Blehmo 3y ago
I'm watching nvidia researchers doing a TON of AI at a wide spectrum, that your 'lucky situation' is a hard understatement what nvidia is doing on Research & Development area.
Gh0stRAT 3y ago
There's a quote that I love: >I'm a great believer in luck. The harder I work, the more of it I seem to have. Nvidia has been working hard(er than their competition) on the software side for almost 2 decades to be in the position they find themselves today. 16 years ago, they released CUDA for general-purpose computing on GPUs, and then 9 years ago they followed that up with cuDNN. They have a consistent pattern of making a intentional, long-term bets to diversify their market exposure and unlock new product areas while building a software ecosystem moat. Yes, they obviously got super lucky with the cryptocurrency frenzy, but there's a reason all the miners were mostly buying Nvidia cards instead of AMD cards.
- kranke155 3y ago
  The miners were buying both Nvidia and AMD I think?
  - Gh0stRAT 3y ago
    Yeah, I'm going to be honest: I'm a cryptocurrency doomer so I've not followed things super closely, but a quick Google search turned up this article[0] from 2022: >While it is true that Nvidia cards are generally preferred by miners due to better price-to-performance, AMD GPUs such as the Radeon RX 6600 XT could still be mined on profitably until recently. […] So yes, carefully consider the condition of all used graphics cards—Nvidia or AMD. [0] https://www.pcworld.com/article/395149/should-you-buy-a-used-mining-gpu.html https://www.pcworld.com/article/395149/should-you-buy-a-used...
  - Slartie 3y ago
    Yes, they bought any supply they could get their hands on. Also, it's not true that Nvidia cards were generally better for (Eth) mining. The Radeon VII for example outclassed Nvidia cards sold at the same time. It's just that Nvidia was able to supply more cards, since they had won the bigger gaming marketshare a while ago and thus were producing more.
Maken 3y ago
Not really. They did push CUDA and GPGPU on their own hardware while AMD and Intel offered a barely functional OpenCL. Of course they had no idea about how big AI and crypto would become, but they were there offering their cards to whoever wanted to run calculations on them.
KaoruAoiShiho 3y ago
Completely ridiculous lol. They built the AI ecosystem from the ground up. https://www.youtube.com/watch?v=WLq9zv3k5n0 https://www.youtube.com/watch?v=WLq9zv3k5n0 The race was on, but nobody else was running. https://www.youtube.com/watch?v=Yhg3IEpl60M https://www.youtube.com/watch?v=Yhg3IEpl60M
infecto 3y ago
No, Nvidia has spent considerable resources to support ML for I think a decade now? They certainly got lucky in the crypto craze and of course the timing of AI off the failings of crypto is extremely lucky but you are wrong in the rest of the argument.
mliker 3y ago
No, Nvidia decided to change their GPU architectures to be more suitable for neural networks, so they did have to bet that this shift would have to pay off. They spoke to many leading AI experts and came to this conclusion. They should be commended for the risk they took. If Nvidia solely ended up being "lucky", then how come AMD didn't take off?
- malfist 3y ago
  One company being lucky doesn't mean another company will be similarly lucky.
  - mliker 3y ago
    Sure, but Nvidia wasn't just lucky. They invested heavily in R&D.
- bryanlarsen 3y ago
  > how come AMD didn't take off? Because they were broke and didn't have the resources to invest properly even if they wanted to. That's not quite true -- they bet the farm on Zen, and that bet paid off. Which means that now they have the resources to also invest in AI. I'm fairly sure if they had bet the farm on AI instead of Zen or if they had tried splitting that bet they'd be bankrupt now.
throw0101c 3y ago
> Crypto then came along and swiped up a bunch of GPUs. When the self mining craze somewhat waned the AI craze started […] Copy-pasting a comment from a discussion a little while[1] ago: CUDA was first released in 2007: * https://en.wikipedia.org/wiki/CUDA https://en.wikipedia.org/wiki/CUDA * https://developer.download.nvidia.com/compute/cuda/1.0/NVIDIA_CUDA_Programming_Guide_1.0.pdf https://developer.download.nvidia.com/compute/cuda/1.0/NVIDI... Two years before the Bitcoin paper (2009): * https://en.wikipedia.org/wiki/Bitcoin https://en.wikipedia.org/wiki/Bitcoin They had a presentation called "The Era of the Personal Supercomputing" at SIGGRAPH 2007: * https://dl.acm.org/doi/10.1145/1281500.1281647 https://dl.acm.org/doi/10.1145/1281500.1281647 * https://www.nvidia.com/content/events/siggraph_2007/supercomputing.html https://www.nvidia.com/content/events/siggraph_2007/supercom... Ian Buck (co-?)creator of CUDA speaking in 2008: > Ian Buck talks about his background developing Brook for GPUs at Stanford university and what paths were taken for developing a C platform for GPUs. * https://www.youtube.com/watch?v=Cmh1EHXjJsk https://www.youtube.com/watch?v=Cmh1EHXjJsk > In 2003, a team of researchers led by Ian Buck unveiled Brook, the first widely adopted programming model to extend C with data-parallel constructs. Ian Buck later joined NVIDIA and led the launch of CUDA in 2006, the world's first solution for general-computing on GPUs. * https://developer.nvidia.com/cuda-zone https://developer.nvidia.com/cuda-zone * http://graphics.stanford.edu/~ianbuck/ http://graphics.stanford.edu/~ianbuck/ Nvidia purposefully went after parallel computing. Specific applications (cryptocurrency, ML/AI) appeared later. [1] https://news.ycombinator.com/item?id=38446957#unv_38447944 https://news.ycombinator.com/item?id=38446957#unv_38447944
didip 3y ago
Nah… It is true that they got lucky several times in a big way. But CUDA was an expensive R&D for many years without clear payouts.
latchkey 3y ago
> When the self mining craze somewhat waned It didn't wane, it was decimated when ETH switched to PoS, and off of GPU mining entirely. All of the other mined coins dropped in value as miners moved to them and dumped all their rewards, making mining those coins unprofitable as well. It was ETH that was propping up the entire GPU mining ecosystem.
mattnewton 3y ago
This timeline seems completely wrong to me. Nvidia’s cudnn has been the only game in town for NN research since I have been in the field, and predates ethereum and the crypto bull run by a few years. If anything they didn’t waver and jump too hard on the crypto bandwagon when the craze was at its height.
diegof79 3y ago
It wasn't luck. For example, the book "Good Strategy/Bad Strategy" has a chapter about the NVidia strategy in its early days. You'll find a lot of similarities to what happened with CUDA. In its early days, NVidia focused on its delivery speed by having a unified software driver/integration strategy. The TNT video card was meh, the TNT2 a little better but inferior to 3Dfx... by the time they launched GeForce, 3Dfx didn't have a product ready to compete. Their driver/integration/product test strategy made that speed possible. With that background you can understand why CUDA wasn't lucky, they repeated the same approach: combine their hardware with the software.
lofaszvanitt 3y ago
A tad bit too many coincidences.
wslh 3y ago
Don't forget the "Artificial Intelligence and Japan's Fifth Generation [Project]" [1] launched in 1982. Bad timing ;-). Wikipedia includes a specific page [2]. Finally, we cannot forget Transputers [3]. [1] https://www.jstor.org/stable/26861060 https://www.jstor.org/stable/26861060 [2] https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Systems https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Syst... [3] https://en.wikipedia.org/wiki/Transputer https://en.wikipedia.org/wiki/Transputer
Bayart 3y ago
I clearly remember AI becoming a big tech subject in the late 2000s/early 2010s following the release of CUDA on consumer hardware, around the same time as Bitcoin and much before GPU-mining was a thing.
gregw2 3y ago
No, Nvidia was not lucky. They are a strong engineering culture who also has deep marketing expertise in 3D in all its facets for decades . Nvidia wisely recognized that only so much horsepower could be used by a conventional 3d graphics pipeline with a given screen resolution, and they needed to invest in growing future compute-heavy adjacent markets. They invested in generalizing their GPU into a more flexible vector coprocessor for HPC and then adjacent markets. They convinced fundamental engineers and researchers in this area to come work for them. There was deep fundamental work done by Ian Buck in 2004 on leveraging GPUs as general vector processors ( https://graphics.stanford.edu/papers/brookgpu/ https://graphics.stanford.edu/papers/brookgpu/ ) and that leadership and deep thinking went to Nvidia, not to Intel. Intel did not have the passion from the top to care about this. They couldn't even care enough to field competitive 3D chips (and associated software), much less extend their thinking to generalize beyond it skillfully. Nvidia did. Anyone who spent every day thinking about how to grow the vector coprocessor market would have pursued crypto and AI when they came along but Nvidia's strong engineering and profits from a leading 3D position gave them competitive advantages which they are, for now, reaping.

by 1-6 3y ago

IG_Semmelweiss 3y ago

You are right. Even more interesting, is the timing itself: The crypto craze waned just as LLMs were picking up. Any more delay with LLMs, and perhaps nvidia would have been overextended with no demand to take their inventory. There may in fact , be no nvidia in that future depending on the level of the bet. Or take it 1 step further . if COVID hadnt happened, the crypto craze would never have occurred, and nvidia would have only been a bit-level player in the LLM craze we are now in. I still remember nvidia ads in pc game magazines. That and 3dfx. Who knew?

qwytw 3y ago
> and nvidia would have only been a bit-level player in the LLM craze we are now in Why? They were the only ones taking GPGPU seriously? AMD completely ignored it for years and Intel only became serious about GPUs very recently.
dylan604 3y ago
How are you pinning the crypto craze to COVID? People were mining long before 2020, and that demand was so high putting pressure on GPU inventory that the manufacturers were artificially crippling their performance when used as miners.
- IG_Semmelweiss 3y ago
  Crypto prices really took off when speculation went thru the roof due to govt "stimmies". Look at the increase in $ nominal pricing of non-standard crypto (everything not BTC, ETH, xmr), the largest upward move recorded in history will be the period 20-22 in the aggregate NFTs at 100K , etc. Thats the "craze" Before that it was still a speculative play, but hardly pervasive and available to so many.

mrkstu 3y ago

As Seneca is said to have quipped- “Fortuna est quae fit cum praeparatio in occasionem incidit." or "Luck is what happens when preparation meets opportunity.” Nvidia has been doing the hard work in preparing to succeed in this market. CUDA has been meticulously developed and maintained, creating an adhesion to their hardware that would not otherwise exist in the AI market. It also has been willing and capable of creating lines of business hardware aimed at maximizing utility for their customers. They also have hired and maintained a roster of the best engineers in their specialties, including the software part of the equation. There is no part of their success that they weren't prepared to take advantage of when the opportunity presented itself. They didn't control the size of the opportunity itself, but no greatly successful company does.

dkasper 3y ago
I thought you were gonna go with “Audentes Fortuna Iuvat” for a minute.
dhruvdh 3y ago
Nvidia in the past made only GPUs, CUDA is used to program GPUs and get use out of them. It doesn't take any magical foresight to want to invest in CUDA, it is the most obvious way to expand your TAM.
- throw0101c 3y ago
  > It doesn't take any magical foresight to want to invest in CUDA, it is the most obvious way to expand your TAM. Tell that to ATI/AMD, and all the now-defunct and also-ran (Matrox, 3dfx) graphics companies.
  - mratsim 3y ago
    What if I told you ... Matrox is still alive. Iirc they found a niche in medical device imaging.
    - throw0101c 3y ago

qwytw 3y ago

> No, Nvidia just found themselves in a lucky situation. Well yes and no. They were certainly lucky to be at right place in the right time. But they were also consistently investing into CUDA and the AI/ML ecosystem while their competitor(s) ignored it to such an extent that NVDIA became the only real option (and deservedly so). This is why they can effectively behave like a monopoly these days and just almost inconceivably high margins.

DSingularity 3y ago

Uh no. IIRC were on CUDA 5 or something when imagenet came out and changed the world. They might not have imagined LLMs when they decided to invest in making their GPUs programmable but I guarantee you they extrapolated the future compute potential of vector programmable machines and decided it was not a huge risk to enable it as it is simply betting that some important application would be around to tap into it.

m_mueller 3y ago
Didn't even need to extrapolate future potential. The world of supercomputing was massively sunsetting the old vector processors (Cray, Sparc etc.) and switched to commodity x86 hardware back in that timeframe when CUDA came out. Perfect opening for a new vector processor on steroids, for which the chip development was already payed by gamers...
- Gh0stRAT 3y ago
  IIRC people were already converting their data into texture maps and writing their SIMD instructions as shaders operating on those textures before Nvidia released CUDA.
  - mratsim 3y ago
    Ah yeah there was an article on HN about that a couple years ago. GPGPU before Cuda, now onto a treasure hunt ...
  - hnlmorg 3y ago
    I remember some DAWs supporting pluggins to run PCM over GPU back in the early 00s. It was hugely experimental but does demonstrate that the idea of a GPUs being used for compute isn’t a recent one.
  - PH95VuimJjqBqy 3y ago
    ^ exactly. What people find contentious is this claim that they specifically did it for ML/AI. That's not why they invested the way they did.

mensetmanusman 3y ago

He said it was lucky.

m_mueller 3y ago

you're missing the software part, which is essential to this story. Nvidia was the only GPU maker betting the farm on positioning for GPGPU/HPC all the way back in ~2007. What is happening now and since the last couple of years is just payoff for that massive R&D and software maintenance cost they've been fronting. They bet big and won big.

dataking 3y ago

I think that's oversimplifying things. The acquired podcast has covered Nvidia's different growth periods in depth. I highly recommend anyone interested to give them a listen: https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993-2006 https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993... https://www.acquired.fm/episodes/nvidia-the-machine-learning-company-2006-2022 https://www.acquired.fm/episodes/nvidia-the-machine-learning... https://www.acquired.fm/episodes/nvidia-the-dawn-of-the-ai-era https://www.acquired.fm/episodes/nvidia-the-dawn-of-the-ai-e...

xnx 3y ago

Luck is a big part of it, but doesn't explain why other GPU makers haven't benefited in the same way.

VerminOctopus1 3y ago

Chalking it up to dumb luck is kind of ridiculous. They invested massively in developing an ecosystem for GPU driven computing, which was a well reasoned gamble, and it paid off.

whywhywhywhy 3y ago

If it was pure luck why did they build CUDA over a decade ago? Think you're just looking at this from the angle of a gamer and not someone who's been paying attention to GPGPU compute earlier than the past 6 months.

PheonixPharts 3y ago

> just found themselves in a lucky situation. Then this luck should have equally found AMD, who even today are struggling to pick up the ball they've been dropping for a decade now. My last PC had a Radeon, and I waited the life time of that PC assuming AMD support was just around the corner, all the while renting Nvidia cards in the cloud for any serious projects. I've been in the ML space long enough to remember when people were just speculating about doing ML/computation on GPUs. Nvidia made that much easier and has continued to improve support and features for the past decade+ There insane success is certainly part luck, but I wouldn't be so quick to dismiss all of it as merely happenstance.

Blehmo 3y ago

I'm watching nvidia researchers doing a TON of AI at a wide spectrum, that your 'lucky situation' is a hard understatement what nvidia is doing on Research & Development area.

Gh0stRAT 3y ago

There's a quote that I love: >I'm a great believer in luck. The harder I work, the more of it I seem to have. Nvidia has been working hard(er than their competition) on the software side for almost 2 decades to be in the position they find themselves today. 16 years ago, they released CUDA for general-purpose computing on GPUs, and then 9 years ago they followed that up with cuDNN. They have a consistent pattern of making a intentional, long-term bets to diversify their market exposure and unlock new product areas while building a software ecosystem moat. Yes, they obviously got super lucky with the cryptocurrency frenzy, but there's a reason all the miners were mostly buying Nvidia cards instead of AMD cards.

kranke155 3y ago
The miners were buying both Nvidia and AMD I think?
- Gh0stRAT 3y ago
  Yeah, I'm going to be honest: I'm a cryptocurrency doomer so I've not followed things super closely, but a quick Google search turned up this article[0] from 2022: >While it is true that Nvidia cards are generally preferred by miners due to better price-to-performance, AMD GPUs such as the Radeon RX 6600 XT could still be mined on profitably until recently. […] So yes, carefully consider the condition of all used graphics cards—Nvidia or AMD. [0] https://www.pcworld.com/article/395149/should-you-buy-a-used-mining-gpu.html https://www.pcworld.com/article/395149/should-you-buy-a-used...
- Slartie 3y ago
  Yes, they bought any supply they could get their hands on. Also, it's not true that Nvidia cards were generally better for (Eth) mining. The Radeon VII for example outclassed Nvidia cards sold at the same time. It's just that Nvidia was able to supply more cards, since they had won the bigger gaming marketshare a while ago and thus were producing more.

Maken 3y ago

Not really. They did push CUDA and GPGPU on their own hardware while AMD and Intel offered a barely functional OpenCL. Of course they had no idea about how big AI and crypto would become, but they were there offering their cards to whoever wanted to run calculations on them.

KaoruAoiShiho 3y ago

Completely ridiculous lol. They built the AI ecosystem from the ground up. https://www.youtube.com/watch?v=WLq9zv3k5n0 https://www.youtube.com/watch?v=WLq9zv3k5n0 The race was on, but nobody else was running. https://www.youtube.com/watch?v=Yhg3IEpl60M https://www.youtube.com/watch?v=Yhg3IEpl60M

infecto 3y ago

No, Nvidia has spent considerable resources to support ML for I think a decade now? They certainly got lucky in the crypto craze and of course the timing of AI off the failings of crypto is extremely lucky but you are wrong in the rest of the argument.

mliker 3y ago

No, Nvidia decided to change their GPU architectures to be more suitable for neural networks, so they did have to bet that this shift would have to pay off. They spoke to many leading AI experts and came to this conclusion. They should be commended for the risk they took. If Nvidia solely ended up being "lucky", then how come AMD didn't take off?

malfist 3y ago
One company being lucky doesn't mean another company will be similarly lucky.
- mliker 3y ago
  Sure, but Nvidia wasn't just lucky. They invested heavily in R&D.
bryanlarsen 3y ago
> how come AMD didn't take off? Because they were broke and didn't have the resources to invest properly even if they wanted to. That's not quite true -- they bet the farm on Zen, and that bet paid off. Which means that now they have the resources to also invest in AI. I'm fairly sure if they had bet the farm on AI instead of Zen or if they had tried splitting that bet they'd be bankrupt now.

throw0101c 3y ago

> Crypto then came along and swiped up a bunch of GPUs. When the self mining craze somewhat waned the AI craze started […] Copy-pasting a comment from a discussion a little while[1] ago: CUDA was first released in 2007: * https://en.wikipedia.org/wiki/CUDA https://en.wikipedia.org/wiki/CUDA * https://developer.download.nvidia.com/compute/cuda/1.0/NVIDIA_CUDA_Programming_Guide_1.0.pdf https://developer.download.nvidia.com/compute/cuda/1.0/NVIDI... Two years before the Bitcoin paper (2009): * https://en.wikipedia.org/wiki/Bitcoin https://en.wikipedia.org/wiki/Bitcoin They had a presentation called "The Era of the Personal Supercomputing" at SIGGRAPH 2007: * https://dl.acm.org/doi/10.1145/1281500.1281647 https://dl.acm.org/doi/10.1145/1281500.1281647 * https://www.nvidia.com/content/events/siggraph_2007/supercomputing.html https://www.nvidia.com/content/events/siggraph_2007/supercom... Ian Buck (co-?)creator of CUDA speaking in 2008: > Ian Buck talks about his background developing Brook for GPUs at Stanford university and what paths were taken for developing a C platform for GPUs. * https://www.youtube.com/watch?v=Cmh1EHXjJsk https://www.youtube.com/watch?v=Cmh1EHXjJsk > In 2003, a team of researchers led by Ian Buck unveiled Brook, the first widely adopted programming model to extend C with data-parallel constructs. Ian Buck later joined NVIDIA and led the launch of CUDA in 2006, the world's first solution for general-computing on GPUs. * https://developer.nvidia.com/cuda-zone https://developer.nvidia.com/cuda-zone * http://graphics.stanford.edu/~ianbuck/ http://graphics.stanford.edu/~ianbuck/ Nvidia purposefully went after parallel computing. Specific applications (cryptocurrency, ML/AI) appeared later. [1] https://news.ycombinator.com/item?id=38446957#unv_38447944 https://news.ycombinator.com/item?id=38446957#unv_38447944

didip 3y ago

Nah… It is true that they got lucky several times in a big way. But CUDA was an expensive R&D for many years without clear payouts.

latchkey 3y ago

> When the self mining craze somewhat waned It didn't wane, it was decimated when ETH switched to PoS, and off of GPU mining entirely. All of the other mined coins dropped in value as miners moved to them and dumped all their rewards, making mining those coins unprofitable as well. It was ETH that was propping up the entire GPU mining ecosystem.

mattnewton 3y ago

This timeline seems completely wrong to me. Nvidia’s cudnn has been the only game in town for NN research since I have been in the field, and predates ethereum and the crypto bull run by a few years. If anything they didn’t waver and jump too hard on the crypto bandwagon when the craze was at its height.

diegof79 3y ago

It wasn't luck. For example, the book "Good Strategy/Bad Strategy" has a chapter about the NVidia strategy in its early days. You'll find a lot of similarities to what happened with CUDA. In its early days, NVidia focused on its delivery speed by having a unified software driver/integration strategy. The TNT video card was meh, the TNT2 a little better but inferior to 3Dfx... by the time they launched GeForce, 3Dfx didn't have a product ready to compete. Their driver/integration/product test strategy made that speed possible. With that background you can understand why CUDA wasn't lucky, they repeated the same approach: combine their hardware with the software.

lofaszvanitt 3y ago

A tad bit too many coincidences.

wslh 3y ago

Don't forget the "Artificial Intelligence and Japan's Fifth Generation [Project]" [1] launched in 1982. Bad timing ;-). Wikipedia includes a specific page [2]. Finally, we cannot forget Transputers [3]. [1] https://www.jstor.org/stable/26861060 https://www.jstor.org/stable/26861060 [2] https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Systems https://en.wikipedia.org/wiki/Fifth_Generation_Computer_Syst... [3] https://en.wikipedia.org/wiki/Transputer https://en.wikipedia.org/wiki/Transputer

Bayart 3y ago

I clearly remember AI becoming a big tech subject in the late 2000s/early 2010s following the release of CUDA on consumer hardware, around the same time as Bitcoin and much before GPU-mining was a thing.

gregw2 3y ago

No, Nvidia was not lucky. They are a strong engineering culture who also has deep marketing expertise in 3D in all its facets for decades . Nvidia wisely recognized that only so much horsepower could be used by a conventional 3d graphics pipeline with a given screen resolution, and they needed to invest in growing future compute-heavy adjacent markets. They invested in generalizing their GPU into a more flexible vector coprocessor for HPC and then adjacent markets. They convinced fundamental engineers and researchers in this area to come work for them. There was deep fundamental work done by Ian Buck in 2004 on leveraging GPUs as general vector processors ( https://graphics.stanford.edu/papers/brookgpu/ https://graphics.stanford.edu/papers/brookgpu/ ) and that leadership and deep thinking went to Nvidia, not to Intel. Intel did not have the passion from the top to care about this. They couldn't even care enough to field competitive 3D chips (and associated software), much less extend their thinking to generalize beyond it skillfully. Nvidia did. Anyone who spent every day thinking about how to grow the vector coprocessor market would have pursued crypto and AI when they came along but Nvidia's strong engineering and profits from a leading 3D position gave them competitive advantages which they are, for now, reaping.