6 ms·
Investing billionS over the course of CUDA and its predecessors. The whole concept about wanting to use GPU for not just Graphics but High performance or Highly
by ksec 3y ago
Investing billionS over the course of CUDA and its predecessors. The whole concept about wanting to use GPU for not just Graphics but High performance or Highly parallel computing started before 2000s. CUDA announced in 2007, and most of the work predate back to Cg in early 2000s. Even Intel who were already very late to the party made the decision to go invest and start Larrabee in 2005. And there was PS3 Cell, which started development in 2001.
And yet all these work and success from Nvidia was because of, if you read 90% of HN comments for the past 2 years; Luck.
They could have given up at any point in time for the past 20 years and simply not do anything CUDA or GPGPU related. Because who would want to do that when vast majority of those investment were not even bringing in much revenue. Like Intel decided to cancel Larrabee. They persevere and hit the Jackpot some 10-15 years later. But all of this was because of; Luck.
Yes. Luck plays a big part. They could have continue another 10 years and they may never find the Killer App for it. But to ignore all the investment and work for such a long time and pin it down to Luck was about as rude and as disrespectful it can be. Especially on a forum which was started by VC with the spirit of entrepreneurship.
- bryanlarsen 3y agoIt does help to have billions to invest. AMD in the last 20 years was basically broke and they bet the farm on Zen. If they split their limited resources between Zen and CUDA they probably would have failed at both. But their Zen bet has paid off and now they're playing catch up. Intel, OTOH, I'm not sure what their excuse is.
- narrator 3y agoIMHO, Intel lost a couple of years because the CEO had a consensual affair with a subordinate so they fired him and put the CFO in charge for a while and puttered around unable to fix the serious process manufacturing problems getting to 10nm they encountered which caused them to fall behind in manufacturing to TSMC. Finally, after a few years of little progress, Gelsinger was hired, and we'll see what he can do.
- baq 3y agoIf the process worked out Brian wouldn’t get fired, probably. The affair was very convenient for the board. Intel had grown accustomed to internal mediocrity and got too big to root it out; the fact that accounting runs the company instead of someone with a vision doesn’t help. Accounting doesn’t like risky bets and Intel needed some to work out instead of cancelling everything left and right.
- antonjs 3y agoIt seems pretty possible to me that the reason Nvidia pulled off this bet is that Jensen still basically unilaterally controls what NV is working on, and Jensen is still fundamentally a nerd (in the best way).
- FirmwareBurner 3y ago>It does help to have billions to invest. AMD in the last 20 years was basically broke You're making it sound like AMD being broke at that time was some unfortunate accident due to external events, and not the result of their own blunders. Nvidia had billions due to great products and great business decisions on their part, and AMD was broke entirely due to it's own actions, by having average products on the CPU side and making bad business decisions at the time by spending way too much money acquiring ATI, and then selling off their golden goose, the Imageon mobile GPU division to Qualcomm for pennies right at the beginning of the smartphone revolution. It's a miracle they managed to turn things around and not end up like SGI and 3dfx, bankrupt and having their carcass devoured by Intel and Nvidia.
- Teever 3y agoWhile AMD did make mistakes, there were external factors: https://en.wikipedia.org/wiki/Advanced_Micro_Devices,_Inc._v._Intel_Corp https://en.wikipedia.org/wiki/Advanced_Micro_Devices,_Inc._v....
- tharmas 3y agoMaybe what happened to Intel is what happened to GE under Jack Welch. Edit: just saw the CFO put in charge reference in other comment. There's your answer: financialization of the company.
- lumost 3y agoThis is likely a testament to the need for large tech companies to choose core differentiated competencies, and consistently invest in those competencies over long periods of time. It's so tempting for the finance minded to take a large high margin business like NVidia and cut R&D spending to boost profits. Or alternately allow the R&D spend become lazy and ineffective - This happened to Intel, Boeing, and arguably Google. That NVidia has maintained this push for the last 2 decades makes one wonder what other tricks they'll have up their sleeve.
- ativzzz 3y agoSo all of us making fun of Meta for spending billions on AR/VR will be wrong in 10 years? Sometimes the R&D pays off, sometimes it doesn't
- bsdpufferfish 3y agoYes, minus the deadline. You would be crazy to not see the long term value of literally being able to enter virtual worlds, like a Star Trek holodeck. Even something as simple as replacing TVs and monitors is a no-brainer. It's just a question of whether comfort and quality can be achieved.
- adamc 3y agoI don't see that there is enough value there to motivate the investment. For games, to the extent that VR works for them... sure, I could see that use case. But for, say, business meetings? Uh, no. Or to hang out with friends? Sure, maybe, but how would they monetize it well enough to make the investment worthwhile? To make it compelling enough that I would visit? I'm not saying that the use cases don't exist. But either making compelling VR experiences has to be cheap[1] (which I don't see on the near horizon), or they have to add enough value that people will pay for them in some way. That strikes me as uncertain. [1] If they get cheap, then sure, although then it will have the same problem CGI now has... there is so much of it that it has lost a lot of its magic. It's hard to be common and compelling.
- gosub100 3y agoRe: Intel, it's worth mentioning their Phi coprocessor board. They zig-ed when they should have zag-ed. The Phi essentially had thousands of i486 CPUs on it, but apparently that isn't as useful for ML because (apparently) it's all about the FLOPs, whereas the less-mathematical but more logical Phi cores could do (something?) better? I just think it's an honorable mention because had things gone a little differently, perhaps Intel could have been king of the hill instead.
- dzdt 3y agoIntel didn't just miss by a little, they missed by a lot. Nvidia has more software engineers than hardware engineers. Building the software ecosystem is the thing that makes the whole thing work. Intel didn't just have the wrong product with Phi, they had a product with no ecosystem. It was impossible to develop for. None of Nvidia's competitors have a serious answer to the ecosystem problem. Its not clear that Intel has even ever understood this.
- baq 3y agoIntel has had more software engineers than hardware since forever. They know the ecosystem argument in and out - they made x86, they are the argument. They lost the process lead and can’t figure it out for almost a decade now - we wouldn’t be having this discussion if their hardware worked out according to their plans.
- amadeuspagel 3y ago> Nvidia has more software engineers than hardware engineers. Building the software ecosystem is the thing that makes the whole thing work. One thing I never see people discuss in the context is that from the beginning NVidia drivers were closed source while Intel drivers were open source. If you don't see software as a competitive advantage that obviously limits the resources you can invest in it. Linus Torvalds said "Fuck you, NVidia" but who's fucked now?
- sailingparrot 3y ago> success from Nvidia was because of, if you read 90% of HN comments for the past 2 years; Luck. Never once read a comment attributing Nvidia success to luck. The only luck Nvidia has is the luck that AMD fell asleep at the wheel and couldn't get bothered to put more than 2 engineers on a CUDA competitor even when it was getting apparent that AI was worth billions (i.e. ~4-5 years ago).
- ZekeSulastin 3y agoAn example is the top level comment immediately below this thread (on my screen at least): https://news.ycombinator.com/item?id=38721311 https://news.ycombinator.com/item?id=38721311 “No, Nvidia just found themselves in a lucky situation.”
- PH95VuimJjqBqy 3y agowhich means they had reasons for doing what they did and found themselves well positioned for ML/AI but that those reasons were not specifically ML/AI. You have to keep in mind that for a long time concurrency was thought of as the only way to keep getting increasing performance out of CPU's, what they invested in is massive concurrency and when ML/AI hit they were well-positioned for it due to their investments in massive concurrency.
- deleted 3y ago[deleted]
- meragrin_ 3y ago> Never once read a comment attributing Nvidia success to luck. The vast majority of the comments on this article prior to the one you were replying to were attributing it solely/largely to luck.
- adamc 3y agoI think it is obviously true that they have worked hard and built great devices. That was evident even when they only sold products for gaming. But this is what most companies do, or try to do: take their core products and invest in making them better. CUDA was impressive but not shocking. When people say it's luck, I think they are reacting to the reality that Nvidia couldn't know, when they were doing this investing, that there was a big AI market waiting to take off. They were doing good work, but they were also very, very lucky that circumstances granted them this opportunity. There is no shame in that -- few companies achieve great success without some opportunity manifesting. But it's a mistake to pat yourself on the back too hard, either. Without the opportunity, they'd still be making gpus with some other applications.
- mrandish 3y agoAs you note, the "luck" debate often bogs down into a false dichotomy of extremes when the reality is usually in-between and complicated. In my experience, most people (and companies) have the opportunity to encounter approximately similar amounts of "good" and "bad" luck, when averaged over the long run. However... * This is gated by the ability to recognize those opportunities when they appear, willingness to act decisively to maximize the probability of positive outcomes and the preparedness to exploit such advantages. This tends to require mental preparedness, emotional maturity and a willingness to invest scarce resources and/or time - in advance - toward maintaining situational awareness and some excess reserve resources. Doing this is hard but these traits are learnable. * Similarly, a portion of available conscious effort and scarce resources must be continuously expended toward being resilient to bad luck when it inevitably strikes. The net impact of misfortune can vary substantially depending on mitigation steps taken in advance. This requires accurate awareness of ambient risk factors and careful balancing of where you choose to place your limited 'air bags' and 'ounces of prevention.' Most of these things are at least somewhat within your ability to influence, with the exception of initial conditions. At the "opening deal" of life some people are dealt better cards and some people are dealt worse cards. This is not fair, but it is what it is. The silver-lining is that, after the initial cards are dealt, it can still be a long game with many rounds. How you choose to play the cards you have in each of those rounds can lead to substantially different outcomes. Because it's a game like poker with randomness, hidden variables, subtle cues and second-order probabilities - it's easy to conclude it's almost all luck. This is unfortunate because not understanding the 'meta' of the game, or even knowing there is a meta, does make it mostly luck for some. I think NVidia's 'good fortune' is the cumulative result of playing the meta-game effectively for a long-time and thus leading to them having the capability to maximize their outcomes when eventually finding themselves in a high-opportunity environment (aka "lucky").
- PH95VuimJjqBqy 3y agonot luck, but the idea that they predicted the ML/AI renaissance 10-20 years out is laughable considering it was google that kicked it off because google had big data and showed it could be done. They had their reasons for doing what they did and I'm sure they eventually realized they were well positioned for ML/AI, but there's no way they planned that out before ML/AI was a viable thing.
- paulmd 3y agobelieve it or not they did predict it 10 years out, NVIDIA bet the farm on AI/ML after seeing AlexNet. This has been covered in other articles too. And sure, AlexNet was good but remember this is the maxwell days, tensor cores aren't even a thing yet, it was at minimum a very bold bet on the basis of "some image classifier model thing". Nobody else saw it as more than an academic toy (obviously, or they'd have jumped in too). https://www.newyorker.com/magazine/2023/12/04/how-jensen-huangs-nvidia-is-powering-the-ai-revolution https://www.newyorker.com/magazine/2023/12/04/how-jensen-hua... > Within a couple of years, every entrant in the ImageNet competition was using a neural network. By the mid-twenty-tens, neural networks trained on G.P.U.s were identifying images with ninety-six-per-cent accuracy, surpassing humans. Huang’s ten-year crusade to democratize supercomputing had succeeded. “The fact that they can solve computer vision, which is completely unstructured, leads to the question ‘What else can you teach it?’ ” Huang said to me. > The answer seemed to be: everything. Huang concluded that neural networks would revolutionize society, and that he could use CUDA to corner the market on the necessary hardware. He announced that he was once again betting the company. “He sent out an e-mail on Friday evening saying everything is going to deep learning, and that we were no longer a graphics company,” Greg Estes, a vice-president at Nvidia, told me. “By Monday morning, we were an A.I. company. Literally, it was that fast.”
- PH95VuimJjqBqy 3y agoI said 10-20 years out because 10 years out, yes, it was starting to become obvious, but their journey started long before that. 10 years out is 2014.
- 3y ago
- happytiger 3y agoWell it’s just another variation of overnight success taking a decade or two of preparation and, honestly, quiet pain and suffering. I remember talking to Jason Fried about this with 37 signals. A lot of people didn’t realize the grind that team had before things finally clicked. It’s a common pattern to see people gloss over the insane lengths and foresite required to achieve overnight success. :)
- bcrosby95 3y agoThe hand wringing about luck can be annoying. Bringing it up is mostly useful to empathize with other people and be humble. Hard work without luck rarely works out. But luck without hard work rarely works out either (unless you're insanely lucky). Never doing anything because "luck" will guarantee you're never lucky. Use luck to be humble, use luck to empathize, use luck to build people up, but don't use luck to tear people down.
- pjmlp 3y agoYes many are very eager to blame the winner, usually without seeing the work that took them there, and even more relevant, the competition keeps doing a mess.
- ksec 3y agoPrecisely. And this isn't just about Nvidia either. It is every single topic on successful companies. These sort of discussion are common or the norm on 99.999999999% of internet. But we are on HN, the bar needs to be higher, a lot higher. And I cant accept this without some rebuttal. At least judging from the upvote it does seems I have the backing from silent majority. All is not lost.
- GabeIsko 3y agoBut it has nothing to do with AI, it was an investment in optimized hard for floating point calculations for completely unrelated applications to AI. Not that they don't deserve it. It's great hardware. But it would be a shame for them to tie their business to AI and then there is a huge bust from creditors who realize that LLMs don't really do anything.
- todd3834 3y agoLuck is spelled W.O.R.K.
- FergusArgyll 3y agoThis is the first time I saw a high school football coach comment on HN
- iforgotpassword 3y agoYeah, I wouldn't call it just luck, but I refuse to accept that they saw AI coming and blowing up like that any more than anybody else. It worked out for them that they created cuda (btw how much of that was related to acquiring that company that made physX and the according accelerator card?) and decided to stick with it, improve on it, and supply developers with superb tools. But again, I don't think there ever specifically was AI in that planning. It seemed like a step-by-step thing, provide additional value eg to games with physics simulation that would give gamers a reason to buy Nvidia over ati (as if driver quality alone wasn't already a strong argument ;)). The wise decision was to keep pushing cuda and not dropping the ball on it to cut costs in the short run, and realizing the potential of using it for scientific computation early on. Then one thing led to another. When AI came around, cuda was the only mature and serious framework for the job.