5 ms·
Intel's CEO wanted to buy NVidia 20 years ago. Unfortunately for Intel, the board of directors killed the proposal as "too expensive" at $20B. Granted no one co
by insane_dreamer 2y ago
Intel's CEO wanted to buy NVidia 20 years ago. Unfortunately for Intel, the board of directors killed the proposal as "too expensive" at $20B. Granted no one could see the future of AI on GPUs, but think of the ROI on that investment.
- ipsum2 2y agoNvidia would've acquired the culture of Intel, so it wouldn't be the company it was today if that happened.
- insane_dreamer 2y agoNot sure that would have mattered once LLMs emerged. NVidia's success is mostly a matter of being the only major advanced GPU player at the time when LLMs exploded onto the scene. That won't be the case for long. The big players are already heavily investing in high performance AI chips to wean themselves off a dependance on NVidia.
- kiratp 2y agoNvidia’s success is due to a decade+ of investment into their software stack (CUDA and friends), built on a world view that more software can and should be massively parallel. They were a key cause for LLMs being a thing in the first place.
- BobbyJo 2y agoThis. I worked in HPC between 2010 and 2013, and people were trying to compete with NVidia in the GPGPU space even back then, seeing the way the wind was blowing. Compute bandwidth, and FLOPS/watt has steadily been more and more important the last 15 years, even before AI. NVidia has continued to stay ahead because every alternative to CUDA is half baked trash, even when the silicone make sense. As a tech company, trading $$ for time not spent dealing with compatibility bugs and broken drivers pretty much always makes business sense.
- otabdeveloper4 2y ago> half baked trash > dealing with compatibility bugs > broken drivers Describes my experience trying to use CUDA perfectly. We have a long way to go and we haven't even started yet.
- initplus 2y agoGiven that experience, think about what state the alternatives must be in!
- disgruntledphd2 2y agoTo be fair, Cuda has improved a lot since 2014 or so. I messed up my Linux box multiple times trying to install Cuda but the last time it was just apt install and maybe setting ld library and it all just worked.
- yieldcrv 2y agoIt’s pretty bad. just when you think you can order AMD chips since there is no shortage, and use a translation layer and have a cheap AI datacenter, it turns out AMD is fumbling the ball at every step of the way
- bottled_poe 2y agoIt’s interesting. They have had plenty of time and resources available to mount solid competition. Why haven’t they? Is it a talent hiring problems or some more fundamental problem with their engineering processes? The writing has been on the wall for gpgpu for more than 10 years. Definitely enough time to catch up.
- BobbyJo 2y agoIts a commitment problem IMO. NVidia stuck with CUDA for a long time before it started paying them what it cost. AMD and Intel have both launched and killed initiatives a couple times each, but abandon them within a few years because adoption didn't happen overnight. If you need people to abandon an ecosystem thats been developed steadily over nearly 20 years for your shiny new thing in order to keep it around, you'll never compete.
- llm_trw 2y agoMore than a decade. I was using it as soon as it came out in 2007 and my dinky desktop workstation was out performing the main frame in the basement.
- mullingitover 2y ago> a world view that more software can and should be massively parallel. Twenty years ago I was thinking we'd be speccing machines in kilocores by now.
- epcoa 2y agohttps://en.wikipedia.org/wiki/Connection_Machine https://en.wikipedia.org/wiki/Connection_Machine
- otabdeveloper4 2y ago128 and 512 cores are normal now, so not that far off.
- Keyframe 2y ago4090 has 16384 cuda cores, so there's that!
- atq2119 2y agoNvidia's marketing is misleading. Those "cuda cores" are more SIMD lanes than cores. Number of SMs is a more appropriate equivalent to CPU core count.
- Keyframe 2y agoyou're right
- dahart 2y agoAre you sure that isn’t what @mullingitover meant? > Number of SMs is a more appropriate equivalent to CPU core count. What do you mean by this? Why should an SM be considered equivalent to a CPU core? An SM can do 128 simultaneous adds and/or multiplies in a single cycle, where a CPU core can do, what, 2 or maybe 4? Obviously depends on the CPU / core / hyperthreading / # math pipelines / etc., but the SM to CPU-core ratio of the number of simultaneous calculation is in the double digits. It’s a tradeoff where the GPU has some restrictions in return for being able to do many multiples more at the same time. If you consider an SM and a CPU equivalent, then the SM’s perf can exceed the CPU core by ~2 orders of magnitude — is that the comparison you want? If you consider a GPU thread lane and a CPU thread lane equivalent, the the GPU thread lane is slower and more restricted. Neither comparison is apples to apples, CPUs and GPUs are made for different workloads, but arguing that an SM is equivalent to a CPU core seems equally or more “misleading” when you’re leaving out the tradeoff. I’d argue that comparing SMs to cores is misleading and that it makes more sense to compare chips is by their thread counts. Or, don’t compare cores at all and just look at the performance in, say, FLOPS.
- seanhunter 2y ago100%. If you ever tried to build pytorch (or tensorflow) gpu accellerated using NVidia vs AMD it was absolutely chalk and cheese for the longest time. Nvidia/CUDA process: Download package. Run the build. It works. Run your thing- it's GPU accelerated. Go get a beer/coffee/whatever while your net runs. AMD process: Download package. Run the build. Debug failure. Read lots of articles about which patches to apply. Apply the patches. Run the build. It fails again. Shit. OK ok now I know what to do. I need a special fork of the package. go get that. Find it doesn't actually have the same API that the latest pytorch/tf relies on. OK downgrade those to an earlier version. OK now we're good. Run the build again. Aw shit. That failed again. More web searches. Oh ok now I know - there's some other patches you need to apply to this branch. OK cool. Now it compiles. Install the package. Run the thing. Huh. That's weird. GPU accelleration isn't on.... sigh....
- KaoruAoiShiho 2y agoThis seems to have cause and effect backwards. LLMs emerged because of Nvidia.
- dagmx 2y agoLLMs absolutely did not emerge because of NVIDIA. you’re the one imho who is mistaking correlation with causality. The first transformer models were developed at Google. NVIDIA were the card du jour for accelerating it in the years since and have contributed research too, but your statement goes way too far
- BobbyJo 2y agoI think their point is that easily accessible hardware capable of supporting the research is the reason the research has come as far as it has, and I would tend to agree. At the very least, GPUs keeping the PCI ecosystem going has played a major role in allowing the specialized accelerator market to flourish.
- dagmx 2y agoBut that could apply to any of the GPU manufacturers. CUDA made for an easier ecosystem but if it didn’t exist it would have been any of the other APIs. The first transformer models didn’t even use CUDA and CUDA didn’t have mass ecosystem inroads till years later. I’m not trying to downplay NVIDIA but they specifically mentioned cause and effect, and then said it was because of NVIDIA.
- KaoruAoiShiho 2y agoFirst transformer != LLM. Imagine a world where you had to use AMD or CPUs, there would be no AlexNet, there would be no LLMs. Nvidia seeded universities with gifted hardware accelerators for a over a decade. Nvidia built the foundations for modern ML on which transformer lies, it's just one step on a long road to LLMs.
- 2y ago
- deelowe 2y agoAs someone who's intimately familiar with both cultures, I'm convinced Intel would have killed any innovation Nvidia had going for it before it had a chance to really take off. Management at the two could not be more opposite if they tried.
- daedrdev 2y agoSurely intel would have killed off CUDA
- grogenaut 2y agoIntels 3rd wave management by MBA would have ruined nvidia way back then forcing them into something dumb and blocked all of the R&D they did since since that's what they also did at intel.
- astrange 2y agoOther companies are capable of making big GPUs; they aren't the only TSMC customer. Intel themselves have perfectly fine GPUs. Their issue is that their management never allocates enough space in their chips to let them perform well. Nvidia's advantage is that they have by far the most complete programming ecosystem for them. (Also honestly… they're a meme stock.)
- DeathArrow 2y agoOther companies don't have CUDA.
- DeathArrow 2y agoNvidia will make general purpose GPUs while the AI players will make ASICS. I guess most data center will prefer GPUs so customers can run whatever AI model they need.
- TrainedMonkey 2y agoI think this undersells the story. NVidia's success is built on innovating and scaling for 20 years. Vastly oversimplifying it: - CUDA conception in 2006 to build super computers for scientific computing. - CUDA influencing CPU designs to be dual purpose, with major distinction of RAM amounts (for scientific compute you need a lot more RAM compared to gaming) - Crypto craze driving extreme consumer GPU demand which enabled them to invest heavily into RND and scale up production. - AI workload explosion arriving right as the crypto demand was dying down. - Consistently great execution, or at least not making any major blunders, during all of the above.
- no_wizard 2y agoIt’s more no major blunders + no real major competition, they have not consistently “executed great”, it’s dumb luck + no extremely stupid decisions. It doesn’t mean they didn’t make a bunch of mistakes its that when they did there was no competition to to realistically turn towards, and they fixed a lot of their mistakes.
- Shorel 2y agoYou are completely ignoring the impacts of Covid and GPU cryptocurrency mining on Nvidia profits. It could have been AMD/ATI profiting from such random events, as they were the ones that financed AI development.
- EVa5I7bHFq9mnYK 2y agoBTW, when GPUs were used for Bitcoin mining (up until year 2013, obsoleted by spacialized chips after that), AMD chips were used exclusively, because they had much better integer math performance, compared to Nvidia cards, which focused on floating point performance.
- kylebenzle 2y agoAnd when the LLM hype dies NVDA will be the hardest to fall.
- basiccalendar74 2y agoNvidia was a $750B company in 2021 (pre-ChatGPT). Cause of gaming and all of pre-LLM machine learning using nvidia gpus.
- albertop 2y ago100% on target. The Intel culture was a big contributor to Intel fall.
- hinkley 2y agoAnd the danger of Nvidia buying Intel is the same. See also MD/Boeing
- ipsum2 2y agoNvidia buying Intel would be doing Intel a great service.
- insane_dreamer 2y agoDoes nvidia contribute anything to OSS? Its drivers are closed, that much I know. Whereas intel is the top corp contributor to the Linux kernel
- DeathArrow 2y agoNvidia buying Intel would mean some damn fast CPUs. I'd like to see that.
- homebrewer 2y agoAnd most likely the death of Intel being one of the largest contributors to the Linux kernel.
- DeathArrow 2y agoCan you expand on what is the Intel culture and why it would hurt growth?
- pixelpoet 2y agoSome of my favourite reading on this topic is Matt Pharr's retrospective on ISPC: https://pharr.org/matt/blog/2018/04/30/ispc-all https://pharr.org/matt/blog/2018/04/30/ispc-all
- toasterlovin 2y agoSecond this. A great read in general and, to the extent that it’s accurate, very instructive about dysfunction at Intel.
- princearthur 2y agoI doubt they would have actually merged operations. The NIH at Intel is so strong that it would have virtually annihilated it otherwise.
- BeetleB 2y agoIndeed. A friend of a friend of a friend was in the room when the decision was made. The reason for not buying: Huang won't be a team player and is hard to work with. So yes, they would have tried to integrate it into the rest of Intel.
- thijson 2y agoJust look at what happened to Altera under Intel. Preacquisition it was similar in size to Xilinx. Now it's just a shadow of its former self, latest few quarters shows it in the red. The person they have in charge of it. Sandra Rivera was formerly the head of HR. From 2019 to 2021, Rivera was Intel’s chief people officer, leading the company’s Human Resources organization worldwide. I heard a rumor that Jensen wouldn't agree to the acquisition unless he became CEO of the combined entity.
- pseudosavant 2y agoIntel of that era would have also required Nvidia to only fab their chips with Intel. That would have been fine initially when Intel was a process leader. But it'd have killed Nvidia for the last 10 years. AMD would have enjoyed a consistent process advantage over Nvidia.
- bogwog 2y agoImagine where we'd (not) be if that happened. Acquisitions/consolidation kill innovation.
- metadat 2y agoAnd if the Intel board jumped on every expensive acquisition novelty that came to the table, how does that go? Everyday I look at stocks and wish I'd known to buy the ones that went up today. Overall, pretty uninteresting and uninsightful, unless you have Doc Brown's DeLorean with the Mr. Fusion upgrade (BTTF II). Even then, would it actually be good if you built your own reality to such an extent? I'm sure life's weird for the UHNW's, unclear if it's actually better. We're all still stuck on the same planet, our kids and kid's kids will all face the same struggles. Even still, today is a pretty interesting temporal location to occupy!
- mrpippy 2y agoNot gonna say they jumped on every one, but…McAfee, Havok(!), Infineon’s wireless business, Mobileye, etc
- metadat 2y agoYeah, McAfee.. https://youtube.com/watch?v=bKgf5PaBzyg https://youtube.com/watch?v=bKgf5PaBzyg metadat slowly saunters off into the nearby foggy embankment, disappearing, Homer Simpson style
- lotsofpulp 2y ago> Everyday I look at stocks and wish I'd known to buy the ones that went up today. You can’t compare yourself to the board of the premiere microchip manufacturing company (at the time). They should have more information than you and they are paid to be making more informed decisions (obviously they can be wrong too).
- DeathArrow 2y agoBefore AI Nvidia made some nice money from the crypto craze. Also, during pandemics and after, but before AI, there was a severe shortage of Nvidia GPUs.
- trhway 2y ago>during pandemics and after, but before AI, there was a severe shortage of Nvidia GPUs. you'd think that such multi-year shortage of a product would be used as an opportunity by other players to jump in and make great money. Yet that major machinery of capitalism is failing here.
- Applejinx 2y agoImplies that 'that major machinery of capitalism' is an assumption. If it doesn't happen even under optimal conditions, you've learned something about the assumptions you've been taught as axioms.
- TrapLord_Rhodo 2y agowell.. capitalism is still what made them happen, so we can still have that as an assumption. I think the real takeaway here is the right conditions within companies can lead to breakthroughs. If a company employs the smartest engineers and gets them to work 12 hours days, it's obvious that they are going to take the lead.
- Shorel 2y agoIntel involvement would have killed any initiative that would lead to what Nvidia is now. So, the ROI would have been much, much lower.
- throwaway48476 2y agoAMD at one point wanted to buy Nvidia too.
- asah 2y agoEvery big techco was the target of numerous buyout offers along the way. Yahoo! tried to buy Google and Facebook got any number of offers. It makes sense when the growing company doesn't have a path forward (e.g. YouTube's bandwidth costs) or the price is truly crazy (e.g. WhatsApp). It's not clear to me why Instagram sold out to Facebook for $1B.
- nickfromseattle 2y agoIt was only a few years old with a 1-2 dozen employees. Social networks hadn't been monetized in the way they are today. The market thought Facebook overpaid for Instagram.
- wellthisisgreat 2y agoAI wouldn’t have had a future in its current form. Or maybe Intel would be partitioning off Nvidia like the memory unit.