9 ms·

Nvidia solved a hard problem and their competitors all failed. I spent years with an AMD card growing progressively more annoyed at their self-inflicted apparen

by roenxi 2y ago

Nvidia solved a hard problem and their competitors all failed. I spent years with an AMD card growing progressively more annoyed at their self-inflicted apparent inability to multiply matrices on demand. Nvidia had nothing to do with their failure, unless they had some sort of high-level mole in AMD's driver teams. Hitting the only successful company in a difficult field with legal assaults is not the obvious path to success. Nvidia has nothing close to a monopoly; there is no moat and they operate in a commodity space that will behave like a normal competitive market. It just happens that their competitors were wall-to-wall complacent about how quickly the hardware world can change and they stupidly thought graphics cards were for displaying graphics. It turns out graphics cards are for building AI and crypto markets.

nimish 2y ago
Yep. AMD and Intel had two decades to come up with a CUDA alternative or even a parallel implementation and both failed to do anything. Incompetence.
- deadbabe 2y ago
  How do we know the strategy wasn’t to avoid releasing a CUDA alternative until NVIDIA is wounded by an anti-trust lawsuit and then when they get broken up deploy your CUDA alternative and speed past them? Could be playing the long game.
  - throwthrowuknow 2y ago
    That’s not the case at all. AMD and Intel could be making money hand over fist right now if they had products that worked with the ML ecosystem. If they had any magic bullet they’d be idiots to not have used it years ago, they’re missing out on tens of billions of dollars on purpose for what? Even if NVIDIA got broken up their cards would still have massive demand and profit margins. I don’t think you understand how much of a shortage of compute there currently is, anyone who isn’t blatantly incompetent or ignoring ML can make money. AMD and Intel are just fat corporations who didn’t see the potential and thought business as usual would stay number one. Now they’re standing on the sidelines with their dicks in their hands while Jensen is leading the parade.
- oytis 2y ago
  The alternative to CUDA is called OpenCL. There has been speculations that poor performance of NVIDIA GPUs with OpenCL compared to CUDA is an intentional anticompetitive practice, but I don't feel confident to tell for sure.
  - mschuetz 2y ago
    OpenCL is an alternative to CUDA just like Legos are an alternative to bricks. The problem with OpenCL isn't even the performance, it's everything. If OpenCL were any good, people could use it to build similarly powerful applications on cheaper AMD GPUs.
    - jb1991 2y ago
      OpenCL is just a spec. It's up to companies to implement it in a successful way or not. There is no reason in and of itself that OpenCL can't compete with CUDA on performance. The fact that Apple's Metal, which is pretty good, is actually implemented with a private OpenCL system is proof that the spec is not to blame.
      - jonas21 2y ago
        > The fact that Apple's Metal, which is pretty good, is actually implemented with a private OpenCL system is proof that the spec is not to blame. I don't understand. If OpenCL was so good, why did Apple create Metal instead of just using OpenCL?
        jb1991 2y ago
        For similar reasons why Microsoft created DirectX. It allows them to have a system where the software and hardware are more tightly integrated than using a cross-platform spec. It also allows them to situate the API within the context of the operating system and other languages that are used on Apple platforms, making things easier on developers. And at least in that regard, they certainly succeeded. Metal is probably the easiest GPU API offered on any platform. Not necessarily the most powerful, but it’s almost trivial to hand write a compute kernel and spin it off.
vlovich123 2y ago
The thing with monopoly laws is it doesn’t matter how you got into pole position. The point is that you’re illegally utilizing tangential benefits of the pole position to maintain that position as well as enter new markets. > including how the company’s software locks customers into using its chips, as well as how Nvidia distributes those chips to customers. The lock-in is probably a losing argument on technical merits unless NVIDIA is doing something nefarious there (which the lawsuit will potentially reveal). If nVidia is distributing chips in preferred ways to partners it wants to succeed (e.g. those partnered with MSFT which it has a very cozy and tight-knit relationship), that could be an anti trust thing. > Microsoft structured its minority stake in OpenAI in part to avoid antitrust scrutiny, The Times has reported. Microsoft also reached a deal in March to hire most of the staff of Inflection AI, another A.I. start-up, and license its technology. Because the deal was not a standard acquisition, it may be harder for regulators to scrutinize. So there is an argument to be made that MSFT is acutely self-aware its behavior could be viewed as engaging in antitrust. If there’s a law against structuring financial transactions, why isn’t similar behavior in the business world viewed similarly as trying to bypass the spirit of the law?
- roenxi 2y ago
  Just on that last paragraph, because the frame annoys me - that is bullying. It isn't fair to insinuate that a company did something wrong by attempting to obey the law. MS knows all about antitrust. They've hired a bunch of lawyers and told them to do what needs to be done to be compliant with the regulations. That shouldn't then be implied as evidence that they are guilty of antitrust activities! What are the lawyers supposed to do, structure the company to maximise the likelihood of an antitrust lawsuit being bought? Nobody should want that, regulator, company or customer.
  - iforgotpassword 2y ago
    Because laws are always flawless and never contain any loopholes, and it has never happened ever that lawyers were specifically instructed by a company to find those loopholes and exploit them as hard as possible.
trhway 2y ago
>Nvidia had nothing to do with their failure, unless they had some sort of high-level mole in AMD's driver teams. I wonder why Jensen wouldn't help his cousin Lisa Su with a good advice :) In general it isn't surprising that hardware companies like Intel and AMD fall at software - just talk to any programmer at such a hardware shop. What striking is that Jensen is able to have such a prominent and huge software development in a naturally hardware company. NVDA is the only hardware company where my programmer acquaintances don't complain about it being a hardware company.
- Dalewyn 2y ago
  >it isn't surprising that hardware companies like Intel and AMD fall at software A lot of what keeps Intel relevant is their prowess in both hardware and software, bringing properly refined products to market when it matters. Intel's failure against Nvidia had far more to do with their business strategy of CPU First missing the broadside of a barn; it's like bringing a knife to a gunfight, of course you're going to get your ass wrecked. AMD fucking sucks at software though, no disagreement there.
  - 7f4784e3f1 2y ago
    Intel has some great software like MKL and related libraries, but, speaking from experience, due to its internal politics Intel is unable to innovate in a meaningful way or build a reasonable SW strategy for GPUs. The oneAPI heavily contributed to their GPU failures IMO. Choosing SYCL and promoting non-existing performance portability and pouring resources in the trying to shove CPU and GPU (and at some point FPGA!) into the same programming model while disinvesting from OpenCL was a big mistake, but it was probably the only way to please both CPU and GPU management. And the problem is not SYCL per se but the fact that they try to make it an open standard rather then trying to provide a way to extract maximum performance from their HW. They needed to build a CUDA alternative, where, after jumping through considerable hoops, you can get to actually programming tensor cores for those who need it. Not surprisingly, Intel is not using SYCL for their GPU kernels (look at oneDNN sources). With Raja gone there may be hope, but I'm not holding my breath. The only thing that they have that is competitive is Habana's stuff which is outside of the oneAPI. I would not be surprised if AMD has a similar issue of infighting due selling both CPUs and GPUs. And NVIDIA does not have this problem.
Dalewyn 2y ago
>not the obvious path to success. It is if success is defined as deliberate failure. Personally, a lot of US government actions of the past few decades in hindsight do not look like they had the interests of America or Americans in mind.
- HPsquared 2y ago
  I wonder if anyone connected to this is short NVDA.
- pas 2y ago
  Can you name a few examples please?
torginus 2y ago
Honestly I don't get how the current situation could develop. Video games have used compute shaders forever, and those work just fine on other cards. I remember doing some quite involved stuff for my thesis using OpenCL a decade ago, and it worked just fine. Nowadays OpenCL is dead for some reason... I just don't get what is there in CUDA that makes it so special. As far as I remember, a GPGPU API consists of a shader language, a way to copy stuff to and from the GPU, a way of scheduling and synchronizing work, atomics, groupshared variables, and some interop stuff. Is the vendor lock-in due to CUDA libs? In that case, the problem is not CUDA, but the libraries themselves. Not sure about today, but performance-portability basically didn't exist back then. You needed to do specialize your code for the GPU arch at hand. It didn't even exist between different generations of cards by the same vendor. Even if you could run CUDA code on AMD, it would be slow, so you need to do a rewrite anyways.
- roenxi 2y ago
  There probably isn't anything in CUDA that makes it special. They are well optimised math libraries and the math for most of the important stuff is somewhat trivial. AI seems to be >80% matrix multiplication - well optimised BLAS is tricky to implement, but even a bad implementation would see all the major libraries support AMD. The vendor "lock in" is because it takes a few years for decisions to be expressed in marketable silicon and literally only Nvidia was trying to be in the market 5 years ago. I've seen a lot of AMD cards that just crashed when used for anything outside OpenGL. I had a bunch of AI related projects die back in 2019 because initialising OpenCL crashed the drivers. If you believe the official docs everything would work fine. Great card except for the fact that compute didn't work. At the time I thought it was maybe just me. After seeing geohotz's saga trying to make tinygrad work on AMD cards and having a feel for how badly unsupported AMD hardware is by the machine learning community, it makes a lot of sense to me that it is a systemic issue and AMD didn't have any corporate sense of urgency about fixing those problems. Maybe there is something magic in CUDA, but if there is it is probably either their memory management model or something quite technical like that. Not the API.
  - mschuetz 2y ago
    The magic is that CUDA actually works well. There is no reason to pick OpenCL, ROCm, Sycl or others if you get a 10x better developer experience with CUDA.
samplatt 2y ago
>Nvidia had nothing to do with their failure, unless they had some sort of high-level mole in AMD's driver teams. You're pretty close, but it's one level of abstraction away from the truth: NVidia spends absolutely earth-shakingly stonking amounts of money working directly with the devs of major studios, fixing issues with their drivers as they come, and in some cases paying studios to lock-in talking to them about the bugs they find. It's why (and how) they release new driver patches for every AAA release, and part of the reason why it triggers a redownload of the shader cache on a whole slew of Steam games every major version release. AMD and intel don't have a chance; NVidia is inside the design loop for game development far more thoroughly than other hardware vendors.
- reissbaker 2y ago
  Of course AMD and Intel could just do the same thing. But they don't. That's not Nvidia's fault. In fact I wish Intel and AMD would do the same! It's great customer service to know that games are going to work well out of the box with Nvidia, because Nvidia spent tons of money working directly with devs on optimization and even per-game bugfixes. As a longtime PC gamer I wish AMD released better GPUs, but they seem incapable of doing so generation after generation. Intel is getting better at the low end, so there's at least some hope there. But Nvidia is just the best in class, because they put in a huge amount of polish and work. My understanding is this is the same approach Nvidia took with CUDA: investing a massive amount to make it fast, accurate, and broadly available, while AMD continued to push out barely-tested drivers — for example, see hotz's woes with tinygrad and finally giving up and labeling the "red" (aka AMD) tinybox driver quality as "Mediocre" on their product page, with the "green" (aka Nvidia) tinybox driver quality listed as "Great" https://tinygrad.org/ https://tinygrad.org/ — and to this day restricts their competing library (ROCm) to specific ultra-expensive server-class GPUs.
  - Dalewyn 2y ago
    >In fact I wish Intel and AMD would do the same! Intel at least works very closely with Microsoft to iron out the details for Windows.
- WheatMillington 2y ago
hgomersall 2y ago
Cory Doctorow argues well why it doesn't matter how you got into a monopoly position: https://pluralistic.net/2024/05/18/market-discipline/#too-big-to-care https://pluralistic.net/2024/05/18/market-discipline/#too-bi...
- aydyn 2y ago
  Still have no idea how this dude is relevant other than he wrote some scifi books one time. What are his technical accomplishments. Oh, nothing? Anyone can talk, especially nowdays because of AI, ironically.
  - eru 2y ago
    Many people like his writing, and his political stances.
  - criddell 2y ago
    Many of us appreciate the work he’s put in with the EFF. When you are right about things over-and-over for 20 or 30 years, people tend to listen to you.
  - hgomersall 2y ago
    I think he is relevant because he writes stuff that resonates. You might disagree, but I am judging him based on his very real accomplishments.
    - aydyn 2y ago
      If his ideas resonate and are so profound, tell us what they are and why. Don't point to his blog and worship him like some minor deity.
      - hgomersall 2y ago
        How does one worship a minor deity? You're not obliged to read what I post any more than I'm obliged to persuade you. Feel free to ignore the links, or feel free to read them. You might gain something from them as I have, but I'm not going to expend any efforts on trying to enlighten you or any such nonsense.
fnordpiglet 2y ago
Some companies such as standard oil had extraordinarily effective vertically integrated monopolies. The issues came into play when they used their market dominance to secure things like rebate structures with rail roads and made agreements to buy up transit capacity at very favorable rates that left competitors at a serious disadvantage. A real question might be how Nvidia prioritizes capacity and delivery and pricing to OpenAI and Microsoft for their superior product that has no meaningful competitor. It’s not an issue that Nvidia, like standard oil, had a superior business and product. It’s an issue when their scale and the scale of their adjacent partners lead to agreements that strangle others that need access to the supply chain. For instance if Anthropic is at a disadvantage to open ai for Nvidia capacity due to agreements between them that are mutually beneficial to the exclusion of Anthropic, that’s anti competitive.
- foota 2y ago
  Say that Nvidia partners with open AI because they think open AI is the best positioned to succeed (and hence result in increase demand for Nvidia) as opposed to if they got some direct financial kickback, it that legal?
  - fnordpiglet 2y ago
    Probably yes if Nvidia has a monopoly. Because competitors of open ai have no alternatives they’ve killed competition. If doing so is to their advantage that doesn’t make it better, that just makes it rational from their perspective. But regulation against anticompetitive behavior targets behavior that structurally kills competition. Most collusion isn’t direct financial transfers but cooperative agreements to dominate adjacent markets for mutual benefit.
- eru 2y ago
  Standard oil wasn't a monopoly. See https://en.wikipedia.org/wiki/Standard_Oil#Legacy_and_criticism_of_breakup https://en.wikipedia.org/wiki/Standard_Oil#Legacy_and_critic... They even lost market share during the time they were alleged of having and exploiting a monopoly: > Although Standard had 90 percent of American refining capacity in 1880, by 1911, that had shrunk to between 60 and 65 percent because of the expansion in capacity by competitors. See https://fee.org/articles/the-myth-that-standard-oil-was-a-predatory-monopoly/ https://fee.org/articles/the-myth-that-standard-oil-was-a-pr... for a more opinionated take.
uoaei 2y ago
> there is no moat and they operate in a commodity space that will behave like a normal competitive market Sure, until you consider the massive difference in accessible capital vs their rivals. Sure, VCs can fund startups, but Nvidia is publicly traded, which is just investment of a different form, and largely a form that you are barely accountable to. And that's on top of all the capital they directly control. That's an incredibly advantageous position to be in, and one that would be too tempting for most to resist exploiting.
- paulmd 2y ago
  AMD actually had more resources than NVIDIA for a good long while (although they did have more places to spread them over, that was their choice to do so), and the balance of power didn't really shift until the 2014-era when AMD really "took their foot off the gas" on R&D. 290X was the last time AMD could be described as "in the lead" and Maxwell opened up a gap that AMD never managed to close. Raja actually specifically called out 2012 as being the timeframe when that happened, because AMD decided the future of GPUs was iGPUs and not dGPUs, that the latter segment was going to go away entirely as iGPUs got faster and faster. Oops. https://youtu.be/590h3XIUfHg?t=1956 https://youtu.be/590h3XIUfHg?t=1956
blueboo 2y ago
Their competitors didn’t win. That’s not the same as failing, insofar as building technology goes. Business, sure
deadbabe 2y ago
AMD is more than just cards.
rafaelmn 2y ago
It's interesting that Jensen seems to be selling this "software development is dead to AI" story when that means his profitability moat (software) is dead as well. All AMD needs to do is "hey Copilot rewrite this CUDA code to run on AMD hardware". Somehow that's not happening.
- belter 2y ago
  AMD had 10 years to get competent on software. It's not happening.
belter 2y ago
It's a bit ironic to talk about lawsuits over monopolies in an emerging AI market, that hardly existed 24 months ago, while ignoring the two worst monopolies in different domains, led by Microsoft and Google.
- hot_gril 2y ago
  They aren't ignoring Microsoft and Google, that's ongoing too. But I agree that they should not be messing with the AI market.
mort96 2y ago
> There is no moat Isn't the CUDA API a fairly significant moat?
- roenxi 2y ago
  I don't think so, no. I had an AMD card; I didn't feel like the lack of CUDA was slowing me down. I managed to implement everything I tried on the CPU just fine. My problem was that every approach I tried to implement my own stuff on the GPU using things like Vulkan or OpenCL led to crashes or frustration. In fairness I didn't know how to do basic operations on a graphics card with OpenGL and would be relying on super basic programming material out of AMD with titles like "how to multiply a matrix on an AMD GPU" [0]. They can be forgiven for not supporting amateurs I suppose ... but now I own a Nvidia card and they have stuff like https://docs.nvidia.com/cuda/cublasdx/introduction1.html https://docs.nvidia.com/cuda/cublasdx/introduction1.html. But I stress that the issue there isn't that CUDA has tutorials. It is that I expect the tutorial code not to lead to a crash or system lockup. There is a well supported path to do those basic operations. If you look at the CUDA webpage [1] you see things like "cuRAND", "cuFFT" and "cuBLAS". That isn't a moat - those are first year software engineering topics. AMD managed to make it look like a moat by not taking the compute market seriously; and anyone else competes at highly parallel compute. [0] As far as I can tell; doesn't exist. [1] https://developer.nvidia.com/cuda-zone https://developer.nvidia.com/cuda-zone
  - hot_gril 2y ago
    You and a few others have said that OpenCL is buggy on AMD. Is it buggy on Nvidia too?
    - roenxi 2y ago
      I don't know. I'd use CUDA rather than experimenting with it; CUDA seems to be where all the investment is and my experience with OpenCL on AMD was so bad I don't feel the cross-platform argument would make any sense. It is a bit of a chicken-egg problem for me. I couldn't make OpenCL work on an AMD GPU, so I didn't manage to learn that much about OpenCL. At the time I assumed it was just me, but in hindsight I never saw an OpenCL-based approach to a compute problem that worked reliably on my machine so maybe it wasn't. But I don't think it really matters. The algorithms in the field don't seem to be hard and I never felt like I was struggling when implementing them on the CPU without any special API at all. My issues were conceptually similar to George Hotz's famous rants where he had crashes when running the demo app in a loop. In the experimenting phase I found I couldn't run code on the GPU with any API. I'm sure the situation improved and part of it was just me; towards the end of my time with AMD I could run stable diffusion inference and it'd work great for 10-40 minutes before the kernel paniced or whatever - so it was definitely technically possible to get a "hello world++" style thing running. But I never felt it was the APIs that were holding me back.
bayindirh 2y ago
> Nvidia solved a hard problem and their competitors all failed. And openly undermined technologies and standards that aimed to do the same in an interoperable way. This is, in my eyes, enough to hit with a litigation, but we'll see.
jb1991 2y ago
> self-inflicted apparent inability to multiply matrices on demand Isn't it one of the fundamental applications of a GPU, whether for ML compute or more commonly for 3D graphics, to multiple matrices? In what way did AMD GPUs fail to do this?
- sebzim4500 2y ago
  He means big matrices. And when you try to do compute on AMD GPUs they crash your kernel if you look at them wrong.
layla5alive 2y ago
You're grossly misinformed. CUDA is a huge anti-competitive moat, and that's no accident.
EasyMark 2y ago
Well I'm sure that the courts will agree with your assertion if it's quick. That's why it's not left up to just one regulatory agency to define an illegal monopoly. I, for one, am glad they are coming under scrutiny. If there's "no there there" then they'll be just fine.
WheatMillington 2y ago
All of that can be true AND they're engaging in anti-competitive behavior, which should not be tolerated.

by roenxi 2y ago

nimish 2y ago

Yep. AMD and Intel had two decades to come up with a CUDA alternative or even a parallel implementation and both failed to do anything. Incompetence.

deadbabe 2y ago
How do we know the strategy wasn’t to avoid releasing a CUDA alternative until NVIDIA is wounded by an anti-trust lawsuit and then when they get broken up deploy your CUDA alternative and speed past them? Could be playing the long game.
- throwthrowuknow 2y ago
  That’s not the case at all. AMD and Intel could be making money hand over fist right now if they had products that worked with the ML ecosystem. If they had any magic bullet they’d be idiots to not have used it years ago, they’re missing out on tens of billions of dollars on purpose for what? Even if NVIDIA got broken up their cards would still have massive demand and profit margins. I don’t think you understand how much of a shortage of compute there currently is, anyone who isn’t blatantly incompetent or ignoring ML can make money. AMD and Intel are just fat corporations who didn’t see the potential and thought business as usual would stay number one. Now they’re standing on the sidelines with their dicks in their hands while Jensen is leading the parade.
oytis 2y ago
The alternative to CUDA is called OpenCL. There has been speculations that poor performance of NVIDIA GPUs with OpenCL compared to CUDA is an intentional anticompetitive practice, but I don't feel confident to tell for sure.
- mschuetz 2y ago
  OpenCL is an alternative to CUDA just like Legos are an alternative to bricks. The problem with OpenCL isn't even the performance, it's everything. If OpenCL were any good, people could use it to build similarly powerful applications on cheaper AMD GPUs.
  - jb1991 2y ago
    OpenCL is just a spec. It's up to companies to implement it in a successful way or not. There is no reason in and of itself that OpenCL can't compete with CUDA on performance. The fact that Apple's Metal, which is pretty good, is actually implemented with a private OpenCL system is proof that the spec is not to blame.
    - jonas21 2y ago
      > The fact that Apple's Metal, which is pretty good, is actually implemented with a private OpenCL system is proof that the spec is not to blame. I don't understand. If OpenCL was so good, why did Apple create Metal instead of just using OpenCL?
      - jb1991 2y ago
        For similar reasons why Microsoft created DirectX. It allows them to have a system where the software and hardware are more tightly integrated than using a cross-platform spec. It also allows them to situate the API within the context of the operating system and other languages that are used on Apple platforms, making things easier on developers. And at least in that regard, they certainly succeeded. Metal is probably the easiest GPU API offered on any platform. Not necessarily the most powerful, but it’s almost trivial to hand write a compute kernel and spin it off.

vlovich123 2y ago

The thing with monopoly laws is it doesn’t matter how you got into pole position. The point is that you’re illegally utilizing tangential benefits of the pole position to maintain that position as well as enter new markets. > including how the company’s software locks customers into using its chips, as well as how Nvidia distributes those chips to customers. The lock-in is probably a losing argument on technical merits unless NVIDIA is doing something nefarious there (which the lawsuit will potentially reveal). If nVidia is distributing chips in preferred ways to partners it wants to succeed (e.g. those partnered with MSFT which it has a very cozy and tight-knit relationship), that could be an anti trust thing. > Microsoft structured its minority stake in OpenAI in part to avoid antitrust scrutiny, The Times has reported. Microsoft also reached a deal in March to hire most of the staff of Inflection AI, another A.I. start-up, and license its technology. Because the deal was not a standard acquisition, it may be harder for regulators to scrutinize. So there is an argument to be made that MSFT is acutely self-aware its behavior could be viewed as engaging in antitrust. If there’s a law against structuring financial transactions, why isn’t similar behavior in the business world viewed similarly as trying to bypass the spirit of the law?

roenxi 2y ago
Just on that last paragraph, because the frame annoys me - that is bullying. It isn't fair to insinuate that a company did something wrong by attempting to obey the law. MS knows all about antitrust. They've hired a bunch of lawyers and told them to do what needs to be done to be compliant with the regulations. That shouldn't then be implied as evidence that they are guilty of antitrust activities! What are the lawyers supposed to do, structure the company to maximise the likelihood of an antitrust lawsuit being bought? Nobody should want that, regulator, company or customer.
- iforgotpassword 2y ago
  Because laws are always flawless and never contain any loopholes, and it has never happened ever that lawyers were specifically instructed by a company to find those loopholes and exploit them as hard as possible.

trhway 2y ago

>Nvidia had nothing to do with their failure, unless they had some sort of high-level mole in AMD's driver teams. I wonder why Jensen wouldn't help his cousin Lisa Su with a good advice :) In general it isn't surprising that hardware companies like Intel and AMD fall at software - just talk to any programmer at such a hardware shop. What striking is that Jensen is able to have such a prominent and huge software development in a naturally hardware company. NVDA is the only hardware company where my programmer acquaintances don't complain about it being a hardware company.

Dalewyn 2y ago
>it isn't surprising that hardware companies like Intel and AMD fall at software A lot of what keeps Intel relevant is their prowess in both hardware and software, bringing properly refined products to market when it matters. Intel's failure against Nvidia had far more to do with their business strategy of CPU First missing the broadside of a barn; it's like bringing a knife to a gunfight, of course you're going to get your ass wrecked. AMD fucking sucks at software though, no disagreement there.
- 7f4784e3f1 2y ago
  Intel has some great software like MKL and related libraries, but, speaking from experience, due to its internal politics Intel is unable to innovate in a meaningful way or build a reasonable SW strategy for GPUs. The oneAPI heavily contributed to their GPU failures IMO. Choosing SYCL and promoting non-existing performance portability and pouring resources in the trying to shove CPU and GPU (and at some point FPGA!) into the same programming model while disinvesting from OpenCL was a big mistake, but it was probably the only way to please both CPU and GPU management. And the problem is not SYCL per se but the fact that they try to make it an open standard rather then trying to provide a way to extract maximum performance from their HW. They needed to build a CUDA alternative, where, after jumping through considerable hoops, you can get to actually programming tensor cores for those who need it. Not surprisingly, Intel is not using SYCL for their GPU kernels (look at oneDNN sources). With Raja gone there may be hope, but I'm not holding my breath. The only thing that they have that is competitive is Habana's stuff which is outside of the oneAPI. I would not be surprised if AMD has a similar issue of infighting due selling both CPUs and GPUs. And NVIDIA does not have this problem.

Dalewyn 2y ago

>not the obvious path to success. It is if success is defined as deliberate failure. Personally, a lot of US government actions of the past few decades in hindsight do not look like they had the interests of America or Americans in mind.

HPsquared 2y ago
I wonder if anyone connected to this is short NVDA.
pas 2y ago
Can you name a few examples please?

torginus 2y ago

Honestly I don't get how the current situation could develop. Video games have used compute shaders forever, and those work just fine on other cards. I remember doing some quite involved stuff for my thesis using OpenCL a decade ago, and it worked just fine. Nowadays OpenCL is dead for some reason... I just don't get what is there in CUDA that makes it so special. As far as I remember, a GPGPU API consists of a shader language, a way to copy stuff to and from the GPU, a way of scheduling and synchronizing work, atomics, groupshared variables, and some interop stuff. Is the vendor lock-in due to CUDA libs? In that case, the problem is not CUDA, but the libraries themselves. Not sure about today, but performance-portability basically didn't exist back then. You needed to do specialize your code for the GPU arch at hand. It didn't even exist between different generations of cards by the same vendor. Even if you could run CUDA code on AMD, it would be slow, so you need to do a rewrite anyways.

roenxi 2y ago
There probably isn't anything in CUDA that makes it special. They are well optimised math libraries and the math for most of the important stuff is somewhat trivial. AI seems to be >80% matrix multiplication - well optimised BLAS is tricky to implement, but even a bad implementation would see all the major libraries support AMD. The vendor "lock in" is because it takes a few years for decisions to be expressed in marketable silicon and literally only Nvidia was trying to be in the market 5 years ago. I've seen a lot of AMD cards that just crashed when used for anything outside OpenGL. I had a bunch of AI related projects die back in 2019 because initialising OpenCL crashed the drivers. If you believe the official docs everything would work fine. Great card except for the fact that compute didn't work. At the time I thought it was maybe just me. After seeing geohotz's saga trying to make tinygrad work on AMD cards and having a feel for how badly unsupported AMD hardware is by the machine learning community, it makes a lot of sense to me that it is a systemic issue and AMD didn't have any corporate sense of urgency about fixing those problems. Maybe there is something magic in CUDA, but if there is it is probably either their memory management model or something quite technical like that. Not the API.
- mschuetz 2y ago
  The magic is that CUDA actually works well. There is no reason to pick OpenCL, ROCm, Sycl or others if you get a 10x better developer experience with CUDA.

samplatt 2y ago

>Nvidia had nothing to do with their failure, unless they had some sort of high-level mole in AMD's driver teams. You're pretty close, but it's one level of abstraction away from the truth: NVidia spends absolutely earth-shakingly stonking amounts of money working directly with the devs of major studios, fixing issues with their drivers as they come, and in some cases paying studios to lock-in talking to them about the bugs they find. It's why (and how) they release new driver patches for every AAA release, and part of the reason why it triggers a redownload of the shader cache on a whole slew of Steam games every major version release. AMD and intel don't have a chance; NVidia is inside the design loop for game development far more thoroughly than other hardware vendors.

reissbaker 2y ago
Of course AMD and Intel could just do the same thing. But they don't. That's not Nvidia's fault. In fact I wish Intel and AMD would do the same! It's great customer service to know that games are going to work well out of the box with Nvidia, because Nvidia spent tons of money working directly with devs on optimization and even per-game bugfixes. As a longtime PC gamer I wish AMD released better GPUs, but they seem incapable of doing so generation after generation. Intel is getting better at the low end, so there's at least some hope there. But Nvidia is just the best in class, because they put in a huge amount of polish and work. My understanding is this is the same approach Nvidia took with CUDA: investing a massive amount to make it fast, accurate, and broadly available, while AMD continued to push out barely-tested drivers — for example, see hotz's woes with tinygrad and finally giving up and labeling the "red" (aka AMD) tinybox driver quality as "Mediocre" on their product page, with the "green" (aka Nvidia) tinybox driver quality listed as "Great" https://tinygrad.org/ https://tinygrad.org/ — and to this day restricts their competing library (ROCm) to specific ultra-expensive server-class GPUs.
- Dalewyn 2y ago
  >In fact I wish Intel and AMD would do the same! Intel at least works very closely with Microsoft to iron out the details for Windows.
WheatMillington 2y ago

hgomersall 2y ago

Cory Doctorow argues well why it doesn't matter how you got into a monopoly position: https://pluralistic.net/2024/05/18/market-discipline/#too-big-to-care https://pluralistic.net/2024/05/18/market-discipline/#too-bi...

aydyn 2y ago
Still have no idea how this dude is relevant other than he wrote some scifi books one time. What are his technical accomplishments. Oh, nothing? Anyone can talk, especially nowdays because of AI, ironically.
- eru 2y ago
  Many people like his writing, and his political stances.
- criddell 2y ago
  Many of us appreciate the work he’s put in with the EFF. When you are right about things over-and-over for 20 or 30 years, people tend to listen to you.
- hgomersall 2y ago
  I think he is relevant because he writes stuff that resonates. You might disagree, but I am judging him based on his very real accomplishments.
  - aydyn 2y ago
    If his ideas resonate and are so profound, tell us what they are and why. Don't point to his blog and worship him like some minor deity.
    - hgomersall 2y ago
      How does one worship a minor deity? You're not obliged to read what I post any more than I'm obliged to persuade you. Feel free to ignore the links, or feel free to read them. You might gain something from them as I have, but I'm not going to expend any efforts on trying to enlighten you or any such nonsense.

fnordpiglet 2y ago

Some companies such as standard oil had extraordinarily effective vertically integrated monopolies. The issues came into play when they used their market dominance to secure things like rebate structures with rail roads and made agreements to buy up transit capacity at very favorable rates that left competitors at a serious disadvantage. A real question might be how Nvidia prioritizes capacity and delivery and pricing to OpenAI and Microsoft for their superior product that has no meaningful competitor. It’s not an issue that Nvidia, like standard oil, had a superior business and product. It’s an issue when their scale and the scale of their adjacent partners lead to agreements that strangle others that need access to the supply chain. For instance if Anthropic is at a disadvantage to open ai for Nvidia capacity due to agreements between them that are mutually beneficial to the exclusion of Anthropic, that’s anti competitive.

foota 2y ago
Say that Nvidia partners with open AI because they think open AI is the best positioned to succeed (and hence result in increase demand for Nvidia) as opposed to if they got some direct financial kickback, it that legal?
- fnordpiglet 2y ago
  Probably yes if Nvidia has a monopoly. Because competitors of open ai have no alternatives they’ve killed competition. If doing so is to their advantage that doesn’t make it better, that just makes it rational from their perspective. But regulation against anticompetitive behavior targets behavior that structurally kills competition. Most collusion isn’t direct financial transfers but cooperative agreements to dominate adjacent markets for mutual benefit.
eru 2y ago
Standard oil wasn't a monopoly. See https://en.wikipedia.org/wiki/Standard_Oil#Legacy_and_criticism_of_breakup https://en.wikipedia.org/wiki/Standard_Oil#Legacy_and_critic... They even lost market share during the time they were alleged of having and exploiting a monopoly: > Although Standard had 90 percent of American refining capacity in 1880, by 1911, that had shrunk to between 60 and 65 percent because of the expansion in capacity by competitors. See https://fee.org/articles/the-myth-that-standard-oil-was-a-predatory-monopoly/ https://fee.org/articles/the-myth-that-standard-oil-was-a-pr... for a more opinionated take.

uoaei 2y ago

> there is no moat and they operate in a commodity space that will behave like a normal competitive market Sure, until you consider the massive difference in accessible capital vs their rivals. Sure, VCs can fund startups, but Nvidia is publicly traded, which is just investment of a different form, and largely a form that you are barely accountable to. And that's on top of all the capital they directly control. That's an incredibly advantageous position to be in, and one that would be too tempting for most to resist exploiting.

paulmd 2y ago
AMD actually had more resources than NVIDIA for a good long while (although they did have more places to spread them over, that was their choice to do so), and the balance of power didn't really shift until the 2014-era when AMD really "took their foot off the gas" on R&D. 290X was the last time AMD could be described as "in the lead" and Maxwell opened up a gap that AMD never managed to close. Raja actually specifically called out 2012 as being the timeframe when that happened, because AMD decided the future of GPUs was iGPUs and not dGPUs, that the latter segment was going to go away entirely as iGPUs got faster and faster. Oops. https://youtu.be/590h3XIUfHg?t=1956 https://youtu.be/590h3XIUfHg?t=1956

blueboo 2y ago

Their competitors didn’t win. That’s not the same as failing, insofar as building technology goes. Business, sure

deadbabe 2y ago

AMD is more than just cards.

rafaelmn 2y ago

It's interesting that Jensen seems to be selling this "software development is dead to AI" story when that means his profitability moat (software) is dead as well. All AMD needs to do is "hey Copilot rewrite this CUDA code to run on AMD hardware". Somehow that's not happening.

belter 2y ago
AMD had 10 years to get competent on software. It's not happening.

belter 2y ago

It's a bit ironic to talk about lawsuits over monopolies in an emerging AI market, that hardly existed 24 months ago, while ignoring the two worst monopolies in different domains, led by Microsoft and Google.

hot_gril 2y ago
They aren't ignoring Microsoft and Google, that's ongoing too. But I agree that they should not be messing with the AI market.

mort96 2y ago

> There is no moat Isn't the CUDA API a fairly significant moat?

roenxi 2y ago
I don't think so, no. I had an AMD card; I didn't feel like the lack of CUDA was slowing me down. I managed to implement everything I tried on the CPU just fine. My problem was that every approach I tried to implement my own stuff on the GPU using things like Vulkan or OpenCL led to crashes or frustration. In fairness I didn't know how to do basic operations on a graphics card with OpenGL and would be relying on super basic programming material out of AMD with titles like "how to multiply a matrix on an AMD GPU" [0]. They can be forgiven for not supporting amateurs I suppose ... but now I own a Nvidia card and they have stuff like https://docs.nvidia.com/cuda/cublasdx/introduction1.html https://docs.nvidia.com/cuda/cublasdx/introduction1.html. But I stress that the issue there isn't that CUDA has tutorials. It is that I expect the tutorial code not to lead to a crash or system lockup. There is a well supported path to do those basic operations. If you look at the CUDA webpage [1] you see things like "cuRAND", "cuFFT" and "cuBLAS". That isn't a moat - those are first year software engineering topics. AMD managed to make it look like a moat by not taking the compute market seriously; and anyone else competes at highly parallel compute. [0] As far as I can tell; doesn't exist. [1] https://developer.nvidia.com/cuda-zone https://developer.nvidia.com/cuda-zone
- hot_gril 2y ago
  You and a few others have said that OpenCL is buggy on AMD. Is it buggy on Nvidia too?
  - roenxi 2y ago
    I don't know. I'd use CUDA rather than experimenting with it; CUDA seems to be where all the investment is and my experience with OpenCL on AMD was so bad I don't feel the cross-platform argument would make any sense. It is a bit of a chicken-egg problem for me. I couldn't make OpenCL work on an AMD GPU, so I didn't manage to learn that much about OpenCL. At the time I assumed it was just me, but in hindsight I never saw an OpenCL-based approach to a compute problem that worked reliably on my machine so maybe it wasn't. But I don't think it really matters. The algorithms in the field don't seem to be hard and I never felt like I was struggling when implementing them on the CPU without any special API at all. My issues were conceptually similar to George Hotz's famous rants where he had crashes when running the demo app in a loop. In the experimenting phase I found I couldn't run code on the GPU with any API. I'm sure the situation improved and part of it was just me; towards the end of my time with AMD I could run stable diffusion inference and it'd work great for 10-40 minutes before the kernel paniced or whatever - so it was definitely technically possible to get a "hello world++" style thing running. But I never felt it was the APIs that were holding me back.

bayindirh 2y ago

> Nvidia solved a hard problem and their competitors all failed. And openly undermined technologies and standards that aimed to do the same in an interoperable way. This is, in my eyes, enough to hit with a litigation, but we'll see.

jb1991 2y ago

> self-inflicted apparent inability to multiply matrices on demand Isn't it one of the fundamental applications of a GPU, whether for ML compute or more commonly for 3D graphics, to multiple matrices? In what way did AMD GPUs fail to do this?

sebzim4500 2y ago
He means big matrices. And when you try to do compute on AMD GPUs they crash your kernel if you look at them wrong.

layla5alive 2y ago

You're grossly misinformed. CUDA is a huge anti-competitive moat, and that's no accident.

EasyMark 2y ago

Well I'm sure that the courts will agree with your assertion if it's quick. That's why it's not left up to just one regulatory agency to define an illegal monopoly. I, for one, am glad they are coming under scrutiny. If there's "no there there" then they'll be just fine.

WheatMillington 2y ago

All of that can be true AND they're engaging in anti-competitive behavior, which should not be tolerated.