11 ms·

The problem with effectively supporting CUDA is that encourages CUDA adoption all the more strongly. Meanwhile, AMD will always be playing catch-up, forever hav

The problem with effectively supporting CUDA is that encourages CUDA adoption all the more strongly. Meanwhile, AMD will always be playing catch-up, forever having to patch issues, work around Nvidia/AMD differences, and accept the performance penalty that comes from having code optimised for another vendor's hardware. AMD needs to encourage developers to use their own ecosystem or an open standard.

slashdev 3y ago
With Nvidia controlling 90%+ of the market, this is not a viable option. They'd better lean hard into CUDA support if they want to be relevant.
- cduzz 3y ago
  A bit of story telling here: IBM and Microsoft made OS/2. The first version worked on 286s and was stable but useless. The second version worked only on 386s and was quite good, and even had wonderful windows 3.x compatibility. "Better windows than windows!" At that point Microsoft wanted out of the deal and they wanted to make their newer version of windows, NT, which they did. IBM now had a competitor to "new" windows and a very compatible version of "old" windows. Microsoft killed OS2 by a variety of ways (including just letting IBM be IBM) but also by making it very difficult for last month's version of OS/2 to run next month's bunch of Windows programs. To bring this back to the point -- IBM vs Microsoft is akin to AMD vs Nvidia -- where nvidia has the standard that AMD is implementing, and so no matter what if you play in the backward compatibility realm you're always going to be playing catch-up and likely always in a position where winning is exceedingly hard. As WOPR once said "interesting game; the only way to win is to not play."
  - panick21_ 3y ago
    IBM also made a whole bunch of strategic mistakes beyond that. Most importantly their hardware division didn't give a flying f about OS/2. Even when they had a 'better Windows' they did not actually use it themselves and didn't push it to other vendors. Windows NT wasn't really relevant in that competition for much longer, only XP was finally for end consumers. > where nvidia has the standard that AMD is implementing, and so no matter what if you play in the backward compatibility realm you're always going to be playing catch-up That's not true. If AMD starts adding their own features and have their own advantages, that can flip. It only takes a single generation of hardware, or a single feature for things to flip. Look at Linux and Unix. Its started out with Linux implementing Unix, and now the Unix are trying to add compatibility with with Linux. Is SGI still the driving force behind OpenGL/Vulcan? Did you think it was a bad idea for other companies to use OpenGL? AMD was successful against Intel with x86_64. There are lots of example of the company making something popular, not being able to take full advantage of it in the long run.
    - chuckadams 3y ago
      Slapping a price tag of over $300 on OS/2 didn’t do IBM any favors either.
      - BizarroLand 3y ago
        That's what happens when your primary business model is selling to the military. They had to pay what IBM charged them (within a small bit of reason) and it was incredibly difficult for them to pivot away from any path they chose in the 80's once they had chosen it. However, that same logic doesn't apply to consumers, and since they continued to fail to learn that lesson now IBM doesn't even target the consumer market given that they never learned how to be competitive and could only ever effectively function when they had a monopoly or at least a vendor lock-in. https://en.wikipedia.org/wiki/Acquisition_of_the_IBM_PC_business_by_Lenovo https://en.wikipedia.org/wiki/Acquisition_of_the_IBM_PC_busi...
  - incrudible 3y ago
    Windows before NT was crap, so users had an incentive to upgrade. If there had existed a Windows 7 alternative that was near fully compatible and FOSS, I would wager Microsoft would have lost to it with Windows 8 and even 10. The only reason to update for most people was Microsoft dropping support. For CUDA, it is not just AMD who would need to catch up. Developers also are not necessarily going to target the latest feature set immediately, especially if it only benefits (or requires) new hardware. I accept the final statement, but that also means AMD for compute is gonna be dead like OS/2. Their stack just will not reach critical mass.
    - BizarroLand 3y ago
      Todays linux OS's would have competed incredibly strongly against Vista and probably would have gone blow for blow against 7. Proton, Wine, and all of the compatibility fixes and drive improvements that the community has made in the last 16 years has been amazing, and every day is another day where you can say that it has never been easier to switch away from Windows. However, Microsoft has definitely been drinking the IBM koolaid a little to long and has lost the mandate of heaven. I think in the next 7-10 years we will reach a point where there is nothing Windows can do that linux cannot do better and easier without spying on you, and we may be 3-5 years from a "killer app" that is specifically built to be incompatible with Windows just as a big FU to them, possibly in the VR world, possibly in AR, and once that happens maybe, maybe, maybe it will finally actually be the year of the linux desktop.
      - paulmd 3y ago
        > However, Microsoft has definitely been drinking the IBM koolaid a little to long and has lost the mandate of heaven. I think in the next 7-10 years we will reach a point where there is nothing Windows can do that linux cannot do better and easier without spying on you that's a fascinating statement with the clear ascendancy of neural-assisted algorithms etc. Things like DLSS are the future - small models that just quietly optimize some part of a workload that was commonly considered impossible to the extent nobody even thinks about it anymore. my prediction is that in 10 years we are looking at the rise of tag+collection based filesystems and operating system paradigms. all of us generate a huge amount of "digital garbage" constantly, and you either sort it out into the important stuff, keep temporarily, and toss, or you accumulate a giant digital garbage pile. AI systems are gonna automate that process, it's gonna start on traditional tree-based systems but eventually you don't need the tree at all, AI is what's going to make that pivot to true tag/collection systems possible. Tags mostly haven't worked because of a bunch of individual issues which are pretty much solved by AI. Tags aren't specific enough: well, AI can give you good guesses at relevance. Tagging files and maintaining collections is a pain: well, the AI can generate tags and assign collections for you. Tags really require an ontology for "fuzzy" matching (search for "food" should return the tag "hot dog") - well, LLMs understand ontologies fine. Etc etc. And if you do it right, you can basically have the AI generate "inbox/outbox" for you, deduplicate files and handle versioning, etc, all relatively seamlessly. microsoft and macos are both clearly racing for this with the "AI os" concept. It's not just better relevance searches etc. And the "generate me a whole paragraph before you even know what I'm trying to type" stuff is not how it's going to work either. That stuff is like specular highlights in video games around 2007 or whatever - once you had the tool, for a few years everything was w e t until developers learned some restraint with it. But there are very very good applications that are going to come out in the 10 year window that are going to reduce operator cognitive load by a lot - that is the "AI OS" concept. What would the OS look like if you truly had the "computer is my secretary" idea? Not just dictating memorandums, but assistance in keeping your life in order and keeping you on-task. I simply cannot see linux being able to keep up with this change, in the same way the kernel can't just switch to rust - at some point you are too calcified to ever do the big-bang rewrite if there is not a BDFL telling you that it's got to happen. the downside of being "the bazaar" is that you are standards-driven and have to deal with corralling a million whiny nerds constantly complaining about "spying on me just like microsoft" and continuing to push in their own other directions (sysvinit/upstart/systemd factions, etc) and whatever else, on top of all the other technical issues of doing a big-bang rewrite. linux is too calcified to ever pivot away from being a tree-based OS and it's going to be another 2-3 decades before they catch up with "proper support for new file-organization paradigms" etc even in the smaller sense. that's really just the tip of the iceberg on the things AI is going to change, and linux is probably going to be left out of most of those commercial applications despite being where the research is done. It's just too much of a mess and too many nerdlingers pushing back to ever get anything done. Unix will be represented in this new paradigm but not Linux - the commercial operators who have the centralization and fortitude to build a cathedral will get there much quicker, and that looks like MacOS or Solaris not linux. Or at least, unless I see some big announcement from KDE or Gnome or Canonical/Red Hat about a big AI-OS rewrite... I assume that's pretty much where the center of gravity is going to stay for linux.
  - foobiekr 3y ago
    IBM was also incompetent and the os/2 team in Boca was had some exceptional engineers but was packed witg mostly mediocre-to-bad ones, which is why so many things in OS/2 were bad and why IBM got upset for Microsoft contributing negative work to the project because their lines of code contribution was negative (they were rewriting a lot of inefficient bloated IBM code). A lot went wrong with os/2. For CUDA, I think a better analogy is vhs. The standard, in the effective not open sense, is what it is. AMD sucks at software and views it as an expense rather than an advantage.
    - AYBABTME 3y ago
      You would think that by now AMD realizes that poor software is what left them behind in the dust, and would have changed that mindset.
      - hyperman1 3y ago
        Most businesses understand the pain points of their suppliers very well, as they feel that pain and gave themselves organized around it. They have a hard time to understand the pain points of their consumers, as they don't feel that pain, look trough their own organisation-coloured glases, and can't see the real pain points from the whiney-customer ones. AMD probably thinks software ecosystems are the easy part, ready to take it on whenever they feel like it and throw a token amount at it. They've built a great engine, see the carossery as beneath them, and don't understand why the lazy customer wants them to build the rest of the car too.
  - neerajsi 3y ago
    I'm not in the gpu programming realm, so this observation might be inaccurate: I think the case of cuda vs an open standard is different from os2 vs Windows because the customers of cuda are programmers with access to source code while the customers of os2 were end users trying to run apps written by others. If your shrink-wrapped software didn't run on os2, you'd have no choice but to go buy Windows. Otoh if your ai model doesn't run on an AMD device and the issue is something minor, you can edit the shader code.
kgeist 3y ago
Intel embraced Amd64 ditching Itanium. Wasn't it a good decision that worked out well? Is it comparable?
- teucris 3y ago
  In hindsight, yes, but just because a specific technology is leading an industry doesn’t mean it’s going to be the best option. It has to play out long enough for the market to indicate a preference. In this case, for better or worse, it looks like CUDA’s the preference.
  - diggan 3y ago
    > It has to play out long enough for the market to indicate a preference By what measures hasn't that happened already? CUDA been around and constantly improving for more than 15 years, and there is no competitors in sight so far. It's basically the de facto standard in many ecosystems.
    - teucris 3y ago
      There haven’t been any as successful, but there have been competitors. OpenCL, DirectX come to mind.
      - cogman10 3y ago
        SYCL is the latest attempt that I'm aware of. It's still pretty active and may just work as it doesn't rely on video card manufactures to work out.
        zozbot234 3y ago
        SYCL is the quasi-successor to OpenCL, built on the same flavor of SPIR-V. Various efforts are trying to run it on top of Vulkan Compute (which tends to be broadly support by modern GPU's) but it's non-trivial because the technologies are independently developed and there are some incompatibilities.
bachmeier 3y ago
> The problem with effectively supporting CUDA is that encourages CUDA adoption all the more strongly. I'm curious about this. Sure some CUDA code has already been written. If something new comes along that provides better performance per dollar spent, why continue writing CUDA for new projects? I don't think the argument that "this is what we know how to write" works in this case. These aren't scripts you want someone to knock out quickly.
- Uehreka 3y ago
  > If something new comes along that provides better performance per dollar spent They won’t be able to do that, their hardware isn’t fast enough. Nvidia is beating them at hardware performance, AND ALSO has an exclusive SDK (CUDA) that is used by almost all deep learning projects. If AMD can get their cards to run CUDA via ROCm, then they can begin to compete with Nvidia on price (though not performance). Then, and only then, if they can start actually producing cards with equivalent performance (also a big stretch) they can try for an Embrace Extend Extinguish play against CUDA.
  - bachmeier 3y ago
    > They won’t be able to do that, their hardware isn’t fast enough. Well, then I guess CUDA is not really the problem, so being able to run CUDA on AMD hardware wouldn't solve anything. > try for an Embrace Extend Extinguish play against CUDA They wouldn't need to go that route. They just need a way to run existing CUDA code on AMD hardware. Once that happens, their customers have the option to save money by writing ROCm or whatever AMD is working on at that time.
    - deleted 3y ago
      [deleted]
    - foobiekr 3y ago
      Intel has the same software issue as AMD but their hardware is genuinely competitive if a generation behind. Cost and power wise, Intel is there; software? No.
panick21_ 3y ago
That's not guaranteed at all. One could make the same argument about Linux vs Commercial Unix. If the put their stuff as OpenSource, including firmware, I think they will win out eventually. And its also not a guarantee that Nvidia will always produce the superior hardware for that code.
throwoutway 3y ago
Is it? Apple Silicon exists, but Apple created a translation layer above it so the transition could be smoother.
- jack_pp 3y ago
  not really the same in that Apple was absolutely required to do this in order for people to transition smoothly and it wasn't competing against another company / platform, it just needed apps from its previous platform to work while people recompile apps for the current one which they will
- Jorropo 3y ago
  This is extremely different, apple was targeting end consumers that just want their app to run. The performance between apple rosetta and native cpu were still multiple times different. People writing CUDA apps don't just want stuff to run, performance is an extremely important factor else they would target CPUs which are easier to program for. From their readme: > On Server GPUs, ZLUDA can compile CUDA GPU code to run in one of two modes: > Fast mode, which is faster, but can make exotic (but correct) GPU code hang. > Slow mode, which should make GPU code more stable, but can prevent some applications from running on ZLUDA.
  - hamandcheese 3y ago
    > The performance between apple rosetta and native cpu were still multiple times different. Rosetta 2 runs apps at 80-90% their native speed.
    - Jorropo 3y ago
      Indeed I got that wrong. Sadly minimal SIMD and hardware acceleration support.
  - piva00 3y ago
    > The performance between apple rosetta and native cpu were still multiple times different. Not at all, the performance hit was in the low 10s %, before natively supporting Apple Silicon most of the apps I use for music/video/photography didn't seem to have a performance impact at all, even more when the M1 machines were so much faster than the Intels.
coldtea 3y ago
>The problem with effectively supporting CUDA is that encourages CUDA adoption all the more strongly Worked fine for MS with Excel supporting Lotus 123 and Word supporting WordPerfect's formats when those were dominant...
- Dork1234 3y ago
  Microsoft could do that because they had the Operating System monopoly to leverage and take out both Lotus 123 and WordPerfect. Without the monopoly of the operating system they wouldn't of been able to Embrace, Extend, Extinguish. https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
- bell-cot 3y ago
  But MS controlled the underlying OS. Letting them both throw money at the problem, and (by accounts at the time) frequently tweak the OS in ways that made life difficult for Lotus, WordPerfect, Ashton-Tate, etc.
  - p_l 3y ago
    Last I checked, Lotus did themselves by not innovating, and betting on the wrong horse (OS/2) then not doing well on a pivot to Windows. Meanwhile Excel was gaining features and winning users with them even before Windows was in play.
    - dadadad100 3y ago
      This is a key point. Before windows we had all the dos players - WordPerfect was king. Microsoft was more focused on the Mac. I’ve always assumed that Microsoft understood that a GUI was coming and trained a generation of developers on the main gui of the day. Once windows came out the dos focused apps could not adapt in time
    - robocat 3y ago
      > betting on the wrong horse (OS/2) Ahhhh, your hindsight is well developed. I would be interested to know the background on the reasons why Lotus made that bet. We can't know the counterfactual, but Lotus delivering on a platform owned by their deadly competitor Microsoft would seem to me to be a clearly worrysome idea to Lotus at the time. Turned out it was an existentially bad idea. Did Lotus fear Microsoft? "DOS ain't done till Lotus won't run" is a myth[1] for a reason. Edit: DRDOS errors[2] were one reason Lotus might fear Microsoft. We can just imagine a narritive of a different timeline where Lotus delivered on Windows but did some things differently to beat Excel. I agree, Lotus made other mistakes and Microsoft made some great decisions, but the point remains. We can also suspect that AMD have a similar choice now where they are forked. Depending on Nvidea/CUDA may be a similar choice for AMD - fail if they do and fail if they don't. [1] http://www.proudlyserving.com/archives/2005/08/dos_aint_done_t.html http://www.proudlyserving.com/archives/2005/08/dos_aint_done... [2] https://www.theregister.com/1999/11/05/how_ms_played_the_incompatibility/ https://www.theregister.com/1999/11/05/how_ms_played_the_inc...
andy_ppp 3y ago
When the alternative is failure I suppose you choose the least bad option. Nobody is betting the farm on ROCm!
- hjabird 3y ago
  True. This is the big advantage of an open standard instead jumping from one vendors walled garden to another.
more_corn 3y ago
They have already lost. The question is do they want to come in second in the game to control the future of the world or not play at all?
bick_nyers 3y ago
The latest version of CUDA is 12.3, and version 12.2 came out 6 months prior. How many people are running an older version of CUDA right now on NVIDIA hardware for whatever particular reason? Even if AMD lagged support on CUDA versioning, I think it would be widely accepted if the performance per dollar at certain price points was better. Taking the whole market from NVIDIA is not really an option, it's better to attack certain price points and niches and then expand from there. The CUDA ship sailed a long time ago in my view.
- swozey 3y ago
  I just went through this this weekend - If you're running in Windows and want to use deepspeed, you have to still use Cuda 12.1 because deepspeed 13.1 is the latest that works with 12.1. There's no deepspeed for windows that works with 12.3. I tried to get it working this weekend but it was a huge PITA so I switched to putting everything into WSL2 then in arch on there pytorch etc in containers so I could flip versions easily now that I know how SPECIFIC the versions are to one another. I'm still working on that part, halfway into it my WSL2 completely broke and I had to reinstall windows. I'm scared to mount the vhdx right now. I did ALL of my work and ALL of my documentation is inside of the WSL2 archlinux and NOT on my windows machine. I have EVERYTHING I need to quickly put another server up (dotfiles, configs) sitting in a chezmoi git repo ON THE VM. That I only git committed one init like 5 mins into everything. THAT was a learning experience, now I have no idea if I should follow the "best practice" of keeping projects in wsl or having wsl reach out to windows, there's a performance drop. The 9p networking stopped working and no matter what I reinstalled, reset, removed features, reset windows, etc, it wouldn't start. But at least I have that WSL2 .vhdx image that will hopefully mount and start. And probably break WSL2 again. I even SPECIFICALLY took backups of the image as tarballs every hour in case I broke LINUX, not WSL. If anyone has done sd containers in wsl2 already let me know. I've tried to use WSL for dev work (i use osx) like this 2-3 times in the last 4-5 years and I always run into some catastrophically broken thing that makes my WSL stop working. I hadn't used it in years so hoped it was super reliable by now. This is on 3 different desktops with completely different hardware, etc. I was terrified it would break this weekend and IT DID. At least I can be up in windows in 20 minutes thanks to chocolately and chezmoi. Wiped out my entire gaming desktop. Sorry I'm venting now this was my entire weekend. This repo is from a deepspeed contrib (iirc) and lists the reqs for deepspeed + windows that mention the version matches https://github.com/S95Sedan/Deepspeed-Windows https://github.com/S95Sedan/Deepspeed-Windows > conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia It may sound weird to do any of this in Windows, or maybe not, but if it does just remember that it's a lot of gamers like me with 4090s who just want to learn ML stuff as a hobby. I have absolutely no idea what I'm doing but thank god I know containers and linux like the back of my hand.
hjabird 3y ago
There are some great replies to my comment - my original comment was too reductive. However, I still think that entrenching CUDA as the de-facto language for heterogeneous computing is a mistake. We need an open ecosystem for AI and HPC, where vendors compete on producing the best hardware.
- ethbr1 3y ago
  The problem with open standards is that someone has to write them. And that someone usually isn't a manufacturer, lest the committee be accused of bias. Consequently, you get (a) outdated features that SotA has already moved beyond, (b) designed in a way that doesn't correspond to actual practice, and (c) that are overly generalized. There are some notable exceptions (e.g. IETF), but the general rule has been that open specs please no one, slowly. IMHO, FRAND and liberal cross-licensing produce better results.
  - jchw 3y ago
    Vulkan already has some standard compute functionality. Not sure if it's low level enough to be able to e.g. recompile and run CUDA kernels, but I think if people were looking for a vendor-neutral standard to build GPGPU compute features on top of, I mean, that seems to be the obvious modern choice.
    - zozbot234 3y ago
      There is already a work-in-progress implementation of HIP on top of OpenCL https://github.com/CHIP-SPV/chipStar https://github.com/CHIP-SPV/chipStar and the Mesa RustiCL folks are quite interested in getting that to run on top of Vulkan. (To be clear, HIP is about converting CUDA source code not running CUDA-compiled binaries but the Zluda project discussed in OP heavily relies on it.)
jvanderbot 3y ago
If you replace CUDA -> x86 and NVIDIA -> Intel, you'll see a familiar story which AMD has already proved it can work through. These were precisely the arguments for 'x86 will entrench Intel for all time', and we've seen AMD succeed at that game just fine.
- ianlevesque 3y ago
  And indeed more than succeed, they invented x86_64.
  - stcredzero 3y ago
    And indeed more than succeed, they invented x86_64. If AMD invented the analogous to x86_64 for CUDA, this would increase competition and progress in AI by some huge fraction.
    - pjmlp 3y ago
      Only works if NVidia misteps and creates the Itanium version of CUDA.
      - stcredzero 3y ago
        You don't think someone would welcome the option to have more hardware buying options, even if the "Itanium version" didn't happen?
    - jvanderbot 3y ago
      Why "if only". Intel had been around forever when AMD showed up. CUDA isn't unassailable
  - sangnoir 3y ago
    x86_64's win was helped by Intel's Itanium misstep. AMD can't bank on Nvidia making a mistake, and Nvidia seems content with incremental changes to CUDA, contrasted with Intel's 32-bit to 64-bit transition. It is highly unlikely that AMD can find and exploit a similar chink in the amor against CUDA.
mindcrime 3y ago
Yep. This is very similar to the "catch-22" that IBM wound up in with OS/2 and the Windows API. On the one hand, by supporting Windows software on OS/2, they gave OS/2 customers access to a ready base of available, popular software. But in doing so, they also reduced the incentive for ISV's to produce OS/2 native software that could take advantage of unique features of OS/2. It's a classic "between a rock and a hard place" scenario. Quite a conundrum.
- ianlevesque 3y ago
  Thinking about the highly adjacent graphics APIs history, did anyone really 'win' the Direct3D, OpenGL, Metal, Vulkan war? Are we benefiting from the fragmentation? If the players in the space have naturally coalesced around one over the last decade, can we skip the thrashing and just go with it this time?
  - tadfisher 3y ago
    The game engines won. Folks aren't building Direct3D or Vulkan renderers; they're using Unity or Unreal or Godot and clicking "export" to target whatever API makes sense for the platform. WebGPU might be the thing that unifies the frontend API for folks writing cross-platform renderers, seeing as browsers will have to implement it on top of the platform APIs anyway.
  - pjmlp 3y ago
    You missed the Nintendo and Sony APIs as well. FOSS folks make this a bigger issue than it really is, game studios make a pluggable API on their engine and call it a day, move on into everything else that matters in actually delivering a game.
mqus 3y ago
If their primary objective is to sell cards, then they should make it as easy as possible to switch cards. If their primary objective is to break the CUDA monopoly, they should up their game in software, which means going as far as implementing support for their hardware in the most popular user apps themselves, if necessary. But since they don't seem to want to do that, they should really go for option one, especially if a single engineer already got so far. Let's say AMD sold a lot of cards with CUDA support. Now nvidia tries to cut them off. What will happen next? A lot of people will replace their cards with nvidia ones. But a lot of the rest will try to make their expensive AMD cards work regardless. And if AMD provides a platform for that, they will get that work for free.

slashdev 3y ago

With Nvidia controlling 90%+ of the market, this is not a viable option. They'd better lean hard into CUDA support if they want to be relevant.

cduzz 3y ago
A bit of story telling here: IBM and Microsoft made OS/2. The first version worked on 286s and was stable but useless. The second version worked only on 386s and was quite good, and even had wonderful windows 3.x compatibility. "Better windows than windows!" At that point Microsoft wanted out of the deal and they wanted to make their newer version of windows, NT, which they did. IBM now had a competitor to "new" windows and a very compatible version of "old" windows. Microsoft killed OS2 by a variety of ways (including just letting IBM be IBM) but also by making it very difficult for last month's version of OS/2 to run next month's bunch of Windows programs. To bring this back to the point -- IBM vs Microsoft is akin to AMD vs Nvidia -- where nvidia has the standard that AMD is implementing, and so no matter what if you play in the backward compatibility realm you're always going to be playing catch-up and likely always in a position where winning is exceedingly hard. As WOPR once said "interesting game; the only way to win is to not play."
- panick21_ 3y ago
  IBM also made a whole bunch of strategic mistakes beyond that. Most importantly their hardware division didn't give a flying f about OS/2. Even when they had a 'better Windows' they did not actually use it themselves and didn't push it to other vendors. Windows NT wasn't really relevant in that competition for much longer, only XP was finally for end consumers. > where nvidia has the standard that AMD is implementing, and so no matter what if you play in the backward compatibility realm you're always going to be playing catch-up That's not true. If AMD starts adding their own features and have their own advantages, that can flip. It only takes a single generation of hardware, or a single feature for things to flip. Look at Linux and Unix. Its started out with Linux implementing Unix, and now the Unix are trying to add compatibility with with Linux. Is SGI still the driving force behind OpenGL/Vulcan? Did you think it was a bad idea for other companies to use OpenGL? AMD was successful against Intel with x86_64. There are lots of example of the company making something popular, not being able to take full advantage of it in the long run.
  - chuckadams 3y ago
    Slapping a price tag of over $300 on OS/2 didn’t do IBM any favors either.
    - BizarroLand 3y ago
      That's what happens when your primary business model is selling to the military. They had to pay what IBM charged them (within a small bit of reason) and it was incredibly difficult for them to pivot away from any path they chose in the 80's once they had chosen it. However, that same logic doesn't apply to consumers, and since they continued to fail to learn that lesson now IBM doesn't even target the consumer market given that they never learned how to be competitive and could only ever effectively function when they had a monopoly or at least a vendor lock-in. https://en.wikipedia.org/wiki/Acquisition_of_the_IBM_PC_business_by_Lenovo https://en.wikipedia.org/wiki/Acquisition_of_the_IBM_PC_busi...
- incrudible 3y ago
  Windows before NT was crap, so users had an incentive to upgrade. If there had existed a Windows 7 alternative that was near fully compatible and FOSS, I would wager Microsoft would have lost to it with Windows 8 and even 10. The only reason to update for most people was Microsoft dropping support. For CUDA, it is not just AMD who would need to catch up. Developers also are not necessarily going to target the latest feature set immediately, especially if it only benefits (or requires) new hardware. I accept the final statement, but that also means AMD for compute is gonna be dead like OS/2. Their stack just will not reach critical mass.
  - BizarroLand 3y ago
    Todays linux OS's would have competed incredibly strongly against Vista and probably would have gone blow for blow against 7. Proton, Wine, and all of the compatibility fixes and drive improvements that the community has made in the last 16 years has been amazing, and every day is another day where you can say that it has never been easier to switch away from Windows. However, Microsoft has definitely been drinking the IBM koolaid a little to long and has lost the mandate of heaven. I think in the next 7-10 years we will reach a point where there is nothing Windows can do that linux cannot do better and easier without spying on you, and we may be 3-5 years from a "killer app" that is specifically built to be incompatible with Windows just as a big FU to them, possibly in the VR world, possibly in AR, and once that happens maybe, maybe, maybe it will finally actually be the year of the linux desktop.
    - paulmd 3y ago
      > However, Microsoft has definitely been drinking the IBM koolaid a little to long and has lost the mandate of heaven. I think in the next 7-10 years we will reach a point where there is nothing Windows can do that linux cannot do better and easier without spying on you that's a fascinating statement with the clear ascendancy of neural-assisted algorithms etc. Things like DLSS are the future - small models that just quietly optimize some part of a workload that was commonly considered impossible to the extent nobody even thinks about it anymore. my prediction is that in 10 years we are looking at the rise of tag+collection based filesystems and operating system paradigms. all of us generate a huge amount of "digital garbage" constantly, and you either sort it out into the important stuff, keep temporarily, and toss, or you accumulate a giant digital garbage pile. AI systems are gonna automate that process, it's gonna start on traditional tree-based systems but eventually you don't need the tree at all, AI is what's going to make that pivot to true tag/collection systems possible. Tags mostly haven't worked because of a bunch of individual issues which are pretty much solved by AI. Tags aren't specific enough: well, AI can give you good guesses at relevance. Tagging files and maintaining collections is a pain: well, the AI can generate tags and assign collections for you. Tags really require an ontology for "fuzzy" matching (search for "food" should return the tag "hot dog") - well, LLMs understand ontologies fine. Etc etc. And if you do it right, you can basically have the AI generate "inbox/outbox" for you, deduplicate files and handle versioning, etc, all relatively seamlessly. microsoft and macos are both clearly racing for this with the "AI os" concept. It's not just better relevance searches etc. And the "generate me a whole paragraph before you even know what I'm trying to type" stuff is not how it's going to work either. That stuff is like specular highlights in video games around 2007 or whatever - once you had the tool, for a few years everything was w e t until developers learned some restraint with it. But there are very very good applications that are going to come out in the 10 year window that are going to reduce operator cognitive load by a lot - that is the "AI OS" concept. What would the OS look like if you truly had the "computer is my secretary" idea? Not just dictating memorandums, but assistance in keeping your life in order and keeping you on-task. I simply cannot see linux being able to keep up with this change, in the same way the kernel can't just switch to rust - at some point you are too calcified to ever do the big-bang rewrite if there is not a BDFL telling you that it's got to happen. the downside of being "the bazaar" is that you are standards-driven and have to deal with corralling a million whiny nerds constantly complaining about "spying on me just like microsoft" and continuing to push in their own other directions (sysvinit/upstart/systemd factions, etc) and whatever else, on top of all the other technical issues of doing a big-bang rewrite. linux is too calcified to ever pivot away from being a tree-based OS and it's going to be another 2-3 decades before they catch up with "proper support for new file-organization paradigms" etc even in the smaller sense. that's really just the tip of the iceberg on the things AI is going to change, and linux is probably going to be left out of most of those commercial applications despite being where the research is done. It's just too much of a mess and too many nerdlingers pushing back to ever get anything done. Unix will be represented in this new paradigm but not Linux - the commercial operators who have the centralization and fortitude to build a cathedral will get there much quicker, and that looks like MacOS or Solaris not linux. Or at least, unless I see some big announcement from KDE or Gnome or Canonical/Red Hat about a big AI-OS rewrite... I assume that's pretty much where the center of gravity is going to stay for linux.
- foobiekr 3y ago
  IBM was also incompetent and the os/2 team in Boca was had some exceptional engineers but was packed witg mostly mediocre-to-bad ones, which is why so many things in OS/2 were bad and why IBM got upset for Microsoft contributing negative work to the project because their lines of code contribution was negative (they were rewriting a lot of inefficient bloated IBM code). A lot went wrong with os/2. For CUDA, I think a better analogy is vhs. The standard, in the effective not open sense, is what it is. AMD sucks at software and views it as an expense rather than an advantage.
  - AYBABTME 3y ago
    You would think that by now AMD realizes that poor software is what left them behind in the dust, and would have changed that mindset.
    - hyperman1 3y ago
      Most businesses understand the pain points of their suppliers very well, as they feel that pain and gave themselves organized around it. They have a hard time to understand the pain points of their consumers, as they don't feel that pain, look trough their own organisation-coloured glases, and can't see the real pain points from the whiney-customer ones. AMD probably thinks software ecosystems are the easy part, ready to take it on whenever they feel like it and throw a token amount at it. They've built a great engine, see the carossery as beneath them, and don't understand why the lazy customer wants them to build the rest of the car too.
- neerajsi 3y ago
  I'm not in the gpu programming realm, so this observation might be inaccurate: I think the case of cuda vs an open standard is different from os2 vs Windows because the customers of cuda are programmers with access to source code while the customers of os2 were end users trying to run apps written by others. If your shrink-wrapped software didn't run on os2, you'd have no choice but to go buy Windows. Otoh if your ai model doesn't run on an AMD device and the issue is something minor, you can edit the shader code.

kgeist 3y ago

Intel embraced Amd64 ditching Itanium. Wasn't it a good decision that worked out well? Is it comparable?

teucris 3y ago
In hindsight, yes, but just because a specific technology is leading an industry doesn’t mean it’s going to be the best option. It has to play out long enough for the market to indicate a preference. In this case, for better or worse, it looks like CUDA’s the preference.
- diggan 3y ago
  > It has to play out long enough for the market to indicate a preference By what measures hasn't that happened already? CUDA been around and constantly improving for more than 15 years, and there is no competitors in sight so far. It's basically the de facto standard in many ecosystems.
  - teucris 3y ago
    There haven’t been any as successful, but there have been competitors. OpenCL, DirectX come to mind.
    - cogman10 3y ago
      SYCL is the latest attempt that I'm aware of. It's still pretty active and may just work as it doesn't rely on video card manufactures to work out.
      - zozbot234 3y ago
        SYCL is the quasi-successor to OpenCL, built on the same flavor of SPIR-V. Various efforts are trying to run it on top of Vulkan Compute (which tends to be broadly support by modern GPU's) but it's non-trivial because the technologies are independently developed and there are some incompatibilities.

bachmeier 3y ago

> The problem with effectively supporting CUDA is that encourages CUDA adoption all the more strongly. I'm curious about this. Sure some CUDA code has already been written. If something new comes along that provides better performance per dollar spent, why continue writing CUDA for new projects? I don't think the argument that "this is what we know how to write" works in this case. These aren't scripts you want someone to knock out quickly.

Uehreka 3y ago
> If something new comes along that provides better performance per dollar spent They won’t be able to do that, their hardware isn’t fast enough. Nvidia is beating them at hardware performance, AND ALSO has an exclusive SDK (CUDA) that is used by almost all deep learning projects. If AMD can get their cards to run CUDA via ROCm, then they can begin to compete with Nvidia on price (though not performance). Then, and only then, if they can start actually producing cards with equivalent performance (also a big stretch) they can try for an Embrace Extend Extinguish play against CUDA.
- bachmeier 3y ago
  > They won’t be able to do that, their hardware isn’t fast enough. Well, then I guess CUDA is not really the problem, so being able to run CUDA on AMD hardware wouldn't solve anything. > try for an Embrace Extend Extinguish play against CUDA They wouldn't need to go that route. They just need a way to run existing CUDA code on AMD hardware. Once that happens, their customers have the option to save money by writing ROCm or whatever AMD is working on at that time.
  - deleted 3y ago
    [deleted]
  - foobiekr 3y ago
    Intel has the same software issue as AMD but their hardware is genuinely competitive if a generation behind. Cost and power wise, Intel is there; software? No.

panick21_ 3y ago

That's not guaranteed at all. One could make the same argument about Linux vs Commercial Unix. If the put their stuff as OpenSource, including firmware, I think they will win out eventually. And its also not a guarantee that Nvidia will always produce the superior hardware for that code.

throwoutway 3y ago

Is it? Apple Silicon exists, but Apple created a translation layer above it so the transition could be smoother.

jack_pp 3y ago
not really the same in that Apple was absolutely required to do this in order for people to transition smoothly and it wasn't competing against another company / platform, it just needed apps from its previous platform to work while people recompile apps for the current one which they will
Jorropo 3y ago
This is extremely different, apple was targeting end consumers that just want their app to run. The performance between apple rosetta and native cpu were still multiple times different. People writing CUDA apps don't just want stuff to run, performance is an extremely important factor else they would target CPUs which are easier to program for. From their readme: > On Server GPUs, ZLUDA can compile CUDA GPU code to run in one of two modes: > Fast mode, which is faster, but can make exotic (but correct) GPU code hang. > Slow mode, which should make GPU code more stable, but can prevent some applications from running on ZLUDA.
- hamandcheese 3y ago
  > The performance between apple rosetta and native cpu were still multiple times different. Rosetta 2 runs apps at 80-90% their native speed.
  - Jorropo 3y ago
    Indeed I got that wrong. Sadly minimal SIMD and hardware acceleration support.
- piva00 3y ago
  > The performance between apple rosetta and native cpu were still multiple times different. Not at all, the performance hit was in the low 10s %, before natively supporting Apple Silicon most of the apps I use for music/video/photography didn't seem to have a performance impact at all, even more when the M1 machines were so much faster than the Intels.

coldtea 3y ago

>The problem with effectively supporting CUDA is that encourages CUDA adoption all the more strongly Worked fine for MS with Excel supporting Lotus 123 and Word supporting WordPerfect's formats when those were dominant...

Dork1234 3y ago
Microsoft could do that because they had the Operating System monopoly to leverage and take out both Lotus 123 and WordPerfect. Without the monopoly of the operating system they wouldn't of been able to Embrace, Extend, Extinguish. https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
bell-cot 3y ago
But MS controlled the underlying OS. Letting them both throw money at the problem, and (by accounts at the time) frequently tweak the OS in ways that made life difficult for Lotus, WordPerfect, Ashton-Tate, etc.
- p_l 3y ago
  Last I checked, Lotus did themselves by not innovating, and betting on the wrong horse (OS/2) then not doing well on a pivot to Windows. Meanwhile Excel was gaining features and winning users with them even before Windows was in play.
  - dadadad100 3y ago
    This is a key point. Before windows we had all the dos players - WordPerfect was king. Microsoft was more focused on the Mac. I’ve always assumed that Microsoft understood that a GUI was coming and trained a generation of developers on the main gui of the day. Once windows came out the dos focused apps could not adapt in time
  - robocat 3y ago
    > betting on the wrong horse (OS/2) Ahhhh, your hindsight is well developed. I would be interested to know the background on the reasons why Lotus made that bet. We can't know the counterfactual, but Lotus delivering on a platform owned by their deadly competitor Microsoft would seem to me to be a clearly worrysome idea to Lotus at the time. Turned out it was an existentially bad idea. Did Lotus fear Microsoft? "DOS ain't done till Lotus won't run" is a myth[1] for a reason. Edit: DRDOS errors[2] were one reason Lotus might fear Microsoft. We can just imagine a narritive of a different timeline where Lotus delivered on Windows but did some things differently to beat Excel. I agree, Lotus made other mistakes and Microsoft made some great decisions, but the point remains. We can also suspect that AMD have a similar choice now where they are forked. Depending on Nvidea/CUDA may be a similar choice for AMD - fail if they do and fail if they don't. [1] http://www.proudlyserving.com/archives/2005/08/dos_aint_done_t.html http://www.proudlyserving.com/archives/2005/08/dos_aint_done... [2] https://www.theregister.com/1999/11/05/how_ms_played_the_incompatibility/ https://www.theregister.com/1999/11/05/how_ms_played_the_inc...

andy_ppp 3y ago

When the alternative is failure I suppose you choose the least bad option. Nobody is betting the farm on ROCm!

hjabird 3y ago
True. This is the big advantage of an open standard instead jumping from one vendors walled garden to another.

more_corn 3y ago

They have already lost. The question is do they want to come in second in the game to control the future of the world or not play at all?

bick_nyers 3y ago

The latest version of CUDA is 12.3, and version 12.2 came out 6 months prior. How many people are running an older version of CUDA right now on NVIDIA hardware for whatever particular reason? Even if AMD lagged support on CUDA versioning, I think it would be widely accepted if the performance per dollar at certain price points was better. Taking the whole market from NVIDIA is not really an option, it's better to attack certain price points and niches and then expand from there. The CUDA ship sailed a long time ago in my view.

swozey 3y ago
I just went through this this weekend - If you're running in Windows and want to use deepspeed, you have to still use Cuda 12.1 because deepspeed 13.1 is the latest that works with 12.1. There's no deepspeed for windows that works with 12.3. I tried to get it working this weekend but it was a huge PITA so I switched to putting everything into WSL2 then in arch on there pytorch etc in containers so I could flip versions easily now that I know how SPECIFIC the versions are to one another. I'm still working on that part, halfway into it my WSL2 completely broke and I had to reinstall windows. I'm scared to mount the vhdx right now. I did ALL of my work and ALL of my documentation is inside of the WSL2 archlinux and NOT on my windows machine. I have EVERYTHING I need to quickly put another server up (dotfiles, configs) sitting in a chezmoi git repo ON THE VM. That I only git committed one init like 5 mins into everything. THAT was a learning experience, now I have no idea if I should follow the "best practice" of keeping projects in wsl or having wsl reach out to windows, there's a performance drop. The 9p networking stopped working and no matter what I reinstalled, reset, removed features, reset windows, etc, it wouldn't start. But at least I have that WSL2 .vhdx image that will hopefully mount and start. And probably break WSL2 again. I even SPECIFICALLY took backups of the image as tarballs every hour in case I broke LINUX, not WSL. If anyone has done sd containers in wsl2 already let me know. I've tried to use WSL for dev work (i use osx) like this 2-3 times in the last 4-5 years and I always run into some catastrophically broken thing that makes my WSL stop working. I hadn't used it in years so hoped it was super reliable by now. This is on 3 different desktops with completely different hardware, etc. I was terrified it would break this weekend and IT DID. At least I can be up in windows in 20 minutes thanks to chocolately and chezmoi. Wiped out my entire gaming desktop. Sorry I'm venting now this was my entire weekend. This repo is from a deepspeed contrib (iirc) and lists the reqs for deepspeed + windows that mention the version matches https://github.com/S95Sedan/Deepspeed-Windows https://github.com/S95Sedan/Deepspeed-Windows > conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia It may sound weird to do any of this in Windows, or maybe not, but if it does just remember that it's a lot of gamers like me with 4090s who just want to learn ML stuff as a hobby. I have absolutely no idea what I'm doing but thank god I know containers and linux like the back of my hand.

hjabird 3y ago

There are some great replies to my comment - my original comment was too reductive. However, I still think that entrenching CUDA as the de-facto language for heterogeneous computing is a mistake. We need an open ecosystem for AI and HPC, where vendors compete on producing the best hardware.

ethbr1 3y ago
The problem with open standards is that someone has to write them. And that someone usually isn't a manufacturer, lest the committee be accused of bias. Consequently, you get (a) outdated features that SotA has already moved beyond, (b) designed in a way that doesn't correspond to actual practice, and (c) that are overly generalized. There are some notable exceptions (e.g. IETF), but the general rule has been that open specs please no one, slowly. IMHO, FRAND and liberal cross-licensing produce better results.
- jchw 3y ago
  Vulkan already has some standard compute functionality. Not sure if it's low level enough to be able to e.g. recompile and run CUDA kernels, but I think if people were looking for a vendor-neutral standard to build GPGPU compute features on top of, I mean, that seems to be the obvious modern choice.
  - zozbot234 3y ago
    There is already a work-in-progress implementation of HIP on top of OpenCL https://github.com/CHIP-SPV/chipStar https://github.com/CHIP-SPV/chipStar and the Mesa RustiCL folks are quite interested in getting that to run on top of Vulkan. (To be clear, HIP is about converting CUDA source code not running CUDA-compiled binaries but the Zluda project discussed in OP heavily relies on it.)

jvanderbot 3y ago

If you replace CUDA -> x86 and NVIDIA -> Intel, you'll see a familiar story which AMD has already proved it can work through. These were precisely the arguments for 'x86 will entrench Intel for all time', and we've seen AMD succeed at that game just fine.

ianlevesque 3y ago
And indeed more than succeed, they invented x86_64.
- stcredzero 3y ago
  And indeed more than succeed, they invented x86_64. If AMD invented the analogous to x86_64 for CUDA, this would increase competition and progress in AI by some huge fraction.
  - pjmlp 3y ago
    Only works if NVidia misteps and creates the Itanium version of CUDA.
    - stcredzero 3y ago
      You don't think someone would welcome the option to have more hardware buying options, even if the "Itanium version" didn't happen?
  - jvanderbot 3y ago
    Why "if only". Intel had been around forever when AMD showed up. CUDA isn't unassailable
- sangnoir 3y ago
  x86_64's win was helped by Intel's Itanium misstep. AMD can't bank on Nvidia making a mistake, and Nvidia seems content with incremental changes to CUDA, contrasted with Intel's 32-bit to 64-bit transition. It is highly unlikely that AMD can find and exploit a similar chink in the amor against CUDA.

mindcrime 3y ago

Yep. This is very similar to the "catch-22" that IBM wound up in with OS/2 and the Windows API. On the one hand, by supporting Windows software on OS/2, they gave OS/2 customers access to a ready base of available, popular software. But in doing so, they also reduced the incentive for ISV's to produce OS/2 native software that could take advantage of unique features of OS/2. It's a classic "between a rock and a hard place" scenario. Quite a conundrum.

ianlevesque 3y ago
Thinking about the highly adjacent graphics APIs history, did anyone really 'win' the Direct3D, OpenGL, Metal, Vulkan war? Are we benefiting from the fragmentation? If the players in the space have naturally coalesced around one over the last decade, can we skip the thrashing and just go with it this time?
- tadfisher 3y ago
  The game engines won. Folks aren't building Direct3D or Vulkan renderers; they're using Unity or Unreal or Godot and clicking "export" to target whatever API makes sense for the platform. WebGPU might be the thing that unifies the frontend API for folks writing cross-platform renderers, seeing as browsers will have to implement it on top of the platform APIs anyway.
- pjmlp 3y ago
  You missed the Nintendo and Sony APIs as well. FOSS folks make this a bigger issue than it really is, game studios make a pluggable API on their engine and call it a day, move on into everything else that matters in actually delivering a game.

mqus 3y ago

If their primary objective is to sell cards, then they should make it as easy as possible to switch cards. If their primary objective is to break the CUDA monopoly, they should up their game in software, which means going as far as implementing support for their hardware in the most popular user apps themselves, if necessary. But since they don't seem to want to do that, they should really go for option one, especially if a single engineer already got so far. Let's say AMD sold a lot of cards with CUDA support. Now nvidia tries to cut them off. What will happen next? A lot of people will replace their cards with nvidia ones. But a lot of the rest will try to make their expensive AMD cards work regardless. And if AMD provides a platform for that, they will get that work for free.