9 ms·

Ask HN: Why hasn’t AMD made a viable CUDA alternative?

194 points by spacebanana7 2y ago

I appreciate developing ROCm into something competitive with CUDA would require a lot of work, both internally within AMD and with external contributions to the relevant open source libraries. However the amount of resources at stake is incredible. The delta between NVIDIA's value and AMD's is bigger than the annual GDP of Spain. Even if they needed to hire a few thousand engineers at a few million in comp each, it'd still be a good investment.

whywhywhywhy 2y ago
The answer is in the question, because if they had the foresight to do such a thing the tech would already be here, instead they thought 1 dimensionally about their product, were part of the group that fumbled OpenCL and now they're a decade behind playing catch up.
- bluGill 2y ago
  A good group can catch up significantly in 2 years. They will still be behind, but if they are cheaper (or just you can buy them) that would still go a long way.
- Symmetry 2y ago
  The second article I ever submitted to Hacker News back in 2011 was on AMD's efforts to build a CUDA competitor[1]. I don't think it's lack of foresight. [1]https://www.semiaccurate.com/2011/06/22/amd-and-arm-join-forces-at-last/ https://www.semiaccurate.com/2011/06/22/amd-and-arm-join-for...
echelon 2y ago
Leadership. At the end of the day, the buck stops with leadership. If they wanted to prioritize this, they would. They're simply not taking it seriously.
- spacebanana7 2y ago
  I always thought of Lisa Su as an effective leader, but this does make me question it.
  - setgree 2y ago
    I thought so too until I read a Stratechery interview that painted a pretty bad portrait of her time at Sony [0]. I tend to think that AMD looks well run when compared with Intel [1], but when you consider Nvidia as the relevant counterfactual [2], things don't look so good. I wrote about this here [3]. [0] https://news.ycombinator.com/item?id=40697341 https://news.ycombinator.com/item?id=40697341 [1] https://news.ycombinator.com/item?id=41446766 https://news.ycombinator.com/item?id=41446766 [2] https://news.ycombinator.com/item?id=39344815 https://news.ycombinator.com/item?id=39344815 [3] https://setharielgreen.com/blog/amd-also-seems-to-be-floundering/ https://setharielgreen.com/blog/amd-also-seems-to-be-flounde...
  - adra 2y ago
    Why, because 90% of her job is talking to and appeasing shareholders, grand standing with fat whales, and what else.. what do you think a CEO at these companies actually does? They aren't in the trenches of each subdivision nurturing and cracking whips. She likely attends a 2 hour briefing with a line item: CUDA parity project: on schedule release date not set
    - fulladder 2y ago
      Fair points. I think the difference is that AMD has other businesses that are much larger than GPUs, whereas NVIDIA was always just about GPUs.
  - grg0 2y ago
    And why do you think that? Have you worked with her personally, or is it maybe because Fortune spent a decade preaching "women in tech" regardless of what those women may actually be doing?
deleted 2y ago
[deleted]
postalrat 2y ago
Because they dont want nvidia to be in control of their own development efforts.
- jsheard 2y ago
  The question was why don't they have anything as good as CUDA, not why don't they adopt CUDA itself.
  - cmrdporcupine 2y ago
    Isn't the "goodness" of CUDA really down to its mass adoption -- and therefore its community and network effects -- not strictly its technical attributes? If I recall, there are various "GPU programming" and "AI" efforts that have existed for AMD GPUs, but none of them have had the same success in large part because they're simply non-"standard?"
    - em500 2y ago
      I thought OpenCL was supposed to be the "standard"? From the Wikipedia page, it's largely vendor neutral and not that much younger than CUDA (initial release Aug 2009 vs Feb 2007). Maybe some more knowledgeable people can comment why it seems to have been outcompeted by the proprietary CUDA?
      - the__alchemist 2y ago
        CUDA has a comparatively nicer user experience. If you would like to understand tacitly and have an nvidia GPU, try writing a simple program using both. (Something highly parallel, like nbody, for example)
      - fulladder 2y ago
        OpenCL was definitely supposed to be the standard, but as the sibling suggests, it's just much harder to use than CUDA. Also, it doesn't feel like OpenCL has much of a community around it, documentation doesn't seem great, and just the overall experience is very frustrating. I tried to implement something in OpenCL about 5 years ago, thinking it would be fairly trivial to port a simple compute shader from CUDA, and ended up giving up. Just too difficult.
deleted 2y ago
[deleted]
bee_rider 2y ago
NVIDIA does GPUs and software. Intel does CPUs and software. AMD does GPUs and CPUs.
christkv 2y ago
I don't think its that bad. The focus will turn to inference going forward and that eventually means a place for AMD and maybe even Intel. Eventually it will be all about the efficiency of inference in watts. That switch will reduce the NVIDIA margins by a lot. NVIDIA probably has 2 years left of being the only one with golden shovels.
- electriclove 2y ago
  Until they have another hit (gaming -> mining -> AI -> ?)
fancyfredbot 2y ago
There is more than one way to answer this. They have made an alternative to the CUDA language with HIP, which can do most of the things the CUDA language can. You could say that they haven't released supporting libraries like cuDNN, but they are making progress on this with AiTer for example. You could say that they have fragmented their efforts across too many different paradigms but I don't think this is it because Nvidia also support a lot of different programming models. I think the reason is that they have not prioritised support for ROCm across all of their products. There are too many different architectures with varying levels of support. This isn't just historical. There is no ROCm support for their latest AI Max 395 APU. There is no nice cross architecture ISA like PTX. The drivers are buggy. It's just all a pain to use. And for that reason "the community" doesn't really want to use it, and so it's a second class citizen. This is a management and leadership problem. They need to make using their hardware easy. They need to support all of their hardware. They need to fix their driver bugs.
- thrtythreeforty 2y ago
  This ticket, finally closed after being open for 2 years, is a pretty good micocosm of this problem: https://github.com/ROCm/ROCm/issues/1714 https://github.com/ROCm/ROCm/issues/1714 Users complaining that the docs don't even specify which cards work. But it goes deeper - a valid complaint is that "this only supports one or two consumer cards!" A common rebuttal is that it works fine on lots of AMD cards if you set some environment flag to force the GPU architecture selection. The fact that this is so close to working on a wide variety of hardware, and yet doesn't, is exactly the vibe you get with the whole ecosystem.
  - CoastalCoder 2y ago
    I had a similar (I think) experience when building LLVM from source a few years ago. I kept running into some problem with LLVM's support for HIP code, even though I had not interest in having that functionality. I realize this isn't exactly an AMD problem, but IIRC it was they were who contributed the troublesome code to LLVM, and it remained unfixed. Apologies if there's something unfair or uninformed in what I wrote, it's been a while.
danielmarkbruce 2y ago
They likely haven't put even close to enough money behind it. This isn't a unique situation - you'll see in corporate america a lot of CEOs who say "we are investing in X" and they really believe they are. But the required size is billions (like, hundreds of really insanely talented engineers being paid 500k-1m, lead by a few being paid $3-10m), and they are instead investing low 10's of millions. They can't bring themselves to put so much money into it that it would be an obvious fail if it didn't work.
- spacebanana7 2y ago
  Given how the big tech companies are buying hundreds of thousands of GPUs at huge prices, most of which is pure margin, I wonder whether it'd make sense for Microsoft to donate a couple billion to make the market competitive. https://www.datacenterdynamics.com/en/news/microsoft-bought-twice-as-many-nvidia-hopper-gpus-as-other-big-tech-companies-report/ https://www.datacenterdynamics.com/en/news/microsoft-bought-...
  - danielmarkbruce 2y ago
    The big players are all investing in building chips themselves. And probably not putting enough money behind it... it takes enormous courage as a CEO to walk into a boardroom and say "I'm going to spend $50 billion, I think it will probably work, I'm... 60% certain".
    - spacebanana7 2y ago
      You're probably correct, but I feel like I have to raise the issue of Zuckerberg spending a comparable amount on VR which was much more speculative.
      - hnlmorg 2y ago
        Zuckerberg owns Facebook though. It’s a lot easier to make bold decisions when you’re the majority shareholder. Edit: though emphasis should be put on “easIER” because it’s still far from easy.
johnnyjeans 2y ago
> The delta between NVIDIA's value and AMD's is bigger than the annual GDP of Spain. Nvidia is massively overvalued right now. AI has rocketed them into absolute absurdity, and it's not sustainable. Put aside the actual technology for a second and realize that public image of AI is at rock bottom. Every single time a company puts out AI-generated materials, they receive immense public backlash. That's not going away any time soon and it's only likely to get worse. Speaking as someone that's not even remotely anti-AI, I wouldn't touch the shit with a 10 foot pole because of how bad the public image is. The moment that capital realizes this, that bubble is going to pop and it's going to pop hard.
- the__alchemist 2y ago
  You state this very confidently. Are you shorting Nvidia stock? If not, why not?
  - chrisan 2y ago
    You imply you either would believe his word or would short nvidia yourself if he said he was. If not, why not?
    - the__alchemist 2y ago
      Close - If I had the degree of confidence that post implies about Nvidia being overvalued, I would take an aggressive short position.
      - dagw 2y ago
        I would take an aggressive short position. Lots of very smart people have lost a lot of money by being completely right about the destination, but wrong about the path and how long it will take to get there.
      - the__alchemist 2y ago
        > Lots of very smart people have lost a lot of money by being completely right about the destination, but wrong about the path and how long it will take to get there. If you make a habit of this and still lose money, then either you statistically were very unlucky, or did not have a history of being right.
dagmx 2y ago
AMD have actually made several attempts at it. The first time, they went ahead and killed off their effort to consolidate on OpenCL. OpenCL went terribly (in no small part because NVIDIA held out on OpenCL 2 support) and that set AMD back a long ways. Beyond that, AMD does not have a strong software division or one with the teeth to really influence hardware to their needs . They have great engineers but leadership doesn’t know how to get them to where they need to be.
- WithinReason 2y ago
  This is it, it's an organisational skill issue. To be fair, being a HW company and a SW company at the same time is very difficult.
  - dagmx 2y ago
    It is but you have to be. It’s been key to the success of their peers. NVIDIA and Apple are the best examples but even Intel to a smaller degree.
    - latchkey 2y ago
      https://x.com/AnushElangovan https://x.com/AnushElangovan is now in charge of the software part of things and is making great progress very quickly.
bryanlarsen 2y ago
There's a massive amount of effort in https://github.com/orgs/ROCm/repositories?type=all https://github.com/orgs/ROCm/repositories?type=all Throwing a vast amount of effort at something isn't sufficient.
atemerev 2y ago
HIP is now somewhat viable (and ROCm is now all HIP). But — too late. First versions of ROCm were terrible. Too much boilerplate. 1200 lines of template-heavy C++ for a simple FFT. Can't just start hacking around. Since then, the CUDA way is cemented in minds of developers. Intel now has oneAPI, and it is not too bad, and hackable, but there is no hardware and no one will learn it. And HIP is "CUDA-like", so why not CUDA, unless you _have to_ use AMD hardware. Tl;dr first versions of ROCm were bad. Now they are better, but it is too late.
Cieric 2y ago
I can't contribute much to this discussion due to bias and NDAs, but I just wanted to mention, technically HIP is our CUDA competitor. ROCm is the foundation that HIP is being built on.
- johnnyjeans 2y ago
  I wonder what the purpose is behind creating a whole new API? Why not just focus on getting Vulkan compute on AMD GPUs to have the data throughput of CUDA?
  - Const-me 2y ago
    I don’t know answer to your question, but I recalled something relevant. Some time ago, Microsoft had a tech which compiled almost normal looking C++ into Direct3D 11 compute shaders: https://learn.microsoft.com/en-us/cpp/parallel/amp/cpp-amp-overview?view=msvc-170 https://learn.microsoft.com/en-us/cpp/parallel/amp/cpp-amp-o... The compute kernels are integrated into CPU-running C++ in the similar fashion CUDA does. As you see, the technology deprecated in Visual Studio 2022. I don’t know why but I would guess people just didn’t care. Maybe because it only run on Windows.
    - Wumpnot 2y ago
      Yes I found CPP AMP really interesting, but since it only ran on Windows..never used it for anything.
      - Const-me 2y ago
        It’s unfortunate they have deprecated it. We how have DXVK which implements D3D11, including compute shaders, for any platform which supports Vulkan. Making that (or a conceptually similar) thing work across platforms is no longer prohibitively expensive. I believe that approach, i.e. the compute shaders, is the correct thing to do because modern videogames use them a lot, the runtime support is stable and performant now. No need for special HPC-only drivers or runtime components.
- 2y ago
gmm1990 2y ago
The idea that CUDA is the main reason behind Nvidia dominance seems strange to me. If most of the money is coming from Facebook and Microsoft they have their own teams writing code at a lower level than CUDA anyway. Even deepseek was writing stuff lower than that.
- brcmthrowaway 2y ago
  What is lower than CUDA? Compute shaders?
  - a5ehren 2y ago
    PTX assembly. Deepseek used some of it to do a little bit of work that CUDA didn't have APIs for.
    - brcmthrowaway 2y ago
      Sadly platform specific
- ActorNightly 2y ago
  Its not efficient to write low level code. Its like trying to build a web application with C. CUDA is basically the equivalent of "hey you wanna do matrix multiply fast? just use this and we handle the rest". And its standardized, and when Nvidia comes out with new cards, all the old shit still works.
petesergeant 2y ago
Also why haven’t MS released a decent set of ML-tooling for TypeScript?
dlewis1788 2y ago
CUDA is an entire ecosystem - not a single programming language extension (C++) or a single library, but a collection of libraries & tools for specific use cases and optimizations (cuDNN, CUTLASS, cuBLAS, NCCL, etc.). There is also tooling support that Nvidia provides, such as profilers, etc. Many of the libraries build on other libraries. Even if AMD had the decent, reliable language extensions for general-purpose GPU programming, they still don't have the libraries and the supporting ecosystem to provide anything to the level that CUDA provides today, which is a decade plus of development effort from Nvidia to build.
- guywithahat 2y ago
  The counter point is they could make a higher level version of CUDA which wouldn't necessitate all the other supporting libraries. The draw of cuBLAS is that CUDA is a confusing pain. It seems reasonable to think they could write a better, higher level language (in the same vein as triton) and not have to write as many support libraries
  - dlewis1788 2y ago
    100% valid - Nvidia is trying to address that now with cuTile and the new Python front-end for CUTLASS.
- general_reveal 2y ago
  [dead]
Vvector 2y ago
Back in 2015, they were a quarter or two from bankruptcy, saved by the XBOX and Playstation contracts. Those years saw several significant layoffs, and talent leaving for greener pastures. Lisa Su has done a great job at rebuilding the company. But not in a position to hire 2000 engineers x few million comp (~$4 billion annually) even if there were people readily available. "it'd still be a good investment." - that's definitely not a sure thing. Su isn't a risk taker, seems to prefer incremental growth, mainly focused on the CPU side.
- ninetyninenine 2y ago
  This is the difference between Jensen and Su. It’s not that Jensen is a risk taker. No. Jensen focused on incremental growth of the core business while slowly positioning the company for growth in other verticals as well should the landscape change. Jensen never said… hey I’m going to bet it all on AI and cuda. Let’s go all in. This never happened. Both Jensen and Su are not huge risk takers imo. Additionally there’s a lot of luck involved with the success of NVIDIA.
  - kbolino 2y ago
    I think this broaches the real matter, which is that nVidia's core business is GPUs while AMD's core business is CPUs. And, frankly, AMD has lately been doing a great job at its core business. The problem is that GPUs are now much more profitable than CPUs, both in terms of unit economics and growth potential. So they are winning a major battle (against Intel) even as they are losing a different one (against nVidia). I'm not sure there's a strategy they could have adopted to win both at the same time. However, the next big looming problem for them is likely to be the shrinking market for x86 vs. the growing market for Arm etc. So they might very well have demonstrated great core competence, that ends up being completely swept away by not just one but two major industry shifts.
- fancyfredbot 2y ago
  Where does the idea that engineers cost "a few million" come from? You might pay that much to senior engineering management, big names who can attract other talent, but normal engineers cost much less than a million dollars a year.
shmerl 2y ago
What about Rust for GPU programming? I wonder why AMD doesn't back such kind of effort as an alternative.
x0nr8 2y ago
HIP is definitely a viable option. In fact with some effort you can port large CUDA projects to be compilable with the HIP/ AMD-clang toolchain. This way you don’t have to rewrite the world from scratch in a new language but still be able to run GPU workloads on AMD hardware.
WithinReason 2y ago
This is the best article on why OpenCL failed: https://www.modular.com/blog/democratizing-ai-compute-part-5-what-about-cuda-c-alternatives https://www.modular.com/blog/democratizing-ai-compute-part-5...
btown 2y ago
AMD was investing in a drop-in CUDA compatibility layer & cross-compiler! Perhaps in keeping with the broader thread here, they had only ever funded a single contract developer working on it, and then discontinued the project (for who-knows-what legal or political reasons). But the developer had specified that he could open-source the pre-AMD state if the contract was dissolved, and he did exactly that! The project is active with an actively contributing community, and is rapidly catching up to where it was. https://www.phoronix.com/review/radeon-cuda-zluda https://www.phoronix.com/review/radeon-cuda-zluda https://vosen.github.io/ZLUDA/blog/zludas-third-life/ https://vosen.github.io/ZLUDA/blog/zludas-third-life/ https://vosen.github.io/ZLUDA/blog/zluda-update-q4-2024/ https://vosen.github.io/ZLUDA/blog/zluda-update-q4-2024/ IMO it's vital that even if NVIDIA's future falters in some way, the (likely) collective millennia of research built on top of CUDA will continue to have a path forward on other constantly-improving hardware. It's frustrating that AMD will benefit from this without contributing - but given the entire context of this thread, maybe it's best that they aren't actively managing the thing that gives their product a future!
- bryanlarsen 2y ago
  ZLUDA is built on HIP which is built on ROCm. Both of the latter are significant efforts that AMD is pouring significant resources into.
londons_explore 2y ago
AMD's CEO is the cousin of Nvidias CEO. Neither will encroach too much on the others turf. The two companies don't want to directly compete on the things that really drive the share price.
- a5ehren 2y ago
  Distant cousins, and they had never met until they were both CEOs. This is a stupid trope and people need to stop using it.
  - londons_explore 2y ago
    The long history of both companies doing a half-assed job of entering the others markets speaks differently. Every 'run cuda on AMD' project gets bought then cancelled. Every cuda alternative is understaffed and cancelled as soon as it starts to gain any traction, etc. To me it looks like a clear attempt to not hurt each others businesses whilst being deny-able if anyone accuses them of unfair market practices/collusion.
singhrac 2y ago
I want to argue that graphics cards are really 3 markets: integrated, gaming (dedicated), and compute. Not only do these have different hardware (fixed function, ray tracing cores, etc.) but also different programming and (importantly) distribution models. NVIDIA went from 2 to 3. Intel went from 1 to 2, and bought 3 (trying to merge). AMD started with 2 and went to 1 (around Llano) and attempted the same thing as NVIDIA via GCN (please correct me if I'm wrong). My understanding is that the reason is that the real market for 3 (GPUs for compute) didn't show up until very late, so AMD's GCN bet didn't pay off. Even in 2021, NVIDIA's revenue from gaming was above data center revenue (a segment they basically had no competition in, and 100% of their revenue was from CUDA). AMD meanwhile won the battle for Playstation and Xbox consoles, and was executing a turnaround in data centers with EPYC and CPUs (with Zen). So my guess as to why they might have underinvested is basically: for much of the 2010s they were just trying to survive, so they focused on battles they could win that would bring them revenue. This high level prioritization would explain a lot of "misexecution", e.g. if they underhired for ROCm, or prioritized APU SDK experience over data center, their testing philosophy ("does this game work ok? great").
- brudgers 2y ago
  The market segmentation you describe makes a lot of sense to me. But I don't think the situation is a matter of under-investment and is instead just fundamental market economics. Nvidia can afford to develop a comprehensive software platform for the compute market segment because it has a comprehensive share of that segment. AMD cannot afford it because it does not have the market share. Or to put it another way, I assume that AMD's efforts are motivated rational economic behavior and it has not been economically rational to compete heavily with Nvidia in the compute segment. AMD was able to buy ATI because ATI could not compete with Nvidia. So AMD's graphics business started out trailing Nvidia. AMD has had a viable graphics strategy without trying to beat Nvidia...which makes sense since the traditional opponent is Intel and the ATI purchase has allowed AMD to compete with them pretty well. Finally, most of the call for AMD to develop a CUDA alternative is based on a desire for cheaper compute. That's not a good business venture to invest in against a dominate player because price sensitive customers are poor customers.
spmurrayzzz 2y ago
CUDA isn't the moat people think it is. NVIDIA absolutely has the best dev ergonomics for machine learning, there's no question about that. Their driver is also far more stable than AMD's. But AMD is also improving, they've made some significant strides over the last 12-18 months. But I think more importantly, what is often missed in this analysis is that most programmers doing ML work aren't writing their own custom kernels. They're just using pytorch (or maybe something even more abstracted/multi-backend like keras 3.x) and let the library deal with implementation details related to their GPU. That doesn't mean there aren't footguns in that particular land of abstraction, but the delta between the two providers is not nearly as stark as its often portrayed. At least not for the average programmer working with ML tooling. (EDIT: also worth noting that the work being done in the MLIR project has a role to play in closing the gap as well for similar reasons)
- martinpw 2y ago
  > But I think more importantly, what is often missed in this analysis is that most programmers doing ML work aren't writing their own custom kernels. They're just using pytorch (or maybe something even more abstracted/multi-backend like keras 3.x) and let the library deal with implementation details related to their GPU. That would imply that AMD could just focus on implementing good PyTorch support on their hardware and they would be able to start taking market share. Which doesn't sound like much work compared with writing a full CUDA competitor. But that does not seem to be the strategy, which implies it is not so simple? I am not an ML engineer so don't have first hand experience, but those I have talked to say they depend on a lot more than just one or two key libraries. But my sample size is small. Interested in other perspectives...
  - spmurrayzzz 2y ago
    > But that does not seem to be the strategy, which implies it is not so simple? That is exactly what has been happening [1], and not just in pytorch. Geohot has been very dedicated in working with AMD to upgrade their station in this space [2]. If you hang out in the tinygrad discord, you can see this happening in real time. > those I have talked to say they depend on a lot more than just one or two key libraries. Theres a ton of libraries out there yes, but if we're talking about python and the libraries in question are talking to GPUs its going to be exceedingly rare that theyre not using one of these under the hood: pytorch, tensorflow, jax, keras, et al. There are of course exceptions to this, particularly if you're not using python for your ML work (which is actually common for many companies running inference at scale and want better runtime performance, training is a different story). But ultimately the core ecosystem does work just fine with AMD GPUs, provided you're not doing any exotic custom kernel work. (EDIT: just realized my initial comment unintentionally borrowed the "moat" commentary from geohot's blog. A happy accident in this case, but still very much rings true for my day to day ML dev experience) [1] https://github.com/pytorch/pytorch/pulls?q=is%3Aopen+is%3Apr+label%3A%22module%3A+rocm%22 https://github.com/pytorch/pytorch/pulls?q=is%3Aopen+is%3Apr... [2] https://geohot.github.io//blog/jekyll/update/2025/03/08/AMD-YOLO.html https://geohot.github.io//blog/jekyll/update/2025/03/08/AMD-...
euos 2y ago
CUDA is over a decade of investment. I left CUDA toolkit team in 2014 and it was probably around 10 years old back then. Can't build something comparable fast.
dachworker 2y ago
Another possible reason might be outreach. NVIDIA spends big money on getting people to use their products. I have worked at two HPC centers and at both we had NVIDIA employees stationed there, whose job it was to help us get the most out of the hardware. Besides that, they also organize Hackatrons and they have dedicated software developer programs for each common application, be it LLMs, Weather Prediction or Protein Folding, not to mention dedicated libraries for pretty much every domain.
waltercool 2y ago
[dead]
noboostforyou 2y ago
Maybe this is an overly cynical response but the answer is simply that they cannot (at least not immediately). They have not invested enough into engineering talent with this specific goal in mind.
spellbaker 2y ago
If AMD developers use AI deployed on nvidia hardware to create tools that complete against nvidia as a company but overall improves outcomes because of competition, would this be an example of co evolution observable in human time standards... I feel like ai is evolving, taking a stable form in this complex multi dimension multi paradigm sweet spot of an environment we have created, on top of this technical, social and governmental infrastructure and we're watching it live on discovery tech filtered into a 2d video narrated by some idiot who has no right to be as confident as he sounds. I'm sorry I'm on withdrawal from quitting mass media and I'm very bored.
- fulladder 2y ago
  >I'm sorry I'm on withdrawal from quitting mass media and I'm very bored. Good choice! So many people doing that these days.
fulladder 2y ago
I've been telling people for years that NVIDIA is actually a software company, but nobody ever listens. My argument is that their silicon is nothing special and could easily be replicated by others, and therefore their real value is in their driver+CUDA layer. (Maybe "nothing special" is a little bit strong, but as a chip designer I've never seen the actual NVIDIA chips as all that much of a moat. What makes it hard to find alternatives to NVIDIA is their driver and CUDA stack.) Curious to hear others' opinions on this.
- deleted 2y ago
  [deleted]
z3phyr 2y ago
In turn I will raise you the following: Why are GPU ISA trade secrets at all? Why not open them up like CPU ISAs, get rid of specialized cores and let compiler writers port their favorite languages to compile into native GPU programs? Everyone will be happy. Game devs will be happy with more control over the hardware, Compiler devs will be happy to run haskell or prolog natively on GPUs, ML devs will be happier, NVIDIA/AMD will be happier with taking the MainStage.
Lerc 2y ago
I made a post here a while back suggesting an investment strategy of spending one billion on AMD shares and one billion on software developers to 3rd party write a quality support stack for their hardware. I'm still not sure if its a crazy idea. Actually it might be better to spend 1B on shares and 10x 100M on development and take ten attempts in parallel and use the best of them.
- whiplash451 2y ago
  What’s the point of buying the 1B worth of AMD shares in your strategy ? (I get the other half)
  - Lerc 2y ago
    That's your profit source. Make AMD as viable an option as nVidia and you massively increase their stock price. (and probably reduce nVidia at the same time if you're in the shorting game)
geor9e 2y ago
If it's a question about entrenched corporate dysfunction, I can't answer it. Most people's answers are wild guesses at best. If it's a question of first principles, there is a small glimmer of hope in a company called tinygrad making the attempt - https://geohot.github.io//blog/jekyll/update/2025/03/08/AMD-YOLO.html https://geohot.github.io//blog/jekyll/update/2025/03/08/AMD-... If the current 1:16 AMD:NVIDIA stock value difference is entirely due to the CUDA moat, you might make some money if the tide turns. But who can say…
jtatarchuk 2y ago
Ya'll should have come to Beyond CUDA. https://www.linkedin.com/posts/jtatarchuk_beyondcuda-democratizecompute-thefutureisopen-activity-7312527320743907330-NDkp?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAtWOS4BaMaYiX7S5F24HG_eBOc4OsXef9k https://www.linkedin.com/posts/jtatarchuk_beyondcuda-democra...
cavisne 2y ago
The problem is the hardware not the software, and specifically not CUDA. Triton for example writes PTX directly (a level below CUDA). Trying to copy Nvidia hardware exactly means you will always be a generation behind, so they are forced to try and guess what different direction to take that will be useful. So far those guesses haven't worked out (not surprising as they have no specific ML expertise and are not partnered with any frontier lab), and no amount of papering over with software will help. That said I'm hopeful the rise of reasoning models can help, no one wants to bet the farm on their untested clusters but buying some chips for inference is much safer.
mancerayder 2y ago
Is this why the stock is in the toilet for years now? It seems it missed the AI bubble, at least from an investor standpoint, like there's tremendous skepticism.
latchkey 2y ago
Nobody has mentioned this, but https://docs.scale-lang.com/ https://docs.scale-lang.com/ is doing some amazing work on this front. Take CUDA code, compile it and output a binary that runs on AMD. Michael and his team working on this are brilliant engineers.
mayerwin 2y ago
Tinycorp (owned by George Hotz, also behind Comma.ai) is working on it after AMD finally understood that it was a no-brainer: https://geohot.github.io/blog/jekyll/update/2025/03/08/AMD-YOLO.html https://geohot.github.io/blog/jekyll/update/2025/03/08/AMD-Y... Exciting times ahead!
- ActorNightly 2y ago
  I doubt anything will come from this, granted how much he hyped Comma and where its at now. Namely, the fact that this is still on the website: > How is tinygrad faster than PyTorch? For most use cases it isn't yet, but it will be Despite being in development for so long is kinda telling. Like at least prove that you can make it work on CUDA as well as Pytorch. Otherwise its just an personal project based on your ideology, and then when other companies don't play along with what you think is right, you get to blame their culture because you can never be wrong.
rinka_singh 2y ago
There's OpenCL https://en.wikipedia.org/wiki/OpenCL https://en.wikipedia.org/wiki/OpenCL - this BTW, also runs on NVIDIA GPUs too... OpenCL is completely open (source) and so why wouldn't we, all of us, throw our weight behind OpenCL. (no, I have no connection with them and have nothing to do with them, other than having learned a bit).
kara4151 2y ago
[dead]

mook 2y ago

I suspect part of it is also that Nvidia actually does a lot of things in firmware that can be upgraded. The new Nvidia Linux drivers (the "open" ones) support Turing cards from 2018. That means chips that old already do much of the processing in firmware. AMD keeps having issues because their drivers talk to the hardware directly so their drivers are massive bloated messes, famous for pages of auto-generated register definitions. Likely it's much more difficult to fix anything.

bgnn 2y ago
Hmm that is interesting. Can you elaborate what is exactly different between them? I'm asking because I think a firmware has to directly talk to hardware through lower HAL (hardware abstraction layer), while customer facing parts should be fairly isolated in the upper HAL. Some companies like to add direct HW acces to customer interface via more complex functions (often a recipe made out of lower HAL functions), which I always disliked. I prefer to isolate lower level functions and memory space from the user. In any case, both Nvidia and AMD should have very similar FW capabilities. I don't know what I'm missing here.
- Evil_Saint 2y ago
  I worked on at both companies on drivers. The programming models are quite different. Both make GPUs but they were designed by different groups of people who made different decisions. For example: Nvidia cards are much easier to program in the user mode driver. You cannot hang a Nvidia GPU with a bad memory access. You can hang the display engine with one though. At least when I was there. You can hang an AMD GPU with a bad memory access. At least up to the Navi 3x.
- raxxorraxor 2y ago
  Why isolate these functions? That will always cripple capabilities. With well designed interfaces, it doesn't lead to a mess and a more powerful device. Of course these lower level functions shouldn't be essential, but especially in these times you almost have to provide an interface here or be left behind by other environments.
Evil_Saint 2y ago
Having worked at both Nvidia and AMD I can assure you that they both feature lots of generated header files.

iforgotpassword 2y ago

What I don't get is why they don't at least assign a dev or two to make the poster child of this work: llama.cpp It's the first thing anyone tries when trying to dabble in AI or compute on the gpu, yet it's a clusterfuck to get to work. A few blessed cards work, with proper drivers and kernel; others just crash, perform horribly slow, or output GGGGGGGGGGGGGG to every input (I'm not making this up!) Then you LOL, dump it and go buy nvidia et voila, stuff works first try.

wkat4242 2y ago
It does work, I have it running on my Radeon VII Pro
- Filligree 2y ago
  It sometimes works.
  - wkat4242 2y ago
    How so? It's rock solid for me. I use ollama but it's based on llama.cpp It's quite fast also, probably because that card has fast HBM2 memory (it has the same memory bandwidth as a 4090). And it was really cheap as it was on deep sale as an outgoing model.
    - Filligree 2y ago
      "Sometimes" as in "on some cards". You're having luck with yours, but that doesn't mean it's a good place to build a community.
      - wkat4242 2y ago
        Ah I see. Yes, but you pick the card for the purpose of course. I also don't like the way they have such limited support on ROCm. But when it works it works well. I have Nvidia cards too by the way, a 4090 and a 3060 (the latter I use for AI also, but more for Whisper because faster-whisper doesn't do ROCm right now).
    - halJordan 2y ago
      Aside from the fact that gfx906 is one of the blessed architecture mentioned (so why would it not work). Like how do you look at your specific instance and then turn around and say "All of you are lying, it works perfectly." How do you square that circle in your head
      - wkat4242 2y ago
        No I was just a bit thrown by the "sometimes". I thought they were referring to a reliability issue. I am aware of the limited card support with ROCm and I complained about this elsewhere in the thread too. Also I didn't accuse anyone of lying. No need to be so confrontational. And my remark to the original poster at the top was from before they clarified their post. I just don't really see what AMD can do to make ollama work better other than porting ROCm to all their cards which is definitely something they should do. And no I'm not an AMD fanboi. I have no loyalty to anyone, any company or any country.

trod1234 2y ago

It is a little bit more complicated than ROCm simply not having support because ROCm has at a point claimed support, and they've had to walk it back painfully (multiple times). Its not a driver issue, nor a hardware issue on their side. There has been a long-standing issue between AMD and its mainboard manufacturers. The issue has to do with features required for ROCm, namely PCIe Atomics. AMD has been unable or unwilling to hold the mainboard manufacturers to account for advertising features the mainboard does not support. The CPU itself must support this feature, but the mainboard must as well (in firmware). One of the reasons why ROCm hasn't worked in the past is because the mainboard manufacturers have claimed and advertised support for PCIe Atomics, and the support they've claimed has been shown to be false, and the software fails in non-deterministic ways when tested. This is nightmare fuel for the few AMD engineers tasked with ROCm. PCIe Atomics requires non-translated direct IO to operate correctly, and in order to support the same CPU models from multiple generations they've translated these IO lines in firmware. This has left most people that query their system to check this showing PCIAtomics is supported, while when actual tests that rely on that support are done they fail, in chaotic ways. There is no technical specification or advertising that the mainboard manufacturers provide showing whether this is supported. Even the boards with multiple x16 slots and the many technologies related to it such as Crossfire/SLI/mGPU brandings these don't necessarily show whether PCIAtomics is properly supported. In other words, the CPU is supported, the firmware/mainboard fail with no way to differentiate between the two at the upper layers of abstraction. All in all. You shouldn't be blaming AMD for this. You should be blaming the three mainboard manufacturers who chose to do this. Some of these manufacturers have upper end boards where they actually did do this right they just chose to not do this for any current gen mainboard costing less than ~$300-500.

spacebanana7 2y ago
How does NVIDIA manage this issue? I wonder whether they have a very different supply chain or just design software that puts less trust in the reliability of those advertised features.
- trod1234 2y ago
  Its an open question they have never answered afaik. I would speculate that their design is self-contained in hardware.
- bigyabai 2y ago
  I should point out here, if nobody has already; Nvidia's GPU designs are extremely complicated compared to what AMD and Apple ship. The "standard" is to ship a PCIe card with display handling drivers and some streaming multiprocessor hardware to process your framebuffers. Nvidia goes even further by adding additional accelerators (ALUs by way of CUDA core and tensor cores), onboard RTOS management hardware (what Nvidia calls GPU System Processor), and more complex userland drivers that very well might be able to manage atomics without any PCIe standards. This is also one of the reasons AMD and Apple can't simply turn their ship around right now. They've both invested heavily in simplifying their GPU and removing a lot of the creature-comforts people pay Nvidia for. 10 years ago we could at least all standardize on OpenCL, but these days it's all about proprietary frameworks and throwing competitors under the bus.
  - kimixa 2y ago
    FYI AMD also has similar "accelerators", with the 9070 having separate ALU paths for wmma ("tensor") operations much like Nvidia's model - older RDNA2/3 architectures had accelerated instructions but used the "normal" shader ALUs, if a bit beefed up and tweaked to support multiple smaller data types. And CUDA cores are just what Nvidia call their normal shader cores. Pretty much every subunit on a geforce has a direct equivalent on a radeon - they might be faster/slower or more/less capable, but they're there and often at an extremely similar level of the design. AMD also have on-die microcontrollers (multiple, actually) that do things like scheduling or pipeline management, again just like Nvidia's GSP. It's been able to schedule new work on-GPU with zero host system involvement since the original GCN, something that Nvidia advertise as "new" with them introducing their GSP (which just replaced a slightly older, slightly less capable controller rather than being /completely/ new too) The problem is that AMD are a software follower right now - after decades of under-investment they're behind on the treadmill just trying to keep up, so when the Next Big Thing inevitably pops up they're still busy polishing off the Last Big Thing. I've always seen AMD as a hardware company, with the "build it and they will come" approach - which seems to have worked for the big supercomputers who likely find it worth investing in their own modified stack to get that last few %, but clearly falls down selling to "mere" professionals. Nvidia, however, support the same software APIs on even the lowest end hardware, while nobody is likely running much on their laptop's 3050m in anger, it offers a super easy on-ramp for developers - and it's easy to mistake familiarity with superiority - you already know to avoid the warts so you don't get burned by them. And believe me, CUDA has plenty of warts. And marketing - remember "Technical Marketing" is still marketing - and to this day lots of people believe that the marketing name for something, or branding a feature, implies anything about the underlying architecture design - go to an "enthusiast" forum and you'll easily find people claiming that because Nvidia call their accelerator a "core" means it's somehow superior/better/"more accelerated" than the direct equivalent on a competitor, or actually believe that it just doesn't support hardware video encoding as it "Doesn't Have NVENC" (again, GCN with video encoding was released before a Geforce with NVENC). Same with branding - AMD hardware can already read the display block's state and timestamp in-shader, but Everyone Knows Nvidia Introduced "Flip Metering" With Blackwell!
pjc50 2y ago
So .. how's Nvidia dealing with this? Or do they benefit from motherboard manufacturers doing preferential integration testing?
fancyfredbot 2y ago
Look, this sounds like a frustrating nightmare, but the way it seems to us consumers is that AMD chose to rely on poorly implemented and supported technology, and Nvidia didn't. I can't blame AMD for the poor support by motherboards manufacturers but I can and will blame AMD for relying on it.
- trod1234 2y ago
  While we won't know for sure, unless someone from AMD comments on this; in fairness there may not have been any other way. Nvidia has a large number of GPU related patents. The fact that AMD chose to design their system this way, in such a roundabout and brittle manner, which is contrary to how engineer's approach things, may have been a direct result of being unable to design such systems any other way because of broad patents tied to the interface/GPU.
  - fancyfredbot 2y ago
    I feel like this issue is to at least some extent a red herring. Even accepting that ROCm doesn't work on some motherboards, this can't explain why so few of AMD's GPUs have official ROCm support. I notice that at one point there was a ROCm release which said it didn't require atomics for gfx9 GPUs, but the requirement was reintroduced in a later version of ROCm. Not sure what happened there but this seems to suggest AMD might have had a workaround at some point (though possibly it didn't work). If this really is due to patent issues AMD can likely afford to licence or cross-license the patent given potential upside. It would be in line with other decisions taken by AMD if they took this decision because it works well with their datacentre/high-end GPUs, and they don't (or didn't) really care about offering GPGPU to the mass/consumer GPU market.
    - trod1234 2y ago
      > I feel like this issue is to at least some extent a red herring. I don't see that, these two issues adequately explain why so few GPUs have official support. They don't want to get hit with a lawsuit, as a result of issues outside their sphere of control. > If this really is due to patent issues AMD can likely afford to license or cross-license the patent given potential upside. Have you ever known any company willing to cede market dominance and license or cross-license a patent letting competition into a market that they hold an absolute monopoly over, let alone in an environment where antitrust is non-existent and fang-less? There is no upside for NVIDIA to do that. If you want to do serious AI/ML work you currently need to use NVIDIA hardware, and they can charge whatever they want for that. The moment you have a competitor, demand is halved at a bare minimum depending on how much the competitor undercuts you by. Any agreement on coordinating prices leads to price-fixing indictments.
wongarsu 2y ago
There are so many hardware certification programs out there, why doesn't AMD run one to fix this? Create a "ROCm compatible" logo and a list of criteria. Motherboard manufacturers can send a pre-production sample to AMD along with a check for some token amount (let's say $1000). AMD runs a comprehensive test suite to check actual compatibility, if it passes the mainboard is allowed to be advertised and sold with the previously mentioned logo. Then just tell consumers to look for that logo if they want to use ROCm. If things go wrong on a mainboard without the certification, communicate that it's probably the mainboard's fault. Maybe add some kind of versioning scheme to allow updating requirements in the future
zozbot234 2y ago
AIUI, AMD documentation claims that the requirement for PCIe Atomics is due to ROCm being based on Heterogeneous System Architecture, https://en.wikipedia.org/wiki/Heterogeneous_System_Architecture https://en.wikipedia.org/wiki/Heterogeneous_System_Architect... which allows for a sort of "unified memory" (strictly speaking, a unified address space) across CPU and GPU RAM. Other compute API's such as CUDA, OpenCL, SYCL or Vulkan Compute don't have HSA as a strict requirement but ROCm apparently does.

bee_rider 2y ago

I see this form of argument sometimes here but I really don’t get it. Lots of people don’t play the stock market or just invest in funds. It seems like just a way of challenging somebody that looks vaguely clever, or calls them out in a “put your money where your mouth is” sense, but actually presents no argument. Anyway, if you want to short Nvidia you have to know when their bubble is going to pop to get much benefit out of it, right? The market can remain stupid for longer than you can remain solvent or whatever.

the__alchemist 2y ago
Spot on on the timing being important. I don't think you need to fine-tune it that much; short and hold until the pop happens. If you hold off for a the pop could happen at an indefinite time; maybe very far from now, then I think that invalidates the individual prediction. One frustrating aspect of investing is that confident information is tough to come by. It's my take that if you have any (I personally rarely do), you should act on it. So, when someone claims confidently (e.g. with adjectives that imply confidence) that something's going to happen, then that's better than the default. I don't have the insight the claimer does; my thought is: "I am jealous. I with I could be that confident about a stock's trajectory. I would act on it."
- deleted 2y ago
  [deleted]
- themaninthedark 2y ago
  I was a student up until 2009; watching people talk about buying houses for 50K and selling them for 100K, everyone talking about easy money. I knew things were bad when a friend of my sister was complaining that her father(a building framer) was not able to get a loan for a 500K house, something that his colleagues had been able to get. It took another 6 months before the collapse started to hit and the banks when up. Timing is hard.
- pointyfence 2y ago
  What you're describing is what Nassim Taleb calls "tawkers." People like economists, journalists, pundits, etc. who talk big but don't have any skin in the game for being wrong or irrelevant (time is a great example). Since there is no feedback loop to punish bad or irrelevant takes, they can continue tawking for a long time. When I have similarly strong opinions, I do act on it because I enjoy seeing how right or wrong I was. Markets are a harsh, expensive teacher. You either learn a trick or two about uncertainty, overconfidence, humility, etc, or you run out of money. I think you're better at it than you're letting on even if you decided to not play. You already understand the properties of the game.
mancerayder 2y ago
I agree with most everything you said, but the timing doesn't have to be exact and you don't have to short the stock to profit on its downfall. You can buy long-dated puts. Alas, they're not cheap because the risk is very real.
ActorNightly 2y ago
You can short over any period of time. You just pay interest for borrowing the stock. Basically, its a really good question to ask, because even in the case that the person doesn't have investments because they don't play the stock market, it shows that they are not motivated enough to actually go short the security, which means that they truly aren't that sure.

spacebanana7 2y ago

> AI has rocketed them into absolute absurdity, and it's not sustainable Why isn't it sustainable? Their biggest customers all have strong finances and legitimate demand. Google and Facebook would happily run every piece of user generated content through an LLM if they had enough GPUs. Same with Microsoft and every enterprise document. The VC backed companies and Open AI are more fragile, but they're comparatively small customers.

pjc50 2y ago
> Google and Facebook would happily run every piece of user generated content through an LLM if they had enough GPUs. Same with Microsoft and every enterprise document. .. But how much actual value derives from this?
- spacebanana7 2y ago
  Youtube could conceivably put multi language subtitles on every video. Potentially even dub them. But the "real value" would come from making adverts better targeted and more interactive. It's hard to quantity as a person outside of the companies, but the intuition for a positive value is pretty strong.
  - johnnyjeans 2y ago
    > Youtube could conceivably put multi language subtitles on every video. They already do this, it's opt-in. > But the "real value" would come from making adverts better targeted and more interactive. Is there any evidence to suggest that a transformer would be better at collaborative filtering than the current deep learning system that was custom engineered and built for this?
ndiddy 2y ago
IMO the closest analogue for Nvidia now is Cisco during the dot-com boom. Cisco sold the physical infrastructure required for Internet companies to operate. Investors all bought in because they figured it was a safe bet. Individual companies may come and go, but if the Internet keeps growing, companies will always need to buy networking equipment. Despite the Internet being way bigger than it was in 2000, and Cisco being highly profitable, Cisco's share price has never exceeded the peak it was at during the dot-com boom.
fancyfredbot 2y ago
Google may well want to run more of their content through an LLM, but they will not be using Nvidia hardware to do it, they'll be using their TPUs. Amazon are on their third generation of in-house AI chips and Anthropic will be using those chips to train the next generation of Claude. In other words, their biggest customers are looking for cheaper alternatives and are already succeeding in finding them.
- a5ehren 2y ago
  Google and Amazon still have to buy tons of Nvidia HW to provide in their clouds. No one writes to their custom chips besides internal teams because the software stack doesn't exist.