12 ms·
A lot of people think AMD should support these translation layers but I think it's a bad idea. CUDA is not designed to be vendor agnostic and Nvidia can make th
by modeless 2y ago
A lot of people think AMD should support these translation layers but I think it's a bad idea. CUDA is not designed to be vendor agnostic and Nvidia can make things arbitrarily difficult both technically and legally. For example I think it would be against the license agreement of cuDNN or cuBLAS to run them on this. So those and other Nvidia libraries would become part of the API boundary that AMD would need to reimplement and support.
Chasing bug-for-bug compatibility is a fool's errand. The important users of CUDA are open source. AMD can implement support directly in the upstream projects like pytorch or llama.cpp. And once support is there it can be maintained by the community.
- DeepYogurt 2y agoYa, honestly better to leave that to third parties who can dedicate themselves to it and maybe offer support or whatever. Let AMD work on good first party support first.
- fngjdflmdflg 2y ago>Nvidia can make things arbitrarily difficult both technically and legally. I disagree. AMD can simply not implement those APIs, similar to how game emulators implement the most used APIs first and sometimes never bother implementing obscure ones. It would only matter that NVIDIA added eg. patented APIs to CUDA if those APIs were useful. In which case AMD should have a way to do them anyway. Unless NVIDIA comes up with a new patented API which is both useful and impossible to implement in any other way, which would be bad for AMD in any event. On the other hand, if AMD start supporting CUDA and people start using AMD cards, then developers will be hesitant to use APIs that only work on NVIDIA cards. Right now they are losing billions of dollars on this. Then again they barely seem capable of supporting RocM on their cards, much less CUDA. You have a fair point in terms of cuDNN and cuBLAS but I don't know that that kind of ToS is actually binding.
- avidphantasm 2y agoPatented API? I thought Google v. Oracle settled this? Making an implementation of an API spec is fair use, is it not?
- fngjdflmdflg 2y agoMy understanding is that Google v. Oracle only applies to copyright.
- nl 2y agoWell you can't patent an API so....
- fngjdflmdflg 2y agoYou can patent the implementation. You can't patent the API name DecodeH265Video() but you can still sue someone for implementing that function correctly.
- anticensor 2y agoIf there is only one way to solve a problem, there is nothing to invent, just discover, and discoveries are decidedly not patentable.
- fngjdflmdflg 2y ago>If there is only one way to solve a problem h.265 is one way, av1 is another way.
- dietr1ch 2y agoHow's this situation different than the one around Java, Sun/Oracle and Google?
- blitzar 2y agoIt would be good if AMD did something, anything. Support this, reimplement that, support upstream efforts, dont really care. Any of those would cost a couple of million and be worth a trillion dollars to AMD shareholders.
- oezi 2y agoA couple of million doesn't get you anything in corporate land
- spacebanana7 2y agoA couple dozen billion for a 10% chance of becoming NVIDIA competitive is worth it, looking at the stock prices.
- slashdave 2y agoROCm counts as "something"
- curt15 2y agoPretty much any modern NVIDIA GPU supports CUDA. You don't have to buy a datacenter-class unit to get your feet wet with CUDA programming. ROCm will count as "something" when the same is true for AMD GPUs.
- muxr 2y agoI don't think AMD needs to support 5+ year old GPUs personally. And all the recent generations are already practically supported. AMD only claims support for a select few GPUs, but in my testing I find all the GPUs work fine if the architecture is supported. I've tested rx6600, rx6700xt for example and even though they aren't officially supported, they work fine on ROCm.
- Const-me 2y ago> Nvidia can make things arbitrarily difficult both technically and legally Pretty sure APIs are not copyrightable, e.g. https://www.law.cornell.edu/supremecourt/text/18-956 https://www.law.cornell.edu/supremecourt/text/18-956 > against the license agreement of cuDNN or cuBLAS to run them on this They don’t run either of them, they instead implement an equivalent API on top of something else. Here’s a quote: “Open-source wrapper libraries providing the "CUDA-X" APIs by delegating to the corresponding ROCm libraries. This is how libraries such as cuBLAS and cuSOLVER are handled.”
- dralley 2y agoI believe it was decided that they are copyrightable but that using them for compatibility purposes is fair use.
- kbolino 2y agoNo, it's stranger than that: SCOTUS did not rule on copyrightability of APIs at all, but simply ruled that even if they are copyrightable, what Google did (completely reimplement Sun/Oracle's public API) was still fair use.
- mrandish 2y agoIt would have been nice to get a clear SCOTUS precedent on this. On the other hand, I also value a SCOTUS which rules minimally and narrowly by default (I also appreciate SCOTUS' return to stricter constitutional grounding in the past decade).
- amelius 2y agoLike supporting x86 was a bad idea as well?
- karolist 2y agoWas there a large entity steering x86 spec alone with a huge feature lead against their competition, free to steer the spec in any ways they choose? Also, hardware is not opensource software, you get big players onboard and they will be able to implement the spec they want every gen, software has more moving parts and unaligned parties involved.
- cherryteastain 2y ago> Was there a large entity steering x86 spec alone with a huge feature lead against their competition, free to steer the spec in any ways they choose? Ever heard of Intel?
- karolist 2y agoI had't considered that angle. Is your point that Intel was the creator of x86, but software chose to support it, then AMD had nothing else but to play catch up in x86 support to be part of the software target market? If so and factual (I've no idea), fair point, I didn't know.
- marshray 2y agoIt was exactly the same instruction set. C compilers didn't offer an "AMD" CPU target* until AMD came out with the "AMD64" instruction set. Today we call this "x86_64" or "x64". * Feel free to point out some custom multimedia vector extensions for Athlons or something, but the point remains.
- gmokki 2y agoAnd Intel named its licenced implementation of AMD64 as IA-32e, just to make it clear to everyone that it is based on Intel architecture 32bit version with an extension. Luckily they dropped that name few years later
- eslaught 2y agoAre you aware of HIP? It's officially supported and, for code that avoids obscure features of CUDA like inline PTX, it's pretty much a find-and-replace to get a working build: https://github.com/ROCm/HIP https://github.com/ROCm/HIP Don't believe me? Include this at the top of your CUDA code, build with hipcc, and see what happens: https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/hip_cuda_compat/hip_cuda.h https://gitlab.com/StanfordLegion/legion/-/blob/master/runti... It's incomplete because I'm lazy but you can see most things are just a single #ifdef away in the implementation.
- currymj 2y agoif you're talking about building anything, that is already too hard for ML researchers. you have to be able to pip install something and just have it work, reasonably fast, without crashing, and also it has to not interfere with 100 other weird poorly maintained ML library dependencies.
- bootsmann 2y agoDon’t most orgs that are deep enough to run custom cuda kernels have dedicated engineers for this stuff. I can’t imagine a person who can write raw cuda not being able to handle things more difficult than pip install.
- gaogao 2y agoEngineers who are really, really good at CUDA are worth their weight in gold, so there's more projects for them than they have time. Worth their weight in gold isn't figurative here – the one I know has a ski house more expensive than 180 lbs of gold (~$5,320,814).
- bbkane 2y agoWould you (or your friend) be able to drop any good CUDA learning resources? I'd like to be worth my weight in gold...
- viraptor 2y agoIsn't cuDNN a much better case for reimplementing than CUDA? It has much more choice in how things actually happen and cuDNN itself chooses different implementations at runtime + does fusing. It seems way more generic and the reimplementation would allow using the best AMD-targeted kernel rather than one the original has.
- ckitching 2y agoAMD have "MIOpen" which is basically cuDNN-for-AMD. Ish.
- mmis1000 2y agoAnd that thing is left for unreleased on windows for almost a whole year for unknown reason. Even though there is activity on github and build fix frequently. There is just no .exe or .msi for you to download. In fact, the rocm for linux is on major 6 release (which includes miopen). But somehow windows is still on major 5 (don't have miopen) for almost a whole year. It almost make me wonder. Is there a shady trade somewhere to ask amd never release sdk for Windows to hike the price of nvidia card higher? Why they keep developing these without release it at all?
- flamedoge 2y agoprobably because their focus is on data centers that mostly run linux
- viraptor 2y agoSince they cancelled the work on zluda and absolutely fail to do anything about other options, I really believe there's some "don't do it or you'll get sued to hell and back" agreement. They can't be so dumb they just miss it by accident.
- anigbrowl 2y agoGiven AMDs prior lack of interest I'll take whatever options there are. My daily driver has a Vega 10 GPU and it's been quite frustrating not to be able to easily leverage it for doing basic ML tasks, to the point that I've been looking at buying an external nvidia GPU instead just to try out some of the popular Python libraries.
- Wowfunhappy 2y ago> CUDA is not designed to be vendor agnostic and Nvidia can make things arbitrarily difficult [...] technically. (Let's put the legal questions aside for a moment.) nVidia changes GPU architectures every generation / few generations, right? How does CUDA work across those—and how can it have forwards compatibility in the future—if it's not designed to be technologically agnostic?
- andy_ppp 2y agoOne way is to make sure the hardware team does certain things to support easy transition to new architectures, we have seen this with Apple Silicon for example!
- saagarjha 2y agoPTX is meant to be portable across GPU microarchitectures. That said, Nvidia owns the entire spec, so they can just keep adding new instructions that their GPUs now support but AMD GPUs don't.
- koolala 2y agoCUDA v1...CUDA v2... CUDA v... CUDA isnt commonly assosiated with a version number...
- Uehreka 2y ago…yes it is? https://developer.nvidia.com/cuda-toolkit-archive https://developer.nvidia.com/cuda-toolkit-archive
- neutrinobro 2y agoCries in OpenCL
- rjurney 2y agoNot having a layer like this has left AMD completely out of the AI game that has made NVDA the world's most valuable company.
- ChoGGi 2y agoSelf-inflicted wounds hurt the most.
- HarHarVeryFunny 2y agoWell, they kinda have it with their hipify tool, although this is for porting CUDA code to AMD's HIP which supports both AMD and NVIDIA. This supports CUDA C code and libraries with AMD equivalents like cuDNN, cuBLAS, cuRAND, but doesn't support porting of CUDA C inline PTX assembler. AMD have their own inline GCN assembler, but seem to discourage it's use. There are also versions of PyTorch, TensorFlow and JAX with AMD support. PyTorch's torch.compile can generate Triton (OpenAI's GPU compiler) kernels, with Triton also supporting AMD.
- varelse 2y ago[dead]
- apatheticonion 2y agoAgreed. Rather than making CUDA the standard; AMD should push/drive an open standard that can be run on any hardware. We have seen this succeed multiple times: FreeSync vs GSync, DLSS vs FSR, (not AMD but) Vulkan vs DirectX & Metal. All of the big tech companies are obsessed with ring-fencing developers behind the thin veil of "innovation" - where really it's just good for business (I swear it should be regulated because it's really bad for consumers). A CUDA translation layer is okay for now but it does risk CUDA becoming the standard API. Personally, I am comfortable with waiting on an open standard to take over - ROCm has serviced my needs pretty well so far. Just wish GPU sharing with VMs was as easy as CPU sharing.
- amy-petrik-214 2y agowe actually also saw this historically with openGL. openGL comes from an ancient company whispered about by the elderly programmers (30 + year old) known as SGI. Originally it was CLOSED SOURCE and SGI called it "SGI-GL" for a computer codename IRIS which was cool looking with bright popping color plastic and faux granite keyboard. Good guy SGI open sourced SGI-GL to become what we called "openGL" (get it, now it's open), and then it stuck. That's all to say NVIDIA could pull a SGI and open their stuff, but they're going more sony style and trying to monopolize. Oh, and SGI also wrote another ancient lore library known as "STL" or the "SGI Template Library" which is like the original boost template metaprogramming granddaddy
- adrian_b 2y agoAlso the XFS file system.
- usr1106 2y agoNice story, but is it correct? Wikipedia says STL was first implemented by HP and later by the same authors at SGI.
- adrian_b 2y agoSTL started even earlier, obviously without using the name "STL", as a library of generic algorithms for the programming language Ada (David R. Musser & Alexander A. Stepanov, 1987).
- magic_hamster 2y agoCUDA is the juice that built Nvidia in the AI space and allowed them to charge crazy money for their hardware. To be able to run CUDA on cost effective AMD hardware can be a big leap forward, allow more people to research, and break away from Nvidia's stranglehold over VRAM. Nvidia will never open source their own platform unless their hand is forced. I think we all should support this endeavor and contribute where possible.
- Sparkyte 2y agoThat is why an open standard should be made so it isn't locked to a particular piece of hardware and then allow modular support for different hardware to interface with supported drivers.
- raxxorraxor 2y agoI really hope they will do what you suggested. With some innovative product placement, GPUs with a lot of memory for example, they could dethrone nvidia if it doesn't change strategy. That said, easier said than done. You need very specialized developers to build a CUDA equivalent and have people start using it. AMD could do it with a more open development process leveraging the open source community. I believe this will happen at some point anyway by AMD or someone else. The market just gets more attractive by the day and at some point the high entry barrier will not matter much. So why should AMD skimp on their ambitions here? This would be a most sensible investment, few risks and high gains if successful.
- consf 2y ago[dead]
- consf 2y ago[dead]
- hot_gril 2y agoProviding support in pytorch sounds obvious, but I have very little experience here. Why is this still a problem?