10 ms·
So to make sure I understand, this would mean: 1. Programs built against MLX -> Can take advantage of CUDA-enabled chips but not: 2. CUDA programs -> Can now
by lukev 1y ago
So to make sure I understand, this would mean:
1. Programs built against MLX -> Can take advantage of CUDA-enabled chips
but not:
2. CUDA programs -> Can now run on Apple Silicon.
Because the #2 would be a copyright violation (specifically with respect to NVidia's famous moat).
Is this correct?
- ls612 1y ago#2 would be Google v. Oracle wouldn’t it?
- saagarjha 1y agoNo, it's because doing 2 would be substantially harder.
- lukev 1y agoThere's a massive financial incentive (billions) to allow existing CUDA code to run on non-NVidia hardware. Not saying it's easy, but is implementation difficulty really the blocker?
- ivell 1y agoModular is trying with Mojo + Max offering. It has taken quite a bit of effort to target NVidia and get parity. They are now focusing on other hardware.
- saagarjha 1y agoYes. See: AMD
- lukev 1y agoAMD has never implemented the CUDA API. And not for technical reasons.
- gpm 1y agoThey did, or at least they paid someone else to. https://www.techpowerup.com/319016/amd-develops-rocm-based-solution-to-run-unmodified-nvidias-cuda-binaries-on-amd-graphics https://www.techpowerup.com/319016/amd-develops-rocm-based-s...
- Imustaskforhelp 1y agoBut I think then there was some lawsuit and the rocm guy/team had gone really ahead but amd dropped it because of either fear of lawsuit or lawsuit in general. Then, now they had to stop working on some part of the source code and had to rewrite a lot of things again, they are still not as close to as they were before amd lawyer shenanigan
- pjmlp 1y agoPartially, the CUDA C++ API, not CUDA APIs.
- lmm 1y agoI think it's ultimately a project management problem, like all hard problems. Yes it's a task that needs skilled programmers, but if an entity was willing to pay what programmers of that caliber cost and give them the conditions to let them succeed they could get it done.
- fooker 1y agoExisting high performance cuda code is almost all first party libraries, written by NVIDIA and uses weird internal flags and inline ptx. You can get 90% of the way there with a small team of compiler devs. The rest 10% would take hundreds of people working ten years. The cost of this is suspiciously close to the billions in financial incentive you mentioned, funny how efficient markets work.
- lcnielsen 1y ago> funny how efficient markets work. Can one really speak of efficient markets when there are multiple near molopolies at various steps in the production chain with massive integration, and infinity amounts of state spending in the process?
- bigyabai 1y agoSure they can. CUDA used to have a competitor, sponsored by Apple. It's name is OpenCL.
- dannyw 1y agoAnd after Apple dropped NVIDIA, they stopped caring about openCL performance on their GPUs.
- pjmlp 1y agoIt was never really a competitor, as the other two sponsors Intel and AMD, never deliverd anything great with it. Additionally the tooling is horrendous, plain old C, with the same compilation model as OpenGL. It took getting a hard beating from CUDA, to finally add a bytecode format (SPIR), and at least support C++ as well. Additionally the other mobile OS big name never cared about OpenCL, rather pushed their own thing, Renderscript.
- bigyabai 1y agoSo, to reiterate: Apple used to have a CUDA competitor that was so bad you guys get mad when I call it competition.
- int_19h 1y agoFrom the market perspective, it's down to whether the amount of money needed to get there and stay there (keeping in mind that this would have to be an ongoing effort given that CUDA is not a static target) is more or less than the amount of money needed to just buy NVIDIA GPUs.
- hangonhn 1y agoIs CUDA tied very closely to the Nvidia hardware and architecture so that all the abstraction would not make sense on other platforms? I know very little about hardware and low level software. Thanks
- saagarjha 1y agoYes, also it's a moving target where people don't just want compatibility but also good performance.
- dagmx 1y agoCUDA isn’t really that hyper specific to NVIDIA hardware as an api. But a lot of the most useful libraries are closed source and available on NVIDIA hardware only. You could probably get most open source CUDA to run on other vendors hardware without crazy work. But you’d spend a ton more work getting to parity on ecosystem and lawyer fees when NVIDIA come at you.
- lcnielsen 1y agoThe kind of CUDA you or I would write is not very hardware specific (a few constants here and there) but the kind of CUDA behind cuBLAS with a million magic flags, inline PTX ("GPU assembly") and exploitation of driver/firmware hacks is. It's like the difference between numerics code in C and and numerics code in C with tons of in-line assembly code for each one of a number of specific processors. You can see similar things if you buy datacenter-grade CPUs from AMD or Intel and compare their per-model optimized BLAS builds and compilers to using OpenBLAS or swapping them around. The difference is not world ending but you can see maybe 50% in some cases.
- pjmlp 1y agoCUDA is an ecosystem, many keep failing to understand that, trying to copy only the C++ compiler.
- quitit 1y agoIt's 1. It means that a developer can use their relatively low-powered Apple device (with UMA) to develop for deployment on nvidia's relatively high-powered systems. That's nice to have for a range of reasons.
- _zoltan_ 1y ago"relatively high powered"? there's nothing faster out there.
- chvid 1y agoRelative to what you can get in the cloud or on a desktop machine.
- MangoToupe 1y agoIs this true per watt?
- spookie 1y agoIt doesn't matter for a lot of applications. But fair, for a big part of them it is either essential or a nice to have. But completely off the point if we are waging fastest compute no matter what.
- johnboiles 1y ago...fastest compute no matter watt
- sgt101 1y agoI wonder what Apple would have to do to make metal + its processors run faster than nVidia? I guess that it's all about the interconnects really.
- summarity 1y agoRight now, for LLMs, the only limiting factor on Apple Silicon is memory bandwidth. There hasn’t been progress on this since the original M1 Ultra. And since abandoning UltraFusion, we won’t see progress here anytime soon either.
- dagmx 1y ago2 also further cements CUDA as the de facto API to target, and nobody would write MLX targeted code instead. This way, you’re more incentivized to write MLX and have it run everywhere. It’s a situation of everyone wins, especially Apple because they can optimize it further for their platforms.
- tekawade 1y agoI want #3 be able to connect NVIDIA GPU with Apple Silicon and run CUDA. Take advantage of apple silicon + unified memory + GPU + CUDA with PyTorch, JAX or TensorFlow. Haven’t really explored MLX so can’t speak about it.
- tho234j2344 1y agoI don't think #2 is really true - AMDs HIP is doing this exact thing after giving up on OpenCL way back in ~'17/'18.
- NekkoDroid 1y agoI haven't looked into it, but doesn't HIP need everything to be recompiled against it? To my understanding it was mostly a source code translation effectivly.
- pjmlp 1y agoFor CUDA C++, not CUDA the ecosystem.
- sitkack 1y ago#2 is not a copyright violation. You can reimplement APIs.
- 7734128 1y agoThe famous Android Java fight is probably the most important case of that discussion.
- hnfong 1y agoIndeed. Unfortunately when that case went to the Supreme Court they basically just said "yeah for this case it's fair use, but we're not going to comment on whether APIs in general are copyrightable"...
- adastra22 1y agoCUDA is not an API, it is a set of libraries written by NVIDIA. You'd have to reimplement those libraries, and for people to care at all you'd have to reimplement the optimizations in those libraries. That does get into various IP issues.
- Imustaskforhelp 1y agoEven if its not as optimized, it would still be nice to see a CUDA alternative really Also I do wonder what the difference b/w a API and a set of libraries are, couldn't an API be exposed from that set of libraries which could be used? Its a little confusing I guess
- adastra22 1y ago> couldn't an API be exposed from that set of libraries which could be used And now you've entered that copyright violation territory.
- Someone 1y agoIP infringement, not copyright violation. A clean room reimplementation of cuda would avoid any copyright claims, but would not necessary avoid patents infringement. https://en.wikipedia.org/wiki/Clean-room_design https://en.wikipedia.org/wiki/Clean-room_design: “Clean-room design is useful as a defense against copyright infringement because it relies on independent creation. However, because independent invention is not a defense against patents, clean-room designs typically cannot be used to circumvent patent restrictions.”
- pxc 1y agoCopyright can't prohibit compatible implementations that are developed independently through reverse engineering, if the implementers are very careful about the way they work. (Software patents can, though.)
- deleted 1y ago[deleted]