7 ms·
I think the point of open cuda is to run it on non NVIDIA gpus. Once you have to buy NVIDIA gpus what’s the point. If we had true you competition I think it w
by daft_pink 2y ago
I think the point of open cuda is to run it on non NVIDIA gpus. Once you have to buy NVIDIA gpus what’s the point. If we had true you competition I think it would be far easier to buy devices with more vram and thus we might be able to run llama 405b someday locally.
Once you already bought the NVIDIA cards what’s the point
- londons_explore 2y agoThe NVidia software stack has the "no use in datacenters" clause. Is this a workaround for that?
- why_only_15 2y agoSpecifically the clause is that you cannot use their consumer cards (e.g. RTX 4090) in datacenters.
- candiddevmike 2y agoThat's why we run all of our ML workloads in a distributed GPU cluster located in every employee's house
- pplante 2y agoThe bonus is free heating for every employees household!
- rurban 2y agoFree cooling also. You cannot really run a big GPU with external cooling. I needed rather big 15cm isolated cooling tubes to get the heat out of the building.
- seniorThrowaway 2y agoyou joke but I've thought about doing this
- bee_rider 2y agoThe employees can also store their desktops in specially cooled, centrally located lockers at work if they want. And as a perk, we’ll buy and administrate these computers for them.
- paulmd 2y agouse the open kernel driver, which is MIT/GPL and thus cannot impose usage restrictions. it's worth noting that "NVIDIA software stack" is an imprecise term. the driver is the part that has the datacenter usage term, and the open-kernel-driver bypasses that. the CUDA stack itself does not have the datacenter driver clause, the only caveat is that you can't run it on third-party hardware. So ZLUDA/GpuOcelot is still verboten, if you are using the CUDA libraries. https://docs.nvidia.com/cuda/eula/index.html https://docs.nvidia.com/cuda/eula/index.html
- mike64_t 2y agoTo what degree the GeForce software license is applicable even if you don't download the GeForce driver is debatable. It technically governs all software downloadable from nvidia.com including CUDA. The fact that you are only presented with the CUDA license while downloading might not stop Nvidia's lawyers. You might be safe as a consumer against EULAs without the ability to review and reject, but as a company, this might not slide.
- mike64_t 2y agoIt seems to me at least, yes. You still need ptxas, but this piece of Software technically isn't deployed in the datacenter, if you AOT compile your kernels. Its usage seems more than fine, especially considering you could just run it on a system without Nvidia GPUs or old Tesla GPUs while still targeting eg. sm_89. If using ptxas compiled kernels in the datacenter counts as indirect datacenter usage, I don't know. Also, technically you are never presented with the GeForce software license during the CUDA download and installation process, which sparks the question if it is even applicable. In this case, all you would need is the open source driver and you could stuff as many consumer GPUs in your datacenter as you want. However, it technically governs all software downloadable from nvidia.com. I'm no legal expert if this matters, but I would assume consumers would be fine, but companies may be held to a higher standard of seeking out licenses which might govern what they are about to use.
- kelnos 2y agoSome people believe being able to build on fully-open software stacks has value in and of itself. (I happen to be one of those people.) Another benefit could be support for platforms that nvidia doesn't care to release CUDA SDKs for.
- IgorPartola 2y agoHear hear. Yes practically if you need to run a workload on a closed source system or if that’s your only option to get the performance then you have to do what you have to do. But in the long run open source wins because once an open source alternative exists it is just the better option. As a bonus, with open source platforms you are much less subject to whims of company licensing. If tomorrow Nvidia decided to change their licensing strategy and pricing, how many here will be affected by it? OSS doesn’t do that. And even if the project goes in a random direction you don’t like, someone likely forks it to keep going in the right direction (see pfsense/opnsense).
- galkk 2y ago> But in the long run open source wins because once an open source alternative exists it is just the better option. This is just wishful thinking. Anything close to real professional use, not related to IT, and closed source is king: office work, CAD, video editing, music production, and those domains immediately came to mind. Nowhere there open source can seriously challenge commercial, closed sourced competitors. Yes, in any of those domains one can name open source products, but they are far from "winning" or "the better option".
- funcDropShadow 2y agoCounter example: blender. It may not be winning in video editing, but it has serious market share in 3d rendering. Different players are investing money in it and extend it with their own stuff.
- galkk 2y ago
- actionfromafar 2y agoYeah like running Linux on a MacBook…
- Q6T46nT668w6i3m 2y agoCUDA is ubiquitous in science and an open source alternative to the CUDA runtime is useful, even if the use is limited to verifying expected behavior.
- segmondy 2y agoSome of us are running llama 405B locally already. All my GPUs are ancient Nvidai GPUs. IMO, the point of an open cuda is to force Nvidia to stop squeezing us. You get more performance for the buck for AMD. If I could run cuda on AMD, I would have bought new AMD gpus instead. Have enough people do that and Nvidia might take note and stop squeezing us for cash.
- smokel 2y ago> the point of an open cuda is to force Nvidia to stop squeezing us Nobody is forcing you to buy GPUs. Your logic is flawed in the sense that enough people could also simply write alternatives to Torch, which, by the way, is already open source.
- funcDropShadow 2y agoNobody is forcing you to buy a computer. Nobody is forcing you to live under a roof. Nobody is forcing you to eat.
- smokel 2y agoSorry for the harsh comment. I just found it highly unlikely that Nvidia would change its ways due to this, and I don't really see how we're being "squeezed". Nvidia are delivering amazing products (as are AMD), and it is not going to be any cheaper this way. Building this kind of hardware is not something a hacker can do over the weekend.
- Wytwwww 2y agoNvidia is charging what they are entirely because there is no or very little competition. There isn't much else to it, if AMD/Intel caught up Nvidia would suddenly start selling their GPUs for way less...
- deleted 2y ago[deleted]
- btbuildem 2y ago> Once you already bought the NVIDIA cards what’s the point Good luck getting a multi-user GPU setup going, for example. It super sucks when the hardware is capable, but licensing doesn't "allow" it.
- jedberg 2y agoStep 1: Run on NVIDIA gpus until it works just as well as real CUDA. Step 2: Port to other GPUs. At least I assume that is the plan.
- chii 2y ago> Step 2: Port to other GPUs. why not do this first? because the existing closed sourced CUDA already runs well on nvidia chips. Replicating it with an open stack, while ideologically useful, is going to sap resources away from the porting of it to other GPUs (where the real value can be had - by stopping the nvidia monopoly on ai chips).
- jfim 2y agoI'm not involved with the project but I'd assume it's helpful as a reference implementation. If there's a bug on a non-nvidia GPU, the same test case can be run on Nvidia GPUs to see if the bug is in common code or specific to that GPU vendor.
- kstenerud 2y agoI think the point of Linux is to run it on non-Intel CPUs. Once you have to buy Intel CPUs what's the point.
- lambdaone 2y agoYou have it exactly backwards. The original goal of Linux was to create a Unix-like operating system on Linus Torvald's own Intel 80386 PC. Once the original Linux had been created, it was then ported to other CPUs. The joy of a portable operating system is that you can run it on any CPU, including Intel CPUs.
- heavyset_go 2y agoThe closed platform is not without its pitfalls.
- lmpdev 2y agoThe point might not necessarily be for consumers Linus wasn’t writing Linux for consumers (arguably the Linux kernel team still isn’t), he needed a Unix-like kernel on a platform which didn’t support it Nvidia is placed with CUDA in a similar way to how Bell was with Unix in the late 1980s. I’m not sure if a legal “CUDA Wars” is possible in the way the Unix Wars was, but something needs to give Nvidia has a monopoly and many organisations and projects will come about to rectify it, I think this is one example The most interesting thing to see moving forward is where the most just place is to draw the line for Nvidia they deserve remuneration for CUDA, but the question is how much? The axe of the Leviathan (US government) is slowly swinging towards them, and I expect Nvidia to pre-emptively open up CUDA just enough to keep them (and most of us) happy After a certain point for a technology so low in the “stack” of the global economy, more powerful actors than Nvidia will have to step in and clear the IP bottleneck Tech giants are powerful and influence people more than the government, but I think people forget how powerful the government can be when push comes to shove over such an important piece of technology ————— PS my comparison of CUDA to Unix isn’t perfect, mostly as Nvidia has a hardware monopoly as it stands, but as they don’t fab it themselves it’s just a design/information at the end of the day. There’s nothing physically preventing other companies producing CUDA hardware, just obvious legal and business obstacles Perhaps a better comparison would be Texas Instruments trying to monopolise integrated circuits (they never tried). But if Fairchild Semiconductors hadn’t’ve independently discovered ICs, we might have seen a much slower logistic curve than we have had with Moore’s law (assuming competition is proportional to innovation)
- talldayo 2y ago> I expect Nvidia to pre-emptively open up CUDA just enough to keep them (and most of us) happy Besides how they've "opened" their drivers by moving all the proprietary code on-GPU, I don't expect this to happen at all. Nvidia has no incentive to give away their IP, and the antitrust cases that people are trying to build against them border on nonsense. Nvidia monopolizes CUDA like Amazon monopolizes AWS, their "abuse" is the specialization they offer to paying customers... which harms the market how? What really makes me lament the future is the fact that we had a chance to kill CUDA. Khronos wanted OpenCL to be a serious competitor, and if it wasn't specifically for the fact that Apple and AMD stopped funding it we might have a cross-platform GPU compute layer that outperforms CUDA. Today's Nvidia dominance is a result of the rest of the industry neglecting their own GPGPU demand. Nvidia only "wins" because their adversaries would rather fight each other than work together to beat a common competitor. It's an expensive lesson for the industry about adopting open standards when people ask you to, or you suffer the consequences of having nothing competitive.
- jokoon 2y agoI guess this framework was made by amd engineers. Anyway I wonder why amd never challenged nvidia on that market... It smells a bit like amd and nvidia secretly agreed to not compete against each other. Opencl exists but is abandoned.
- deleted 2y ago[deleted]