3 ms·
CUDA is a moat because AMD and Intel are run by morons^W^W^W run by people who can't swallow the fact that software is more important than hardware. Intel shou
by bsder 3y ago
CUDA is a moat because AMD and Intel are run by morons^W^W^W run by people who can't swallow the fact that software is more important than hardware.
Intel should be shoveling out 16GB Arc graphics cards for free to every graduate program in the country who can fill out a web form. In a couple years, they'd displace NVIDIA.
AMD needs to be funding a CUDA shim that allows people to port stuff directly to their cards. And they need to NOT be segmenting the consumer and professional cards software ecosystems.
Yes, there has been progress. However, when you look at the amount of money that AMD and Intel throw at software vs how much NVIDIA throws at software, it's an instant facepalm moment.
NVIDIA is 100% vulnerable--if it weren't for the fact that their competitors are idiots.
- froonly 3y agoCUDA is a shallow moat whose effectiveness depends entirely on NVidia convincing people to be mortally fearful of water.
- kkielhofner 3y agoGenuine questions. What are your use cases? What do you do? How much experience? My personal experience shows CUDA to in fact be a very deep moat. In ~12 years CUDA and ~6 ROCm (since Vega) I’ve never met a professional who says otherwise, including those at top500.org AMD sites. From what I’ve seen online this take really seems to come from some kind of Linux desktop Nvidia grudge/bad experience or just good ‘ol gaming/desktop team red vs green vs blue nonsense. Many things can be said about Nvidia and all kinds of things can be debated but suggesting that Nvidia has > 90% market share simply and solely because people drink Nvidia kool-aid is a wild take.
- froonly 3y agoI have 40+ yrs of HPC/AI apps/performance engineering experience & I was one of the 1st people to port LAPACK and a number of other numerical libs to CUDA. Moreover, many of those major DoE + AI sites are my customers. You should not confuse AMD's general & long-standing indifference/incompetence wrt SW with the actual difficulty of providing a portable SW path for acceleration. As Woody Allen once said: "90% of success is showing up" But what happened in AI, when, in a very short period of time, almost everyone moved away from writing their directly in CUDA, to writing them in frameworks like Tensorflow & PyTorch is all the evidence anyone need to show just how unsound that SW obstacle is.
- teleforce 3y agoI almost spew my coffee when reading your grand parent comments. One of the first teams that ported LAPACK to CUDA or CULA are apparently being paid handsomely by Nividia [1],[2]. Interestingly, DCompute is a little known effort to support compute on CUDA and OpenCL in D language, and it was done by a part-time undergrad student [3]. I strongly believe we need a very capable language to make advancement much easier in HPC/AI/etc, and D language fit the bill very much and then some. Heck it even beat other BLAS libraries that other so called data languages namely Matlab and Julia still heavily depended on for their performances to this very day. It does it in style back in 2016 more than seven years ago [4]. The DCompute implementation by the part-timer in 2017 actually depended on this native D implementation of these linear algebra routines in Mir [5]. [1] CULA: hybrid GPU accelerated linear algebra routines: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/7705/770502/CULA-hybrid-GPU-accelerated-linear-algebra-routines/10.1117/12.850538.short https://www.spiedigitallibrary.org/conference-proceedings-of... [2] CUDA Spotlight: John Humphrey: https://www.nvidia.com/content/cuda/spotlights/john-humphrey.html https://www.nvidia.com/content/cuda/spotlights/john-humphrey... [3] DCompute: GPGPU with Native D for OpenCL and CUDA: https://dlang.org/blog/2017/07/17/dcompute-gpgpu-with-native-d-for-opencl-and-cuda/ https://dlang.org/blog/2017/07/17/dcompute-gpgpu-with-native... [4] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen: http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/glas-gemm-benchmark.html http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/... [5] DCompute: Native execution of D on GPUs and other Accelerators: https://github.com/libmir/dcompute https://github.com/libmir/dcompute
- froonly 3y agoI got paid to do the LAPACK port, back in the mid 2000s, for a federal contractor working on satellite imaging type apps. I was still a good coder, back then... Took me about a month, as I recall. Maybe 6 weeks. But I'm one of those old-school HPC guys who believes that libraries are mostly irrelevant, and absolutely no substitute for compilers and targeted code generation. Julia is cool, btw. It could very well end up supplanting Fortran, once they fix the poor performance code generation issues.
- paulddraper 3y agoIt's effectiveness depends on there being nothing on the other side.
- froonly 3y agoMostly true, and you'll get no argument from me on the AMD & Intel are fuckwits front. Intel does ok, but AMD in particular has completely dropped the ball on the SW front, and has been doing so for at least 25 yrs. The point I was glibly trying to get across was that even a small effort on the part of AMD to treat the SW side as seriously as NVidia does would have yielded great benefits, and not have left them so far behind. Also, there is a lot of work going on in the gcc & llvm toolchain to not only use OpenMP to target accelerators in computationally intensive loops but, in the case of llvm, to also target tensor instructions for more efficient code generation (https://lists.llvm.org/pipermail/llvm-dev/2021-November/153725.html https://lists.llvm.org/pipermail/llvm-dev/2021-November/1537...). It took the AI folk less than 18 months to almost completely move away from CUDA to Tensorflow and then PyTorch... LLVM, imho, is going to do the same for Sci/Eng and general code bases in the next 2 years.
- anon291 3y agoNvidia are arge contributors to these llvm changes, so again I don't see how this is going to hurt them.
- froonly 3y agoNever said it would hurt NVidia, only that CUDA itself isn't as strong a barrier as people seem to think it is. But with GPU target support in LLVM, in most cases you won't need to resort to CUDA anymore.
- lumost 3y agoHow hard do you think it is to find engineers who are - Great at legacy C++ code. - Great at new C++ code. - Great at embedded/high performance/distributed code. - Are experts in Linear Algebra and Calculus - Are competent at Machine Learning and similar problems. Now imagine, that after you find ~10-50 competent senior engineers who can each segment and train 1-5 engineers, you also need to hire 10-20 managers, PMs and directors who are smart enough to do more than "copy NVidia's offering from last year", and wise enough to still build a 1:1 compatibility layer. Apple is likely seeing more traction on their metal API by virtue that it is reasonably well guaranteed to be around in ~5 years, and is common on multiple device platforms that students/devs use or customers deploy.
- Tiktaalik 3y agoAll this describes video game programmers to me (well 4/5 at least). Given that there's been thousands of game layoffs recently anyone looking to build their AI teams should be diving through linkedin looking for laid off video game programmers.
- lumost 3y agoMy understanding of the games industry is that a small fraction of game programmers are working on core game engines and low level graphics kernels. Is that inaccurate?
- klooney 3y agoEven people working on the higher level stuff have more exposure to matrix multiplication than people working on CRUD apps.
- dannyw 3y agoWe're talking about trillion dollars of market cap here. If the difficulty is in hiring up to ~70 people, with somewhat but not obscure skills, perhaps the executives should be revisited.
- deleted 3y ago[deleted]
- aurareturn 3y ago>NVIDIA is 100% vulnerable--if it weren't for the fact that their competitors are idiots. I think Nvidia sees it too. That's why they're moving upstream by providing the entire stack from CUDA, GPUs, interconnects chips, networking chips, racks, OS, software, models. I think the "CUDA moat" people like OP are underselling Nvidia. They're positioning themselves as the full-stack AI provider. Forget CUDA.
- chii 3y agoThe moat that CUDA currently provides is what gives nvidia the room to move up. CUDA is a stepping stone - something stable they can rely on to cement an even higher position (tell me that the full stack is less vendor locked-in than CUDA, and i'll have a bridge to sell you).
- aurareturn 3y agoBy the time some companies figure out how to replace CUDA, Nvidia has already moved well above the CUDA layer.
- lardo 3y ago> AMD needs to be funding a CUDA shim that allows people to port stuff directly to their cards. And they need to NOT be segmenting the consumer and professional cards software ecosystems. Isn't that what HIPIFY does? https://github.com/ROCm/HIPIFY https://github.com/ROCm/HIPIFY
- paulmd 3y agoproblem being that despite years of work and despite all the marketing hype, it’s still missing basic feature that are over 10+ years old on the nvidia side. If you can’t do dynamic parallelism then kernels can never launch kernels, for example. It has “partial support” for texture unit access. Inter-process communication is not supported. Etc. https://rocm.docs.amd.com/projects/HIP/en/latest/user_guide/faq.html https://rocm.docs.amd.com/projects/HIP/en/latest/user_guide/...