3 ms·
No, because there is no CUDA. We have fast and cheap alternatives to NVIDIA, but they do not have CUDA. This is why NVIDIA has 90% margins on its hardware.
by ZeroTalent 2y ago
No, because there is no CUDA. We have fast and cheap alternatives to NVIDIA, but they do not have CUDA. This is why NVIDIA has 90% margins on its hardware.
- jauntywundrkind 2y agoCUDA is simply not important for modern vLLM and many many others. DeepSeek V3 works great on SGLang. https://www.amd.com/en/developer/resources/technical-articles/amd-instinct-gpus-power-deepseek-v3-revolutionizing-ai-development-with-sglang.html https://www.amd.com/en/developer/resources/technical-article... Can you do absolutely everything? No. But most models will run or retrain fine now without CUDA. This premise keeps getting recycled from the past, even as that past has grown ever more distant.
- physicsguy 2y agoCUDA is incredibly important still. It's still an incredible amount of work to get packages working on multiple GPU paradigms, and by default everyone still starts with CUDA. The example I always give is FFT libraries - if you compare cuFFT to rocFFT. rocFFT only just released support for distributed transforms in December 2024, something you've been able to do since CUDA Toolkit v8.0, released in 2017. It's like this across the whole AMD toolkit, they're so far behind CUDA it's kind of laughable.
- ZeroTalent 2y agoCUDA is becoming more critical, not less, every day. Software developed around CUDA is vastly outpacing what other companies produce. And saving a few millions when creating new models doesn't matter; NVIDIA is pretty efficient at scale. I don't know if you've heard, but NVIDIA is about to add a monthly payment for additional CUDA features and I'm almost certain that many big companies will be happy to pay for them. > But most models will run or retrain fine now without CUDA. This is correct for some small startups, not big companies.