6 ms·
The silicon is just one piece of the puzzle. CUDA and the rest of the software stack is huge advantage for NVIDIA.
by chrsw 9mo ago
The silicon is just one piece of the puzzle. CUDA and the rest of the software stack is huge advantage for NVIDIA.
- bhouston 9mo agoyup, which is why AMD struggles so much even though its hardware is usually within 30% of the performance (give or take) of NVIDIA. (Replaced "with 30%" with "within 30%")
- bigyabai 9mo agoIt's also complicated by the notion that raster performance doesn't directly translate to tensor performance. Apple and AMD both make excellent raster GPUs, but still lose in efficiency to the CUDA's architecture in rendering and compute. I'd really like AMD and Apple to start from scratch with a compute-oriented GPU architecture, ideally standardized with Khronos. The NPU/tensor coprocessor architecture has already proven itself to be a bad idea.
- Asmod4n 9mo agoAMD is doing that for their next gen, no idea if khronos is involved.
- roysting 9mo agoThat may be true, but assuming you meant "within 30% of the performance" ... can we just acknowledge that is a rather significant handicap, even ignoring CUDA.
- epolanski 9mo agoThe customers are players that can throw money into the software stack, hell, they are even throwing lots of money in the hardware one too with proprietary tensors and such. And the big players don't necessarily care about the full software stack, they are likely to optimize the hardware for single usage (e.g. inference or specific steps of the training).
- Qision 9mo agoWhy doesn't AMD make a similar framework than CUDA? Is this so much of a task? But if that increases their market share that should be financially viable, no?
- 4fterd4rk 9mo agoThey do. It's called ROCm. It works, it's open source, but CUDA is so entrenched it's like a Windows vs. Linux kind of thing.
- PrivateButts 9mo agoROCm is their CUDA-like and imo it's been a buggy mess, and I'm talking bugs that make your entire system lock up until you hard reboot. Same with their media encoders. Vulkan compute is starting to recieve support by stuff like llama.cpp and ollama and I've had way better luck with that on non-nvidia hardware. Probably for the best that we have a single cross-vendor standard for this.
- yencabulator 8mo ago> bugs that make your entire system lock up until you hard reboot That has historically been the experience with Nvidia GPUs on Linux also.
- hipster001 9mo agoThey even make two! rocm and hip. And intel has one…api.
- yolostar1 9mo agoExactly. CUDA is huge moat and all competitors must be adopting SOFTWARE first approach similar to what tinycorp is trying to do. Find one single thing that makes CUDA bad to use and TRIPLE DOWN on that.
- nszceta 9mo agoIntel people would point at SYCL if you told them this
- bfrog 9mo agoI which no one cares about. As a 1% player having a convoluted C++ centric stack when the 99% player has something different e ouch porting requires critical thinking means no one gives a damn about it. ZLUDA has more interest that SyCL and that should say it all right there.
- bfrog 9mo agoIntel focused on SyCL which not many people seem to actually care about. It looks far enough removed from CUDA you’d have to think hard about porting things as well. From what I understand ROCm looks very close to CUDA.