9 ms·

I really wish more GPU libraries had focused on vulkan instead of CUDA ...

by lalaland1125 4y ago

rowanG077 4y ago
It's one of the reasons Nvidia is basically untouchable at this time. The AI field willingly enslaved itself to NVidia.
- my123 4y ago
  It's because NVIDIA actually cared and AMD does not where it matters (customer HW). Openness is totally secondary to functional. You know, the same kind of reasons as of why Linux on the desktop is not a mass market thing compared to Windows for a very long time.
  - rowanG077 4y ago
    OpenCL is totally functional, on AMD and even iGPU Intel. The reason Nvidia won was because they made it easier. And the AI people ate it up. The tooling Nvidia offers is second to none. But you can build almost anything CUDA does with openCL. It's simply harder to do. The AI crowd cared more about that then the impact of tieing the entire ecosystem to a single company. Who knows what openCL might have been if it would be the premier implementation language. I'd wager it would have gotten a LOT more love.
    - deleted 4y ago
      [deleted]
    - CamperBob2 4y ago
      When I catch myself writing a sentence like It's simply harder to do in order to promote or justify an alternative engineering approach, I try to, well, catch myself before making a really weak argument in favor of an inferior solution. Making life easy for developers is important.
      - rowanG077 4y ago
        I'm not claiming it's not important and it's not very nice that you say I did. When you basically have doomed humanity to rely on a single (malicious) company for a technology that is as important as AI. Then maybe, just maybe, the trade off that it is a harder to implement is worth it.
        my123 4y ago
        The tradeoff was not that. The OpenCL SW ecosystem was just not there at all. It's not a coincidence that nobody has a good AI training on OpenCL stack even today. The cross-vendor infrastructure for that doesn't exist. And NV was far from malicious here, they are who made building this ecosystem possible. Without NV what would have plausibly happened was not having AI training on GPUs at all, but on bespoke accelerators (which _did_ exist back then) at a totally inaccessible cost to customers. It's hard to understate their role in building this ecosystem.
        rowanG077 4y ago
        What exactly is the issue? I use OpenCl without significant issues everyday.
        my123 4y ago
        The whole library ecosystem, for example (but far from only) if you want a BLAS. OpenCL only provides much lower level infrastructure bricks. With CUDA having unmatched performance compared to alternatives too.
        CamperBob2 4y ago
        When you basically have doomed humanity to rely on a single (malicious) company for a technology that is as important as AI I don't disagree, but how did this argument fare against Microsoft? Is there a reason you expect it to fare better against Nvidia? That sweaty guy jumping around yelling "Developers! Developers! Developers!" had a point.
    - frognumber 4y ago
      No. It's not. AMD only officially supports GPGPU headless. That discounts 90% of the market. Old graphics cards lose support randomly. That discounts much of the rest. The whole thing is a horrible, bug-ridden mess. I'd pick AMD over NVidia if it was e.g. 50% slower at the same price point -- open source is worth waiting for -- but I can't take nonworking. AMD also has no support. I'm now building tooling reliant on NVidia, so if AMD ever gets their stuff working, we're many backports away from a working ecosystem. The longer AMD takes, the deeper the hole.
    - paulmd 4y ago
      > OpenCL is totally functional, on AMD and even iGPU Intel. The fact that it’s not functional on non-NVIDIA platforms is the whole reason Blender dropped OpenCL support. If you’re going to write a bunch of implementation-specific code to handle AMD’s bugs/non-compliant runtime, why not just target CUDA directly? “it’s open-source, if you want it fixed then pay someone to do it or just spend a month making a patch instead of doing your work???” bugs-bunny-no.gif like, same with ROCm, is AMD just wants to externalize all their costs onto customers yet still wants them to adopt it instead of the turnkey solution that everyone else already uses. Why would anyone do that? It’s great for NVIDIA but terrible for users. (and as for the “AMD drivers have been good for like ten years now!!!” crowd… counterpoint: the entire 5700XT thing, drivers broken for the first 18 months, just like Vega before it. And oh look 7900XTX is turning into a trainwreck too. There’s just constant showstopping bugs with AMD drivers. Just like with ROCm too… patchwork support and endless bugs that don’t exist in the industry-standard solution. Nobody wants to spend their time doing AMD’s job for them.) To their credit this is one thing Intel got right… they probably spent more dev time on oneAPI in the last year than AMD spent on ROCm and all their previous attempts/projects/resume-driven-development fodder combined.
andy_ppp 4y ago
I thought Vulkan was a graphics specific layer and CUDA was specifically for machine learning?
- mhh__ 4y ago
  CUDA is for GPGPU (general purpose GPU) which includes machine learning. Vulkan is a primarily for graphics but does have options for GPGPU too. Vulkan is however not like OpenGL in that it's fairly close to the hardware in terms of abstraction.
  - my123 4y ago
    Vulkan has a very atrocious developer experience by GPGPU standards. The chance of it winning over CUDA is at zero. And that's _before_ considering its API gaps compared to modern OpenCL. (Yes, even OpenCL is a much better compute API choice than Vulkan. Vulkan does not even have SVM)
    - lalaland1125 4y ago
      Does CUDA have SVM either? Seems like a pretty niche feature IMHO
      - my123 4y ago
        Yes CUDA does, since Kepler, under the CUDA Unified Memory naming. It's not a niche feature at all, but one that is essential to lower the barrier for developer adoption.
    - VHRanger 4y ago
      You should think more of vulkan as an IR endpoint than the actually usable API here. Vulkan is well supported by most GPUs because it's so low level. Performance tends to be good everywhere. What would make vulkan succesful is having APIs that "compile" to this IR. Stuff like vulkan Kompute are good ideas in this direction.
AceJohnny2 4y ago
CUDA predates Vulkan by over 8 years. There's a lot of established ecosystem for CUDA, thanks to Nvidia's investment.

I really wish more GPU libraries had focused on vulkan instead of CUDA ...

rowanG077 4y ago

It's one of the reasons Nvidia is basically untouchable at this time. The AI field willingly enslaved itself to NVidia.

my123 4y ago
It's because NVIDIA actually cared and AMD does not where it matters (customer HW). Openness is totally secondary to functional. You know, the same kind of reasons as of why Linux on the desktop is not a mass market thing compared to Windows for a very long time.
- rowanG077 4y ago
  OpenCL is totally functional, on AMD and even iGPU Intel. The reason Nvidia won was because they made it easier. And the AI people ate it up. The tooling Nvidia offers is second to none. But you can build almost anything CUDA does with openCL. It's simply harder to do. The AI crowd cared more about that then the impact of tieing the entire ecosystem to a single company. Who knows what openCL might have been if it would be the premier implementation language. I'd wager it would have gotten a LOT more love.
  - deleted 4y ago
    [deleted]
  - CamperBob2 4y ago
    When I catch myself writing a sentence like It's simply harder to do in order to promote or justify an alternative engineering approach, I try to, well, catch myself before making a really weak argument in favor of an inferior solution. Making life easy for developers is important.
    - rowanG077 4y ago
      I'm not claiming it's not important and it's not very nice that you say I did. When you basically have doomed humanity to rely on a single (malicious) company for a technology that is as important as AI. Then maybe, just maybe, the trade off that it is a harder to implement is worth it.
      - my123 4y ago
        The tradeoff was not that. The OpenCL SW ecosystem was just not there at all. It's not a coincidence that nobody has a good AI training on OpenCL stack even today. The cross-vendor infrastructure for that doesn't exist. And NV was far from malicious here, they are who made building this ecosystem possible. Without NV what would have plausibly happened was not having AI training on GPUs at all, but on bespoke accelerators (which _did_ exist back then) at a totally inaccessible cost to customers. It's hard to understate their role in building this ecosystem.
        rowanG077 4y ago
        What exactly is the issue? I use OpenCl without significant issues everyday.
        my123 4y ago
        The whole library ecosystem, for example (but far from only) if you want a BLAS. OpenCL only provides much lower level infrastructure bricks. With CUDA having unmatched performance compared to alternatives too.
      - CamperBob2 4y ago
        When you basically have doomed humanity to rely on a single (malicious) company for a technology that is as important as AI I don't disagree, but how did this argument fare against Microsoft? Is there a reason you expect it to fare better against Nvidia? That sweaty guy jumping around yelling "Developers! Developers! Developers!" had a point.
  - frognumber 4y ago
    No. It's not. AMD only officially supports GPGPU headless. That discounts 90% of the market. Old graphics cards lose support randomly. That discounts much of the rest. The whole thing is a horrible, bug-ridden mess. I'd pick AMD over NVidia if it was e.g. 50% slower at the same price point -- open source is worth waiting for -- but I can't take nonworking. AMD also has no support. I'm now building tooling reliant on NVidia, so if AMD ever gets their stuff working, we're many backports away from a working ecosystem. The longer AMD takes, the deeper the hole.
  - paulmd 4y ago
    > OpenCL is totally functional, on AMD and even iGPU Intel. The fact that it’s not functional on non-NVIDIA platforms is the whole reason Blender dropped OpenCL support. If you’re going to write a bunch of implementation-specific code to handle AMD’s bugs/non-compliant runtime, why not just target CUDA directly? “it’s open-source, if you want it fixed then pay someone to do it or just spend a month making a patch instead of doing your work???” bugs-bunny-no.gif like, same with ROCm, is AMD just wants to externalize all their costs onto customers yet still wants them to adopt it instead of the turnkey solution that everyone else already uses. Why would anyone do that? It’s great for NVIDIA but terrible for users. (and as for the “AMD drivers have been good for like ten years now!!!” crowd… counterpoint: the entire 5700XT thing, drivers broken for the first 18 months, just like Vega before it. And oh look 7900XTX is turning into a trainwreck too. There’s just constant showstopping bugs with AMD drivers. Just like with ROCm too… patchwork support and endless bugs that don’t exist in the industry-standard solution. Nobody wants to spend their time doing AMD’s job for them.) To their credit this is one thing Intel got right… they probably spent more dev time on oneAPI in the last year than AMD spent on ROCm and all their previous attempts/projects/resume-driven-development fodder combined.

andy_ppp 4y ago

I thought Vulkan was a graphics specific layer and CUDA was specifically for machine learning?

mhh__ 4y ago
CUDA is for GPGPU (general purpose GPU) which includes machine learning. Vulkan is a primarily for graphics but does have options for GPGPU too. Vulkan is however not like OpenGL in that it's fairly close to the hardware in terms of abstraction.
- my123 4y ago
  Vulkan has a very atrocious developer experience by GPGPU standards. The chance of it winning over CUDA is at zero. And that's _before_ considering its API gaps compared to modern OpenCL. (Yes, even OpenCL is a much better compute API choice than Vulkan. Vulkan does not even have SVM)
  - lalaland1125 4y ago
    Does CUDA have SVM either? Seems like a pretty niche feature IMHO
    - my123 4y ago
      Yes CUDA does, since Kepler, under the CUDA Unified Memory naming. It's not a niche feature at all, but one that is essential to lower the barrier for developer adoption.
  - VHRanger 4y ago
    You should think more of vulkan as an IR endpoint than the actually usable API here. Vulkan is well supported by most GPUs because it's so low level. Performance tends to be good everywhere. What would make vulkan succesful is having APIs that "compile" to this IR. Stuff like vulkan Kompute are good ideas in this direction.

AceJohnny2 4y ago

CUDA predates Vulkan by over 8 years. There's a lot of established ecosystem for CUDA, thanks to Nvidia's investment.