3 ms·

Is there any potential on the horizon for an Intel card to work with CUDA?

snnn 3y ago
Bet on WebGPU. Intel and Google are the two main drafters of WebGPU spec. I can tell you that the card definitely will support WebGPU well, and WebGPU is not just for web. It will run on any device with a very small footprint runtime library and does not need a browser.
- pjmlp 3y ago
  WebGPU isn't polyglot as CUDA, so already out of question, let alone non existing tooling like NInsights.
  - WanderPanda 3y ago
    What do you mean by polyglot? Things like CUDA.jl?
    - pjmlp 3y ago
      Being able to write C, C++, Fortran, Haskell, Java, C#, F#, Futhark, Julia code to run on the GPU, or any other language with a compiler toolchain able to target PTX bytecode. And the related graphical tooling for debugging such workflows when stuff goes wrong on the GPU.
monocasa 3y ago
Not any time soon. CUDA is very Nvidia specific.
- deleted 3y ago
  [deleted]
- jms55 3y ago
  When people say CUDA is Nvidia specific, I don't think they realize _how_ specific. It's not just a matter of having the API implementation. CUDA does things like depend on specific scheduler behavior for GPU threads in order to guarantee forward progress, allowing more efficient single-pass computation routines. Or allowing CUDA kernels to launch additional kernels or allocate GPU memory from within the kernel, without host communication. Or checking the GPU architecture and performing specific micro-optimizations designed around that hardware's internal design. GPU's are nothing like CPU's, in that the x86 architecture is fairly stable. There's not a _ton_ of difference between different CPUs. Sure, performance characteristics and cache sizes might be different, but generally they have the same instruction set. Each GPU generation is basically a completely different architecture, much less between GPU companies. That's (partly) why Vulkan is so complicated - it tries to support the lowest spec 2013 mobile GPU, and the highest spec 2023 desktop GPU in one single API.
  - nologic01 3y ago
    This is the result of GPU computationally intensive tasks not being as simply defined as the CPU workloads. It was always thus for the various accelerators and HPC designs. They must make design choices that favor a relatively narrow range of computational patterns whereas the CPU will happily chew through a wider range of instruction sequences with the same performance. The corresponding GPU software complexity is something that feels not quite appreciated. Yet if we are going to see the massive adoption of GPU's that the market projects, with multiple providers of hardware, we'll need some serious advances of the software side.
    - pjmlp 3y ago
      I think we need more stuff like Futhark or StarLisp. Being able to describe GPU computations in a declarative way, with a driver similar to an SQL query optimizer, distributing the load.
- Grum9 3y ago
  CUDA on AMD attempt https://community.amd.com/t5/rocm/available-now-new-hip-sdk-helps-democratize-gpu-computing/ba-p/621029/jump-to/first-unread-message https://community.amd.com/t5/rocm/available-now-new-hip-sdk-...
pjmlp 3y ago
Any CUDA alternative has to be on the same level supporting polyglot programming, IDE and graphical debugging, libraries. Alternatives based on C or shading languages, have already lost before the game has even started.
- kristianp 3y ago
  What's so important about polyglot support? Isn't CUDA code similar to C?
  - pjmlp 3y ago
    Yes, if you are stuck on a version older than CUDA 3.0. CUDA supports C, C++20 (minus modules currently), Fortran and has Python bindings, additional plenty of alternative toolchains for PTX bytecode exist, some of which with NVidia's official backing, for Haskell, .NET, Java, Futhark, Julia, and whoever wants to target PTX.
- imtringued 3y ago
  You keep posting this but the problem with AMD isn't some luxury problem like that. The problem is that the driver crashes.
  - pjmlp 3y ago
    Drivers not crashing is basic stuff, alone fixing them isn't going to change anything, unless ROCm can actually provide similar tooling.
    - catchnear4321 3y ago
      “drivers not crashing is basic stuff” sounds like an argument for not bothering with anything that would rely on the cards, like tooling, because what’s the point if the drivers are just going to crash. it would be like saying it isn’t enough for your car to run to win a race. while that may be true, if you enter a non-running car into a race, your car isn’t the biggest hindrance to your success.
brucethemoose2 3y ago
OpenVINO is super easy in PyTorch and a few other places. And there are other specific efforts like Embree (which now supports Intel GPU rendering), and Vukan ML efforts like Apache TVM and MLIR-based projects.

snnn 3y ago

Bet on WebGPU. Intel and Google are the two main drafters of WebGPU spec. I can tell you that the card definitely will support WebGPU well, and WebGPU is not just for web. It will run on any device with a very small footprint runtime library and does not need a browser.

pjmlp 3y ago
WebGPU isn't polyglot as CUDA, so already out of question, let alone non existing tooling like NInsights.
- WanderPanda 3y ago
  What do you mean by polyglot? Things like CUDA.jl?
  - pjmlp 3y ago
    Being able to write C, C++, Fortran, Haskell, Java, C#, F#, Futhark, Julia code to run on the GPU, or any other language with a compiler toolchain able to target PTX bytecode. And the related graphical tooling for debugging such workflows when stuff goes wrong on the GPU.

monocasa 3y ago

Not any time soon. CUDA is very Nvidia specific.

deleted 3y ago
[deleted]
jms55 3y ago
When people say CUDA is Nvidia specific, I don't think they realize _how_ specific. It's not just a matter of having the API implementation. CUDA does things like depend on specific scheduler behavior for GPU threads in order to guarantee forward progress, allowing more efficient single-pass computation routines. Or allowing CUDA kernels to launch additional kernels or allocate GPU memory from within the kernel, without host communication. Or checking the GPU architecture and performing specific micro-optimizations designed around that hardware's internal design. GPU's are nothing like CPU's, in that the x86 architecture is fairly stable. There's not a _ton_ of difference between different CPUs. Sure, performance characteristics and cache sizes might be different, but generally they have the same instruction set. Each GPU generation is basically a completely different architecture, much less between GPU companies. That's (partly) why Vulkan is so complicated - it tries to support the lowest spec 2013 mobile GPU, and the highest spec 2023 desktop GPU in one single API.
- nologic01 3y ago
  This is the result of GPU computationally intensive tasks not being as simply defined as the CPU workloads. It was always thus for the various accelerators and HPC designs. They must make design choices that favor a relatively narrow range of computational patterns whereas the CPU will happily chew through a wider range of instruction sequences with the same performance. The corresponding GPU software complexity is something that feels not quite appreciated. Yet if we are going to see the massive adoption of GPU's that the market projects, with multiple providers of hardware, we'll need some serious advances of the software side.
  - pjmlp 3y ago
    I think we need more stuff like Futhark or StarLisp. Being able to describe GPU computations in a declarative way, with a driver similar to an SQL query optimizer, distributing the load.
Grum9 3y ago
CUDA on AMD attempt https://community.amd.com/t5/rocm/available-now-new-hip-sdk-helps-democratize-gpu-computing/ba-p/621029/jump-to/first-unread-message https://community.amd.com/t5/rocm/available-now-new-hip-sdk-...

pjmlp 3y ago

Any CUDA alternative has to be on the same level supporting polyglot programming, IDE and graphical debugging, libraries. Alternatives based on C or shading languages, have already lost before the game has even started.

kristianp 3y ago
What's so important about polyglot support? Isn't CUDA code similar to C?
- pjmlp 3y ago
  Yes, if you are stuck on a version older than CUDA 3.0. CUDA supports C, C++20 (minus modules currently), Fortran and has Python bindings, additional plenty of alternative toolchains for PTX bytecode exist, some of which with NVidia's official backing, for Haskell, .NET, Java, Futhark, Julia, and whoever wants to target PTX.
imtringued 3y ago
You keep posting this but the problem with AMD isn't some luxury problem like that. The problem is that the driver crashes.
- pjmlp 3y ago
  Drivers not crashing is basic stuff, alone fixing them isn't going to change anything, unless ROCm can actually provide similar tooling.
  - catchnear4321 3y ago
    “drivers not crashing is basic stuff” sounds like an argument for not bothering with anything that would rely on the cards, like tooling, because what’s the point if the drivers are just going to crash. it would be like saying it isn’t enough for your car to run to win a race. while that may be true, if you enter a non-running car into a race, your car isn’t the biggest hindrance to your success.

brucethemoose2 3y ago

OpenVINO is super easy in PyTorch and a few other places. And there are other specific efforts like Embree (which now supports Intel GPU rendering), and Vukan ML efforts like Apache TVM and MLIR-based projects.