7 ms·

>So today Khronos is doing something for which I’m not sure there’s any parallel for in the computing industry – and certainly, there’s never been anything like

>So today Khronos is doing something for which I’m not sure there’s any parallel for in the computing industry – and certainly, there’s never been anything like it in the GPU computing ecosystem: the framework is taking a large step backwards. Looking to reset the ecosystem, The people who picked up CUDA can now easily compile it to target NVIDA and AMD GPUs, as well as x86-64 CPUs, using a pletora of compilers (nvcc, hip's framework, PGI compiler for CUDA->x86, Intel's CUDA compiler, ...). Each upgrade to the CUDA platform has been backward compatible, and improved performance by quite a bit. Those who went for OpenCL lost Apple's platform support, have so-so AMD and Intel CPU support, and not so great AMD and NVIDIA GPU support, and now are getting an "ecosystem reset". I find this quite ironic TBH. The main advantage of OpenCL over CUDA on paper was supposed to be the vendor portability, but CUDA is delivering better on that "feature" than OpenCL. And now they land an ecosystem reset. I hope that people can easily migrate their OpenCL 2.x code to OpenCL 3.0. If doing that requires a rewrite, they might as well rewrite their code in CUDA..

skohan 7y ago
> CUDA can now easily compile it to target NVIDA and AMD GPUs, as well as x86-64 CPUs Are these solutions "legal"? I.e. is there a risk NVIDIA can crack down on tools like this from an IP standpoint if they want to at some point in the future?
- fluffything 7y ago
  IANAL, but: - NVIDIA themselves ship multiple CUDA to multi-core x86-64 compilers for different programming languages CUDA C/C++/Fortran - AMD has been shiping hip for a while, advertising it as production ready, without any lawsuits (AFAIK). hip is also being used in production. - LLVM and clang have had open-source CUDA (and PTX) support for years, and nvidia have not enforced any kind of patents there.
  - pjmlp 6y ago
    That clang/LLVM support was contributed by NVidia.
    - fluffything 6y ago
      I think it was contributed by Google.
      - rrss 6y ago
        The original version of the NVPTX backend was contributed by nvidia, but iiuc they abandoned it afterwards and I think all the work on it since then has been by Google.
  - adev_ 6y ago
    > AMD has been shiping hip for a while, advertising it as production ready, without any lawsuits (AFAIK). hip is also being used in production. Hip is not Cuda and even remove most mention if cuda from its API. It just mimic cuda API to try to provide compatibility. And the result is that almost nobody use it today.
    - skohan 6y ago
      How do people do GPGPU on AMD today? Because I thought AMD has a huge cloud GPU market share for things like ML
      - adev_ 6y ago
        > How do people do GPGPU on AMD today? Because I thought AMD has a huge cloud GPU market share for things like ML ROCm and HIP [^1] or OpenCL. OpenCL is the hated solution of GPGPU. It is low level, incomplete and definitively not with a mature ecosystem of tools like Cuda. But it is currently the only one existing if you want your program to run on something else than Nvidia GPU ( Meaning AMD, Intel, phone GPUs or FPGA accelerators). [^1] https://github.com/ROCmSoftwarePlatform/ https://github.com/ROCmSoftwarePlatform/
        pjmlp 6y ago
        Android phone GPUs only do Renderscript, OpenCL is not officially supported on Android.
      - rrss 6y ago
        AMD has a tiny market share in the cloud GPU market. AWS, Google Cloud, and Azure all have compute instances with NVIDIA GPUs, and I don't think any with AMD GPUs (IIRC Azure maybe has windows instances with AMD GPUs for graphics). According to https://www.techquila.co.in/nvidia-cloud-ai-gpu-share/ https://www.techquila.co.in/nvidia-cloud-ai-gpu-share/, AMD is at ~1%.
- Bayart 6y ago
  Can you patent a formal language or API calls ? That's a real question. I'm sure NVIDIA IP includes the CUDA trademark, their own compilers, SDKs and so on... but the language itself ?
  - raphlinus 6y ago
    It's a gray area and currently being litigated. If Oracle prevails, then NVidia absolutely would be in a position to shut down compatible implementations of CUDA, or dictate the terms under which they might or might not be able to continue to exist. Using open standards avoids this risk, though some companies are unwilling to work with Khronos, citing (not publicly detailed) legal concerns. [1]: https://en.wikipedia.org/wiki/Google_v._Oracle_America https://en.wikipedia.org/wiki/Google_v._Oracle_America
sorenjan 7y ago
I just want to be able to install a program or library, like Tensorflow, and have it work on my GPU whether it's from Nvidia or AMD without having to first find and install the right compiler. How do I run a Python program that imports Tensorflow on my AMD GPU today?
- fluffything 7y ago
  That's something you need to ask to your package manager. Making sure that all dependencies of a package (e.g. drivers, compilers, etc.) are installed is the package manager's job. If you are installing TF from source, then it becomes your job. TF docs should help you here, and TF's build-system should be able to guide you through what you need to install.
  - adev_ 6y ago
    > That's something you need to ask to your package manager. Making sure that all dependencies of a package (e.g. drivers, compilers, etc.) are installed is the package manager's job. No that's bullshit and it shows you never have done it. Tensorflow is CUDA only and does not run over AMD GPU. Because CUDA is proprietary and has a monopoly on GPGPU for now. It is such a big deal that AMD had to fork tensorflow and try to make their own version it in order to make it "partially" run over HIP. This naturally makes it behind upstream, buggy and partially updated [^1] CUDA is not portable and will never be. The sad truth is that, due to CUDA, a big majority of GPGPU programs are now vendor lock-in on Nvidia hardware. That is unacceptable, but it is like that. [^1]: https://github.com/ROCmSoftwarePlatform/tensorflow-upstream https://github.com/ROCmSoftwarePlatform/tensorflow-upstream
    - fluffything 6y ago
      > It is such a big deal that AMD had to fork tensorflow and try to make their own version it in order to make it "partially" run over HIP. That's because HIP requires changes to the program source code, but AMD has managed to upstream those patches to other projects. If they haven't done that for Tensorflow, that's kind of their problem. Clang can compile CUDA code, and it has an AMDGpu backend. AMD could compile that CUDA code to SPIRV if they wanted. Intel and PGI do compile CUDA to x86-64 using LLVM without problems.
pjmlp 7y ago
It is like Khronos keeps missing the point why developers jump into more productive eco-systems. Lets see how they manage Vulkan, which keeps getting extensions that NVidia and AMD first develop in DirectX anyway, like Raytracing and mesh shaders. Google and Samsung had to adapt Microsoft's HLSL compiler to target SPIR-V, because plenty of game developers weren't willing to write GLSL, or to port their shaders. For those that don't know shading languages HLSL is closer to C++ in language features, also similar to what PS4 uses, while GLSL is pretty much still stuck in being a C derived shading language.
- z3phyr 7y ago
  Well, now that SPIRV is upon us, we can write a custom shading language which targets SPIRV
  - pjmlp 7y ago
    You can, the point is who will care to use it, and which graphical tools across the industry will care that it exists. Hence why Google and Samsung spent about one year doing it for HLSL, and it still isn't 100% compliant with DXIC.
- fluffything 7y ago
  > It is like Khronos keeps missing the point why developers jump into more productive eco-systems. Implementation and standardization should go hand-in-hand, to make sure that you don't standardize something that's unimplementable. If you can't find companies to implement the standard while it's being developed, then maybe nobody cares?
  - pjmlp 7y ago
    Indeed, just take a look at the slide about which OpenCL version each company is supporting.
- ksec 7y ago
  >It is like Khronos keeps missing the point why developers jump into more productive eco-systems. They have a history of doing it for the past 20 years. So when the article said "Taking some hard earned (and hard learned) lessons to heart,". I laughed. I am not even sure they learned anything. And it is the same reason I dont understand why everyone is chanting Vulkan. It is the same /similar story with OpenGL vs 3DFx Glide.
fizixer 7y ago
Honestly the whole thing makes me sick. And now effin Khronos comes up with their 15th grand new idea. Every party in this is a PoS in once sense or another.
simias 7y ago
You'd think Khronos would learn from their (many) past mistakes with OpenGL but apparently they seem to miraculously have found some leftover foot left to shoot. I guess hindsight is 20/20 and all that but it's pretty crazy that they seem to miss the mark so reliably over the years.
- pjmlp 7y ago
  Nah, just check the Vulkan extensions registry.
  - slezyr 6y ago
    Issues 1) Are there issues? RESOLVED: Yes. https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_NV_ray_tracing.html https://www.khronos.org/registry/vulkan/specs/1.2-extensions...
m12k 7y ago
As an outsider reading this, it sounds like OpenGL vs DirectX all over again - an open API is good, but a good API is better.
- phiresky 7y ago
  Alternative take: An Open API is good, but Microsoft and Nvidia are too powerful to allow it http://blog.wolfire.com/2010/01/Why-you-should-use-OpenGL-and-not-DirectX http://blog.wolfire.com/2010/01/Why-you-should-use-OpenGL-an...
  - m12k 7y ago
    I'm not much of a graphics programmer, so take this with a grain of salt, but from talking to people who are, the impression I've gotten is that OpenGL is stuck in this weird middle state where it still has two ways of doing things, a simple and easy, but inefficient way where you specify everything from scratch each frame, and one where you use more persistent state like vertex buffer objects to minimize copying and speed things up, but the latter has a cumbersome bolted-on-top API. By comparison DirectX only supports the latter, but was from the start designed with a nicer API to deal with this persistent state, so engine programmers tend to prefer it. And rumor has it that it was the CAD companies that were preventing a modernization of the OpenGL API to properly compete with the ease-of-use of DirectX. So this wasn't just a fight of open vs proprietary, it was also a fight of design-by-committee vs single coherent architectural vision.
    - glowcoil 6y ago
      This isn't really accurate; the immediate-mode approach you're talking about was deprecated in OpenGL 3.0 and was never supported in OpenGL ES, and in fact, Direct3D started out with an immediate-mode API just like OpenGL before moving to retained vertex buffers.
    - skohan 6y ago
      As someone who spent quite a bit of time with OpenGL, this criticism never really scanned with me. Yes there were multiple ways to do things, but it was fairly easy to understand what the modern parts of the API were and just use that subset. If you run through any modern OpenGL tutorial it will explain this very concisely, and I never had an issue being unsure which parts of the API to use. That said, Vulkan really does solve all of the real pain-points which existed with OpenGL. It’s very well documented and nice to use, and I really don’t understand why anyone would choose a proprietary api over it.
snicker7 6y ago
With regards to HIP, here is a quote from AMD's FAQ: > Most developers will port their code from CUDA to HIP and then maintain the HIP version. HIP code provides the same performance as native CUDA code, plus the benefits of running on AMD platforms. E.g. AMD wants developers to develop primarily in HIP. The CUDA->HIP transpilation tools is to help developers make the switch.

skohan 7y ago

> CUDA can now easily compile it to target NVIDA and AMD GPUs, as well as x86-64 CPUs Are these solutions "legal"? I.e. is there a risk NVIDIA can crack down on tools like this from an IP standpoint if they want to at some point in the future?

fluffything 7y ago
IANAL, but: - NVIDIA themselves ship multiple CUDA to multi-core x86-64 compilers for different programming languages CUDA C/C++/Fortran - AMD has been shiping hip for a while, advertising it as production ready, without any lawsuits (AFAIK). hip is also being used in production. - LLVM and clang have had open-source CUDA (and PTX) support for years, and nvidia have not enforced any kind of patents there.
- pjmlp 6y ago
  That clang/LLVM support was contributed by NVidia.
  - fluffything 6y ago
    I think it was contributed by Google.
    - rrss 6y ago
      The original version of the NVPTX backend was contributed by nvidia, but iiuc they abandoned it afterwards and I think all the work on it since then has been by Google.
- adev_ 6y ago
  > AMD has been shiping hip for a while, advertising it as production ready, without any lawsuits (AFAIK). hip is also being used in production. Hip is not Cuda and even remove most mention if cuda from its API. It just mimic cuda API to try to provide compatibility. And the result is that almost nobody use it today.
  - skohan 6y ago
    How do people do GPGPU on AMD today? Because I thought AMD has a huge cloud GPU market share for things like ML
    - adev_ 6y ago
      > How do people do GPGPU on AMD today? Because I thought AMD has a huge cloud GPU market share for things like ML ROCm and HIP [^1] or OpenCL. OpenCL is the hated solution of GPGPU. It is low level, incomplete and definitively not with a mature ecosystem of tools like Cuda. But it is currently the only one existing if you want your program to run on something else than Nvidia GPU ( Meaning AMD, Intel, phone GPUs or FPGA accelerators). [^1] https://github.com/ROCmSoftwarePlatform/ https://github.com/ROCmSoftwarePlatform/
      - pjmlp 6y ago
        Android phone GPUs only do Renderscript, OpenCL is not officially supported on Android.
    - rrss 6y ago
      AMD has a tiny market share in the cloud GPU market. AWS, Google Cloud, and Azure all have compute instances with NVIDIA GPUs, and I don't think any with AMD GPUs (IIRC Azure maybe has windows instances with AMD GPUs for graphics). According to https://www.techquila.co.in/nvidia-cloud-ai-gpu-share/ https://www.techquila.co.in/nvidia-cloud-ai-gpu-share/, AMD is at ~1%.
Bayart 6y ago
Can you patent a formal language or API calls ? That's a real question. I'm sure NVIDIA IP includes the CUDA trademark, their own compilers, SDKs and so on... but the language itself ?
- raphlinus 6y ago
  It's a gray area and currently being litigated. If Oracle prevails, then NVidia absolutely would be in a position to shut down compatible implementations of CUDA, or dictate the terms under which they might or might not be able to continue to exist. Using open standards avoids this risk, though some companies are unwilling to work with Khronos, citing (not publicly detailed) legal concerns. [1]: https://en.wikipedia.org/wiki/Google_v._Oracle_America https://en.wikipedia.org/wiki/Google_v._Oracle_America

sorenjan 7y ago

I just want to be able to install a program or library, like Tensorflow, and have it work on my GPU whether it's from Nvidia or AMD without having to first find and install the right compiler. How do I run a Python program that imports Tensorflow on my AMD GPU today?

fluffything 7y ago
That's something you need to ask to your package manager. Making sure that all dependencies of a package (e.g. drivers, compilers, etc.) are installed is the package manager's job. If you are installing TF from source, then it becomes your job. TF docs should help you here, and TF's build-system should be able to guide you through what you need to install.
- adev_ 6y ago
  > That's something you need to ask to your package manager. Making sure that all dependencies of a package (e.g. drivers, compilers, etc.) are installed is the package manager's job. No that's bullshit and it shows you never have done it. Tensorflow is CUDA only and does not run over AMD GPU. Because CUDA is proprietary and has a monopoly on GPGPU for now. It is such a big deal that AMD had to fork tensorflow and try to make their own version it in order to make it "partially" run over HIP. This naturally makes it behind upstream, buggy and partially updated [^1] CUDA is not portable and will never be. The sad truth is that, due to CUDA, a big majority of GPGPU programs are now vendor lock-in on Nvidia hardware. That is unacceptable, but it is like that. [^1]: https://github.com/ROCmSoftwarePlatform/tensorflow-upstream https://github.com/ROCmSoftwarePlatform/tensorflow-upstream
  - fluffything 6y ago
    > It is such a big deal that AMD had to fork tensorflow and try to make their own version it in order to make it "partially" run over HIP. That's because HIP requires changes to the program source code, but AMD has managed to upstream those patches to other projects. If they haven't done that for Tensorflow, that's kind of their problem. Clang can compile CUDA code, and it has an AMDGpu backend. AMD could compile that CUDA code to SPIRV if they wanted. Intel and PGI do compile CUDA to x86-64 using LLVM without problems.

pjmlp 7y ago

It is like Khronos keeps missing the point why developers jump into more productive eco-systems. Lets see how they manage Vulkan, which keeps getting extensions that NVidia and AMD first develop in DirectX anyway, like Raytracing and mesh shaders. Google and Samsung had to adapt Microsoft's HLSL compiler to target SPIR-V, because plenty of game developers weren't willing to write GLSL, or to port their shaders. For those that don't know shading languages HLSL is closer to C++ in language features, also similar to what PS4 uses, while GLSL is pretty much still stuck in being a C derived shading language.

z3phyr 7y ago
Well, now that SPIRV is upon us, we can write a custom shading language which targets SPIRV
- pjmlp 7y ago
  You can, the point is who will care to use it, and which graphical tools across the industry will care that it exists. Hence why Google and Samsung spent about one year doing it for HLSL, and it still isn't 100% compliant with DXIC.
fluffything 7y ago
> It is like Khronos keeps missing the point why developers jump into more productive eco-systems. Implementation and standardization should go hand-in-hand, to make sure that you don't standardize something that's unimplementable. If you can't find companies to implement the standard while it's being developed, then maybe nobody cares?
- pjmlp 7y ago
  Indeed, just take a look at the slide about which OpenCL version each company is supporting.
ksec 7y ago
>It is like Khronos keeps missing the point why developers jump into more productive eco-systems. They have a history of doing it for the past 20 years. So when the article said "Taking some hard earned (and hard learned) lessons to heart,". I laughed. I am not even sure they learned anything. And it is the same reason I dont understand why everyone is chanting Vulkan. It is the same /similar story with OpenGL vs 3DFx Glide.

fizixer 7y ago

Honestly the whole thing makes me sick. And now effin Khronos comes up with their 15th grand new idea. Every party in this is a PoS in once sense or another.

simias 7y ago

You'd think Khronos would learn from their (many) past mistakes with OpenGL but apparently they seem to miraculously have found some leftover foot left to shoot. I guess hindsight is 20/20 and all that but it's pretty crazy that they seem to miss the mark so reliably over the years.

pjmlp 7y ago
Nah, just check the Vulkan extensions registry.
- slezyr 6y ago
  Issues 1) Are there issues? RESOLVED: Yes. https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_NV_ray_tracing.html https://www.khronos.org/registry/vulkan/specs/1.2-extensions...

m12k 7y ago

As an outsider reading this, it sounds like OpenGL vs DirectX all over again - an open API is good, but a good API is better.

phiresky 7y ago
Alternative take: An Open API is good, but Microsoft and Nvidia are too powerful to allow it http://blog.wolfire.com/2010/01/Why-you-should-use-OpenGL-and-not-DirectX http://blog.wolfire.com/2010/01/Why-you-should-use-OpenGL-an...
- m12k 7y ago
  I'm not much of a graphics programmer, so take this with a grain of salt, but from talking to people who are, the impression I've gotten is that OpenGL is stuck in this weird middle state where it still has two ways of doing things, a simple and easy, but inefficient way where you specify everything from scratch each frame, and one where you use more persistent state like vertex buffer objects to minimize copying and speed things up, but the latter has a cumbersome bolted-on-top API. By comparison DirectX only supports the latter, but was from the start designed with a nicer API to deal with this persistent state, so engine programmers tend to prefer it. And rumor has it that it was the CAD companies that were preventing a modernization of the OpenGL API to properly compete with the ease-of-use of DirectX. So this wasn't just a fight of open vs proprietary, it was also a fight of design-by-committee vs single coherent architectural vision.
  - glowcoil 6y ago
    This isn't really accurate; the immediate-mode approach you're talking about was deprecated in OpenGL 3.0 and was never supported in OpenGL ES, and in fact, Direct3D started out with an immediate-mode API just like OpenGL before moving to retained vertex buffers.
  - skohan 6y ago
    As someone who spent quite a bit of time with OpenGL, this criticism never really scanned with me. Yes there were multiple ways to do things, but it was fairly easy to understand what the modern parts of the API were and just use that subset. If you run through any modern OpenGL tutorial it will explain this very concisely, and I never had an issue being unsure which parts of the API to use. That said, Vulkan really does solve all of the real pain-points which existed with OpenGL. It’s very well documented and nice to use, and I really don’t understand why anyone would choose a proprietary api over it.

snicker7 6y ago

With regards to HIP, here is a quote from AMD's FAQ: > Most developers will port their code from CUDA to HIP and then maintain the HIP version. HIP code provides the same performance as native CUDA code, plus the benefits of running on AMD platforms. E.g. AMD wants developers to develop primarily in HIP. The CUDA->HIP transpilation tools is to help developers make the switch.