6 ms·

> the consumer cards don't do compute? Typically, software developers only support a single GPGPU API, and that API is nVidia CUDA. Technically, consumer AMD

> the consumer cards don't do compute? Typically, software developers only support a single GPGPU API, and that API is nVidia CUDA. Technically, consumer AMD cards are awesome at compute. For example, UE5 renders triangle meshes with compute instead of graphics https://www.youtube.com/watch?v=TMorJX3Nj6U https://www.youtube.com/watch?v=TMorJX3Nj6U Moreover, AMD cards often outperform nVidia equivalents because nVidia prioritized ray tracing and DLSS over compute power and memory bandwidth. The issue is, no tech company is interested in adding D3D or Vulkan backend to AI libraries like PyTorch. nVidia is not doing that because they are happy with the status quo. Intel and AMD are not doing that because both hope to replace CUDA with their proprietary equivalents, instead of an open GPU API.

boppo1 3y ago
>Vulkan backend to AI libraries like PyTorch How many man hours is a project like this?
- Const-me 3y ago
  Couple times in the past I wanted to port open source ML models from CUDA/Python to a better technology stack. I have ported Whisper https://github.com/Const-me/Whisper/ https://github.com/Const-me/Whisper/ and Mistral https://github.com/Const-me/Cgml/ https://github.com/Const-me/Cgml/ to D3D11. I don’t remember how much time I spent, but given both were unpaid part-time hobby projects, probably under 160 hours / each. These software projects were great to validate the technology choices, but note I only did bare minimum to implement specific ML models. Implementing a complete PyTorch backend gonna involve dramatically more work. I can’t even estimate how much more because I’m not an expert in Python or these Python-based ML libraries.
  - versteegen 3y ago
    Wow, a very nice reimplementation. To go on a tangent, I note your custom 'BCML1' 5bit per weight compression codec and your optimised hand-coded AVX2 to encode it... was that really needed? Are the weights encoded on every startup? Why not do it once and save to disk?
    - Const-me 3y ago
      > Are the weights encoded on every startup? Not really, that code only runs while importing the PyTorch format. See readme for the frontend app: https://github.com/Const-me/Cgml/tree/master/Mistral/MistralChat https://github.com/Const-me/Cgml/tree/master/Mistral/Mistral... When loading the model from *.cgml, the model file already contains compressed tensors. That’s how that file is only 4.55 GB, versus 13.4 GB in the original model. > was that really needed? For desktops with many CPU cores, a simpler scalar version would probably work equally well. Still, low-end computers don’t always have many cores to use by these background encoding tasks. Also, CPU usage on laptops translates to battery drain.
  - spookie 3y ago
    This is great!
- deleted 3y ago
  [deleted]
GeekyBear 3y ago
> The issue is, no tech company is interested in adding D3D or Vulkan backend to AI libraries like PyTorch. AMD is interested in making PyTorch take advantage of their chips, and has done so. > we are delighted that the PyTorch 2.0 stable release includes support for AMD Instinct™ and Radeon™ GPUs that are supported by the ROCm™ software platform. When NVIDIA is charging 1000% profit margins for a GPU aimed at use as an AI accelerator, you can expect competitors to be willing to do what it takes to move into that market. https://www.tomshardware.com/news/nvidia-makes-1000-profit-on-h100-gpus-report https://www.tomshardware.com/news/nvidia-makes-1000-profit-o...
- MrBuddyCasino 3y ago
  You would think so, and yet AMD dropped the ball so hard on the software side of things, the resulting hole in the ground might as well be bottomless.
  - jorvi 3y ago
    Is that really just on AMD? All the companies that are now screaming hellfire because of Nvidia's market maker position are also the companies that gave Nvidia a warchest filled with billions of dollars. How is AMD supposed to compete when the whole market is funding their rival?
    - HideousKojima 3y ago
      The whole market funded Nvidia because AMD failed to provide a viable alternative for 10+ years
      - deleted 3y ago
        [deleted]
      - doikor 3y ago
        AMD was almost bankrupt 10 years ago. Basically AMD had to bet the whole company on their next CPU or GPU and their choice was the CPUs and put all their efforts into that and it payed off (Zen architecture). It has only been the last 4 or 5 years that AMD has had any real money to put into their GPU/AI accelerator sector and that seems to be developing quite well now (though they seem to be mostly interested in super computer/massive data center deployments for now)
zozbot234 3y ago
Vulkan Compute backends for numerical compute (as typified by both OpenCL and SYCL) are challenging, you can look at clspv https://github.com/google/clspv https://github.com/google/clspv project for the nitty gritty details. The lowest-effort path so far is most likely via some combination of Rocm/HIP (for hardware that AMD bothers to support themselves) and the Mesa project's RustiCL backend (for everything else).
- Const-me 3y ago
  > Vulkan Compute backends for numerical compute (as typified by both OpenCL and SYCL) are challenging Microsoft has an offline dxc.exe compiler which compiles HLSL to Spir-V. Also, DXVK has a JIT compiler which recompiles DXBC byte codes to Spir-V. Both technologies are old, stable and reliable, for example the DXVK’s JIT compiler is a critical software component of the SteamDeck console. > The lowest-effort path so far is most likely I agree that’s most likely to happen, but the outcome is horrible from consumer PoV. Mesa is Linux-only, Rust is too hard to use for vast majority of developers (myself included), AMD will never support older cards with ROCm, and we now have the third discrete GPU vendor, Intel.
  - my123 3y ago
    SPIR-V for OpenCL and for Vulkan are substantially different, with the translation between the two being quite non-trivial. (note that rusticl + zink does deal with it _partially_ to some extent nowadays) + Vulkan memory management doesn't expose unified address space primitives
    - Const-me 3y ago
      Why would you want OpenCL? Pretty sure D3D11 compute shaders gonna be adequate for a Torch backend, and they even work on Linux with Wine: https://github.com/Const-me/Whisper/issues/42 https://github.com/Const-me/Whisper/issues/42 Native Vulkan compute shaders would be even better. Why would you want unified address space? At least in my experience, it’s often too slow to be useful. DMA transfers (CopyResource in D3D11, copy command queue in D3D12, transfer queue in VK) are implemented by dedicated hardware inside GPUs, and are way more efficient.
einpoklum 3y ago
> Typically, software developers only support a single GPGPU API, and that API is nVidia CUDA. There is very little NVIDIA and CUDA cards off of X86_64 and maybe OpenPower architectures. So I disagree. Also, OpenCL, despite being kind of a "betrayed standard", enjoys quite a lot of popularity even on x86_64 (sometimes even with NVIDIA hardware) - even if it is not as popular there. > AMD cards often outperform nVidia equivalents Can you link to benchmarks or other analysis supporting this claim? This has not been my impression in recent years, though I don't routinely look at high-end AMD hardware. > because nVidia prioritized ray tracing and DLSS over compute power and memory bandwidth. Has it really?
- Const-me 3y ago
  > Can you link to benchmarks or other analysis supporting this claim? Current generation nVidia: https://en.wikipedia.org/wiki/GeForce_40_series#Desktop https://en.wikipedia.org/wiki/GeForce_40_series#Desktop Current generation AMD: https://en.wikipedia.org/wiki/Template:AMD_Radeon_RX_7000 https://en.wikipedia.org/wiki/Template:AMD_Radeon_RX_7000 The key performance characteristics are processing power TFlops, and memory bandwidth GB/s. nVidia 4080 which costs $1200: 43 TFlops FP16, 43 TFlops FP32, 0.672 TFlops FP64, 717 GB/s memory. AMD 7900 XTX which costs $1000: 93 TFlops FP16, 47 TFlops FP32, 1.5 TFlops FP64, 960 GB/s memory. Note that for applications which bottleneck on FP16 compute (many ML workloads) or FP64 compute (many traditional HPC workloads: numerical solvers, fluid dynamics, etc), the 7900 XTX even outperforms the 4090 which costs $1600.
  - ColonelPhantom 3y ago
    In ML workloads, usually the FP16 operations are matrix operations. On RDNA3, these execute at the same rate as normal shader/vector operations, but on Nvidia RTX cards there are Tensor cores which accelerate them. The Ada whitepaper lists 48.7 shader TFlops (not 43 because boost vs base clock), and 195 TFlops for FP16 Tensor with FP16 Accumulate. That's 4 times faster than regular, and almost double what the XTX lists! Ampere and newer also have native sparsity support which means that you can skip over the zeroes 'for free', which Nvidia uses to market double the TFlops, which is kind of misleading imo. But the 195 TFlops are even before sparsity is included! I'm not sure if the 93 TFlops (120 with boost clocks) on AMD are with FP16 or FP32 accumulation, as with FP32 accumulation the 4080 slows down significantly and gets much closer with 97.5 TFlops. Intel Xe-HPG (used in the Arc A cards) also offers very aggressive matrix acceleration via XMX, with 137.6 FP16 TFlops at base clock, vs. 17.2 FP32 TFlops.
jorvi 3y ago
> Intel and AMD are not doing that because both hope to replace CUDA with their proprietary equivalents, instead of an open GPU API Like AMD did with FreeSync (now VRR) and FSR3 (whatever open standard it will morph into)? I'll never understand the hate AMD gets from the open source community. Mind-boggling.
- 7speter 3y ago
  Isn’t Intel’s solution oneAPI, which is supposed to be open and have a sort of translation layer for CUDA?
- trelane 3y ago
  > I'll never understand the hate AMD gets from the open source community. Mind-boggling. A large part of that hate is that, for a very long time, they had the worst of both worlds: a fragile proprietary driver and subpar performance. Only recently have they fixed both. Indeed I only found out about their newfound openness a few years ago. Not everyone has gotten the memo.
spookie 3y ago
> They aim to replace CUDA with their proprietary equivalents Neither oneAPI, nor ROCm are proprietary?
- einpoklum 3y ago
  They're single-vendor-controlled (and vendor-specific implementation-wise). The question of whether the spec is proprietary is less significant.
  - hjabird 3y ago
    SYCL (closely related to oneAPI) isn't single-vendor-controlled. It's a Khronos open standard. If you take a look at the spec, you'll see contributions from various universities, Qualcomm, Huawei, Argonne, Altera, and AMD (Xilinx I think). Intel just adopted it (and bought Codeplay, the original contributor). oneAPI is a set of SYCL related standards. Originally that was Intel, but now that's owned by the UXL foundation, which is part of the Linux foundation.
pjmlp 3y ago
DirectML is part of DirectX 12.
0xDEADFED5 3y ago
> The issue is, no tech company is interested in adding D3D or Vulkan backend i'm using Apache TVM right now to run LLMs via Vulkan on Intel Arc cards. Intel has their own pytorch wheels here for their GPUs: https://developer.intel.com/ipex-whl-stable-xpu https://developer.intel.com/ipex-whl-stable-xpu
paulmd 3y ago
AMD's products have generally failed to catch traction because their implementations are halfassed and buggy and incomplete (despite promising more features, these are often paper features or resume-driven development from now-departed developers). all of the same "developer B" stuff from openGL really applies to openCL as well. http://richg42.blogspot.com/2014/05/the-truth-on-opengl-driver-quality.html http://richg42.blogspot.com/2014/05/the-truth-on-opengl-driv... that's the reason blender kicked their openCL implemention to the curb after years of futzing with it. it wasn't portable because AMD's openCL was so broken they had to have their own special codepath anyway, and it didn't perform well and wasn't worth the effort. AMD has left a trail of abandoned code and disappointed developers in their wake. These two repos are the same thing for AMD's ecosystem and NVIDIA's ecosystem, how do you think the support story compares? https://github.com/HSA-Libraries/Bolt https://github.com/HSA-Libraries/Bolt https://github.com/NVIDIA/thrust https://github.com/NVIDIA/thrust in the last few years they have (once again) dumped everything and started over, ROCm supported essentially no consumer cards and rotated support rapidly even in the CDNA world. It offers no binary compatibility support story, it has to be compiled for specific chips within a generation, not even just "RDNA3" but "Navi 31 specifically". Etc etc. And nobody with consumer cards could access it until like, six months ago, and that still is only on windows, consumer cards are not even supported on linux (!). That's fine for non-commercial and HPC, but it makes it difficult to imagine ever shipping commercial products to end-users on it. I guess you... ship as source, and your users have to figure out ROCm and compile it in-situ? Or ship a binary for every single GPU to ever release (including NVIDIA, Intel, ....) https://geohot.github.io/blog/jekyll/update/2023/06/07/a-dive-into-amds-drivers.html https://geohot.github.io/blog/jekyll/update/2023/06/07/a-div... This is on top of the actual problems that still remain, as geohot found out. Installing ROCm is a several-hour process that will involve debugging the platform just to get it to install, and then you will probably find that the actual code demos segfault when you run them. AMD's development processes are not really open, and actual development is silo'd inside the company with quarterly code dumps outside. The current code is not guaranteed to run on the actual driver itself, they do not test it even in the supported configurations, because all the development is happening on internal builds etc, not release code. oh and AMD themselves isn't using the open driver of course... they only test and validate on AMDGPU-PRO. it hasn't got traction because it's a low-quality product and nobody can even access it and run it anyway, let alone ship commercial products on it. and now everyone regrets it because it's turned into a money fountain.

boppo1 3y ago

>Vulkan backend to AI libraries like PyTorch How many man hours is a project like this?

Const-me 3y ago
Couple times in the past I wanted to port open source ML models from CUDA/Python to a better technology stack. I have ported Whisper https://github.com/Const-me/Whisper/ https://github.com/Const-me/Whisper/ and Mistral https://github.com/Const-me/Cgml/ https://github.com/Const-me/Cgml/ to D3D11. I don’t remember how much time I spent, but given both were unpaid part-time hobby projects, probably under 160 hours / each. These software projects were great to validate the technology choices, but note I only did bare minimum to implement specific ML models. Implementing a complete PyTorch backend gonna involve dramatically more work. I can’t even estimate how much more because I’m not an expert in Python or these Python-based ML libraries.
- versteegen 3y ago
  Wow, a very nice reimplementation. To go on a tangent, I note your custom 'BCML1' 5bit per weight compression codec and your optimised hand-coded AVX2 to encode it... was that really needed? Are the weights encoded on every startup? Why not do it once and save to disk?
  - Const-me 3y ago
    > Are the weights encoded on every startup? Not really, that code only runs while importing the PyTorch format. See readme for the frontend app: https://github.com/Const-me/Cgml/tree/master/Mistral/MistralChat https://github.com/Const-me/Cgml/tree/master/Mistral/Mistral... When loading the model from *.cgml, the model file already contains compressed tensors. That’s how that file is only 4.55 GB, versus 13.4 GB in the original model. > was that really needed? For desktops with many CPU cores, a simpler scalar version would probably work equally well. Still, low-end computers don’t always have many cores to use by these background encoding tasks. Also, CPU usage on laptops translates to battery drain.
- spookie 3y ago
  This is great!
deleted 3y ago
[deleted]

GeekyBear 3y ago

> The issue is, no tech company is interested in adding D3D or Vulkan backend to AI libraries like PyTorch. AMD is interested in making PyTorch take advantage of their chips, and has done so. > we are delighted that the PyTorch 2.0 stable release includes support for AMD Instinct™ and Radeon™ GPUs that are supported by the ROCm™ software platform. When NVIDIA is charging 1000% profit margins for a GPU aimed at use as an AI accelerator, you can expect competitors to be willing to do what it takes to move into that market. https://www.tomshardware.com/news/nvidia-makes-1000-profit-on-h100-gpus-report https://www.tomshardware.com/news/nvidia-makes-1000-profit-o...

MrBuddyCasino 3y ago
You would think so, and yet AMD dropped the ball so hard on the software side of things, the resulting hole in the ground might as well be bottomless.
- jorvi 3y ago
  Is that really just on AMD? All the companies that are now screaming hellfire because of Nvidia's market maker position are also the companies that gave Nvidia a warchest filled with billions of dollars. How is AMD supposed to compete when the whole market is funding their rival?
  - HideousKojima 3y ago
    The whole market funded Nvidia because AMD failed to provide a viable alternative for 10+ years
    - deleted 3y ago
      [deleted]
    - doikor 3y ago
      AMD was almost bankrupt 10 years ago. Basically AMD had to bet the whole company on their next CPU or GPU and their choice was the CPUs and put all their efforts into that and it payed off (Zen architecture). It has only been the last 4 or 5 years that AMD has had any real money to put into their GPU/AI accelerator sector and that seems to be developing quite well now (though they seem to be mostly interested in super computer/massive data center deployments for now)

zozbot234 3y ago

Vulkan Compute backends for numerical compute (as typified by both OpenCL and SYCL) are challenging, you can look at clspv https://github.com/google/clspv https://github.com/google/clspv project for the nitty gritty details. The lowest-effort path so far is most likely via some combination of Rocm/HIP (for hardware that AMD bothers to support themselves) and the Mesa project's RustiCL backend (for everything else).

Const-me 3y ago
> Vulkan Compute backends for numerical compute (as typified by both OpenCL and SYCL) are challenging Microsoft has an offline dxc.exe compiler which compiles HLSL to Spir-V. Also, DXVK has a JIT compiler which recompiles DXBC byte codes to Spir-V. Both technologies are old, stable and reliable, for example the DXVK’s JIT compiler is a critical software component of the SteamDeck console. > The lowest-effort path so far is most likely I agree that’s most likely to happen, but the outcome is horrible from consumer PoV. Mesa is Linux-only, Rust is too hard to use for vast majority of developers (myself included), AMD will never support older cards with ROCm, and we now have the third discrete GPU vendor, Intel.
- my123 3y ago
  SPIR-V for OpenCL and for Vulkan are substantially different, with the translation between the two being quite non-trivial. (note that rusticl + zink does deal with it _partially_ to some extent nowadays) + Vulkan memory management doesn't expose unified address space primitives
  - Const-me 3y ago
    Why would you want OpenCL? Pretty sure D3D11 compute shaders gonna be adequate for a Torch backend, and they even work on Linux with Wine: https://github.com/Const-me/Whisper/issues/42 https://github.com/Const-me/Whisper/issues/42 Native Vulkan compute shaders would be even better. Why would you want unified address space? At least in my experience, it’s often too slow to be useful. DMA transfers (CopyResource in D3D11, copy command queue in D3D12, transfer queue in VK) are implemented by dedicated hardware inside GPUs, and are way more efficient.

einpoklum 3y ago

> Typically, software developers only support a single GPGPU API, and that API is nVidia CUDA. There is very little NVIDIA and CUDA cards off of X86_64 and maybe OpenPower architectures. So I disagree. Also, OpenCL, despite being kind of a "betrayed standard", enjoys quite a lot of popularity even on x86_64 (sometimes even with NVIDIA hardware) - even if it is not as popular there. > AMD cards often outperform nVidia equivalents Can you link to benchmarks or other analysis supporting this claim? This has not been my impression in recent years, though I don't routinely look at high-end AMD hardware. > because nVidia prioritized ray tracing and DLSS over compute power and memory bandwidth. Has it really?

Const-me 3y ago
> Can you link to benchmarks or other analysis supporting this claim? Current generation nVidia: https://en.wikipedia.org/wiki/GeForce_40_series#Desktop https://en.wikipedia.org/wiki/GeForce_40_series#Desktop Current generation AMD: https://en.wikipedia.org/wiki/Template:AMD_Radeon_RX_7000 https://en.wikipedia.org/wiki/Template:AMD_Radeon_RX_7000 The key performance characteristics are processing power TFlops, and memory bandwidth GB/s. nVidia 4080 which costs $1200: 43 TFlops FP16, 43 TFlops FP32, 0.672 TFlops FP64, 717 GB/s memory. AMD 7900 XTX which costs $1000: 93 TFlops FP16, 47 TFlops FP32, 1.5 TFlops FP64, 960 GB/s memory. Note that for applications which bottleneck on FP16 compute (many ML workloads) or FP64 compute (many traditional HPC workloads: numerical solvers, fluid dynamics, etc), the 7900 XTX even outperforms the 4090 which costs $1600.
- ColonelPhantom 3y ago
  In ML workloads, usually the FP16 operations are matrix operations. On RDNA3, these execute at the same rate as normal shader/vector operations, but on Nvidia RTX cards there are Tensor cores which accelerate them. The Ada whitepaper lists 48.7 shader TFlops (not 43 because boost vs base clock), and 195 TFlops for FP16 Tensor with FP16 Accumulate. That's 4 times faster than regular, and almost double what the XTX lists! Ampere and newer also have native sparsity support which means that you can skip over the zeroes 'for free', which Nvidia uses to market double the TFlops, which is kind of misleading imo. But the 195 TFlops are even before sparsity is included! I'm not sure if the 93 TFlops (120 with boost clocks) on AMD are with FP16 or FP32 accumulation, as with FP32 accumulation the 4080 slows down significantly and gets much closer with 97.5 TFlops. Intel Xe-HPG (used in the Arc A cards) also offers very aggressive matrix acceleration via XMX, with 137.6 FP16 TFlops at base clock, vs. 17.2 FP32 TFlops.

jorvi 3y ago

> Intel and AMD are not doing that because both hope to replace CUDA with their proprietary equivalents, instead of an open GPU API Like AMD did with FreeSync (now VRR) and FSR3 (whatever open standard it will morph into)? I'll never understand the hate AMD gets from the open source community. Mind-boggling.

7speter 3y ago
Isn’t Intel’s solution oneAPI, which is supposed to be open and have a sort of translation layer for CUDA?
trelane 3y ago
> I'll never understand the hate AMD gets from the open source community. Mind-boggling. A large part of that hate is that, for a very long time, they had the worst of both worlds: a fragile proprietary driver and subpar performance. Only recently have they fixed both. Indeed I only found out about their newfound openness a few years ago. Not everyone has gotten the memo.

spookie 3y ago

> They aim to replace CUDA with their proprietary equivalents Neither oneAPI, nor ROCm are proprietary?

einpoklum 3y ago
They're single-vendor-controlled (and vendor-specific implementation-wise). The question of whether the spec is proprietary is less significant.
- hjabird 3y ago
  SYCL (closely related to oneAPI) isn't single-vendor-controlled. It's a Khronos open standard. If you take a look at the spec, you'll see contributions from various universities, Qualcomm, Huawei, Argonne, Altera, and AMD (Xilinx I think). Intel just adopted it (and bought Codeplay, the original contributor). oneAPI is a set of SYCL related standards. Originally that was Intel, but now that's owned by the UXL foundation, which is part of the Linux foundation.

pjmlp 3y ago

DirectML is part of DirectX 12.

0xDEADFED5 3y ago

> The issue is, no tech company is interested in adding D3D or Vulkan backend i'm using Apache TVM right now to run LLMs via Vulkan on Intel Arc cards. Intel has their own pytorch wheels here for their GPUs: https://developer.intel.com/ipex-whl-stable-xpu https://developer.intel.com/ipex-whl-stable-xpu

paulmd 3y ago

AMD's products have generally failed to catch traction because their implementations are halfassed and buggy and incomplete (despite promising more features, these are often paper features or resume-driven development from now-departed developers). all of the same "developer B" stuff from openGL really applies to openCL as well. http://richg42.blogspot.com/2014/05/the-truth-on-opengl-driver-quality.html http://richg42.blogspot.com/2014/05/the-truth-on-opengl-driv... that's the reason blender kicked their openCL implemention to the curb after years of futzing with it. it wasn't portable because AMD's openCL was so broken they had to have their own special codepath anyway, and it didn't perform well and wasn't worth the effort. AMD has left a trail of abandoned code and disappointed developers in their wake. These two repos are the same thing for AMD's ecosystem and NVIDIA's ecosystem, how do you think the support story compares? https://github.com/HSA-Libraries/Bolt https://github.com/HSA-Libraries/Bolt https://github.com/NVIDIA/thrust https://github.com/NVIDIA/thrust in the last few years they have (once again) dumped everything and started over, ROCm supported essentially no consumer cards and rotated support rapidly even in the CDNA world. It offers no binary compatibility support story, it has to be compiled for specific chips within a generation, not even just "RDNA3" but "Navi 31 specifically". Etc etc. And nobody with consumer cards could access it until like, six months ago, and that still is only on windows, consumer cards are not even supported on linux (!). That's fine for non-commercial and HPC, but it makes it difficult to imagine ever shipping commercial products to end-users on it. I guess you... ship as source, and your users have to figure out ROCm and compile it in-situ? Or ship a binary for every single GPU to ever release (including NVIDIA, Intel, ....) https://geohot.github.io/blog/jekyll/update/2023/06/07/a-dive-into-amds-drivers.html https://geohot.github.io/blog/jekyll/update/2023/06/07/a-div... This is on top of the actual problems that still remain, as geohot found out. Installing ROCm is a several-hour process that will involve debugging the platform just to get it to install, and then you will probably find that the actual code demos segfault when you run them. AMD's development processes are not really open, and actual development is silo'd inside the company with quarterly code dumps outside. The current code is not guaranteed to run on the actual driver itself, they do not test it even in the supported configurations, because all the development is happening on internal builds etc, not release code. oh and AMD themselves isn't using the open driver of course... they only test and validate on AMDGPU-PRO. it hasn't got traction because it's a low-quality product and nobody can even access it and run it anyway, let alone ship commercial products on it. and now everyone regrets it because it's turned into a money fountain.