5 ms·

Introducing NVK

134 points by phire 4y ago

homarp 4y ago
NVK is a new open source Mesa Vulkan driver for NVIDIA GPUs, written almost entirely from scratch using the new official headers from NVIDIA.
- tomcam 4y ago
  Ah… relief. Thank you.
neogodless 4y ago
Off Topic: I did a double take and thought I had opened the article in a Private Window in Firefox. But no, that's the Collabora logo!
shmerl 4y ago
Nice. Hopefully this will replace the blob in the future for Nvidia users.
- robert_foss 4y ago
  It will probably exist as a parallel implementation much like amdvlk-pro & radv for Radeon devices on Linux.
  - ColonelPhantom 4y ago
    The AMD situation is even more complicated than that: you have both AMDGPU/AMDVLK and AMDGPU-Pro/AMDVLK-Pro. This exists in parallel to Mesa's RADV, so there are three Vulkan drivers for AMD for Linux. I do believe that AMD actively supports the radeonsi Gallium driver in Mesa, so I don't think that AMDGPU ships with a custom OpenGL driver, but I'm not too sure on that.
    - shmerl 4y ago
      AMD supports readeonsi, but they also have a proprietary OpenGL stack that's not based on llvm. I think their plan is to eventually get rid of the latter and just have a unified and open OpenGL implementation in Mesa.
    - TingPing 4y ago
      The pro variant of the vulkan driver is the same as the open one, just packaged by AMD and sometimes slightly newer than what they've open sourced.
      - shmerl 4y ago
        I think their shader compiler in amdvkl-pro is different, it's not using llvm unlike amdvlk (open).
      - ColonelPhantom 4y ago
        That's not quite true, the other commenter already mentioned LLVM vs. AMD's proprietary shader compiler, and I think ray-tracing support has also been present in the AMDGPU-PRO driver for quite a while but is only just now starting to appear in the open AMDVLK.
        my123 4y ago
        Note also some sizeable perf differences between AMDGPU-PRO and AMDVLK. The raytracing impl for example is just not the same. https://www.phoronix.net/image.php?id=2022&image=radv_rt_performance https://www.phoronix.net/image.php?id=2022&image=radv_rt_per...
  - shmerl 4y ago
    It might, but I mean for all practical purposes. Practically no one uses amdvlk / pro for Linux desktop and gaming.
- rektide 4y ago
  > Hopefully this will replace the blob in the future for Nvidia users. Seems deeply unlikely, having read the article. quote: > There are a few things which have changed recently to make the technical landscape a bit more friendly. The first change actually happened a few years ago when NVIDIA launched their Turing class of hardware. Turing GPUs use a new unified firmware blob. The GSP firmware contains everything needed for bringing up the GPU, and once the GSP firmware is loaded, it loads all the other firmwares needed by various pieces of the GPU. If we can get GSP working, this will allow us to use the same firmware as the proprietary NVIDIA drivers and should solve many of the mystery bugs. Dave Airlie gave a very good talk at LPC about this. I'm not aware of any efforts to replace the firmware blob. This work is based off of a complete blob finally actually possibly being semi useful, and not some stripped-down ultra-minimal open-source blob that barely sets up anything.
  - wmf 4y ago
    It replaces the proprietary driver. People don't care about firmware.
  - pabs3 4y ago
    The firmware blob is signed, so replacing it is very unlikely unless there is a security issue in the signing, but exploiting it and telling others how to exploit it would be a DMCA violation.
  - shmerl 4y ago
    I mean the Vulkan implementation, not the firmware.
- aseipp 4y ago
  For graphics users I think it'll happen surprisingly quick. These days I think you can get away with Docker for the userspace SDK bits like CUDA, so once there's a reasonable open source graphics stack, you'll be able to just shove everything into a container or virtual machine or whatever, I think. Some features like vGPU/SR-IOV are segregated by product line and it's unclear how that will impact the driver/userspace stack. But that's a smaller set of users.
- dyingkneepad 4y ago
  The problem here is that this is just the user space driver and it's on top of nouveau, which is the not-so-ideal open source implementation based on reverse-engineering. We really need some kind of Kernel solution from Nvidia here. Maybe at some point in the near future Nvidia is going to announce a real Kernel driver and then Jason is going to magically announce he also had that as a backend for NVK which he couldn't share before :) I also want a Pony.
  - shmerl 4y ago
    I think the post mentions they are planning to refactor kernel portion.
Vt71fcAqt7 4y ago
I think the idea of using Zink instead of rewriting OpenGL drivers is a good idea. Semantically it makes sense because Vulkan is much more low level. They really shouldn't be compiling to the same depth. Just like how the android runtime/open JDK is written in c and c++. The performance of Zink has been pretty good, although as mentioned it requires more of the vulkan spec to be implemented before using it. Better to use that time imrpoving the vulkan driver.
- bno1 4y ago
  Just yesterday zmike held a presentation about Zink at XDC, it's very interesting to hear about all the issues they had to deal with (he's also a great person to listen to): https://youtu.be/0XSJG5xGZfU?t=6h49m36s https://youtu.be/0XSJG5xGZfU?t=6h49m36s
  - Vt71fcAqt7 4y ago
    Thanks for this link!
deleted 4y ago
[deleted]
mikessoft_gmail 4y ago
mikessoft_gmail 4y ago
pedrocr 4y ago
Is gallium now deprecated within Mesa? The way this post is written it seems that way but I could swear I had read a similar post recently about other hardware in mesa leveraging gallium to get to a working driver faster.
- dyingkneepad 4y ago
  Gallium is not used in Vulkan drivers. But Zink (OpenGL on top of Vulkan) does use Gallium, so with NVK, if you want OpenGL you can use Zink which uses Gallium. Edit: Gallium is not only not deprecated, but it's also used by basically every GL driver withing Mesa these days.
- amaranth 4y ago
  Vulkan is a lower level API than Gallium so they've had to create new abstractions at the Vulkan level. This is why the ANV (Intel) driver is what everyone copy/pasted from, there was no shared code at the Vulkan level at that point. Over time they've taken the parts people were copy/pasting and turned them into generic things all the Vulkan drivers can call instead. Gallium is still the way everything else is implemented (OpenGL, Direct3D 9, video decoding, OpenCL) for drivers. One of those drivers is Zink which implements the Gallium OpenGL state tracker on top of Vulkan and another is D3D12 which does the same on top of Direct3D 12 (for WSL2). Not every driver supports every state tracker, in general all you can count on is some version of OpenGL to be supported (4.6 for Zink, 3.3 for D3D12).
  - my123 4y ago
    At least 4.1 for D3D12 nowadays. Also got a sizeable perf bump.
XorNot 4y ago
Does this mean Nvidia on Desktop Linux won't be a huge pain anymore? I'm on AMD because ultimately the Linux support just works reliably whatever I install with the open source drivers (different from great, but hey, I can run Stable Diffusion in podman so not terribly either). But the ML space is going to Nvidia, so I suspect my next card will be an NV - but not at the cost of my "work" space becoming unreliable.
- wmf 4y ago
  Yeah, give it four or five years and I expect Nvidia to be as well-supported as AMD and Intel.
  - uluyol 4y ago
    It took much longer than that for the AMD driver's to get to where they are today. Newer drivers get to piggyback off the work done for others (yay open source) but I think it will likely take longer than 5 years before NVK and friends are competitive with the proprietary driver. I would love to be proven wrong though.
    - capableweb 4y ago
      NVIDIA probably has north of 70% of the discrete graph card market share, so a lot more developers and users use NVIDIA cards than AMD. Probably will be much faster to reach the same state as AMD.
- lbotos 4y ago
  I'm curious, how are you running SD on podman? I'm looking to build an ML rig, preferably linux, and AMD cards looked... like a hassle compared to Nvidia in my cloud GPU instance testing.
  - XorNot 4y ago
    The trick is to use the PyTorch ROCm docker image (rocm/rocm-terminal) and then run with podman run -it --rm --device=/dev/kfd --device=/dev/dri --ipc=host --group-add keep-groups --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -p 127.0.0.1:7860:7860 $(cat .sd-image.iid) It has other wrinkles - i.e. your memory use is functionally doubled because you can't use float16's for whatever reason (seems to NaN bomb but I've not found an answer). So it does work, but yeah, it's a hassle compared to Nvidia where you can just run things...although, I did not need to install anything special into my host environment which I believe isn't quite as true with NV. Dockerfile I've been using lately is https://pastebin.com/BSYN8062 https://pastebin.com/BSYN8062 but I think I need to update it to incorporate the AUTOMATIC111 repo at some point.
vulkan_future 4y ago
Vulkan is years away from meaningful support. Graphically immersive Android games don’t make sense to develop. So what platform is left? Stadia died. Along with it the last best hope for Vulkan “HD.” There will really only be 4 low level graphics APIs for the next decade: DirectX, GNM (PS5), NVN (Nintendo Switch) and Metal.
- wmf 4y ago
  The future of desktop Linux is probably OpenGL and DirectX layered on top of Vulkan. Of course desktop Linux is a niche platform compared to, say, the PS5 or Switch but it's surviving.

134 points by phire 4y ago

NVK is a new open source Mesa Vulkan driver for NVIDIA GPUs, written almost entirely from scratch using the new official headers from NVIDIA.

tomcam 4y ago
Ah… relief. Thank you.

neogodless 4y ago

Off Topic: I did a double take and thought I had opened the article in a Private Window in Firefox. But no, that's the Collabora logo!

shmerl 4y ago

Nice. Hopefully this will replace the blob in the future for Nvidia users.

robert_foss 4y ago
It will probably exist as a parallel implementation much like amdvlk-pro & radv for Radeon devices on Linux.
- ColonelPhantom 4y ago
  The AMD situation is even more complicated than that: you have both AMDGPU/AMDVLK and AMDGPU-Pro/AMDVLK-Pro. This exists in parallel to Mesa's RADV, so there are three Vulkan drivers for AMD for Linux. I do believe that AMD actively supports the radeonsi Gallium driver in Mesa, so I don't think that AMDGPU ships with a custom OpenGL driver, but I'm not too sure on that.
  - shmerl 4y ago
    AMD supports readeonsi, but they also have a proprietary OpenGL stack that's not based on llvm. I think their plan is to eventually get rid of the latter and just have a unified and open OpenGL implementation in Mesa.
  - TingPing 4y ago
    The pro variant of the vulkan driver is the same as the open one, just packaged by AMD and sometimes slightly newer than what they've open sourced.
    - shmerl 4y ago
      I think their shader compiler in amdvkl-pro is different, it's not using llvm unlike amdvlk (open).
    - ColonelPhantom 4y ago
      That's not quite true, the other commenter already mentioned LLVM vs. AMD's proprietary shader compiler, and I think ray-tracing support has also been present in the AMDGPU-PRO driver for quite a while but is only just now starting to appear in the open AMDVLK.
      - my123 4y ago
        Note also some sizeable perf differences between AMDGPU-PRO and AMDVLK. The raytracing impl for example is just not the same. https://www.phoronix.net/image.php?id=2022&image=radv_rt_performance https://www.phoronix.net/image.php?id=2022&image=radv_rt_per...
- shmerl 4y ago
  It might, but I mean for all practical purposes. Practically no one uses amdvlk / pro for Linux desktop and gaming.
rektide 4y ago
> Hopefully this will replace the blob in the future for Nvidia users. Seems deeply unlikely, having read the article. quote: > There are a few things which have changed recently to make the technical landscape a bit more friendly. The first change actually happened a few years ago when NVIDIA launched their Turing class of hardware. Turing GPUs use a new unified firmware blob. The GSP firmware contains everything needed for bringing up the GPU, and once the GSP firmware is loaded, it loads all the other firmwares needed by various pieces of the GPU. If we can get GSP working, this will allow us to use the same firmware as the proprietary NVIDIA drivers and should solve many of the mystery bugs. Dave Airlie gave a very good talk at LPC about this. I'm not aware of any efforts to replace the firmware blob. This work is based off of a complete blob finally actually possibly being semi useful, and not some stripped-down ultra-minimal open-source blob that barely sets up anything.
- wmf 4y ago
  It replaces the proprietary driver. People don't care about firmware.
- pabs3 4y ago
  The firmware blob is signed, so replacing it is very unlikely unless there is a security issue in the signing, but exploiting it and telling others how to exploit it would be a DMCA violation.
- shmerl 4y ago
  I mean the Vulkan implementation, not the firmware.
aseipp 4y ago
For graphics users I think it'll happen surprisingly quick. These days I think you can get away with Docker for the userspace SDK bits like CUDA, so once there's a reasonable open source graphics stack, you'll be able to just shove everything into a container or virtual machine or whatever, I think. Some features like vGPU/SR-IOV are segregated by product line and it's unclear how that will impact the driver/userspace stack. But that's a smaller set of users.
dyingkneepad 4y ago
The problem here is that this is just the user space driver and it's on top of nouveau, which is the not-so-ideal open source implementation based on reverse-engineering. We really need some kind of Kernel solution from Nvidia here. Maybe at some point in the near future Nvidia is going to announce a real Kernel driver and then Jason is going to magically announce he also had that as a backend for NVK which he couldn't share before :) I also want a Pony.
- shmerl 4y ago
  I think the post mentions they are planning to refactor kernel portion.

Vt71fcAqt7 4y ago

I think the idea of using Zink instead of rewriting OpenGL drivers is a good idea. Semantically it makes sense because Vulkan is much more low level. They really shouldn't be compiling to the same depth. Just like how the android runtime/open JDK is written in c and c++. The performance of Zink has been pretty good, although as mentioned it requires more of the vulkan spec to be implemented before using it. Better to use that time imrpoving the vulkan driver.

bno1 4y ago
Just yesterday zmike held a presentation about Zink at XDC, it's very interesting to hear about all the issues they had to deal with (he's also a great person to listen to): https://youtu.be/0XSJG5xGZfU?t=6h49m36s https://youtu.be/0XSJG5xGZfU?t=6h49m36s
- Vt71fcAqt7 4y ago
  Thanks for this link!

deleted 4y ago

[deleted]

mikessoft_gmail 4y ago

pedrocr 4y ago

Is gallium now deprecated within Mesa? The way this post is written it seems that way but I could swear I had read a similar post recently about other hardware in mesa leveraging gallium to get to a working driver faster.

dyingkneepad 4y ago
Gallium is not used in Vulkan drivers. But Zink (OpenGL on top of Vulkan) does use Gallium, so with NVK, if you want OpenGL you can use Zink which uses Gallium. Edit: Gallium is not only not deprecated, but it's also used by basically every GL driver withing Mesa these days.
amaranth 4y ago
Vulkan is a lower level API than Gallium so they've had to create new abstractions at the Vulkan level. This is why the ANV (Intel) driver is what everyone copy/pasted from, there was no shared code at the Vulkan level at that point. Over time they've taken the parts people were copy/pasting and turned them into generic things all the Vulkan drivers can call instead. Gallium is still the way everything else is implemented (OpenGL, Direct3D 9, video decoding, OpenCL) for drivers. One of those drivers is Zink which implements the Gallium OpenGL state tracker on top of Vulkan and another is D3D12 which does the same on top of Direct3D 12 (for WSL2). Not every driver supports every state tracker, in general all you can count on is some version of OpenGL to be supported (4.6 for Zink, 3.3 for D3D12).
- my123 4y ago
  At least 4.1 for D3D12 nowadays. Also got a sizeable perf bump.

XorNot 4y ago

Does this mean Nvidia on Desktop Linux won't be a huge pain anymore? I'm on AMD because ultimately the Linux support just works reliably whatever I install with the open source drivers (different from great, but hey, I can run Stable Diffusion in podman so not terribly either). But the ML space is going to Nvidia, so I suspect my next card will be an NV - but not at the cost of my "work" space becoming unreliable.

wmf 4y ago
Yeah, give it four or five years and I expect Nvidia to be as well-supported as AMD and Intel.
- uluyol 4y ago
  It took much longer than that for the AMD driver's to get to where they are today. Newer drivers get to piggyback off the work done for others (yay open source) but I think it will likely take longer than 5 years before NVK and friends are competitive with the proprietary driver. I would love to be proven wrong though.
  - capableweb 4y ago
    NVIDIA probably has north of 70% of the discrete graph card market share, so a lot more developers and users use NVIDIA cards than AMD. Probably will be much faster to reach the same state as AMD.
lbotos 4y ago
I'm curious, how are you running SD on podman? I'm looking to build an ML rig, preferably linux, and AMD cards looked... like a hassle compared to Nvidia in my cloud GPU instance testing.
- XorNot 4y ago
  The trick is to use the PyTorch ROCm docker image (rocm/rocm-terminal) and then run with podman run -it --rm --device=/dev/kfd --device=/dev/dri --ipc=host --group-add keep-groups --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -p 127.0.0.1:7860:7860 $(cat .sd-image.iid) It has other wrinkles - i.e. your memory use is functionally doubled because you can't use float16's for whatever reason (seems to NaN bomb but I've not found an answer). So it does work, but yeah, it's a hassle compared to Nvidia where you can just run things...although, I did not need to install anything special into my host environment which I believe isn't quite as true with NV. Dockerfile I've been using lately is https://pastebin.com/BSYN8062 https://pastebin.com/BSYN8062 but I think I need to update it to incorporate the AUTOMATIC111 repo at some point.

vulkan_future 4y ago

Vulkan is years away from meaningful support. Graphically immersive Android games don’t make sense to develop. So what platform is left? Stadia died. Along with it the last best hope for Vulkan “HD.” There will really only be 4 low level graphics APIs for the next decade: DirectX, GNM (PS5), NVN (Nintendo Switch) and Metal.

wmf 4y ago
The future of desktop Linux is probably OpenGL and DirectX layered on top of Vulkan. Of course desktop Linux is a niche platform compared to, say, the PS5 or Switch but it's surviving.