4 ms·

On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support

by shmerl 3y ago

On Windows Vulkan video would work. But not on macOS (that mpv thread mentions it). On Linux it should be a better option than VAAPI once drivers will support it. Since Firefox is using ffmpeg anyway, it probably shouldn't be too hard to make it an option.

jorvi 3y ago
Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.
- Osiris 3y ago
  Do CPUs have fixed video decoders? I thought those were only included in GPUs, including iGPUs.
  - tadfisher 3y ago
    Yes, every Intel consumer CPU has included a hardware video block since Haswell, called Quick Sync Video.
    - vetinari 3y ago
      That block is a part of the included iGPU though. Intel CPUs without iGPU, like the F-series, do not support QSV.
- kllrnohj 3y ago
  VAAPI is also using the GPU. The fixed-function video decoders are on the GPUs, not the CPU. That just happens to be using the iGPU on Intel so you don't need a discreet GPU necessarily, but this is also missing if you have a CPU-only sku like the -f suffix parts The only difference is the vulkan spec is _in theory_ cross-platform. However, since MacOS doesn't support Vulkan and MoltenVK doesn't implement this extension (and probably won't), and this extension also isn't implemented on Android, then your scope of actual availability is quite pathetic. So it probably doesn't make sense to move off of VAAPI regardless unless you really want Linux & Windows to share a codepath, even though you have something OS-specific anyways for MacOS/iOS & Android.
  - stefan_ 3y ago
    It's unhelpful to bundle this into the "GPU" at large and is only true for desktop systems anyway. On mobile, video decoding, GPU and display pipeline are typically different hardware blocks often made by different companies.
  - brigade 3y ago
    > MoltenVK doesn't implement this extension (and probably won't) Since Khronos appears to have decided to require manual DPB management, this cannot be implemented in HW on macOS by MoltenVK.
  - jorvi 3y ago
    Huh, I stand corrected. I always figured they stuffed it in its own little bit of silicon. Thanks for clearing it up!
    - tadfisher 3y ago
      Intel does do this, they call it Quick Sync Video. There's support for it in vaapi. AV1 is only if you have a Gen12 iGPU or newer, or Arc.
      - jorvi 3y ago
        I might just be bad at Google-fu, but my understanding is that -F processors don’t have Quick Sync, which seems to imply that the iGPU is required in some capacity.
- deleted 3y ago
  [deleted]
- brigade 3y ago
  Vulkan Video Extensions use the same physical hardware as VAAPI, but with a little bit less of forcing you to do codec-specific nonsense. It should be better in that it's much easier to get a zero-copy Vulkan texture from the decoded frame.
- theHamsta 3y ago
  The GPU has also the fixed function video encode like integrated CPUs. Both VAAPI and Vulkan Video address the same hardware. Differences are in the API itself (with Vulkan being more explicit) and potentially in which vendors (or OSS driver developers) support which API in what degree.
- deskamess 3y ago
  Could anyone elaborate on the difference between software decoding and CPU decoding? Software decoding uses the CPU right?
  - mattst88 3y ago
    That's right. "Software decoding" means the decoding algorithms run on the CPU. (As opposed to "hardware decoding" which typically means the work is done by some fixed-function video units on e.g. a GPU)
- shmerl 3y ago
  I think not being tied to VAAPI reduces the chain of development (like you don't need to wait for Intel to add something first since they are the primary developers of VAAPI in general).
slimsag 3y ago
Beware on Windows, out-of-the-box Microsoft will ship e.g. nvidia drivers that have DirectX 12, but not Vulkan. This can lead to confusing user situations ('DirectX 12 games work fine.. but not Vulkan') and probably means DXVA would be a better choice if targeting Windows
- my123 3y ago
  > nvidia drivers that have DirectX 12, but not Vulkan not a thing. at worst you get a situation where the ICD loader isn't installed
  - slimsag 3y ago
    In Windows 10 and above, windows update provides a default graphics driver for each IHV, which comes from the manufacturer, has DirectX support, but doesn't have Vulkan. Often nvidia users will try to debug it, and discover 'the vulkan checkbox is off but I have the latest driver version!' in gpuZ[0][1] - but it happens across all IHVs, AMD, Intel, etc. I was told by someone in the industry the default Microsoft drivers across all IHVs indeed have DirectX, but not Vulkan. To make matters more annoying, Windows Update often _overwrites_ the user's driver which they installed from the manufacturer directly, leading to Vulkan becoming broken overnight[2][3] Easy to find multiple instances of this happening[4][5][6], I became aware of it because of someone asking on /r/vulkan a few weeks ago. [0] https://i.imgur.com/qj3Rn9p.png https://i.imgur.com/qj3Rn9p.png https://rog-forum.asus.com/t5/rog-strix-series/nvidia-driver-without-vulkan-support-solved/td-p/892751 https://rog-forum.asus.com/t5/rog-strix-series/nvidia-driver... [1] https://forums.x-plane.org/index.php?/forums/topic/228597-vulkan-checkbox-greyed-out/ https://forums.x-plane.org/index.php?/forums/topic/228597-vu... [2] https://www.reddit.com/r/Amd/comments/1145ffp/windows_update_may_have_automatically_replaced/ https://www.reddit.com/r/Amd/comments/1145ffp/windows_update... [3] https://answers.microsoft.com/en-us/windows/forum/all/windows-10-update-replaced-amd-graphics-driver/78c071ed-eaa7-4508-8d94-1121669c39cf https://answers.microsoft.com/en-us/windows/forum/all/window... [4] https://i.imgur.com/PXZRxYh.png https://i.imgur.com/PXZRxYh.png [5] https://www.reddit.com/r/pcmasterrace/comments/qhfesc/vulkan_disabled_after_driver_update/ https://www.reddit.com/r/pcmasterrace/comments/qhfesc/vulkan... [6] https://www.reddit.com/r/techsupport/comments/110bytq/vulkan_api_disabled_on_amd_card/ https://www.reddit.com/r/techsupport/comments/110bytq/vulkan...
ComputerGuru 3y ago
But why would Firefox go out of its way to use Vulkan on Windows or macOS when it doesn’t have to because it can just use the native OS functionality and let the OS worry about implementation details?
- shmerl 3y ago
  I don't really care much about Windows and macOS. But I'd like to see it using Vulkan video on Linux in addition or instead of VAAPI. As pointed out elsewhere, it's not possible on macOS either way. Apple are too stuck up not supporting Vulkan and it won't work with MoltenVK.

by shmerl 3y ago

jorvi 3y ago

Why would it be a better option than VAAPI? Decoding h264/h265 in fixed (CPU) hardware is much more efficient than on the GPU. Don’t get me wrong, its still miles and miles better than software decode.

Osiris 3y ago
Do CPUs have fixed video decoders? I thought those were only included in GPUs, including iGPUs.
- tadfisher 3y ago
  Yes, every Intel consumer CPU has included a hardware video block since Haswell, called Quick Sync Video.
  - vetinari 3y ago
    That block is a part of the included iGPU though. Intel CPUs without iGPU, like the F-series, do not support QSV.
kllrnohj 3y ago
VAAPI is also using the GPU. The fixed-function video decoders are on the GPUs, not the CPU. That just happens to be using the iGPU on Intel so you don't need a discreet GPU necessarily, but this is also missing if you have a CPU-only sku like the -f suffix parts The only difference is the vulkan spec is _in theory_ cross-platform. However, since MacOS doesn't support Vulkan and MoltenVK doesn't implement this extension (and probably won't), and this extension also isn't implemented on Android, then your scope of actual availability is quite pathetic. So it probably doesn't make sense to move off of VAAPI regardless unless you really want Linux & Windows to share a codepath, even though you have something OS-specific anyways for MacOS/iOS & Android.
- stefan_ 3y ago
  It's unhelpful to bundle this into the "GPU" at large and is only true for desktop systems anyway. On mobile, video decoding, GPU and display pipeline are typically different hardware blocks often made by different companies.
- brigade 3y ago
  > MoltenVK doesn't implement this extension (and probably won't) Since Khronos appears to have decided to require manual DPB management, this cannot be implemented in HW on macOS by MoltenVK.
- jorvi 3y ago
  Huh, I stand corrected. I always figured they stuffed it in its own little bit of silicon. Thanks for clearing it up!
  - tadfisher 3y ago
    Intel does do this, they call it Quick Sync Video. There's support for it in vaapi. AV1 is only if you have a Gen12 iGPU or newer, or Arc.
    - jorvi 3y ago
      I might just be bad at Google-fu, but my understanding is that -F processors don’t have Quick Sync, which seems to imply that the iGPU is required in some capacity.
deleted 3y ago
[deleted]
brigade 3y ago
Vulkan Video Extensions use the same physical hardware as VAAPI, but with a little bit less of forcing you to do codec-specific nonsense. It should be better in that it's much easier to get a zero-copy Vulkan texture from the decoded frame.
theHamsta 3y ago
The GPU has also the fixed function video encode like integrated CPUs. Both VAAPI and Vulkan Video address the same hardware. Differences are in the API itself (with Vulkan being more explicit) and potentially in which vendors (or OSS driver developers) support which API in what degree.
deskamess 3y ago
Could anyone elaborate on the difference between software decoding and CPU decoding? Software decoding uses the CPU right?
- mattst88 3y ago
  That's right. "Software decoding" means the decoding algorithms run on the CPU. (As opposed to "hardware decoding" which typically means the work is done by some fixed-function video units on e.g. a GPU)
shmerl 3y ago
I think not being tied to VAAPI reduces the chain of development (like you don't need to wait for Intel to add something first since they are the primary developers of VAAPI in general).

slimsag 3y ago

Beware on Windows, out-of-the-box Microsoft will ship e.g. nvidia drivers that have DirectX 12, but not Vulkan. This can lead to confusing user situations ('DirectX 12 games work fine.. but not Vulkan') and probably means DXVA would be a better choice if targeting Windows

my123 3y ago
> nvidia drivers that have DirectX 12, but not Vulkan not a thing. at worst you get a situation where the ICD loader isn't installed
- slimsag 3y ago
  In Windows 10 and above, windows update provides a default graphics driver for each IHV, which comes from the manufacturer, has DirectX support, but doesn't have Vulkan. Often nvidia users will try to debug it, and discover 'the vulkan checkbox is off but I have the latest driver version!' in gpuZ[0][1] - but it happens across all IHVs, AMD, Intel, etc. I was told by someone in the industry the default Microsoft drivers across all IHVs indeed have DirectX, but not Vulkan. To make matters more annoying, Windows Update often _overwrites_ the user's driver which they installed from the manufacturer directly, leading to Vulkan becoming broken overnight[2][3] Easy to find multiple instances of this happening[4][5][6], I became aware of it because of someone asking on /r/vulkan a few weeks ago. [0] https://i.imgur.com/qj3Rn9p.png https://i.imgur.com/qj3Rn9p.png https://rog-forum.asus.com/t5/rog-strix-series/nvidia-driver-without-vulkan-support-solved/td-p/892751 https://rog-forum.asus.com/t5/rog-strix-series/nvidia-driver... [1] https://forums.x-plane.org/index.php?/forums/topic/228597-vulkan-checkbox-greyed-out/ https://forums.x-plane.org/index.php?/forums/topic/228597-vu... [2] https://www.reddit.com/r/Amd/comments/1145ffp/windows_update_may_have_automatically_replaced/ https://www.reddit.com/r/Amd/comments/1145ffp/windows_update... [3] https://answers.microsoft.com/en-us/windows/forum/all/windows-10-update-replaced-amd-graphics-driver/78c071ed-eaa7-4508-8d94-1121669c39cf https://answers.microsoft.com/en-us/windows/forum/all/window... [4] https://i.imgur.com/PXZRxYh.png https://i.imgur.com/PXZRxYh.png [5] https://www.reddit.com/r/pcmasterrace/comments/qhfesc/vulkan_disabled_after_driver_update/ https://www.reddit.com/r/pcmasterrace/comments/qhfesc/vulkan... [6] https://www.reddit.com/r/techsupport/comments/110bytq/vulkan_api_disabled_on_amd_card/ https://www.reddit.com/r/techsupport/comments/110bytq/vulkan...

ComputerGuru 3y ago

But why would Firefox go out of its way to use Vulkan on Windows or macOS when it doesn’t have to because it can just use the native OS functionality and let the OS worry about implementation details?

shmerl 3y ago
I don't really care much about Windows and macOS. But I'd like to see it using Vulkan video on Linux in addition or instead of VAAPI. As pointed out elsewhere, it's not possible on macOS either way. Apple are too stuck up not supporting Vulkan and it won't work with MoltenVK.