4 ms·

Vulkan Video Decoding

89 points by zdw 4y ago

Vt71fcAqt7 4y ago
This was a great read and finally explained why Vulkan Video is good being "just" a layer ontop of existing HW decode/encode. What's interesting, then, is that Nvidia and others are supporting it. As opposed to rendering where Intel, AMD and Nvidia have their own rendering backend in eg. blender. Nvidia has Optix and CUDA, AMD has HIP, Intel has oneAPI (ironic given this discussion) and Apple has Metal.[0] I guess there isn't much value in being unique for something that's anyways implementing an existing standard (in this case the codecs themselves like h.265). [0]https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html https://docs.blender.org/manual/en/latest/render/cycles/gpu_...
- Scaevolus 4y ago
  Video decoding is a massively simpler API surface than a GPU programming framework like CUDA or HIP. Exposing the APIs in Vulkan is not that much extra work.
Const-me 4y ago
All of these APIs are very low-level, that’s why the quirks. There’s one system-specific video acceleration API which is not on the list, supports all GPUs, and just works without quirks, that’s IMFMediaEngine interface https://learn.microsoft.com/en-us/windows/win32/api/mfmediaengine/nn-mfmediaengine-imfmediaengine https://learn.microsoft.com/en-us/windows/win32/api/mfmediae... You give it a media file, and ask to play. The API delivers frames in DXGI surfaces, and you can do whatever you want with the data. Ideally, that’s what I would want in all other OSes.
jacooper 4y ago
Do anything, just please get Video decoding working on Linux, it kills battery life immediately on laptops.
- deleted 4y ago
  [deleted]
- nicolaslem 4y ago
  This is what I thought initially, but in my testing with a modern laptop I saw almost no difference with and without hardware decoding. It is possible that my testing was flawed so I would be curious if anyone has investigated power consumption more thoroughly.
  - cb22 4y ago
    I just did some quick testing on my XPS 7390 2-in-1 (i7 1065G7) with Wayland: * Idle @ 50% brightness, WiFI on: 3.5W * mpv + hardware acceleration: 15W * Firefox + hardware acceleration: 19W * Firefox software decoding: 29W The test video was https://www.youtube.com/watch?v=LXb3EKWsInQ https://www.youtube.com/watch?v=LXb3EKWsInQ - which is 4k 60 FPS and got scaled to 1920x1080 for display.
    - doubled112 4y ago
      Huh, that's the video I use to test hardware acceleration too. Good taste.
    - btdmaster 4y ago
      The difference is much more pronounced at lower resolutions. 4K60 is not handled efficiently by any hardware.
      - FpUser 4y ago
        I have desktop product that among other things decodes 4K60P using MMF. The decoding part either decodes straight into DirectX texture if hardware based video decoding is supported or does it in software otherwise. Software based path causes significant CPU load. Using hardware path keeps CPU at about 0%. Since hardware video decoding supported nearly on any modern Laptop, Desktop I would say 4K60 is handled very efficiently.
      - 4y ago

89 points by zdw 4y ago

This was a great read and finally explained why Vulkan Video is good being "just" a layer ontop of existing HW decode/encode. What's interesting, then, is that Nvidia and others are supporting it. As opposed to rendering where Intel, AMD and Nvidia have their own rendering backend in eg. blender. Nvidia has Optix and CUDA, AMD has HIP, Intel has oneAPI (ironic given this discussion) and Apple has Metal.[0] I guess there isn't much value in being unique for something that's anyways implementing an existing standard (in this case the codecs themselves like h.265). [0]https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html https://docs.blender.org/manual/en/latest/render/cycles/gpu_...

Scaevolus 4y ago
Video decoding is a massively simpler API surface than a GPU programming framework like CUDA or HIP. Exposing the APIs in Vulkan is not that much extra work.

Const-me 4y ago

All of these APIs are very low-level, that’s why the quirks. There’s one system-specific video acceleration API which is not on the list, supports all GPUs, and just works without quirks, that’s IMFMediaEngine interface https://learn.microsoft.com/en-us/windows/win32/api/mfmediaengine/nn-mfmediaengine-imfmediaengine https://learn.microsoft.com/en-us/windows/win32/api/mfmediae... You give it a media file, and ask to play. The API delivers frames in DXGI surfaces, and you can do whatever you want with the data. Ideally, that’s what I would want in all other OSes.

jacooper 4y ago

Do anything, just please get Video decoding working on Linux, it kills battery life immediately on laptops.

deleted 4y ago
[deleted]
nicolaslem 4y ago
This is what I thought initially, but in my testing with a modern laptop I saw almost no difference with and without hardware decoding. It is possible that my testing was flawed so I would be curious if anyone has investigated power consumption more thoroughly.
- cb22 4y ago
  I just did some quick testing on my XPS 7390 2-in-1 (i7 1065G7) with Wayland: * Idle @ 50% brightness, WiFI on: 3.5W * mpv + hardware acceleration: 15W * Firefox + hardware acceleration: 19W * Firefox software decoding: 29W The test video was https://www.youtube.com/watch?v=LXb3EKWsInQ https://www.youtube.com/watch?v=LXb3EKWsInQ - which is 4k 60 FPS and got scaled to 1920x1080 for display.
  - doubled112 4y ago
    Huh, that's the video I use to test hardware acceleration too. Good taste.
  - btdmaster 4y ago
    The difference is much more pronounced at lower resolutions. 4K60 is not handled efficiently by any hardware.
    - FpUser 4y ago
      I have desktop product that among other things decodes 4K60P using MMF. The decoding part either decodes straight into DirectX texture if hardware based video decoding is supported or does it in software otherwise. Software based path causes significant CPU load. Using hardware path keeps CPU at about 0%. Since hardware video decoding supported nearly on any modern Laptop, Desktop I would say 4K60 is handled very efficiently.
    - 4y ago