5 ms·

> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos One vendor specific API, not "various OS' nati

by tagrun 3y ago

> They have perfectly great existing hardware decoder offloading APIs via the various OS' native APIs for videos One vendor specific API, not "various OS' native APIs". Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only on Linux, which is not supported by NVIDIA. (A third-party VA-API to NVDEC translation layer for Linux does exist on GitHub, nvidia-vaapi-driver, but it's not yet reliable as the officially supported VDPAU or NVDEC, and is not included in official linux package repositories.) Intel has VA-API, AMD has AMF, and NVIDIA has VDPAU which is being replaced by NVDEC/NVENC. The idea behind Vulkan Video Extensions is to have a vendor independent and cross-platform video API.

mikepavone 3y ago
> No, Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only, which is not supported by NVIDIA. Intel is behind VA-API originally, but I don't think it's fair to say it's a vendor specific API anymore. It's supported by the open source drivers for GPUs from all 3 vendors. It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc
- tagrun 3y ago
  You can make the same argument for VDPAU. AMD officially supports it, and there is an unofficial translation layer with limited capabilities for some Intel GPUs. Is VDPAU not a vendor specific API anymore then? Intel, AMD and NVIDIA have their own vendor-specific video APIs, and even when they provide official support for the API of another vendor, it tends to expose a limited subset of the full functionality (like the list of available codecs and encoding features). You are free to call these vendor specific APIs for what they are or something else, but the reality has been that there is no single video API officially supported by Intel, AMD and NVIDIA. This changed with Vulkan Video. But Vulkan Video isn't just about desktop: mobile devices, Raspberry Pi, etc. are expected to get on board with it eventually, just like they did with Vulkan. > It's supported by the open source drivers for GPUs from all 3 vendors. Which 3 vendors are you referring to? Intel, AMD, and who? > It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc Why are you bringing up open source drivers, and what is not practical? Both official open source drivers (open-gpu-kernel-module) and unofficial open source drivers (nouveau, through binary firmware) support VDPAU. However, NVIDIA's drivers (open source or binary) does not support VA-API.
  - mikepavone 3y ago
    Nouveau supports va-api on Nvidia. Nouveau is not supported by Nvidia of course.
    - tagrun 3y ago
      What of it? So does nvidia-vaapi-driver. They're third party projects, that's very different from "supported by the vendor", and doesn't change the fact that NVIDIA as a vendor offers no support for VA-API. By the way, nouveau's support is currently limited and not useful: https://nouveau.freedesktop.org/VideoAcceleration.html https://nouveau.freedesktop.org/VideoAcceleration.html see Video engine support status table, only old GPUs and no H.265 or AV1 support.
      - mikepavone 3y ago
        > What of it? It was an answer to this question specifically > Which 3 vendors are you referring to? Intel, AMD, and who? I either missed some of the other text in your post or it was added after I started to reply. > Both official open source drivers (open-gpu-kernel-module) This is not remotely close to being a complete graphics driver. Most of a GPU driver on Linux is in userspace and there is no official open source user space component. > Why are you bringing up open source drivers, and what is not practical? nouveau has never been practical for serious use due to poor performance and mediocre hardware support (as you noted). open-gpu-kernel-module is only practical when paired with a proprietary userspace driver. Anyway, my original point in all this is that describing VA-API as an Intel vendor specific API is unfair given it has been well supported on AMD GPUs for a long time now and on nouveau it's supported as well as VDPAU (i.e. not very well as you note). I did not intend to imply that it was universal. I didn't even intend to imply that VDPAU is a vendor specific API (though as a decode-only API it's not really a complete replacement). Intel tried to make va-api the standard for hardware encode and decode on Linux, Nvidia tried to make VDPAU the standard for hardware decode on Linux. Neither was entirely successful. By contrast, NVENC/NVDEC, AMF and the Intel Media SDK (and whatever they replaced this with) never had such ambitions.
    - deleted 3y ago
      [deleted]
- account42 3y ago
  Right, this is yet another instance of Nvidia not wanting to play nice with the other kids.
kllrnohj 3y ago
> One vendor specific API, not "various OS' native APIs". Incorrect. Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor. Presumably it uses whatever iOS' equivalent is as well. It only uses VA-API on a single OS, Linux, and that's probably more a reflection on the media qualities (or lack thereof) of Linux as a whole. Maybe Vulkan video extensions will be the savior on Linux. Or maybe it won't because it won't be anyone's focus of investment since it's largely a Linux-only problem in the first place.
- Dalewyn 3y ago
  I'm always annoyed how any Linux media player or encoder needs to bring its own entire media operating system, down to each individual nut and bolt. On Windows there's Windows Media Foundation and DirectShow that centrally manage everything and also support the "individual nut and bolt" approach. Android has its own central thing (MediaCodec?) that must be used. MacOS and iOS presumably have their own central manager (Quicktime?) too. But Linux? It doesn't serve as an operating system for media. It's tremendously inconvenient as an admin/user rather than an evangelist.
  - shmerl 3y ago
    You don't need to implement every nut and bolt in the application. Lot's of useful things can do the heavy lifting (Pipewire, ffmpeg, libplacebo, Mesa and so on). Linux isn't after calling it all using some uniform "DirectFoo" naming scheme, but tools are there. Comparison is also invalid. Linux as a whole (not the kernel but OS experience) isn't controlled by some Big Brother who decides what and how it's done single mindedly. So such kind of composite result is somewhat expected.
    - pjmlp 3y ago
      Hence why it will never be embraced by desktop application developers, and Electron it is.
      - 3y ago
vetinari 3y ago
AMD AMF is not open-source; only the SDK part of it is. The runtime part is closed, bundled with the "pro" drivers. It is also intended as a multi-platform abstraction. This makes it a no-go as a platform API. The open drivers for AMD use VA-API.

by tagrun 3y ago

mikepavone 3y ago

> No, Firefox currently supports hardware video decoding with Intel's vendor specific VA-API only, which is not supported by NVIDIA. Intel is behind VA-API originally, but I don't think it's fair to say it's a vendor specific API anymore. It's supported by the open source drivers for GPUs from all 3 vendors. It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc

tagrun 3y ago
You can make the same argument for VDPAU. AMD officially supports it, and there is an unofficial translation layer with limited capabilities for some Intel GPUs. Is VDPAU not a vendor specific API anymore then? Intel, AMD and NVIDIA have their own vendor-specific video APIs, and even when they provide official support for the API of another vendor, it tends to expose a limited subset of the full functionality (like the list of available codecs and encoding features). You are free to call these vendor specific APIs for what they are or something else, but the reality has been that there is no single video API officially supported by Intel, AMD and NVIDIA. This changed with Vulkan Video. But Vulkan Video isn't just about desktop: mobile devices, Raspberry Pi, etc. are expected to get on board with it eventually, just like they did with Vulkan. > It's supported by the open source drivers for GPUs from all 3 vendors. Which 3 vendors are you referring to? Intel, AMD, and who? > It's just that the open source drivers for Nvidia cards are not very practical and the proprietary drivers only support vdpau and nvdec/nvenc Why are you bringing up open source drivers, and what is not practical? Both official open source drivers (open-gpu-kernel-module) and unofficial open source drivers (nouveau, through binary firmware) support VDPAU. However, NVIDIA's drivers (open source or binary) does not support VA-API.
- mikepavone 3y ago
  Nouveau supports va-api on Nvidia. Nouveau is not supported by Nvidia of course.
  - tagrun 3y ago
    What of it? So does nvidia-vaapi-driver. They're third party projects, that's very different from "supported by the vendor", and doesn't change the fact that NVIDIA as a vendor offers no support for VA-API. By the way, nouveau's support is currently limited and not useful: https://nouveau.freedesktop.org/VideoAcceleration.html https://nouveau.freedesktop.org/VideoAcceleration.html see Video engine support status table, only old GPUs and no H.265 or AV1 support.
    - mikepavone 3y ago
      > What of it? It was an answer to this question specifically > Which 3 vendors are you referring to? Intel, AMD, and who? I either missed some of the other text in your post or it was added after I started to reply. > Both official open source drivers (open-gpu-kernel-module) This is not remotely close to being a complete graphics driver. Most of a GPU driver on Linux is in userspace and there is no official open source user space component. > Why are you bringing up open source drivers, and what is not practical? nouveau has never been practical for serious use due to poor performance and mediocre hardware support (as you noted). open-gpu-kernel-module is only practical when paired with a proprietary userspace driver. Anyway, my original point in all this is that describing VA-API as an Intel vendor specific API is unfair given it has been well supported on AMD GPUs for a long time now and on nouveau it's supported as well as VDPAU (i.e. not very well as you note). I did not intend to imply that it was universal. I didn't even intend to imply that VDPAU is a vendor specific API (though as a decode-only API it's not really a complete replacement). Intel tried to make va-api the standard for hardware encode and decode on Linux, Nvidia tried to make VDPAU the standard for hardware decode on Linux. Neither was entirely successful. By contrast, NVENC/NVDEC, AMF and the Intel Media SDK (and whatever they replaced this with) never had such ambitions.
  - deleted 3y ago
    [deleted]
account42 3y ago
Right, this is yet another instance of Nvidia not wanting to play nice with the other kids.

kllrnohj 3y ago

> One vendor specific API, not "various OS' native APIs". Incorrect. Firefox uses Windows Media Foundation, which is cross-vendor, on Windows. It uses MediaCodec on Android which is again cross-vendor. Presumably it uses whatever iOS' equivalent is as well. It only uses VA-API on a single OS, Linux, and that's probably more a reflection on the media qualities (or lack thereof) of Linux as a whole. Maybe Vulkan video extensions will be the savior on Linux. Or maybe it won't because it won't be anyone's focus of investment since it's largely a Linux-only problem in the first place.

Dalewyn 3y ago
I'm always annoyed how any Linux media player or encoder needs to bring its own entire media operating system, down to each individual nut and bolt. On Windows there's Windows Media Foundation and DirectShow that centrally manage everything and also support the "individual nut and bolt" approach. Android has its own central thing (MediaCodec?) that must be used. MacOS and iOS presumably have their own central manager (Quicktime?) too. But Linux? It doesn't serve as an operating system for media. It's tremendously inconvenient as an admin/user rather than an evangelist.
- shmerl 3y ago
  You don't need to implement every nut and bolt in the application. Lot's of useful things can do the heavy lifting (Pipewire, ffmpeg, libplacebo, Mesa and so on). Linux isn't after calling it all using some uniform "DirectFoo" naming scheme, but tools are there. Comparison is also invalid. Linux as a whole (not the kernel but OS experience) isn't controlled by some Big Brother who decides what and how it's done single mindedly. So such kind of composite result is somewhat expected.
  - pjmlp 3y ago
    Hence why it will never be embraced by desktop application developers, and Electron it is.
    - 3y ago

vetinari 3y ago

AMD AMF is not open-source; only the SDK part of it is. The runtime part is closed, bundled with the "pro" drivers. It is also intended as a multi-platform abstraction. This makes it a no-go as a platform API. The open drivers for AMD use VA-API.