4 ms·

Asking here because this might have some visibility: What's the status of hardware acceleration of next-generation video standards (h.265, VP9, something else)

Asking here because this might have some visibility: What's the status of hardware acceleration of next-generation video standards (h.265, VP9, something else)? It's my understanding that after VP8 was pushed and then superseded rapidly, hardware manufacturers are now leery of implementing anything other than h.264. And as a follow-on: what makes generic GPU hardware and software not sufficient for hardware acceleration of video decoding? What is it that makes my GPU stupendously good at pushing pixels and training neural nets and physics calculations and so many other things, but not better than my CPU at video decoding? Is there a reason h.265 and such aren't implemented in CUDA? I don't know much about video encoding, not even enough to be dangerous, so to speak.

deleted 10y ago
[deleted]
ktta 10y ago
>What's the status of hardware acceleration of next-generation video standards (h.265, VP9, something else)? You're only likely to find proper hardware acceleration for the top codecs like h.264, not much else. There's going to be software acceleration for everything else. So from the MPEG[2] group, as you might expect, there's h.265. But then there's a whole new undertaking by the big guys like Netflix, Google, etc. which is called AOMedia[1]. Now they have this new codec called AV1[3] which is still in development. And it's open source and royalty free which is not usually what happens in the MPEG group because they charge royalties and have patents on the codecs/standards they have. >what makes generic GPU hardware and software not sufficient for hardware acceleration of video decoding? Now I'm not a video encoding/decoding expert, but just enough to be dangerous actually. What makes GPUs well suited is that they have many, many more cores so each core can handle a specific area of the video output that needs to be decoded. It's called parallel programming. Whatever can efficiently be implemented on GPUs better than CPUs, people are doing it. This is recently being called GPGPU programming, where tasks that were thought were better done by CPU are being re-programmed for the GPU since the development and performance in GPUs have been going through the roof. Now when it comes to the why, think of it this way. Your average complex computation is more like lifting a couple really heavy boulders. So you need a strong person to lift it. So that strong person is your CPU core, with complexity and high clock rate. But video encoding is not that complex at the pixel level. So it is more like picking lots (thousands) of tennis balls. GPUs are like having a bunch of kids (current gen GPUs have ~2000 cores even at the mid-range) who are no where as strong as the strong CPU guy, but you can guess who will get the task done faster. [1]:http://aomedia.org/about-us/ http://aomedia.org/about-us/ [2]:https://en.wikipedia.org/wiki/Moving_Picture_Experts_Group https://en.wikipedia.org/wiki/Moving_Picture_Experts_Group [3]:https://en.wikipedia.org/wiki/AOMedia_Video_1 https://en.wikipedia.org/wiki/AOMedia_Video_1
- clouddrover 10y ago
  > You're only likely to find proper hardware acceleration for the top codecs like h.264, not much else. Intel, Nvidia, AMD, and ARM all support VP9 acceleration: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video http://www.tomshardware.com/reviews/intel-7th-gen-core-kaby-lake-preview,4728-4.html http://www.tomshardware.com/reviews/intel-7th-gen-core-kaby-... https://en.wikipedia.org/wiki/Nvidia_PureVideo https://en.wikipedia.org/wiki/Nvidia_PureVideo https://en.wikipedia.org/wiki/Unified_Video_Decoder https://en.wikipedia.org/wiki/Unified_Video_Decoder http://www.anandtech.com/show/10428/arm-announces-mali-egil-video-processor http://www.anandtech.com/show/10428/arm-announces-mali-egil-... https://en.wikipedia.org/wiki/VP9#Hardware_encoding.2Fdecoding_support https://en.wikipedia.org/wiki/VP9#Hardware_encoding.2Fdecodi...
dogma1138 10y ago
CUDA supports VP9 decoding since Kepler iirc.
- jsheard 10y ago
  VP9 was added in VDPAU Feature Set F, which was introduced with the later Maxwell chips (950/960 cards have it but 970/980/980ti cards don't).
  - dogma1138 10y ago
    VP9 decoding was supported since Feature Set E, it was running a combination of a native ASIC decoder and a set of shaders running on the GPU. So while it wasn't as power efficient as the dedicated PureVideo hardware block it was still GPU accelerated.
    - jsheard 10y ago
      I'm curious, do you have a source for that? I know that Feature Set E cards used a hybrid ASIC/GPU approach for H.265 but I can't find a mention of hybrid VP9.
      - dogma1138 10y ago
        It's done via DXVA (2.0) if you query DX on a Maxwell card (or even Kepler) you should get something like VP9_VLD_Profile0: DXVA2/D3D11, SD / HD / FHD / 4K (You can either manually do it, or there is a tool called DXVA checker or tester or something along those lines). LAV Filters (an Open Source implementation of DirectShow/DXVA filters) VP9 support works with Maxwell cards for sure, I have a 780ti somewhere I can check Kepler also, but IIRC it should work. That said this would work for many video players that use DXVA or have support for external DS filters e.g. VLC/MPC-HC or proprietary players like Splash, but it won't work in a browser. Chrome for example doesn't allow for external filters iirc so you'll see a pretty big CPU jump. For some reason it seems fine on VP9 1080p videos on a Maxwell card CPU is 2-3% but at 4K it jumps to 70-80% I have a feeling that since Maxwell has partial support in the PV block for HEVC and VP9 they might be able to decode 1080p with it and then do full CPU decoding for 4K, I wouldn't expect CPU decoding for 1080p to be so resource efficient otherwise.
MyOwnSpirit 10y ago
hevc (h.265) has specialized hardware which encodes or decodes much faster than other GPU solutions. I believe typical graphic cards come with hardware codecs within them but I am not sure. Regarding why hevc isn't so easy to use with GPU, it comes from the way video compression is done. In video compression, you don't really process each pixel one by one independently. Instead, you consider blocks of pixels and you look for similar ones in other pictures. So it is not much about having many similar small tasks but really one big one. Besides, to minimize the information to send (i.e. to compress better), the compression of each block is based on what the left and top blocks have done. If I compress all my blocks in parallel, I will not be able to make them use each other's information. Does this help you see the big picture?

deleted 10y ago

[deleted]

ktta 10y ago

>What's the status of hardware acceleration of next-generation video standards (h.265, VP9, something else)? You're only likely to find proper hardware acceleration for the top codecs like h.264, not much else. There's going to be software acceleration for everything else. So from the MPEG[2] group, as you might expect, there's h.265. But then there's a whole new undertaking by the big guys like Netflix, Google, etc. which is called AOMedia[1]. Now they have this new codec called AV1[3] which is still in development. And it's open source and royalty free which is not usually what happens in the MPEG group because they charge royalties and have patents on the codecs/standards they have. >what makes generic GPU hardware and software not sufficient for hardware acceleration of video decoding? Now I'm not a video encoding/decoding expert, but just enough to be dangerous actually. What makes GPUs well suited is that they have many, many more cores so each core can handle a specific area of the video output that needs to be decoded. It's called parallel programming. Whatever can efficiently be implemented on GPUs better than CPUs, people are doing it. This is recently being called GPGPU programming, where tasks that were thought were better done by CPU are being re-programmed for the GPU since the development and performance in GPUs have been going through the roof. Now when it comes to the why, think of it this way. Your average complex computation is more like lifting a couple really heavy boulders. So you need a strong person to lift it. So that strong person is your CPU core, with complexity and high clock rate. But video encoding is not that complex at the pixel level. So it is more like picking lots (thousands) of tennis balls. GPUs are like having a bunch of kids (current gen GPUs have ~2000 cores even at the mid-range) who are no where as strong as the strong CPU guy, but you can guess who will get the task done faster. [1]:http://aomedia.org/about-us/ http://aomedia.org/about-us/ [2]:https://en.wikipedia.org/wiki/Moving_Picture_Experts_Group https://en.wikipedia.org/wiki/Moving_Picture_Experts_Group [3]:https://en.wikipedia.org/wiki/AOMedia_Video_1 https://en.wikipedia.org/wiki/AOMedia_Video_1

clouddrover 10y ago
> You're only likely to find proper hardware acceleration for the top codecs like h.264, not much else. Intel, Nvidia, AMD, and ARM all support VP9 acceleration: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video http://www.tomshardware.com/reviews/intel-7th-gen-core-kaby-lake-preview,4728-4.html http://www.tomshardware.com/reviews/intel-7th-gen-core-kaby-... https://en.wikipedia.org/wiki/Nvidia_PureVideo https://en.wikipedia.org/wiki/Nvidia_PureVideo https://en.wikipedia.org/wiki/Unified_Video_Decoder https://en.wikipedia.org/wiki/Unified_Video_Decoder http://www.anandtech.com/show/10428/arm-announces-mali-egil-video-processor http://www.anandtech.com/show/10428/arm-announces-mali-egil-... https://en.wikipedia.org/wiki/VP9#Hardware_encoding.2Fdecoding_support https://en.wikipedia.org/wiki/VP9#Hardware_encoding.2Fdecodi...

dogma1138 10y ago

CUDA supports VP9 decoding since Kepler iirc.

jsheard 10y ago
VP9 was added in VDPAU Feature Set F, which was introduced with the later Maxwell chips (950/960 cards have it but 970/980/980ti cards don't).
- dogma1138 10y ago
  VP9 decoding was supported since Feature Set E, it was running a combination of a native ASIC decoder and a set of shaders running on the GPU. So while it wasn't as power efficient as the dedicated PureVideo hardware block it was still GPU accelerated.
  - jsheard 10y ago
    I'm curious, do you have a source for that? I know that Feature Set E cards used a hybrid ASIC/GPU approach for H.265 but I can't find a mention of hybrid VP9.
    - dogma1138 10y ago
      It's done via DXVA (2.0) if you query DX on a Maxwell card (or even Kepler) you should get something like VP9_VLD_Profile0: DXVA2/D3D11, SD / HD / FHD / 4K (You can either manually do it, or there is a tool called DXVA checker or tester or something along those lines). LAV Filters (an Open Source implementation of DirectShow/DXVA filters) VP9 support works with Maxwell cards for sure, I have a 780ti somewhere I can check Kepler also, but IIRC it should work. That said this would work for many video players that use DXVA or have support for external DS filters e.g. VLC/MPC-HC or proprietary players like Splash, but it won't work in a browser. Chrome for example doesn't allow for external filters iirc so you'll see a pretty big CPU jump. For some reason it seems fine on VP9 1080p videos on a Maxwell card CPU is 2-3% but at 4K it jumps to 70-80% I have a feeling that since Maxwell has partial support in the PV block for HEVC and VP9 they might be able to decode 1080p with it and then do full CPU decoding for 4K, I wouldn't expect CPU decoding for 1080p to be so resource efficient otherwise.

MyOwnSpirit 10y ago

hevc (h.265) has specialized hardware which encodes or decodes much faster than other GPU solutions. I believe typical graphic cards come with hardware codecs within them but I am not sure. Regarding why hevc isn't so easy to use with GPU, it comes from the way video compression is done. In video compression, you don't really process each pixel one by one independently. Instead, you consider blocks of pixels and you look for similar ones in other pictures. So it is not much about having many similar small tasks but really one big one. Besides, to minimize the information to send (i.e. to compress better), the compression of each block is based on what the left and top blocks have done. If I compress all my blocks in parallel, I will not be able to make them use each other's information. Does this help you see the big picture?