5 ms·

Video codecs: Adding AV1 stateless video decoder support to Linux

87 points by mfilion 3y ago

wmf 3y ago
Does MT8195... exist? It was announced years ago and I still don't see any devices using it.
- CharlesW 3y ago
  It's also called the "Kompanio 1200" and is considered a Chromebook SoC. A quick search that it's used in the HP Chromebook x360 13b, at least.
  - CameronNemo 3y ago
    If you want an SBC, the closest you'll find is the MT8395-based IPi SMARC 1200 (yikes on the price...). AFAICT, the mt8395 is just an mt8195 with an auxiliary AI processor. Device trees published for the IPi device have several references to mt8195. https://www.ipi.wiki/products/i-pi-smarc-1200 https://www.ipi.wiki/products/i-pi-smarc-1200
    - cm2187 3y ago
      I wonder why nvidia doesn't do a pure nvenc card (with several chips on it). In these days of GPU shortage, it feels wasteful to have to buy a full blown gpu of all you plan to use is the nvenc chip.
torrance 3y ago
In case you were wondering like I was: “The Linux kernel's stateless video decoder interface is used for video decoding where no state needs to be kept between processed video frames and allows for independently decoding each video frame.” (https://www.phoronix.com/news/Stateless-H264-Out-Of-Staging https://www.phoronix.com/news/Stateless-H264-Out-Of-Staging)
- yencabulator 3y ago
  That's confusing and if I understand correctly misleading. Here's an alternate summary that I think is more meaningful. tl;dr the state is in the userspace client, and passed to every operation: > A stateless decoder is a decoder that works without retaining any kind of state between processed frames. This means that each frame is decoded independently of any previous and future frames, and that the client is responsible for maintaining the decoding state and providing it to the decoder with each decoding request. https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-stateless-decoder.html https://www.kernel.org/doc/html/latest/userspace-api/media/v...
  - torrance 3y ago
    Thanks for that. That’s quite a different meaning.
cm2187 3y ago
Is there a consensus on whether av1 will become a mainstream codec with broad adoption? As far as I know Apple still doesn't provide hardware acceleration, nvidia only just started (and I don't find their encoding to add much to hevc in term of quality vs file size).
- supertrope 3y ago
  YouTube uses it on the most popular videos (tens of thousands of views). The rest of the massive video library is still vp9 because av1 takes much more CPU time/FPGA/ASIC resources. Netflix uses it as their relatively small catalog means they are insensitive to av1’s slow encoding. There isn’t a public list but they are sending av1 streams to select TVs, phones, and streaming sticks. Broadcasters only upgrade after an act of Congress so they use MPEG-2. The whole ecosystem of cameras, editing tools, broadcast hardware, and consumer boxes means strong path dependence. HEVC is used on UHD Blu-ray Discs and ATSC 3.0 broadcasts. The very latest phones have hardware decoding. Google Pixel 6. Samsung Galaxy S23. Apple iPhone does not. There’s a chicken and egg problem. Chip area for av1 decoding can’t be justified without demand, but use of av1 remains low without widespread hardware support. Mobile battery constraints mean software decoding is a no go. If the SoC maker adds av1 but it’s not used it has a slight negative impact on battery life due to leakage power increasing going up with chip size (despite gating) Intel 11th gen graphics supports hardware decoding. Nvidia 3060 series has it. AMD Radeon since the RX6000 series has it. Intel Arc was the first to support hardware encoding. Nvidia 4000 series and AMD Radeon RX7000 followed (only in OBS).
- 8K832d7tNmiQ 3y ago
  I'd expect for AV1 decode support announcement for their next iphone.
- bjoli 3y ago
  HEVC Vs AV1 means very little for the consumer of the video, but the producer will save a lot of money. HEVC will probably have features that will make sure it stays in use, but most people expect av1 to be the next h264 in terms of ubiquity.
mwinatschek 3y ago
I love AV1 for its small size even compared to HEVC. Great for watching anime.
aargh_aargh 3y ago
Wait... why do I need a video decoder in my kernel?
- porridgeraisin 3y ago
  Hardware acceleration. Hardware so protected api so kernel.
- hulitu 3y ago
  > Wait... why do I need a video decoder in my kernel? Some exploits do not work without kernel support. /s I tought that graphics in the kernel was wrong (win 2k, xp) but what do i know.
- alex_duf 3y ago
  I have no idea what I'm talking about, but could this be to benefit from hardware support?

87 points by mfilion 3y ago

wmf 3y ago

Does MT8195... exist? It was announced years ago and I still don't see any devices using it.

CharlesW 3y ago
It's also called the "Kompanio 1200" and is considered a Chromebook SoC. A quick search that it's used in the HP Chromebook x360 13b, at least.
- CameronNemo 3y ago
  If you want an SBC, the closest you'll find is the MT8395-based IPi SMARC 1200 (yikes on the price...). AFAICT, the mt8395 is just an mt8195 with an auxiliary AI processor. Device trees published for the IPi device have several references to mt8195. https://www.ipi.wiki/products/i-pi-smarc-1200 https://www.ipi.wiki/products/i-pi-smarc-1200
  - cm2187 3y ago
    I wonder why nvidia doesn't do a pure nvenc card (with several chips on it). In these days of GPU shortage, it feels wasteful to have to buy a full blown gpu of all you plan to use is the nvenc chip.

torrance 3y ago

In case you were wondering like I was: “The Linux kernel's stateless video decoder interface is used for video decoding where no state needs to be kept between processed video frames and allows for independently decoding each video frame.” (https://www.phoronix.com/news/Stateless-H264-Out-Of-Staging https://www.phoronix.com/news/Stateless-H264-Out-Of-Staging)

yencabulator 3y ago
That's confusing and if I understand correctly misleading. Here's an alternate summary that I think is more meaningful. tl;dr the state is in the userspace client, and passed to every operation: > A stateless decoder is a decoder that works without retaining any kind of state between processed frames. This means that each frame is decoded independently of any previous and future frames, and that the client is responsible for maintaining the decoding state and providing it to the decoder with each decoding request. https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-stateless-decoder.html https://www.kernel.org/doc/html/latest/userspace-api/media/v...
- torrance 3y ago
  Thanks for that. That’s quite a different meaning.

cm2187 3y ago

Is there a consensus on whether av1 will become a mainstream codec with broad adoption? As far as I know Apple still doesn't provide hardware acceleration, nvidia only just started (and I don't find their encoding to add much to hevc in term of quality vs file size).

supertrope 3y ago
YouTube uses it on the most popular videos (tens of thousands of views). The rest of the massive video library is still vp9 because av1 takes much more CPU time/FPGA/ASIC resources. Netflix uses it as their relatively small catalog means they are insensitive to av1’s slow encoding. There isn’t a public list but they are sending av1 streams to select TVs, phones, and streaming sticks. Broadcasters only upgrade after an act of Congress so they use MPEG-2. The whole ecosystem of cameras, editing tools, broadcast hardware, and consumer boxes means strong path dependence. HEVC is used on UHD Blu-ray Discs and ATSC 3.0 broadcasts. The very latest phones have hardware decoding. Google Pixel 6. Samsung Galaxy S23. Apple iPhone does not. There’s a chicken and egg problem. Chip area for av1 decoding can’t be justified without demand, but use of av1 remains low without widespread hardware support. Mobile battery constraints mean software decoding is a no go. If the SoC maker adds av1 but it’s not used it has a slight negative impact on battery life due to leakage power increasing going up with chip size (despite gating) Intel 11th gen graphics supports hardware decoding. Nvidia 3060 series has it. AMD Radeon since the RX6000 series has it. Intel Arc was the first to support hardware encoding. Nvidia 4000 series and AMD Radeon RX7000 followed (only in OBS).
8K832d7tNmiQ 3y ago
I'd expect for AV1 decode support announcement for their next iphone.
bjoli 3y ago
HEVC Vs AV1 means very little for the consumer of the video, but the producer will save a lot of money. HEVC will probably have features that will make sure it stays in use, but most people expect av1 to be the next h264 in terms of ubiquity.

mwinatschek 3y ago

I love AV1 for its small size even compared to HEVC. Great for watching anime.

aargh_aargh 3y ago

Wait... why do I need a video decoder in my kernel?

porridgeraisin 3y ago
Hardware acceleration. Hardware so protected api so kernel.
hulitu 3y ago
> Wait... why do I need a video decoder in my kernel? Some exploits do not work without kernel support. /s I tought that graphics in the kernel was wrong (win 2k, xp) but what do i know.
alex_duf 3y ago
I have no idea what I'm talking about, but could this be to benefit from hardware support?