6 ms·

Based on AV1's trajectory, hardware encode isn't necessary (though it is nice). The current encoder is a reference encoder. Now that the spec is finalized, expe

Based on AV1's trajectory, hardware encode isn't necessary (though it is nice). The current encoder is a reference encoder. Now that the spec is finalized, expect significant speed improvements from production encoders (realtime likely won't happen until we get it in hardware though)

Gigachad 5mo ago
Hardware encode is required if you want things like video calls, camera recording and such to use it. It isn’t required for content distribution platforms which aren’t realtime and the cost of encode is easily made up by hundreds of thousands of streams.
- Orphis 5mo ago
  One of the interesting usage of AV1 was specifically for low bitrate calls, and software encoding was perfectly fine, even on mobile. With low enough resolution, framerate and bitrate, you can get a quality stream without significant encoding artifacts compared to any other codec. It is in production right now and has been for a while. The tradeoff CPU / bandwidth is quite advantageous in situations like this. And no, AV1 HW encoders cannot usually be used, they are not designed for a tight bitrate control or realtime communications like software encoding is usually.
  - ksec 5mo ago
    Have you said this for Audio Codec I would have agreed. I do not know a single Smartphone Video Conferencing software that uses CPU encoding rather than hardware encoding. Neither WhatsApp or FaceTime, perhaps the largest of the two real time Video Call uses AV1.
    - dannyw 5mo ago
      Yeah, no production or large scale VC system is running software AV1 encoders on smartphones. You will drain a full phone battery in 1-2 hours of calls. It just doesn’t make sense and will result in extraordinary power/battery drainage at best, or output that’s worse than hardware encoding. The only way you could get AV1 to software encode in realtime AND low latency on a mid-range Android chip is by disabling or skipping nearly all of the compression/encoding features that make it good at low bitrate.
      - Andrex 5mo ago
        So, status remains quo, the commons remain tragic, and glory to H.264 forever?
        kergonath 5mo ago
        At least until a better codec has widespread enough hardware support, I think.
        ksec 5mo ago
        > tragic H.264 isn't even that bad at all, if not the best depending on how you look at it. Our Internet bandwidth, both on the backend and front end on Mobile 5G is increasing with plenty more room to grow. While computation decoding and storage isn't. i.e If bandwidth is infinite and free, and we are only optimising for decoding power usage. H.264 wins in a lot of this scenario.
        Orphis 5mo ago
        H264 is lacking a lot of features (behind patents) that are essential to real-time communications. It's available, but by far, the worst offender for call quality. Modern call technology will want to use temporal and spatial scaling which are not available in the profiles supported by most H264 encoders and decoders. Those tools are available for VP8 (temporal only), VP9 and AV1 and improve the quality of calls quite a lot when used right. I don't know about about the internals of H265 and H266 as those are also behind patents and no one wanted to touch them in the real-time conferencing space.
      - tecleandor 5mo ago
        > Yeah, no production or large scale VC system is running software AV1 encoders on smartphones. You will drain a full phone battery in 1-2 hours of calls. Yeah but, half jokingly, Zoom does that (draining the battery in an hour) already :P
    - Orphis 5mo ago
      Google Meet can do it. You don't want the full conference with AV1, just use it for very low bitrate scenarios with a high packet loss possibly. Phones are a good target system. And I know this is quite opposite to expectations. It is a lot better to send a stable and visually ok stream with AV1 at 30KBps than fail to send a VP8 50KBps stream that is unusable anyway and is subject to twice as many packet lost than a lower bitrate solution. It is possible they use AV1 in other scenarios now, but I left the team a while back now and I haven't checked what they are now using under the hood.
  - jech 5mo ago
    > One of the interesting usage of AV1 was specifically for low bitrate calls, and software encoding was perfectly fine, even on mobile. You really want hardware decoding on mobile, otherwise you end up with 40 minutes battery life. Fortunately, for typical videoconference resolutions, VP8 and H.264 are just fine. AV1 is nice to have, though, due to excellent support for synthetic content (screen sharing), and for scalable video coding (a much more elegant solution than simulcast, IMHO). In the world I live in, the general plan is to stick to VP8 and H.264 for the time being, and to skip to AV1 when it's universally available on mobile. I haven't seen any features of AV2 which would justify waiting for it.
    - Orphis 5mo ago
      No, you do NOT want hardware anything on mobile if you are targeting smaller bitrate that are not that taxing on the CPU, when the conditions are otherwise so bad that the call would either drop or be unusable. HW encoders produce bad results at low bitrate. HW decoders usually have issues with the temporal encodings used and they may also just not accept those streams (a lot of test scenarios are movies, and the RTC tools are poorly supported). I worked on shipping it to Chromium, WebRTC and Google Meet many years ago and we had many publications about it: - https://blog.google/products-and-platforms/products/duo/4-new-google-duo-features-help-you-stay-connected/ https://blog.google/products-and-platforms/products/duo/4-ne... - https://webrtchacks.com/the-hidden-av1-gift-in-google-meet/ https://webrtchacks.com/the-hidden-av1-gift-in-google-meet/ The use case is not screensharing or a large conference room, but mainly a simpler talking face for a 1:1 chat, but with good quality as packet loss is then not as impactful on a 30KBps stream with AV1 than a 50KBps VP8 stream.
      - illiac786 5mo ago
        > HW encoders produce bad results at low bitrate. Is that poor implementation or is it inherently harder to implement in hw encoders?
        defrost 5mo ago
        There's a few reasons, I suspect fixed resource depth might be factor in poor hardware single pass encoding ... What does limit them, though, is pseudo real time single pass pipelines. I see the best encoding results from two pass - one fast run to work out the easy compress and hard compress parts of a video and then a second pass to get the optimal results on a stream that's already got a budget in mind for each section through the advantage of foresight as to what's left to do.
- baybal2 5mo ago
  [dead]
diegocg 5mo ago
Anything running on a battery will need hardware acceleration
- Salgat 5mo ago
  Even without encoding, as long as decoding is supported for AV2, streaming sites like Youtube can always transcode uploads. The encoder on mobile hardware is more of a nice bonus as long as we have an AV1 encoder available in the meantime.
  - Gigachad 5mo ago
    Youtube is doing this now. Most semi popular videos have an AV1 transcode, something interesting is I've seen youtube chooses to use the AV1 format even on my macbook which doesn't have a hardware decoder, I had a look at the CPU usage and there is a 50% load on one thread on my M1, but aside from extra battery usage, this is basically negligible since I'm likely not doing any other CPU heavy tasks while watching video.
jjcm 5mo ago
I strongly disagree with it not being required. I run a small social news site - AV1 is still prohibatively expensive both for the server and clients for software encoding/decoding. Without hardware encoding, the tradeoff for better compression ratios in exchange for massive battery use + very long processing times for encoding simply isn't worth it. In order to get AV1 out, I have to often process a h264 version of a video first anyway, just so the client isn't left waiting for their video upload to finish encoding. This means to support AV1 I'm not saving anything on the storage side. Even youtube only does AV1 encodes for extremely popular videos - it only makes sense to do at significant scale. I love AV1, don't get me wrong, and I can't wait til I can switch over to it as a single unified format for both images and video, but for now the cost is too high until hardware acceleration becomes ubiquitous
- Gigachad 5mo ago
  I went and checked some youtube videos on my front page, A video with 15k views had an AV1 encode, while a video with 160 views was h264 only. So extremely popular videos is not how I would describe it, probably by views, almost everything you watch on youtube is AV1. But they skip the extra encodes for videos relatively no one watches.
  - ksec 5mo ago
    Last time I checked they do it for new videos only. Older videos with 1M views aren't even on AV1.
    - Gigachad 5mo ago
      Makes sense, new videos are where most of the streams are happening, I wouldn't be surprised if they start to reduce the number of transcodes to save space as a video drops in popularity. h264 will work on everything so they need that as a minimum, with AV1 just being there to save on data transfer.
- ksec 5mo ago
  Thanks You. I have been saying this since the launch of AV1 on HN, Doom9 and other places. I wanted to mention even Google uses custom dedicated hardware ASIC for AV1 encode. I wish LCEVC is more widespread. For the same H.264 encoding time you get 50% to 60% Bitrate reduction using it with H.264.

Gigachad 5mo ago

Hardware encode is required if you want things like video calls, camera recording and such to use it. It isn’t required for content distribution platforms which aren’t realtime and the cost of encode is easily made up by hundreds of thousands of streams.

Orphis 5mo ago
One of the interesting usage of AV1 was specifically for low bitrate calls, and software encoding was perfectly fine, even on mobile. With low enough resolution, framerate and bitrate, you can get a quality stream without significant encoding artifacts compared to any other codec. It is in production right now and has been for a while. The tradeoff CPU / bandwidth is quite advantageous in situations like this. And no, AV1 HW encoders cannot usually be used, they are not designed for a tight bitrate control or realtime communications like software encoding is usually.
- ksec 5mo ago
  Have you said this for Audio Codec I would have agreed. I do not know a single Smartphone Video Conferencing software that uses CPU encoding rather than hardware encoding. Neither WhatsApp or FaceTime, perhaps the largest of the two real time Video Call uses AV1.
  - dannyw 5mo ago
    Yeah, no production or large scale VC system is running software AV1 encoders on smartphones. You will drain a full phone battery in 1-2 hours of calls. It just doesn’t make sense and will result in extraordinary power/battery drainage at best, or output that’s worse than hardware encoding. The only way you could get AV1 to software encode in realtime AND low latency on a mid-range Android chip is by disabling or skipping nearly all of the compression/encoding features that make it good at low bitrate.
    - Andrex 5mo ago
      So, status remains quo, the commons remain tragic, and glory to H.264 forever?
      - kergonath 5mo ago
        At least until a better codec has widespread enough hardware support, I think.
      - ksec 5mo ago
        > tragic H.264 isn't even that bad at all, if not the best depending on how you look at it. Our Internet bandwidth, both on the backend and front end on Mobile 5G is increasing with plenty more room to grow. While computation decoding and storage isn't. i.e If bandwidth is infinite and free, and we are only optimising for decoding power usage. H.264 wins in a lot of this scenario.
        Orphis 5mo ago
        H264 is lacking a lot of features (behind patents) that are essential to real-time communications. It's available, but by far, the worst offender for call quality. Modern call technology will want to use temporal and spatial scaling which are not available in the profiles supported by most H264 encoders and decoders. Those tools are available for VP8 (temporal only), VP9 and AV1 and improve the quality of calls quite a lot when used right. I don't know about about the internals of H265 and H266 as those are also behind patents and no one wanted to touch them in the real-time conferencing space.
    - tecleandor 5mo ago
      > Yeah, no production or large scale VC system is running software AV1 encoders on smartphones. You will drain a full phone battery in 1-2 hours of calls. Yeah but, half jokingly, Zoom does that (draining the battery in an hour) already :P
  - Orphis 5mo ago
    Google Meet can do it. You don't want the full conference with AV1, just use it for very low bitrate scenarios with a high packet loss possibly. Phones are a good target system. And I know this is quite opposite to expectations. It is a lot better to send a stable and visually ok stream with AV1 at 30KBps than fail to send a VP8 50KBps stream that is unusable anyway and is subject to twice as many packet lost than a lower bitrate solution. It is possible they use AV1 in other scenarios now, but I left the team a while back now and I haven't checked what they are now using under the hood.
- jech 5mo ago
  > One of the interesting usage of AV1 was specifically for low bitrate calls, and software encoding was perfectly fine, even on mobile. You really want hardware decoding on mobile, otherwise you end up with 40 minutes battery life. Fortunately, for typical videoconference resolutions, VP8 and H.264 are just fine. AV1 is nice to have, though, due to excellent support for synthetic content (screen sharing), and for scalable video coding (a much more elegant solution than simulcast, IMHO). In the world I live in, the general plan is to stick to VP8 and H.264 for the time being, and to skip to AV1 when it's universally available on mobile. I haven't seen any features of AV2 which would justify waiting for it.
  - Orphis 5mo ago
    No, you do NOT want hardware anything on mobile if you are targeting smaller bitrate that are not that taxing on the CPU, when the conditions are otherwise so bad that the call would either drop or be unusable. HW encoders produce bad results at low bitrate. HW decoders usually have issues with the temporal encodings used and they may also just not accept those streams (a lot of test scenarios are movies, and the RTC tools are poorly supported). I worked on shipping it to Chromium, WebRTC and Google Meet many years ago and we had many publications about it: - https://blog.google/products-and-platforms/products/duo/4-new-google-duo-features-help-you-stay-connected/ https://blog.google/products-and-platforms/products/duo/4-ne... - https://webrtchacks.com/the-hidden-av1-gift-in-google-meet/ https://webrtchacks.com/the-hidden-av1-gift-in-google-meet/ The use case is not screensharing or a large conference room, but mainly a simpler talking face for a 1:1 chat, but with good quality as packet loss is then not as impactful on a 30KBps stream with AV1 than a 50KBps VP8 stream.
    - illiac786 5mo ago
      > HW encoders produce bad results at low bitrate. Is that poor implementation or is it inherently harder to implement in hw encoders?
      - defrost 5mo ago
        There's a few reasons, I suspect fixed resource depth might be factor in poor hardware single pass encoding ... What does limit them, though, is pseudo real time single pass pipelines. I see the best encoding results from two pass - one fast run to work out the easy compress and hard compress parts of a video and then a second pass to get the optimal results on a stream that's already got a budget in mind for each section through the advantage of foresight as to what's left to do.
baybal2 5mo ago
[dead]

diegocg 5mo ago

Anything running on a battery will need hardware acceleration

Salgat 5mo ago
Even without encoding, as long as decoding is supported for AV2, streaming sites like Youtube can always transcode uploads. The encoder on mobile hardware is more of a nice bonus as long as we have an AV1 encoder available in the meantime.
- Gigachad 5mo ago
  Youtube is doing this now. Most semi popular videos have an AV1 transcode, something interesting is I've seen youtube chooses to use the AV1 format even on my macbook which doesn't have a hardware decoder, I had a look at the CPU usage and there is a 50% load on one thread on my M1, but aside from extra battery usage, this is basically negligible since I'm likely not doing any other CPU heavy tasks while watching video.

jjcm 5mo ago

I strongly disagree with it not being required. I run a small social news site - AV1 is still prohibatively expensive both for the server and clients for software encoding/decoding. Without hardware encoding, the tradeoff for better compression ratios in exchange for massive battery use + very long processing times for encoding simply isn't worth it. In order to get AV1 out, I have to often process a h264 version of a video first anyway, just so the client isn't left waiting for their video upload to finish encoding. This means to support AV1 I'm not saving anything on the storage side. Even youtube only does AV1 encodes for extremely popular videos - it only makes sense to do at significant scale. I love AV1, don't get me wrong, and I can't wait til I can switch over to it as a single unified format for both images and video, but for now the cost is too high until hardware acceleration becomes ubiquitous

Gigachad 5mo ago
I went and checked some youtube videos on my front page, A video with 15k views had an AV1 encode, while a video with 160 views was h264 only. So extremely popular videos is not how I would describe it, probably by views, almost everything you watch on youtube is AV1. But they skip the extra encodes for videos relatively no one watches.
- ksec 5mo ago
  Last time I checked they do it for new videos only. Older videos with 1M views aren't even on AV1.
  - Gigachad 5mo ago
    Makes sense, new videos are where most of the streams are happening, I wouldn't be surprised if they start to reduce the number of transcodes to save space as a video drops in popularity. h264 will work on everything so they need that as a minimum, with AV1 just being there to save on data transfer.
ksec 5mo ago
Thanks You. I have been saying this since the launch of AV1 on HN, Doom9 and other places. I wanted to mention even Google uses custom dedicated hardware ASIC for AV1 encode. I wish LCEVC is more widespread. For the same H.264 encoding time you get 50% to 60% Bitrate reduction using it with H.264.