5 ms·

"Our testing shows AV1 surpasses its stated goal of 30% better compression than VP9, and achieves gains of 50.3%, 46.2% and 34.0%, compared to x264 main profile

"Our testing shows AV1 surpasses its stated goal of 30% better compression than VP9, and achieves gains of 50.3%, 46.2% and 34.0%, compared to x264 main profile, x264 high profile and libvpx-vp9, respectively. This sounds like AV1 is actually worse than H.265, at least at low-ish resolutions. Am I misreading these results? For context, I've moved to using H.265 (via the x265 encoder) for archiving 720p and 1080p video. At least for low bitrates (in the 500-1500 kbit/s range) it gives me over 50% smaller filesizes compared to x264 high profile at roughly the same quality. The encoding speed is much slower, of course, but not by more than 10x on modern CPUs, typically less. I had hoped to move over to some AV1 implementation in a few years, but these results make that look doubtful. Are the supposed advantages of AV1 over H.265 confined to ultra high resolutions and/or high bitrates? Please note that this is a purely technical question. I don't care about the patent issues and for those that do, they've been covered at length in other comments already.

microcolonel 9y ago
> This sounds like AV1 is actually worse than H.265, at least at low-ish resolutions. Am I misreading these results? Yes, VP9 is basically on par with H.265 at standard resolutions, so if AV1 is 34% better, then it is a lot better than H.265. H.265 also doesn't tend to scale as well down to smaller resolutions, a lot of AV1's benefits scale well across the resolution spectrum (it seems). I'm sure a lot of the confusion about this comparison comes from the fact that a) we're not talking about UHD content, which is where H.265 was intended to shine, and b) Facebook probably has never had, and will likely never have, any interest in H.265 because it is simply too legally and practically cumbersome.
- keldaris 9y ago
  > Yes, VP9 is basically on par with H.265 at standard resolutions, so if AV1 is 34% better, then it is a lot better than H.265. Is that really true, though? The most comprehensive test I've seen [1] led Netflix to conclude that H.265 was better than VP9 by about 20%, including resolutions all the way down to 360p. In my own experience and other tests I've seen the gap seems even wider. There's a few usecases where VP9 is superior (most notably screencasts), but generally it seems to lag behind H.265 while also being much slower to encode. [1] http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/Netflix-Finds-x265-20-More-Efficient-than-VP9-113346.aspx http://www.streamingmedia.com/Articles/Editorial/Featured-Ar...
  - microcolonel 9y ago
    I guess it depends on content and your infrastructure. Whenever I put anywhere near similar encoding time and bitrate into a VP9 stream and an H.265 stream, I tend to get very similar results with both.
  - n-gauge 9y ago
    Yes, but VP9 is free.
  - ksec 9y ago
    Sigh. Just reading some of these comments here, and even some in Doom9, turns out we haven't learn much over the past nearly 20 years? Doom9 was started 18 years ago, and many of these has since been discussed many many times. And yet people still look at PSNR and SSIM as gold standard. And many here start to present them as "FACTS", even when they have been told not to repeatedly. Even when we have present them with all the information of why. And this is even happening on HN. May be we dont have a problem of fake news at all, we have a problem with people not willing to learn and understand, and decide to believe what ever their narrative best. This isn't just royalty free ( Not patents Free ) AV1 codec, but also programming languages, etc Sorry I am going a little off topic. I really like you stating patents not an issues, so we throw away one question that does not have a right or wrong answer. To answer your question, if your encoded copy is only viewed by you, then yes you will be the judge whether it is of similar quality, not PSNR or SSIM or even VMAF. We haven't have enough time to fully test out and evolve the VMAF metrics, but so far it is at least a much better metrics then PSNR and SSIM that is being used in EVERY AV1 test that begs about it being superior. Depending on who you ask, but anyone who hate patents or on the AV1 camp will tell you their VP9 is already better then x265, as a matter of facts they have been stating this since day one before VP9 were any good. But as you pointed out using VMAF metrics x265 is better then VP9, and VP9 has been tuned for PSNR, and SSIM values, which is part of the VMAF calculation. This gives a slight disadvantage to x264 or x265, both dont give a damn about PSNR since its inception. I do admit AV1 will likely be better then x265 in encode, given the immense complexity involved. But so far i think it is marginal to slightly better, the 30% better doesn't mean you can get a 1.4Mbps AV1 Encode to look similar to 2Mbps x265, at least in my view. You can test this yourself, but that is assuming you have enough time and spare resources to play around with the AV1 insanely slow encoder.
tveita 9y ago
"compared to x264 high profile at roughly the same quality" Compared how? With what kind of source videos? Unless you are matching SSIM on similar content I don't think you can meaningfully compare your numbers to the ones in the article.
- keldaris 9y ago
  It's certainly not a precise comparison, you're right. Nevertheless, compare for instance x264 and x265. Exactly how large the difference between the two is depends on the type of content, bitrate, resolution and other factors, but it's hard to find any case (encoding speeds aside) where x265 wouldn't be substantially better, to the point where we generally call H.265 a better codec. I was expecting a similar kind of leap with AV1. This comparison may not be conclusive, but it looks like that's probably not going to happen.
- NelsonMinar 9y ago
  Me, I just take x264 video I downloaded off the back of a truck and re-encode it to x265 with default settings on FFmpeg. It looks the same visually to me on my TV but the file size (including audio) is typically 25%. I realize this is the opposite of a carefully controlled codec experiment. Just saying in practice it works pretty well. If you're curious about x265 in practice, the pirate scene group PSA-Rips is doing a good job recoding stuff to x265. http://psarips.com/ http://psarips.com/
  - marcusjt 9y ago
    To be a little fairer you should transcode it to both x264 and x265 so that the outputs are can be compared to each other (rather than comparing the transcoded x265 to the source x264 as you did, which isn't quite the same thing).
    - NelsonMinar 9y ago
      I did that experiment with a file I found online. The source x264 video stream was 1600 MB. My x264 recoding was 290 MB, and the x265 was 80 MB. (There's also a significant savings in ffmpeg recoding the audio, which accounts for the file sizes I reported). In all cases I'm sure my result is lower quality, but still more than acceptable for me watching TV with low expectations.

microcolonel 9y ago

> This sounds like AV1 is actually worse than H.265, at least at low-ish resolutions. Am I misreading these results? Yes, VP9 is basically on par with H.265 at standard resolutions, so if AV1 is 34% better, then it is a lot better than H.265. H.265 also doesn't tend to scale as well down to smaller resolutions, a lot of AV1's benefits scale well across the resolution spectrum (it seems). I'm sure a lot of the confusion about this comparison comes from the fact that a) we're not talking about UHD content, which is where H.265 was intended to shine, and b) Facebook probably has never had, and will likely never have, any interest in H.265 because it is simply too legally and practically cumbersome.

keldaris 9y ago
> Yes, VP9 is basically on par with H.265 at standard resolutions, so if AV1 is 34% better, then it is a lot better than H.265. Is that really true, though? The most comprehensive test I've seen [1] led Netflix to conclude that H.265 was better than VP9 by about 20%, including resolutions all the way down to 360p. In my own experience and other tests I've seen the gap seems even wider. There's a few usecases where VP9 is superior (most notably screencasts), but generally it seems to lag behind H.265 while also being much slower to encode. [1] http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/Netflix-Finds-x265-20-More-Efficient-than-VP9-113346.aspx http://www.streamingmedia.com/Articles/Editorial/Featured-Ar...
- microcolonel 9y ago
  I guess it depends on content and your infrastructure. Whenever I put anywhere near similar encoding time and bitrate into a VP9 stream and an H.265 stream, I tend to get very similar results with both.
- n-gauge 9y ago
  Yes, but VP9 is free.
- ksec 9y ago
  Sigh. Just reading some of these comments here, and even some in Doom9, turns out we haven't learn much over the past nearly 20 years? Doom9 was started 18 years ago, and many of these has since been discussed many many times. And yet people still look at PSNR and SSIM as gold standard. And many here start to present them as "FACTS", even when they have been told not to repeatedly. Even when we have present them with all the information of why. And this is even happening on HN. May be we dont have a problem of fake news at all, we have a problem with people not willing to learn and understand, and decide to believe what ever their narrative best. This isn't just royalty free ( Not patents Free ) AV1 codec, but also programming languages, etc Sorry I am going a little off topic. I really like you stating patents not an issues, so we throw away one question that does not have a right or wrong answer. To answer your question, if your encoded copy is only viewed by you, then yes you will be the judge whether it is of similar quality, not PSNR or SSIM or even VMAF. We haven't have enough time to fully test out and evolve the VMAF metrics, but so far it is at least a much better metrics then PSNR and SSIM that is being used in EVERY AV1 test that begs about it being superior. Depending on who you ask, but anyone who hate patents or on the AV1 camp will tell you their VP9 is already better then x265, as a matter of facts they have been stating this since day one before VP9 were any good. But as you pointed out using VMAF metrics x265 is better then VP9, and VP9 has been tuned for PSNR, and SSIM values, which is part of the VMAF calculation. This gives a slight disadvantage to x264 or x265, both dont give a damn about PSNR since its inception. I do admit AV1 will likely be better then x265 in encode, given the immense complexity involved. But so far i think it is marginal to slightly better, the 30% better doesn't mean you can get a 1.4Mbps AV1 Encode to look similar to 2Mbps x265, at least in my view. You can test this yourself, but that is assuming you have enough time and spare resources to play around with the AV1 insanely slow encoder.

tveita 9y ago

"compared to x264 high profile at roughly the same quality" Compared how? With what kind of source videos? Unless you are matching SSIM on similar content I don't think you can meaningfully compare your numbers to the ones in the article.

keldaris 9y ago
It's certainly not a precise comparison, you're right. Nevertheless, compare for instance x264 and x265. Exactly how large the difference between the two is depends on the type of content, bitrate, resolution and other factors, but it's hard to find any case (encoding speeds aside) where x265 wouldn't be substantially better, to the point where we generally call H.265 a better codec. I was expecting a similar kind of leap with AV1. This comparison may not be conclusive, but it looks like that's probably not going to happen.
NelsonMinar 9y ago
Me, I just take x264 video I downloaded off the back of a truck and re-encode it to x265 with default settings on FFmpeg. It looks the same visually to me on my TV but the file size (including audio) is typically 25%. I realize this is the opposite of a carefully controlled codec experiment. Just saying in practice it works pretty well. If you're curious about x265 in practice, the pirate scene group PSA-Rips is doing a good job recoding stuff to x265. http://psarips.com/ http://psarips.com/
- marcusjt 9y ago
  To be a little fairer you should transcode it to both x264 and x265 so that the outputs are can be compared to each other (rather than comparing the transcoded x265 to the source x264 as you did, which isn't quite the same thing).
  - NelsonMinar 9y ago
    I did that experiment with a file I found online. The source x264 video stream was 1600 MB. My x264 recoding was 290 MB, and the x265 was 80 MB. (There's also a significant savings in ffmpeg recoding the audio, which accounts for the file sizes I reported). In all cases I'm sure my result is lower quality, but still more than acceptable for me watching TV with low expectations.