4 ms·

Interesting introduction. Too bad it doesn't really get into some of the more complicated things an advanced H.264 encoder like x264 is doing, e.g. adaptive qua

Interesting introduction. Too bad it doesn't really get into some of the more complicated things an advanced H.264 encoder like x264 is doing, e.g. adaptive quantization methods. There's also a mistake or two, for example > P-frames are frames that will encode a motion vector for each of the macro-blocks from the previous frame. Actually, P-frames can use data from multiple previous frames, not just the last one. I think it's worth pointing out, as well, that even though we've technically surpassed H.264 with newer codecs like H.265, VP9, and AV1, all of which are roughly 20%-50% more efficient, this has required tremendous increases in encoding complexity. H.264 is special - it seems to occupy a kind of inflection point on the complexity / efficiency curve. It's far more efficient than previous codecs like Xvid, WMV, and so on, but at the same time even a lot of underpowered devices from over a decade ago can easily play it. We're not likely to see tradeoffs that good again in the video codec space.

clouddrover 7y ago
When you control for image quality VP9 outperforms H.264 in single-threaded decoding: https://blogs.gnome.org/rbultje/2015/09/28/vp9-encodingdecoding-performance-vs-hevch-264/ https://blogs.gnome.org/rbultje/2015/09/28/vp9-encodingdecod... You get the same image quality at a lower bitrate and faster decoding. VP9 encoding is much slower, but Intel's SVT-VP9 encoder is getting over 300 frames per second on the right hardware: https://www.phoronix.com/scan.php?page=news_item&px=SVT-VP9-Open-Source https://www.phoronix.com/scan.php?page=news_item&px=SVT-VP9-...
- bscphil 7y ago
  That's an interesting claim. I have to say it doesn't match my experience at all, e.g. when playing VP9 videos on Youtube. Several thoughts: * I'm not sure what the "ffh264" decoder they're talking about is. My understanding is that pretty much everyone is using libx264 for both encoding and decoding. Maybe the latter is more performant? * Using SSIM to pick "equal quality" files will probably not give very accurate results. * SVT-VP9 is, if I'm not mistaken, a hardware encoder. One of the issues with these is that they require some significant tradeoffs to get better speed, so they'll have worse quality than a software encoder at the same bitrate. If you go with software you really pay the price when it comes to encoding time.
  - cornstalks 7y ago
    Some corrections: ffh264 is FFmpeg's decoder. libx264 only does encoding, not decoding. SVT-VP9 is a software encoder, specially optimized for Intel Xeon Scalable and Intel Xeon D processors.
    - pg_is_a_butt 7y ago
      Does it use speculative execution to make those gains?
  - ryantriangles 7y ago
    libx264 isn't used as a decoder, it only encodes.
- cycrutchfield 7y ago
  I thought SSIM was not a very good metric for measuring quality? My impression has always been that the VP9 encoders optimize for these simple “objective quality metrics”, therefore they do well on those metrics. But encoders like x264 don’t, so they do not do as well on those metrics.
  - ahartmetz 7y ago
    This is true according to my observations. And really, nobody cares about "objective" quality (except for quick feedback during development, I guess). It has to look good.
  - clouddrover 7y ago
    VP9 outperforms H.264 on both objective and subjective metrics: http://www.compression.ru/video/codec_comparison/hevc_2018/ http://www.compression.ru/video/codec_comparison/hevc_2018/ http://www.compression.ru/video/codec_comparison/hevc_2018/#subjective_report http://www.compression.ru/video/codec_comparison/hevc_2018/#... Their subjective testing methodology: http://www.compression.ru/video/codec_comparison/hevc_2018/pdf/subjective_comparison_method.pdf http://www.compression.ru/video/codec_comparison/hevc_2018/p...
- _pmf_ 7y ago
  And VP9 is actually usable on TI Sitara platforms, while H.264 and H.265 cause 100% CPU load and still stutter.
- ksec 7y ago
  You measuring the best of VP9 Encoder in Image quality, and then points to SVT-VP9 being the fastest encoder which isn't optimise for quality. The VP9 encoder also tends to smooth ( clean ) the video rather than working with the noise ( Not a problem for most short internet video ), which even as of today x264 is very very good and possibly the best at.
  - clouddrover 7y ago
    I made two separate, unrelated statements. In any case, libvpx isn't the best VP9 encoder for quality. EVE is. The point is that you can select the VP9 encoder that makes the most sense for your use case. You can choose from EVE, libvpx, Intel SVT-VP9, Intel Quick Sync, NGCodec, mobile device encoders, etc.
Causality1 7y ago
That's because even devices from over a decade ago have hardware h264 decoders. Extremely low powered modern devices can play very large 265 videos, if they have a hardware decoder.
- kccqzy 7y ago
  That's exactly true. I still remember the time I managed to have the decade-old iPhone 3GS play an extraordinarily large H.264 encoded video (around 40Mbps) and it absolutely had no problem doing so!
- Dylan16807 7y ago
  Yeah, I remember the first time I ever got an h.264 video, on a somewhat old processor. Couldn't play at all, and it was not a high resolution file. Whether things have a hardware decoder is going to be the biggest factor for a while, and there's plenty of room to improve.
- ksec 7y ago
  Yes, and reason because the decoder was cheap to implement. Transistor isn't getting cheaper like it was in the old days. The complexity of anything post H.264 is much higher, which also meant higher transistor count, higher budget, and higher cost.
petjuh 7y ago
Also the comparison to a PNG instead of a JPEG was a bit disingenuous since PNG is a lossless codec. I understand it was done to sound impressive but if you compare lossless PNG to a codec that just reuses the same frame and uses a lossy codec on top, it's not surprising the video will be smaller.
- bigiain 7y ago
  I'm not convinced. For an article talking about the magic of compression, comparing to uncompressed png makes a lot more sense than, say, starting your comparisons to something that's already compressed like jpeg. That png is a much closer pedagogocal relation to the previously discussed "3D array of pixels, 2 in space and one in time" that he's building on.
  - ukthrowaway123 7y ago
    PNG is compressed, just losslessly. A bitmap image is an uncompressed image format (which is why you don't see them around much anymore).
    - Certhas 7y ago
      But given that the PNG comparison was shown before an extensive discussion on how lossy encryption works by discarding detail that is hard to perceive it's entirely appropriate as a starting point for this article.
dfox 7y ago
> H.264 is special - it seems to occupy a kind of inflection point on the complexity / efficiency curve. 15 years ago there were people who said exactly the same thing about MPEG4-ASP (eg. divx/xvid). It all comes down to Moore's law and hardware acceleration. Another thing about hardware acceleration is that on many platforms you really did not want to push 622Mbit/s worth of raw chroma-subsampled YUV FullHD video across whatever interconnect to your GPU, so moving things like motion compensation into GPU hardware was only way to actually play FullHD content. And this is also part of the reasons why is H.264 somehow special today.
- acdha 7y ago
  > 15 years ago there were people who said exactly the same thing about MPEG4-ASP (eg. divx/xvid). It all comes down to Moore's law and hardware acceleration. There are always format fanboys but it really wasn't anything like that because the MPEG4-ASP experience still had problems for the average user (it wasn't a big win on file size/quality, compatibility was not a given, people still had to pay for licensed players, etc.). In contrast, I think H.264 hit the threshold where it's good enough for the average person and you basically never hit the case where a file doesn't work for the person you shared it with because even though it's license-restricted, most operating systems and browsers include support. Moving away from that is going to be hard because most people will not see much advantage — AV1 is clearly better on file-size but most people aren't hitting limits on a regular basis and the feedback loops are often missing (e.g. you hit play in Netflix on your phone — how many people would have any way to tell how much of their data plan that used?).
  - wmsiler 7y ago
    >you hit play in Netflix on your phone — how many people would have any way to tell how much of their data plan that used? Instead of focusing on the client, what about the server? How happy would Netflix be to reduce their outbound traffic by 20%? I'm guessing it's quite happy.
    - acdha 7y ago
      Oh, don’t get me wrong: I love better codecs and I want them to happen. I’m just with the person upthread who was saying H.264 will stick around: Netflix and YouTube can upgrade rapidly because they control the clients and have robust transcoding pipelines: if it saves 10% on bandwidth costs they’ll recoup the work quickly. It’s upgrading everyone else that’ll take awhile, especially since so much video is natively produced as H.264.
ksec 7y ago
>it seems to occupy a kind of inflection point on the complexity / efficiency curve. I believe it is one reason why MPEG is working on EVC, a Royalty Free Video Codec based on H.264 with a target of 20% improvement. Hopefully the industry as a whole, mainly Apple and Google can settle on something that is better than H.264 and JPEG. Right now Google will not support H.265, and Apple isn't fully on board with AV1 yet.
- clouddrover 7y ago
  > EVC, a Royalty Free Video Codec based on H.264 with a target of 20% improvement. EVC won't deliver much value. VP9 is already royalty-free, already outperforms x264, and already has broad support and use. Apple should add VP9 support now (like everyone else has) and work to add AV1 support in the future.
- kasabali 7y ago
  Correct me if I'm wrong but EVC targets 4x complexity of AVC with 20% improvement. So it has more to do with being royalty free but nothing to do with inflection point on the complexity / efficiency curve of H264.
p0nce 7y ago
HEVC in practice is 3x the encoding complexity for 40% to 50% bitrate reduction.

clouddrover 7y ago

When you control for image quality VP9 outperforms H.264 in single-threaded decoding: https://blogs.gnome.org/rbultje/2015/09/28/vp9-encodingdecoding-performance-vs-hevch-264/ https://blogs.gnome.org/rbultje/2015/09/28/vp9-encodingdecod... You get the same image quality at a lower bitrate and faster decoding. VP9 encoding is much slower, but Intel's SVT-VP9 encoder is getting over 300 frames per second on the right hardware: https://www.phoronix.com/scan.php?page=news_item&px=SVT-VP9-Open-Source https://www.phoronix.com/scan.php?page=news_item&px=SVT-VP9-...

bscphil 7y ago
That's an interesting claim. I have to say it doesn't match my experience at all, e.g. when playing VP9 videos on Youtube. Several thoughts: * I'm not sure what the "ffh264" decoder they're talking about is. My understanding is that pretty much everyone is using libx264 for both encoding and decoding. Maybe the latter is more performant? * Using SSIM to pick "equal quality" files will probably not give very accurate results. * SVT-VP9 is, if I'm not mistaken, a hardware encoder. One of the issues with these is that they require some significant tradeoffs to get better speed, so they'll have worse quality than a software encoder at the same bitrate. If you go with software you really pay the price when it comes to encoding time.
- cornstalks 7y ago
  Some corrections: ffh264 is FFmpeg's decoder. libx264 only does encoding, not decoding. SVT-VP9 is a software encoder, specially optimized for Intel Xeon Scalable and Intel Xeon D processors.
  - pg_is_a_butt 7y ago
    Does it use speculative execution to make those gains?
- ryantriangles 7y ago
  libx264 isn't used as a decoder, it only encodes.
cycrutchfield 7y ago
I thought SSIM was not a very good metric for measuring quality? My impression has always been that the VP9 encoders optimize for these simple “objective quality metrics”, therefore they do well on those metrics. But encoders like x264 don’t, so they do not do as well on those metrics.
- ahartmetz 7y ago
  This is true according to my observations. And really, nobody cares about "objective" quality (except for quick feedback during development, I guess). It has to look good.
- clouddrover 7y ago
  VP9 outperforms H.264 on both objective and subjective metrics: http://www.compression.ru/video/codec_comparison/hevc_2018/ http://www.compression.ru/video/codec_comparison/hevc_2018/ http://www.compression.ru/video/codec_comparison/hevc_2018/#subjective_report http://www.compression.ru/video/codec_comparison/hevc_2018/#... Their subjective testing methodology: http://www.compression.ru/video/codec_comparison/hevc_2018/pdf/subjective_comparison_method.pdf http://www.compression.ru/video/codec_comparison/hevc_2018/p...
_pmf_ 7y ago
And VP9 is actually usable on TI Sitara platforms, while H.264 and H.265 cause 100% CPU load and still stutter.
ksec 7y ago
You measuring the best of VP9 Encoder in Image quality, and then points to SVT-VP9 being the fastest encoder which isn't optimise for quality. The VP9 encoder also tends to smooth ( clean ) the video rather than working with the noise ( Not a problem for most short internet video ), which even as of today x264 is very very good and possibly the best at.
- clouddrover 7y ago
  I made two separate, unrelated statements. In any case, libvpx isn't the best VP9 encoder for quality. EVE is. The point is that you can select the VP9 encoder that makes the most sense for your use case. You can choose from EVE, libvpx, Intel SVT-VP9, Intel Quick Sync, NGCodec, mobile device encoders, etc.

Causality1 7y ago

That's because even devices from over a decade ago have hardware h264 decoders. Extremely low powered modern devices can play very large 265 videos, if they have a hardware decoder.

kccqzy 7y ago
That's exactly true. I still remember the time I managed to have the decade-old iPhone 3GS play an extraordinarily large H.264 encoded video (around 40Mbps) and it absolutely had no problem doing so!
Dylan16807 7y ago
Yeah, I remember the first time I ever got an h.264 video, on a somewhat old processor. Couldn't play at all, and it was not a high resolution file. Whether things have a hardware decoder is going to be the biggest factor for a while, and there's plenty of room to improve.
ksec 7y ago
Yes, and reason because the decoder was cheap to implement. Transistor isn't getting cheaper like it was in the old days. The complexity of anything post H.264 is much higher, which also meant higher transistor count, higher budget, and higher cost.

petjuh 7y ago

Also the comparison to a PNG instead of a JPEG was a bit disingenuous since PNG is a lossless codec. I understand it was done to sound impressive but if you compare lossless PNG to a codec that just reuses the same frame and uses a lossy codec on top, it's not surprising the video will be smaller.

bigiain 7y ago
I'm not convinced. For an article talking about the magic of compression, comparing to uncompressed png makes a lot more sense than, say, starting your comparisons to something that's already compressed like jpeg. That png is a much closer pedagogocal relation to the previously discussed "3D array of pixels, 2 in space and one in time" that he's building on.
- ukthrowaway123 7y ago
  PNG is compressed, just losslessly. A bitmap image is an uncompressed image format (which is why you don't see them around much anymore).
  - Certhas 7y ago
    But given that the PNG comparison was shown before an extensive discussion on how lossy encryption works by discarding detail that is hard to perceive it's entirely appropriate as a starting point for this article.

dfox 7y ago

> H.264 is special - it seems to occupy a kind of inflection point on the complexity / efficiency curve. 15 years ago there were people who said exactly the same thing about MPEG4-ASP (eg. divx/xvid). It all comes down to Moore's law and hardware acceleration. Another thing about hardware acceleration is that on many platforms you really did not want to push 622Mbit/s worth of raw chroma-subsampled YUV FullHD video across whatever interconnect to your GPU, so moving things like motion compensation into GPU hardware was only way to actually play FullHD content. And this is also part of the reasons why is H.264 somehow special today.

acdha 7y ago
> 15 years ago there were people who said exactly the same thing about MPEG4-ASP (eg. divx/xvid). It all comes down to Moore's law and hardware acceleration. There are always format fanboys but it really wasn't anything like that because the MPEG4-ASP experience still had problems for the average user (it wasn't a big win on file size/quality, compatibility was not a given, people still had to pay for licensed players, etc.). In contrast, I think H.264 hit the threshold where it's good enough for the average person and you basically never hit the case where a file doesn't work for the person you shared it with because even though it's license-restricted, most operating systems and browsers include support. Moving away from that is going to be hard because most people will not see much advantage — AV1 is clearly better on file-size but most people aren't hitting limits on a regular basis and the feedback loops are often missing (e.g. you hit play in Netflix on your phone — how many people would have any way to tell how much of their data plan that used?).
- wmsiler 7y ago
  >you hit play in Netflix on your phone — how many people would have any way to tell how much of their data plan that used? Instead of focusing on the client, what about the server? How happy would Netflix be to reduce their outbound traffic by 20%? I'm guessing it's quite happy.
  - acdha 7y ago
    Oh, don’t get me wrong: I love better codecs and I want them to happen. I’m just with the person upthread who was saying H.264 will stick around: Netflix and YouTube can upgrade rapidly because they control the clients and have robust transcoding pipelines: if it saves 10% on bandwidth costs they’ll recoup the work quickly. It’s upgrading everyone else that’ll take awhile, especially since so much video is natively produced as H.264.

ksec 7y ago

>it seems to occupy a kind of inflection point on the complexity / efficiency curve. I believe it is one reason why MPEG is working on EVC, a Royalty Free Video Codec based on H.264 with a target of 20% improvement. Hopefully the industry as a whole, mainly Apple and Google can settle on something that is better than H.264 and JPEG. Right now Google will not support H.265, and Apple isn't fully on board with AV1 yet.

clouddrover 7y ago
> EVC, a Royalty Free Video Codec based on H.264 with a target of 20% improvement. EVC won't deliver much value. VP9 is already royalty-free, already outperforms x264, and already has broad support and use. Apple should add VP9 support now (like everyone else has) and work to add AV1 support in the future.
kasabali 7y ago
Correct me if I'm wrong but EVC targets 4x complexity of AVC with 20% improvement. So it has more to do with being royalty free but nothing to do with inflection point on the complexity / efficiency curve of H264.

p0nce 7y ago

HEVC in practice is 3x the encoding complexity for 40% to 50% bitrate reduction.