5 ms·

The interesting thing about H.264 is that it achieves excellent compression rates not by a single "miracle trick", but by combining multiple small-gain techniqu

by jwr 7y ago

The interesting thing about H.264 is that it achieves excellent compression rates not by a single "miracle trick", but by combining multiple small-gain techniques. Every technique lets you gain on the order of 1-3%, but when you combine them all, you get fantastic results. Another tidbit of (potentially) interesting information: it took many years for H.264 to achieve its full potential, for two reasons: 1) Implementing a decoder is really, really complicated, especially if you want it to run fast, and if you think a miracle "hardware decoder" solves the problem, you need to remember that H.264 places heavy requirements on memory bandwidth and caching (B frames), so it's not like you can just feed a stream through a chip and get video frames on the other side. 2) Implementing an encoder is really, really complicated :-) — and there are many ways to encode a video stream. H.264 delivers the tools, but does not specify how you should use them. It took many years for decoders to get to a state where most of H.264 is used efficiently.

tgtweak 7y ago
It's a tremendous effort to simply ensure standards. There are so many knobs and dials on h264 encoders, and features in each profile, is hard to appreciate the compatability we have today across devices - especially considering how much is in silicon. I remember sponsoring a feature in x264 (the best software encoder) and the level of complexity in that code is mind bending. The patch was to allow better CPU utilization when running with over 16 cores - something the authors hadn't run into at the time and had no systems to test on. I did a simple profile during encode, sent them the trace, in 2 days the patch was tested and working and it hit main in a week's time. Boom, all 48 cores used. Made me realize just how much talent is behind these technologies and I still shake my head at how much money is generated with this and how little of that makes it's way back to the developers at the core of it.
- ahartmetz 7y ago
  x264 is really amazing because it has an amazing team of knowledgeable people who are personally interested in its success. x265 seems to be just some commercially run open source project and I'm not sure if it will ever make as good use of the spec as x264 does. For a time x265 was only about as good as x264 regarding subjective quality per bit.
- ksec 7y ago
  >something the authors hadn't run into at the time and had no systems to test on. Was it Dark Shikari? I remember there were lots of threading optimisation done, but that was... ~2010 time frame. I really wish he is still working in the video encoding scene.
  - tgtweak 7y ago
    Yes, was Jason, in 2012 iirc. Sliced-thread lookahead patch circa may 2012. He was working at gaikai at the time doing amazing things with cloud game streaming.
- da_chicken 7y ago
  > Made me realize just how much talent is behind these technologies and I still shake my head at how much money is generated with this and how little of that makes it's way back to the developers at the core of it. If it's any consolation, not much more makes it to the talented developers in commercial products, either. It mainly ends up going to executives and sales.
  - nojvek 7y ago
    Demand and supply right. How many of these developers started a profit sharing company? As a dev you sign up your life to work for another company where you trade time for money. You sign on that sheet. That's the contract. The company can make bazillions off you but that's the deal you signed. I really really hope with the recent IPOS, all the devs that made good money end up doing startups that are more employee friendly than VC friendly. But then, this is the ugly world we live in. The selfish gene yo!
  - winter_blue 7y ago
    I think in New York and California at least, developers are getting paid closer to what their work is worth. Salaries are often hitting the $300k ballpark for highly talented developers. Average pay itself is now around a $150k. I think it's primarily developers in other countries[1] that are getting severely underpaid. Hopefully that changes in the coming years. [1] The most egregious example might be India. Around 7 years ago, I was on a plane from Kerala (a state in India), and the guy sitting next to me happened to run a software consulting company (in Kerala). His customers were from Japan and some other countries (can't remember), and his company wrote Linux drivers for proprietary hardware. He hired kids right out of college. I asked him how much he paid them, and he said without blinking, Rs. 15000 per month. Type in "INR 15000 to USD" to Google, and you're in for a shock. It's around $2600 per year, but probably closer to $3000 per year at the exchange rates back then. American developers doing the same job could earn up to 100 times that, and at the very least 40 times that amount. I can't understand why on Earth salaries are 40 to a 100 times higher for the same exact work, and for work that could be done by anyone anywhere on the globe, but I hope this massive difference in pay converges and disappears soon.
amelius 7y ago
I don't fully agree that a h.264 system is extremely hard to implement, because it has some nice properties: - once you have a reference implementation, you can have exact input/output pairs, which makes development easy, especially if the reference implementation exposes intermediate results. - because of the previous point, testing is easy. - even if you don't have a reference implementation, you can start by building a slow version of an encoder/decoder, generate testing data (intermediate outputs), and step by step optimize the system, while ensuring the system produces the same outputs at every step.
- wojtekkru 7y ago
  Is it really deterministic mapping from input to output, for given configuration options? I imagine you can build different algorithms that choose what data can be removed, with different tradeoffs between encoding time, decoding time, memory consumption, output file size and experience of quality.
  - dfox 7y ago
    Decoder is completely deterministic. In fact the normative part of most MPEG standards is implementation of the decoder in C-like pseudocode (which is horribly slow because it is not optimized in any way and mostly deals with discrete bits). On the other hand how encoder works is not specified at all apart from the obvious requirement that it has to produce something decodable by the above mentioned decoder.
- jwr 7y ago
  Have you tried to implement one? :-) I worked with a team who did a full implementation from scratch (commercially), and I optimized some parts for certain architectures. The problem is not in getting the output. The reference decoder decodes just fine. The difficult part is getting the output on time, e.g. getting you decoder to run fast. The reference decoder has not been designed for speed, and getting things to run fast requires architecting everything very differently: it's not just a matter of applying micro-optimizations to reference code. Also, the "intermediate results" might be different for each decoder architecture, so you can't really rely on the reference code for those. So yes, while testing is indeed easy, writing a fast decoder is definitely not a "step by step optimization" process. I'd bet every performant decoder out there was rewritten almost completely at least twice.
tootie 7y ago
Where's the reference implementation? And is there actually more than one competing implementation out there?
- deleted 7y ago
  [deleted]
- hathawsh 7y ago
  Google suggests it's this: http://iphome.hhi.de/suehring/tml/download/ http://iphome.hhi.de/suehring/tml/download/ (GitHub: https://github.com/shihuade/JM https://github.com/shihuade/JM )
- jwr 7y ago
  > is there actually more than one competing implementation out there? Yes, there are many software decoders out there, usually specialized for certain architectures (the team I worked with wrote a decoder for Texas Instruments DSPs, for example). Then there are hardware implementations, which are rarely just "hardware", it's usually hardware-assisted software that does the decoding.
canada_dry 7y ago
I run XBMC on an old laptop and it struggles, but manages to play, X.264 encoded videos. However, it doesn't handle X.265 videos very well - pauses every few frames. How come? I'm assuming its because 265 is much more CPU intensive. What's the typical compression benefit of using x.265 vs x.264? ----------------------------------- Update: noticed this comment below from @Causality1 "devices from over a decade ago have hardware h264 decoders. Extremely low powered modern devices can play very large 265 videos, if they have a hardware decoder."
- Thaxll 7y ago
  x265 is not using HW for decoding so it's very heavy on the CPU.
  - Const-me 7y ago
    At least on paper, some hardware h.265 decoding is supported on Intel since Haswell (2013), on nVidia since Maxwell 2-nd gen (2015), on AMD since Polaris (2016).
    - gubbrora 7y ago
      Is it possible to (ab)use that hardware support to do other tasks by mapping them to a decoding task?
      - Const-me 7y ago
        Dunno. Why would you want to? You can already compute lots of stuff on GPUs. Couple generations ago, AMD did the other way, they used GPU shader ALUs for video encoding/decoding. They stopped doing that because power efficiency is worse than custom silicon that’s now in all modern GPUs, even in mobile SoCs.
  - p0nce 7y ago
    Please don't confuse "x265" and "H.265 / HEVC"

by jwr 7y ago

tgtweak 7y ago

It's a tremendous effort to simply ensure standards. There are so many knobs and dials on h264 encoders, and features in each profile, is hard to appreciate the compatability we have today across devices - especially considering how much is in silicon. I remember sponsoring a feature in x264 (the best software encoder) and the level of complexity in that code is mind bending. The patch was to allow better CPU utilization when running with over 16 cores - something the authors hadn't run into at the time and had no systems to test on. I did a simple profile during encode, sent them the trace, in 2 days the patch was tested and working and it hit main in a week's time. Boom, all 48 cores used. Made me realize just how much talent is behind these technologies and I still shake my head at how much money is generated with this and how little of that makes it's way back to the developers at the core of it.

ahartmetz 7y ago
x264 is really amazing because it has an amazing team of knowledgeable people who are personally interested in its success. x265 seems to be just some commercially run open source project and I'm not sure if it will ever make as good use of the spec as x264 does. For a time x265 was only about as good as x264 regarding subjective quality per bit.
ksec 7y ago
>something the authors hadn't run into at the time and had no systems to test on. Was it Dark Shikari? I remember there were lots of threading optimisation done, but that was... ~2010 time frame. I really wish he is still working in the video encoding scene.
- tgtweak 7y ago
  Yes, was Jason, in 2012 iirc. Sliced-thread lookahead patch circa may 2012. He was working at gaikai at the time doing amazing things with cloud game streaming.
da_chicken 7y ago
> Made me realize just how much talent is behind these technologies and I still shake my head at how much money is generated with this and how little of that makes it's way back to the developers at the core of it. If it's any consolation, not much more makes it to the talented developers in commercial products, either. It mainly ends up going to executives and sales.
- nojvek 7y ago
  Demand and supply right. How many of these developers started a profit sharing company? As a dev you sign up your life to work for another company where you trade time for money. You sign on that sheet. That's the contract. The company can make bazillions off you but that's the deal you signed. I really really hope with the recent IPOS, all the devs that made good money end up doing startups that are more employee friendly than VC friendly. But then, this is the ugly world we live in. The selfish gene yo!
- winter_blue 7y ago
  I think in New York and California at least, developers are getting paid closer to what their work is worth. Salaries are often hitting the $300k ballpark for highly talented developers. Average pay itself is now around a $150k. I think it's primarily developers in other countries[1] that are getting severely underpaid. Hopefully that changes in the coming years. [1] The most egregious example might be India. Around 7 years ago, I was on a plane from Kerala (a state in India), and the guy sitting next to me happened to run a software consulting company (in Kerala). His customers were from Japan and some other countries (can't remember), and his company wrote Linux drivers for proprietary hardware. He hired kids right out of college. I asked him how much he paid them, and he said without blinking, Rs. 15000 per month. Type in "INR 15000 to USD" to Google, and you're in for a shock. It's around $2600 per year, but probably closer to $3000 per year at the exchange rates back then. American developers doing the same job could earn up to 100 times that, and at the very least 40 times that amount. I can't understand why on Earth salaries are 40 to a 100 times higher for the same exact work, and for work that could be done by anyone anywhere on the globe, but I hope this massive difference in pay converges and disappears soon.

amelius 7y ago

I don't fully agree that a h.264 system is extremely hard to implement, because it has some nice properties: - once you have a reference implementation, you can have exact input/output pairs, which makes development easy, especially if the reference implementation exposes intermediate results. - because of the previous point, testing is easy. - even if you don't have a reference implementation, you can start by building a slow version of an encoder/decoder, generate testing data (intermediate outputs), and step by step optimize the system, while ensuring the system produces the same outputs at every step.

wojtekkru 7y ago
Is it really deterministic mapping from input to output, for given configuration options? I imagine you can build different algorithms that choose what data can be removed, with different tradeoffs between encoding time, decoding time, memory consumption, output file size and experience of quality.
- dfox 7y ago
  Decoder is completely deterministic. In fact the normative part of most MPEG standards is implementation of the decoder in C-like pseudocode (which is horribly slow because it is not optimized in any way and mostly deals with discrete bits). On the other hand how encoder works is not specified at all apart from the obvious requirement that it has to produce something decodable by the above mentioned decoder.
jwr 7y ago
Have you tried to implement one? :-) I worked with a team who did a full implementation from scratch (commercially), and I optimized some parts for certain architectures. The problem is not in getting the output. The reference decoder decodes just fine. The difficult part is getting the output on time, e.g. getting you decoder to run fast. The reference decoder has not been designed for speed, and getting things to run fast requires architecting everything very differently: it's not just a matter of applying micro-optimizations to reference code. Also, the "intermediate results" might be different for each decoder architecture, so you can't really rely on the reference code for those. So yes, while testing is indeed easy, writing a fast decoder is definitely not a "step by step optimization" process. I'd bet every performant decoder out there was rewritten almost completely at least twice.

tootie 7y ago

Where's the reference implementation? And is there actually more than one competing implementation out there?

deleted 7y ago
[deleted]
hathawsh 7y ago
Google suggests it's this: http://iphome.hhi.de/suehring/tml/download/ http://iphome.hhi.de/suehring/tml/download/ (GitHub: https://github.com/shihuade/JM https://github.com/shihuade/JM )
jwr 7y ago
> is there actually more than one competing implementation out there? Yes, there are many software decoders out there, usually specialized for certain architectures (the team I worked with wrote a decoder for Texas Instruments DSPs, for example). Then there are hardware implementations, which are rarely just "hardware", it's usually hardware-assisted software that does the decoding.

canada_dry 7y ago

I run XBMC on an old laptop and it struggles, but manages to play, X.264 encoded videos. However, it doesn't handle X.265 videos very well - pauses every few frames. How come? I'm assuming its because 265 is much more CPU intensive. What's the typical compression benefit of using x.265 vs x.264? ----------------------------------- Update: noticed this comment below from @Causality1 "devices from over a decade ago have hardware h264 decoders. Extremely low powered modern devices can play very large 265 videos, if they have a hardware decoder."

Thaxll 7y ago
x265 is not using HW for decoding so it's very heavy on the CPU.
- Const-me 7y ago
  At least on paper, some hardware h.265 decoding is supported on Intel since Haswell (2013), on nVidia since Maxwell 2-nd gen (2015), on AMD since Polaris (2016).
  - gubbrora 7y ago
    Is it possible to (ab)use that hardware support to do other tasks by mapping them to a decoding task?
    - Const-me 7y ago
      Dunno. Why would you want to? You can already compute lots of stuff on GPUs. Couple generations ago, AMD did the other way, they used GPU shader ALUs for video encoding/decoding. They stopped doing that because power efficiency is worse than custom silicon that’s now in all modern GPUs, even in mobile SoCs.
- p0nce 7y ago
  Please don't confuse "x265" and "H.265 / HEVC"