4 ms·

Curious, what would that many ffmpeg processes be doing at Netflix? I assume new VOD content gets encoded once per format, and the amount of new content added p

Curious, what would that many ffmpeg processes be doing at Netflix? I assume new VOD content gets encoded once per format, and the amount of new content added per day is not gigantic. Agree with the general premise, of course, if I've got 10 different videos encoding at once then I don't need additional efficiency because the CPU's already maxed out.

asveikau 3y ago
Probably a lot more than once when you consider that different devices have different capabilities, and that they might stream you different bitrates depending on conditions like your network capability, screen resolution, how much you've paid them.. You could also imagine they might apply some kind of heuristic to decide to re-encode something based on some condition... Like fine tune encoder settings when a title becomes popular. No idea if they do that, just using some imagination.
The_Colonel 3y ago
I assume they re-compress for each resolution / format, quite possibly they also have different bitrate levels per resolution. Potentially even variants tweaked for certain classes of device (in cases this is not already covered by combination of format/resolution/bitrate). I would also assume they re-compress with new advances in video processing (things like HDR, improved compression). Also, their devs likely want fast feedback on changes - I imagine they might have CI running changes on some standard movies, checking various stats (like SNR) for regressions. Everybody loves if their CI finishes fast, so you might want to compress even a single movie in multiple threads.
- sanitycheck 3y ago
  They'll be doing VBR encodes to DASH, HLS & (I guess still) MSS which covers the resolutions & formats... DRM will be what prevents high res content from working on some "less-trusted" platforms so the same encodes should work. (Plus a couple more "legacy" encodes with PIFF instead of CENC for ancient devices, probably.) New tech advances, sure, they probably do re-encode everything sometimes - even knocking a few MB off the size of a movie saves a measurable amount of $$ at that scale. But are there frequent enough tech advances to do that more than a couple of times a year..? The amount of difficult testing (every TV model group from the past 10 years, or something) required for an encode change is horrible. I'm sure they have better automation than anyone else, but I'm guessing it's still somewhat of a nightmare. Youtube, OTOH, I really can imagine having thousands of concurrent ffmpeg processes.
  - canucker2016 3y ago
    Why bring up assumptions/suppositions about Netflix's encoding process? Their tech blog and tech presentations discuss many of the requirements and steps involved for encoding source media to stream to all the devices that Netflix supports. The Netflix tech blog: https://netflixtechblog.com/ https://netflixtechblog.com/ or https://netflixtechblog.medium.com/ https://netflixtechblog.medium.com/ Netflix seems to use AWS CPU+GPU for encoding, whereas YouTube has gone to the expense of producing an ASIC to do much of their encoding. 2015 blog entry about their video encoding pipeline: https://netflixtechblog.com/high-quality-video-encoding-at-scale-d159db052746?gi=efeea4b4b5ac https://netflixtechblog.com/high-quality-video-encoding-at-s... 2021 presentation of their media encoding pipeline: https://www.infoq.com/presentations/video-encoding-netflix/ https://www.infoq.com/presentations/video-encoding-netflix/ An example of their FFmpeg usage - a neural-net video frame downscaler: https://netflixtechblog.com/for-your-eyes-only-improving-netflix-video-quality-with-neural-networks-5b8d032da09c https://netflixtechblog.com/for-your-eyes-only-improving-net... Their dynamic optimization encoding framework - allocating more bits for complex scenes and fewer bits for simpler, quieter scenes: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-video-encoding-optimization-framework-e19f1e3a277f https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v... and https://netflixtechblog.com/optimized-shot-based-encodes-now-streaming-4b9464204830 https://netflixtechblog.com/optimized-shot-based-encodes-now... Netflix developed an algorithm for determining video quality - VMAF, which helps determine their encoding decisions: https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652 https://netflixtechblog.com/toward-a-practical-perceptual-vi..., https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12 https://netflixtechblog.com/vmaf-the-journey-continues-44b51..., https://netflixtechblog.com/toward-a-better-quality-metric-for-the-video-community-7ed94e752a30 https://netflixtechblog.com/toward-a-better-quality-metric-f...
    - astrange 3y ago
      > Their dynamic optimization encoding framework - allocating more bits for complex scenes and fewer bits for simpler, quieter scenes: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v... and https://netflixtechblog.com/optimized-shot-based-encodes-now https://netflixtechblog.com/optimized-shot-based-encodes-now... This is overrated - of course that's how you do it, what else would you do? > Mean-squared-error (MSE), typically used for encoder decisions, is a number that doesn’t always correlate very nicely with human perception. Academics, the reference MPEG encoder, and old proprietary encoder vendors like On2 VP9 did make decisions this way because their customers didn't know what they wanted. But people who care about quality, i.e. anime and movie pirate college students with a lot of free time, didn't. It looks like they've run x264 in an unnatural mode to get an improvement here, because the default "constant ratefactor" and "psy-rd" always behaved like this.
banana_giraffe 3y ago
It's been reported in the past that Netflix encodes 120 different variants of each video they have [1] for different bitrates and different device's needs. And that was years ago, I wouldn't be surprised to learn it's a bigger number now. [1] https://news.ycombinator.com/item?id=4946275 https://news.ycombinator.com/item?id=4946275

asveikau 3y ago

Probably a lot more than once when you consider that different devices have different capabilities, and that they might stream you different bitrates depending on conditions like your network capability, screen resolution, how much you've paid them.. You could also imagine they might apply some kind of heuristic to decide to re-encode something based on some condition... Like fine tune encoder settings when a title becomes popular. No idea if they do that, just using some imagination.

The_Colonel 3y ago

I assume they re-compress for each resolution / format, quite possibly they also have different bitrate levels per resolution. Potentially even variants tweaked for certain classes of device (in cases this is not already covered by combination of format/resolution/bitrate). I would also assume they re-compress with new advances in video processing (things like HDR, improved compression). Also, their devs likely want fast feedback on changes - I imagine they might have CI running changes on some standard movies, checking various stats (like SNR) for regressions. Everybody loves if their CI finishes fast, so you might want to compress even a single movie in multiple threads.

sanitycheck 3y ago
They'll be doing VBR encodes to DASH, HLS & (I guess still) MSS which covers the resolutions & formats... DRM will be what prevents high res content from working on some "less-trusted" platforms so the same encodes should work. (Plus a couple more "legacy" encodes with PIFF instead of CENC for ancient devices, probably.) New tech advances, sure, they probably do re-encode everything sometimes - even knocking a few MB off the size of a movie saves a measurable amount of $$ at that scale. But are there frequent enough tech advances to do that more than a couple of times a year..? The amount of difficult testing (every TV model group from the past 10 years, or something) required for an encode change is horrible. I'm sure they have better automation than anyone else, but I'm guessing it's still somewhat of a nightmare. Youtube, OTOH, I really can imagine having thousands of concurrent ffmpeg processes.
- canucker2016 3y ago
  Why bring up assumptions/suppositions about Netflix's encoding process? Their tech blog and tech presentations discuss many of the requirements and steps involved for encoding source media to stream to all the devices that Netflix supports. The Netflix tech blog: https://netflixtechblog.com/ https://netflixtechblog.com/ or https://netflixtechblog.medium.com/ https://netflixtechblog.medium.com/ Netflix seems to use AWS CPU+GPU for encoding, whereas YouTube has gone to the expense of producing an ASIC to do much of their encoding. 2015 blog entry about their video encoding pipeline: https://netflixtechblog.com/high-quality-video-encoding-at-scale-d159db052746?gi=efeea4b4b5ac https://netflixtechblog.com/high-quality-video-encoding-at-s... 2021 presentation of their media encoding pipeline: https://www.infoq.com/presentations/video-encoding-netflix/ https://www.infoq.com/presentations/video-encoding-netflix/ An example of their FFmpeg usage - a neural-net video frame downscaler: https://netflixtechblog.com/for-your-eyes-only-improving-netflix-video-quality-with-neural-networks-5b8d032da09c https://netflixtechblog.com/for-your-eyes-only-improving-net... Their dynamic optimization encoding framework - allocating more bits for complex scenes and fewer bits for simpler, quieter scenes: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-video-encoding-optimization-framework-e19f1e3a277f https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v... and https://netflixtechblog.com/optimized-shot-based-encodes-now-streaming-4b9464204830 https://netflixtechblog.com/optimized-shot-based-encodes-now... Netflix developed an algorithm for determining video quality - VMAF, which helps determine their encoding decisions: https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652 https://netflixtechblog.com/toward-a-practical-perceptual-vi..., https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12 https://netflixtechblog.com/vmaf-the-journey-continues-44b51..., https://netflixtechblog.com/toward-a-better-quality-metric-for-the-video-community-7ed94e752a30 https://netflixtechblog.com/toward-a-better-quality-metric-f...
  - astrange 3y ago
    > Their dynamic optimization encoding framework - allocating more bits for complex scenes and fewer bits for simpler, quieter scenes: https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v https://netflixtechblog.com/dynamic-optimizer-a-perceptual-v... and https://netflixtechblog.com/optimized-shot-based-encodes-now https://netflixtechblog.com/optimized-shot-based-encodes-now... This is overrated - of course that's how you do it, what else would you do? > Mean-squared-error (MSE), typically used for encoder decisions, is a number that doesn’t always correlate very nicely with human perception. Academics, the reference MPEG encoder, and old proprietary encoder vendors like On2 VP9 did make decisions this way because their customers didn't know what they wanted. But people who care about quality, i.e. anime and movie pirate college students with a lot of free time, didn't. It looks like they've run x264 in an unnatural mode to get an improvement here, because the default "constant ratefactor" and "psy-rd" always behaved like this.

banana_giraffe 3y ago

It's been reported in the past that Netflix encodes 120 different variants of each video they have [1] for different bitrates and different device's needs. And that was years ago, I wouldn't be surprised to learn it's a bigger number now. [1] https://news.ycombinator.com/item?id=4946275 https://news.ycombinator.com/item?id=4946275