6 ms·

To stream everywhere, Netflix encodes each movie 120 times

43 points by timf 14y ago

jedberg 14y ago
Using sometimes 1000's of EC2 instances (for a bunch of movies, not just one).
- signa11 14y ago
  this being an ideal application of simd kind of parallelism, i am wondering of a farm of gpu's might be much cheaper (and faster) rather than a general purpose cpu ?
  - dlgeek 14y ago
    AWS has farms of GPUs you can rent by the hour - http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/instance-types/ - Look at "Cluster GPU Instances"
  - jedberg 14y ago
    digeek is right -- they have GPU instances too. We don't actually use the GPU instances yet, but I think I heard we're moving over to them (need to check on that).
  - nodata 14y ago
    The article "The wretched state of GPU transcoding" disagrees: http://www.extremetech.com/computing/128681-the-wretched-state-of-gpu-transcoding http://www.extremetech.com/computing/128681-the-wretched-sta...
ghshephard 14y ago
If this is true, then this should immediately put to rest the explanation (that I'll admit I bought into) that only one video format will win out (H.264 vs Ogg Theora/VP8 is the major discussion point) - as video streaming companies would be unwilling to re-encode their content for more than one format. Apparently they are quite comfortable encoding as many as 120 instantiations of their videos.
- cma 14y ago
  Netflix has a limited catalog... YouTube has a much larger one by many orders of magnitude. However they could still lazily trans code on demand (and cache the result).
  - gillianseed 14y ago
    >Netflix has a limited catalog... YouTube has a much larger one by many orders of magnitude. Youtube offer their videos in vp8 along with h264.
- warfangle 14y ago
  Storage is cheap. Edge connections are not. It makes sense to have 120 instantiations of the videos if that huge multitude is necessary to optimally serve the plethora of mobile devices out there: you don't want to be negotiating for the worst bitrate out of a 720p encode if the device you're delivering too is only capable of 480x320 and can only gain access to 3G networks.
- jedberg 14y ago
  In fact, it is not unheard of for us to re-encode the entire catalog when a new, better format or codec comes along, or we add support for a new device.
  - blhack 14y ago
    Huh. You guys must have quite the rendering farm for that, eh?
    - jedberg 14y ago
      EC2. 1000s of instances. :)
      - toomuchtodo 14y ago
        Spot pricing I hope! =)
mrb 14y ago
A friend of mine, Baptiste Coudurier, is the person at Hulu who manages the transcoding of all their videos. He told me they also transcode a video to multiple versions to accommodate different streaming devices and bandwidth characteristics. I think it only makes sense to lazily-transcode videos on demand (and cache them) when you have millions of videos (eg. Youtube). Netflix and Hulu have much smaller catalogs, so they can afford to transcode ahead of time, and it simplifies their processes.
- morsch 14y ago
  I wonder if there's some lengthy preprocessing step that's common to transcoding regardless of the target resolution/quality. That way you could precompute and store the intermediary format and finish of the encoding to an arbitrary format as required.
  - keeperofdakeys 14y ago
    For two-pass encoding, the video is first scanned to assess complexity, so a second encode pass can work out which frames require a higher bitrate to get the same quality. However this is not used that often, in favour of a quantiser approach, which uses a number to indicate target quality. The encoder then changes the bitrate depending on the frames complexity and the target quality/quantiser. The quantiser approach is much more common because it's easier to specify an output quality that is independent of resolution. As for the actual encoding, nearly all the computation is very specific to the output resolution/quality. Luckily encoding multiple videos can be spread over many machines, and encoders can take advantage of multiple cores.
machrider 14y ago
...everywhere except Linux.
- warfangle 14y ago
  This isn't due to transcode issues. They use Silverlight to stream to full-function PCs (be it OSX or Windows). Microsoft hasn't made Silverlight available to the greater GNU community, so it's pretty obvious that they wouldn't distribute to Linux: their contracts with content-holders probably hold them to constricting DRM demands on the videos streamed to devices capable of ripping them off. Don't blame Netflix for it. Blame the arduous contracts which must be negotiated in order for Netflix to carry the content it does.
  - tkorri 14y ago
    They have a working Android application which uses native linux code to handle the DRM. So Silverlight not beign supported on Linux is not the real issue here.
    - shardling 14y ago
      Right, it's the contracts, exactly like the parent post said...
      - winter_blue 14y ago
        The issue is it not being supported on regular desktop Linux (excl. Chrome OS).
- drivebyacct2 14y ago
  It's all about PlayReady. PlayReady can have backings that tie into hardware protection. That is why it's available for devices that run on Linux - because MS has licensed PlayReady for those implementations. I suspect most, if not every single such device, have hardware support for DRM.
- nnnnni 14y ago
dave1010uk 14y ago
As technology progresses we can probably assume that at some point in the future virtually all devices in use will have fast enough network connections and decoders to play videos that are the highest quality the human eye can see. At this point it would only be worth encoding in this "highest" quality. Of course, by this time we may have new screen technologies such as holographic displays.
- lucian1900 14y ago
  It will likely still be cheaper to send only as little as will actually be rendered.
- winter_blue 14y ago
  Yes I can see you mean a time and day when everyone will gigabit connections and what not. But then you're forgetting: poverty hasn't quite abandoned us yet, has it? Even with all of the modern technology and what not -- people are still dying of hunger. We first need to start with these base human needs. Elevate the standard of living of the people on the other side of digital (and economic) divide, and bring them up to standard of an average first-world-country person.
bambax 14y ago
At the end of the post there's this quote: > Simply put, content owners who can deliver quickly and without error are getting more licensing revenue from Netflix. Really? Can Netflix afford to be picky with content owners? It seems their main problem is the limited size of their online catalog...? They should take anything they can get! I just finished watching "Elite Squad 2"; it's such a great movie (really, it is: see http://www.imdb.com/title/tt1555149/ http://www.imdb.com/title/tt1555149/) that I wanted to watch the first one, but it's "only available on DVD". Why, why, why?
- CaioAlonso 14y ago
  Exactly. When The Pirate Bay has an infinite catalog, they can't afford to be picky.
  - bambax 14y ago
    The Pirate Bay doesn't have everything, but yes, it helps... ;-)
- loceng 14y ago
  Because it's a way to upsell you.
  - jedberg 14y ago
    That's not correct. It's because licensing movies on DVD is monumentally cheaper than licensing content for online streaming. One requires a deal with the studio, the other just requires picking up a DVD somewhere (see http://en.wikipedia.org/wiki/First-sale_doctrine http://en.wikipedia.org/wiki/First-sale_doctrine).
    - loceng 14y ago
      The studio is upselling you - not the middleman - because you'll have to go out somewhere else and spend more money to buy the first version.
- 14y ago

43 points by timf 14y ago

jedberg 14y ago

Using sometimes 1000's of EC2 instances (for a bunch of movies, not just one).

signa11 14y ago
this being an ideal application of simd kind of parallelism, i am wondering of a farm of gpu's might be much cheaper (and faster) rather than a general purpose cpu ?
- dlgeek 14y ago
  AWS has farms of GPUs you can rent by the hour - http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/instance-types/ - Look at "Cluster GPU Instances"
- jedberg 14y ago
  digeek is right -- they have GPU instances too. We don't actually use the GPU instances yet, but I think I heard we're moving over to them (need to check on that).
- nodata 14y ago
  The article "The wretched state of GPU transcoding" disagrees: http://www.extremetech.com/computing/128681-the-wretched-state-of-gpu-transcoding http://www.extremetech.com/computing/128681-the-wretched-sta...

ghshephard 14y ago

If this is true, then this should immediately put to rest the explanation (that I'll admit I bought into) that only one video format will win out (H.264 vs Ogg Theora/VP8 is the major discussion point) - as video streaming companies would be unwilling to re-encode their content for more than one format. Apparently they are quite comfortable encoding as many as 120 instantiations of their videos.

cma 14y ago
Netflix has a limited catalog... YouTube has a much larger one by many orders of magnitude. However they could still lazily trans code on demand (and cache the result).
- gillianseed 14y ago
  >Netflix has a limited catalog... YouTube has a much larger one by many orders of magnitude. Youtube offer their videos in vp8 along with h264.
warfangle 14y ago
Storage is cheap. Edge connections are not. It makes sense to have 120 instantiations of the videos if that huge multitude is necessary to optimally serve the plethora of mobile devices out there: you don't want to be negotiating for the worst bitrate out of a 720p encode if the device you're delivering too is only capable of 480x320 and can only gain access to 3G networks.
jedberg 14y ago
In fact, it is not unheard of for us to re-encode the entire catalog when a new, better format or codec comes along, or we add support for a new device.
- blhack 14y ago
  Huh. You guys must have quite the rendering farm for that, eh?
  - jedberg 14y ago
    EC2. 1000s of instances. :)
    - toomuchtodo 14y ago
      Spot pricing I hope! =)

mrb 14y ago

A friend of mine, Baptiste Coudurier, is the person at Hulu who manages the transcoding of all their videos. He told me they also transcode a video to multiple versions to accommodate different streaming devices and bandwidth characteristics. I think it only makes sense to lazily-transcode videos on demand (and cache them) when you have millions of videos (eg. Youtube). Netflix and Hulu have much smaller catalogs, so they can afford to transcode ahead of time, and it simplifies their processes.

morsch 14y ago
I wonder if there's some lengthy preprocessing step that's common to transcoding regardless of the target resolution/quality. That way you could precompute and store the intermediary format and finish of the encoding to an arbitrary format as required.
- keeperofdakeys 14y ago
  For two-pass encoding, the video is first scanned to assess complexity, so a second encode pass can work out which frames require a higher bitrate to get the same quality. However this is not used that often, in favour of a quantiser approach, which uses a number to indicate target quality. The encoder then changes the bitrate depending on the frames complexity and the target quality/quantiser. The quantiser approach is much more common because it's easier to specify an output quality that is independent of resolution. As for the actual encoding, nearly all the computation is very specific to the output resolution/quality. Luckily encoding multiple videos can be spread over many machines, and encoders can take advantage of multiple cores.

machrider 14y ago

...everywhere except Linux.

warfangle 14y ago
This isn't due to transcode issues. They use Silverlight to stream to full-function PCs (be it OSX or Windows). Microsoft hasn't made Silverlight available to the greater GNU community, so it's pretty obvious that they wouldn't distribute to Linux: their contracts with content-holders probably hold them to constricting DRM demands on the videos streamed to devices capable of ripping them off. Don't blame Netflix for it. Blame the arduous contracts which must be negotiated in order for Netflix to carry the content it does.
- tkorri 14y ago
  They have a working Android application which uses native linux code to handle the DRM. So Silverlight not beign supported on Linux is not the real issue here.
  - shardling 14y ago
    Right, it's the contracts, exactly like the parent post said...
    - winter_blue 14y ago
      The issue is it not being supported on regular desktop Linux (excl. Chrome OS).
drivebyacct2 14y ago
It's all about PlayReady. PlayReady can have backings that tie into hardware protection. That is why it's available for devices that run on Linux - because MS has licensed PlayReady for those implementations. I suspect most, if not every single such device, have hardware support for DRM.
nnnnni 14y ago

dave1010uk 14y ago

As technology progresses we can probably assume that at some point in the future virtually all devices in use will have fast enough network connections and decoders to play videos that are the highest quality the human eye can see. At this point it would only be worth encoding in this "highest" quality. Of course, by this time we may have new screen technologies such as holographic displays.

lucian1900 14y ago
It will likely still be cheaper to send only as little as will actually be rendered.
winter_blue 14y ago
Yes I can see you mean a time and day when everyone will gigabit connections and what not. But then you're forgetting: poverty hasn't quite abandoned us yet, has it? Even with all of the modern technology and what not -- people are still dying of hunger. We first need to start with these base human needs. Elevate the standard of living of the people on the other side of digital (and economic) divide, and bring them up to standard of an average first-world-country person.

bambax 14y ago

At the end of the post there's this quote: > Simply put, content owners who can deliver quickly and without error are getting more licensing revenue from Netflix. Really? Can Netflix afford to be picky with content owners? It seems their main problem is the limited size of their online catalog...? They should take anything they can get! I just finished watching "Elite Squad 2"; it's such a great movie (really, it is: see http://www.imdb.com/title/tt1555149/ http://www.imdb.com/title/tt1555149/) that I wanted to watch the first one, but it's "only available on DVD". Why, why, why?

CaioAlonso 14y ago
Exactly. When The Pirate Bay has an infinite catalog, they can't afford to be picky.
- bambax 14y ago
  The Pirate Bay doesn't have everything, but yes, it helps... ;-)
loceng 14y ago
Because it's a way to upsell you.
- jedberg 14y ago
  That's not correct. It's because licensing movies on DVD is monumentally cheaper than licensing content for online streaming. One requires a deal with the studio, the other just requires picking up a DVD somewhere (see http://en.wikipedia.org/wiki/First-sale_doctrine http://en.wikipedia.org/wiki/First-sale_doctrine).
  - loceng 14y ago
    The studio is upselling you - not the middleman - because you'll have to go out somewhere else and spend more money to buy the first version.
14y ago