6 ms·

> Video encoding is 70% of Netflix’s computing needs, running on 300,000 CPUs in over 1000 different autoscaling groups. This feels crazy to me. They'll have t

> Video encoding is 70% of Netflix’s computing needs, running on 300,000 CPUs in over 1000 different autoscaling groups. This feels crazy to me. They'll have to encode all their content into 15-20 different formats for various devices/bitrates, but surely they cache the output for each of those so it'll only have to be done once for each show/movie. Compare that to serving content, which happens thousands or millions of times for each stream. This isn't Youtube, there's not much of a "long tail" of content that's only watched a few times but has to be encoded anyway.

ubercore 9y ago
I'm assuming because most of the content serving happens in Open Connect [0]. And while there may be a ton of server work involved in billing, api, auth, analytics, etc, it probably pales in comparison to the raw transcoding if you take actually serving the encoded files out to clients. From the article, "9,570 different video, audio, and text files" are generated for one show, from 8k resolution source files. Seems reasonable to me. [0] https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/
mgummelt 9y ago
> Stranger Things season 2 is shot in 8K and has nine episodes. The source files are terabytes and terabytes of data. Each season required 190,000 CPU hours to encode. High resolution video is larger than you might expect.
- abtinf 9y ago
  Is there such a thing as a video, image, or audio format that is inefficient for viewing, but is radically more efficient for transcoding? Such a technique would be useful for a number of applications.
  - aidos 9y ago
    Interesting idea. Could there be a way of making a format requires some sort of extra work to access? So sort of like a bitcoin setup where the local device is a miner (but actually doing useful work) and it gets access to the video as part of that computation. Effectively the content creator would trade computation for content. I'm sure it's totally impossible in theory and in practice.
  - lathiat 9y ago
    In many ways - older video codecs. Newer video codecs gain more compression by doing more and more complicated encoding and decoding. Trading CPU for Size.
  - Retric 9y ago
    Sure, Raw video is one example of that, it takes insane bandwidth and storage, but can load it from SSD fast enough that's not an issue. Really, codecs need to be closely related before encoding X helps make encoding Y.
    - tomjen3 9y ago
      To compare: my camera uses 25 megabytes of storage for each single raw frame. I have no idea how much raw sound takes up, but you would have to move around a lot of SSDs to use that.
      - deleted 9y ago
        [deleted]
      - Retric 9y ago
        It can get worse 100+ Megapixels at 60 fps: http://www.forzasilicon.com/2014/04/forza-silicon-introduces-advanced-cmos-modular-video-camera-platform/ http://www.forzasilicon.com/2014/04/forza-silicon-introduces... Though, I have no idea what network connections they would use. I mean even in black and white this would saturate a 100Gbps link.

ubercore 9y ago

I'm assuming because most of the content serving happens in Open Connect [0]. And while there may be a ton of server work involved in billing, api, auth, analytics, etc, it probably pales in comparison to the raw transcoding if you take actually serving the encoded files out to clients. From the article, "9,570 different video, audio, and text files" are generated for one show, from 8k resolution source files. Seems reasonable to me. [0] https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/

mgummelt 9y ago

> Stranger Things season 2 is shot in 8K and has nine episodes. The source files are terabytes and terabytes of data. Each season required 190,000 CPU hours to encode. High resolution video is larger than you might expect.

abtinf 9y ago
Is there such a thing as a video, image, or audio format that is inefficient for viewing, but is radically more efficient for transcoding? Such a technique would be useful for a number of applications.
- aidos 9y ago
  Interesting idea. Could there be a way of making a format requires some sort of extra work to access? So sort of like a bitcoin setup where the local device is a miner (but actually doing useful work) and it gets access to the video as part of that computation. Effectively the content creator would trade computation for content. I'm sure it's totally impossible in theory and in practice.
- lathiat 9y ago
  In many ways - older video codecs. Newer video codecs gain more compression by doing more and more complicated encoding and decoding. Trading CPU for Size.
- Retric 9y ago
  Sure, Raw video is one example of that, it takes insane bandwidth and storage, but can load it from SSD fast enough that's not an issue. Really, codecs need to be closely related before encoding X helps make encoding Y.
  - tomjen3 9y ago
    To compare: my camera uses 25 megabytes of storage for each single raw frame. I have no idea how much raw sound takes up, but you would have to move around a lot of SSDs to use that.
    - deleted 9y ago
      [deleted]
    - Retric 9y ago
      It can get worse 100+ Megapixels at 60 fps: http://www.forzasilicon.com/2014/04/forza-silicon-introduces-advanced-cmos-modular-video-camera-platform/ http://www.forzasilicon.com/2014/04/forza-silicon-introduces... Though, I have no idea what network connections they would use. I mean even in black and white this would saturate a 100Gbps link.