10 ms·
High Quality Video Encoding at Scale
- spdustin 11y agoIt's a fascinating look into how they produce multiple encoded formats from source video, but something stuck out at me... One feature in their ingestion system is meant to reject source inputs that would lead to a poor viewer experience. This doesn't seem to include interlaced NTSC video as a "poor experience" metric. I'm willing to bet that the majority of the screens on which Star Trek: Deep Space Nine is played are progressive scan, yet that show is interlaced at NTSC resolution. In cases like this, where the source material is analog and, I would have to assume, not available in progressive scan, is there a technical reason why Netflix doesn't de-interlace the source before encoding? DS9 seems big enough of a catalog to be worth encoding in a less jarring scan rate. Not complaining, mind you. The DVDs are interlaced too. Seems that the original recordings were NTSC only. Honestly curious if anyone has any insight.
- castell 11y agoThe probably used DS9 DVDs as source. In fact DVD rental was their original core business.
- jfb 11y agoIt's possible, but the way the rights are assigned, they more than likely went back to tape and had them recaptured.
- dbcooper 11y agoAre you sure they aren't 24p that's been flagged as NTSC interlaced but is actually converted to 30p via 3:2 telecine?
- speakeron 11y agoTNG was like this (sourced from a video version and with a very nasty interlace look) on Netflix up to a couple of months ago. Now they stream the magnificent remastered blu-ray HD version which is sourced from the original film with fresh CGI since the original CGI was tied to the video. It's not clear whether DS9 will get the same treatment.
- dbcooper 11y agoI prefer to watch SD rips from the TNG blu-rays. The merciless clarity of 1080p just kills the magic.
- Keyframe 11y agoI agree. Remastered TNG, in 1080p, looks like a theater-based teleplay.
- knodi123 11y agoat least it's not HFR. I was never able to achieve immersion in the hobbit movies, because I felt like I was watching people on a stage.
- danjc 11y agoI had precisely that problem and this is the first time I've heard someone else mention it.
- superuser2 11y agoYES! I'm insanely sensitive to this too. I have to interrupt group TV watching to play around with the TV's menus and turn off the "automatically simulate 60fps" setting that so many Smart TVs have on by default these days. My mother can't tell the difference at all. I'm pretty sure this is just something that only a subset of the population notices enough to care about.
- Bognar 11y agoThe simulated 60 FPS (or higher) are significantly worse than just 60 FPS source material. TVs attempt to interpolate between the frames automatically and it just screws everything up.
- 11y ago
- endurancerobot 11y agoI’m George Fomitchef and it will be very useful to me to do new acquaintances to people who are interested in high technologies I hope my activity will interests you. I am engaged in high technologies,mainly, lasers, diode laser engravers, telepresence robots and robotized systems and other. And here is a video wich will make a complete idea of what I am engaged. "Endurance project presentation" https://www.youtube.com/watch?v=AzRX1NEYotM&feature=youtu https://www.youtube.com/watch?v=AzRX1NEYotM&feature=youtu. My website http://endurancerobots.com/about-endurance/ http://endurancerobots.com/about-endurance/ where you can find all information. I wrote many articles, and here is the last one ,about A Multifunctional Engraving Machine Endurance Wanhao DuPlicator i3. Maybe it will be interesting to you. https://www.linkedin.com/pulse/multifunctional-engraving-machine-endurance-wanhao-i3-fomitchev https://www.linkedin.com/pulse/multifunctional-engraving-mac.... If you will have any questions i'll answer on them with great pleasure! You can write me on this mail adress gf@endurancerobots.com Thank you and Good luck!
- jfb 11y agoIf size of catalog trumps video quality, you roll with NTSC if NTSC is the best you can get. Deinterlacing is extremely tricky, and if you can deliver interlaced video to playback devices, you're often better off punting.
- lmm 11y agoIf you deinterlace an interlaced source before encoding you either discard information or store twice as much. And potentially people watching on an interlaced screen lose data - not all deinterlace/interlace pairings roundtrip cleanly. Better to store it in the original format, and then it can go through one round of deinterlacing on playback for those screens that need it, and none for those that don't. (Also deinterlacing approaches get better over time - if a particular episode entered their catalogue 10 years ago and was deinterlaced using the state of the art approach at the time, it would look much worse than a modern deinterlace)
- jfb 11y agoOne of the overriding constraints on these sorts of video pipeline tasks is touching the input pixels as few times as possible. Every lossy transformation you do (cropping, scaling, color correction, transcoding) potentially introduces defects; and that's assuming that your lossless transformations (repackaging container formats, for instance) are bug-free. I'd love to see some figures from Netflix' QC team; I bet at their scale they see all kinds of insane edge case problems.
- izacus 11y agoUhm, no, due to the way modern video formats work, you don't store twice as much data - on the contrary, H.264 and similar modern formats are significantly more efficient at storing progressive (including deinterlaced) video than equivalent interlaced stream.
- lmm 11y agoNope. Modern (and even ancient) video codecs can store interlaced data just as efficiently as progressive data - how could it be otherwise? But when you deinterlace e.g. a 30 frames per second interlaced source, either you store the result as 60 frames per second (twice as much data), or you lossily downsample to 30 frames per second.
- Gelob 11y agoThat isn't Netflix's job. A post house delievers the raw .mov files to Netflix. The post house gets the files or tapes from the studio or whomever manages the libary for them (aka another post house). The post house has to comply to a set of standard's set forth by Netflix or it is automatically rejected. Most of the time things like no interlacing is in those requirements but also there can be exceptions if no other source material is available. iTunes works the same way. So say the post house that delievers to Netflix is only getting digital raw .movs and not the actual tapes, they may not be allowed to re-encode those or have access to the tapes to do the correct conversion.
- n0us 11y agoFascinating article. Forgive me for being ignorant about this, but how often do they need to encode video? I would think that it's only done when new titles are added to streaming, and once the video has been encoded into all the required formats they would be done with it. Sure, there is a lot of video content out there to be encoded but it isn't unlimited. Is serving the content and providing the recommendation engine to users at scale not a greater challenge than encoding the video?
- kyriakos 11y agoIts another one of their challenges. Time is an important factor, encoding high quality video is time consuming, multiply that by the many bitrates and codecs it could mean that they need to delay their content availability by days.
- jfb 11y agoWhen I built a similar system (for a large consumer electronics company) we built parallel paths for high-priority content; we'd intelligently split the incoming mezzanine and distribute the 90-120s chunks to a farm of systems, while also completing the multi-pass encodes. When the latter finished, the system would swap them. Because of the business model, we never ran this in full production mode, but it was built and ready to go.
- mkagenius 11y agoYes, encoding needs to be done only once. Generally it would take 2-3x of original duration of video to encode a source into a 1080p, so I am not sure why they take full 1 day? unless they do each bitrate serially which I think is not as hard to parallelize as it is to parallalize single bit rate by chunking. Yes, I believe serving is lot harder, but serving is almost a solved problem since people are dealing with for long time.
- mkaufmann 11y agoFor x264 that is true, HEVC which is also mentioned is much slower. For a 4k source transcoding can take more than a second per frame. For a normal movie this can quickly result in encoding times of more than a day. Another problem is that you have to encode the movie for each codec profile times the number of different bitrates per profile. The article mentions four profiles (VC1, H.264/AVC Baseline, H.264/AVC Main and HEVC) and bitrates ranging from 100 kbps to 16 Mbps. Assuming now there are 20 different bitrates per code you already get 4*20 => 80 encoded copies per source. But of course this can be solved by parallelism.
- mkagenius 11y agoInterestingly, not even a single mention of ffmpeg.
- doppel 11y agoI was wondering the same thing. I find it highly unlikely that they are doing this processing with in-house created tools, although their limited set of input and output codecs shrinks this down from "practically impossible" to "improbable".
- legulere 11y agoI guess using ffmpeg commercially is hard because of patent issues.
- MichaelGG 11y agoI would be surprised if Netflix cannot pay the patent fees. The ffmpeg page mentions that commercial ffmpeg users end up paying the groups like MPEG LA. In general, I'm guessing patent owners don't like saying "no, we don't want to accept your money". https://www.ffmpeg.org/legal.html https://www.ffmpeg.org/legal.html
- astrange 11y agoIt's possible their AVC encoder is not x264, and their VC1 encoder might be from Microsoft, in which case it would be a pretty modified ffmpeg. And some of the input formats might not be using it either. But you can be sure it's involved somewhere, since there's no other ProRes decoder that works on Linux!
- coalescence 11y agoFFmpeg supports Prores decode and encode? FFMBC for sure last time I checked
- garblegarble 11y agoFFmpeg does, it has different decoder implementations with their own quirks (btw, moot AFAIK because I think they use EyeIO): # ffmpeg -codecs | grep prores ffmpeg version 2.8.1 Copyright (c) 2000-2015 the FFmpeg developers built with Apple LLVM version 7.0.0 (clang-700.1.76) configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libtheora --enable-libschroedinger --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-indev=jack --disable-outdev=xv --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --enable-vda --enable-videotoolbox --arch=x86_64 --enable-yasm --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libsmbclient --enable-nonfree --enable-libfdk-aac --enable-libfaac libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100 DEVIL. prores Apple ProRes (iCodec Pro) (decoders: prores prores_lgpl ) (encoders: prores prores_aw prores_ks )
- Keyframe 11y agoHas Netflix started with HEVC h.265 support? I see they mention HEVC briefly.
- virtuallynathan 11y agoI think their 4K streaming to TVs uses HEVC.
- JustSomeNobody 11y agoInteresting post. Netflix's posts usually are. And then you scroll to the comments on the page...
- camperman 11y agoI'm envious of these capabilities. I wrote the backend for a streaming service that streams about forty channels to a quarter of a million mobile users in Africa. We get source video from our content providers that has given me grey hair. The problem is that I simply can't spin off the encoding and source quality checks to the cloud because of bandwidth costs here. So I do the quality checking and compression on local servers and then upload the compressed output to the servers. I'd kill for a 100 megabit line at a decent price.
- whatever_dude 11y agoAs someone who encodes video for work with a ton of ffmpeg-based batch files, this article makes me feel like a child.
- Splines 11y agoI've done a bunch of playing with ffmpeg at home, and I imagine the tech stack is probably similar at Netflix, at least for the Source-->Chunk, Chunk-->Assemble, and Assembled-->Encode steps. The validation done during all of these is interesting. Netflix's early years are probably exactly like what you're doing - single file in, transcode, single file out and deploy. Chunking the pieces up is clever. Getting it right must have been challenging. How do you write an oracle for something that complex?
- jfb 11y agoHow do you write an oracle for something that complex? You start with a much smaller problem and incrementally build up.
- contingencies 11y agoI personally worked on a similar system for a major mobile video solutions provider six years ago: none of this is particularly new.