5 ms·
How Netflix keeps its data infrastructure cost-effective
- joebob42 4y agoIsn't YouTube operating at a larger scale? I'd think certainly in terms of storage and unique bytes served, maybe comparable in terms of total bytes served? Or maybe I'm way underestimating the size of the Netflix library, but my impressions is it extremely tiny by comparison to YouTube.
- ksec 4y agoYes. Netflix's library is tiny by comparison. The article made it as if it is huge. It is so small to the point Netflix could afford to re-encode their entire library every few months when they have new and better encoder or quality measurement matrix. In the end this read like a marketing piece for AccelData. If there is one thing that Netflix technically excel, it is their 800Gbps Serving Appliance [1] with FreeBSD. ( I cant wait until they do 1.6Tbps ) [1] https://news.ycombinator.com/item?id=32519881 https://news.ycombinator.com/item?id=32519881
- isoprophlex 4y ago> It is so small to the point Netflix could afford to re-encode their entire library every few months when they have new and better encoder or quality measurement matrix. I wish they did, their current encoding for some shows is absolutely atrocious. I don't know the terminology for what I'm seeing, but for example in Stranger Things, it seems like the brightness and contrast is boosted for every n-th frame. In general, smooth motion seems to get a stuttering quality whenever Netflix touches it. I don't observe this so much with other video sources on the same device...
- hi_herbert 4y agoI regularly have atrocious transient 144p like compression artefacts and lighting absurdities using Netflix both in chrome and via their native h265 app. It is pathetic. Tangent if they really cared about user experience and Ecology they would implement h265 and h266 support in chromium (aka a mere function call to the embedded ffmpeg). H266 is revolutionary but it's adoption is of zero because of AV1 lobbyists a bit like so called environmentalists once launched a RPG rocket on the superphenix thorium mankind saving reactor.
- ksec 4y ago>H266 is revolutionary but it's adoption is of zero Adoption on the Web. Most broadcasting companies are getting ready to move to H.266. I wish we could get a JPEG XL based Video codec.
- miniman1337 4y agojpeg-xs?
- hi_herbert 4y agoGood point I hope is true but I was also mostly referring to the inexistant (not even in roadmap) hardware acceleration support. > I wish we could get a JPEG XL based Video codec. Well i'm curious about that but FLIF, the predecessor (and maybe XL too) is based on MANIAC which is a variation of H264 encoding.. https://en.wikipedia.org/wiki/Free_Lossless_Image_Format#:~:text=compression%2C%20FLIF%20uses-,MANIAC,-(Meta%2DAdaptive%20Near https://en.wikipedia.org/wiki/Free_Lossless_Image_Format#:~:...
- ksec 4y ago>inexistant (not even in roadmap) hardware acceleration support. Oh they are definitely on the roadmap ( Qualcomm and Mediatek ). I think a lot of people forget there are long lead time in hardware development. Especially with Mobile where energy usage is a key issue. The conformance test of H.266 only finished in mid 2021 if I remember correctly and continued being refined in 2022. If you are going to test your silicon in 2021, your earliest chance of any mobile solution with VVC is going to be 2023. I would guess both AV1 and VVC lands in Qualcomm SoC in ~2023. MediaTek and Apple coming in 2024.
- hi_herbert 4y agoI'm pretty sure you can do that with Linux. The hypothetical BSD networking advantages seems more and more obscolete to me, missing on most modern advances such as e.g MTCP support https://www.multipath-tcp.org/ https://www.multipath-tcp.org/ It's a basic gradient about human resources.
- dilyevsky 4y agoThis switch has throughput of 3.2Tbps and runs linux so evidently doable with hardware offload https://store.nvidia.com/en-us/networking/store/product/MSN2100-CB2RO/NVIDIAMSN2100CB2ROSpectrum100GbE1UOpenEthernetSwitch/ https://store.nvidia.com/en-us/networking/store/product/MSN2...
- hi_herbert 4y ago
- _8j50 4y agoThey probably meant the volume of video served. Youtube is a lot but people don't binge watch youtube as much and certainly not in UHD and similar formats.
- NetOpWibby 4y agoSeems like the original article should be posted instead https://netflixtechblog.com/byte-down-making-netflixs-data-infrastructure-cost-effective-fee7b3235032 https://netflixtechblog.com/byte-down-making-netflixs-data-i...
- ZiiS 4y agoYoutube must get more new videos every day maybe every hour then Netflix has ever had.
- konfusinomicon 4y agoit's a mind boggling amount of bytes. how do the data centers keep up? could there be a scenario where data creation outpaces the rate at which storage medium manufacturing and installation occurs?
- bluedino 4y agoI wonder how many UPS trucks full of hard drives Youtube uses in a day.
- antisthenes 4y agoCheap tape storage, cold storage for videos with 0 views, good compression algorithms, economies of scale with HD manufacturers. But I imagine it's been a challenge, since video sizes have grown considerably as well. Hence why new Youtube algos heavily favor videos that can be easily monetized and thus profitable for them.
- julienreszka 4y agoYouTube and amazon are much larger the claims are just plain false
- Havoc 4y ago>largest video library that is by some estimates almost four times bigger than its closest competitor. Most numbers I've seen suggest prime is significantly larger. (Though in my opinion its quantity over quality unfortunately).
- bombcar 4y agoPrime also has insane amounts of purchaseable and rental shows and movies, and continues to serve purchased shows and movies that are otherwise no longer available.
- arnvald 4y agoYeah, this is a 2022 article that relies on sources from 2020. Prime is now larger than Netflix, and other competitors are not 4x smaller anymore. YouTube is on a completely different level
- dilyevsky 4y ago> from the largest video library that is by some estimates almost four times bigger than its closest competitor. Their source says Netflix has 50K titles which means you can probably store entirety of it on the edge.
- blackoil 4y agoThis is a sad thread. I expected some technical discussion with alternative approaches and some other examples. Nope, only discussion is about who is biggest.
- remram 4y agoUnfortunately, apart from the egregious lie of the very first sentence ("Netflix is unquestionably the largest video provider in the world"), the rest of the article is not that interesting. They seem to leverage off-the-shelf tools and platforms like AWS and Druid, nothing more. This reads like an ad for Acceldata, and not a good one.
- Bluecobra 4y ago> Netflix’s infrastructure is entirely in the cloud This doesn’t sound accurate to me. What about their 400Gb/800Gb FreeBSD boxes serving up content? Surely those are in installed in Netflix managed collocation facilities? Unless the author is suggesting that any on-prem hardware is a ‘private’ cloud. :barf:
- bluedino 4y agoISP facilities, but yes
- beckingz 4y agoNetflix was famously "All in" on AWS for quite a while.
- synicalx 4y agoAs my AWS account manager back then LOVED to tell me every time we spoke
- VirusNewbie 4y agoYou misunderstood the term infrastructure here. Almost no one considers edge services “infra”.
- Bluecobra 4y agoI disagree, as a network engineer the edge devices (e.g. firewalls/routers) that I manage in my company are 100% internal infrastructure. If you go by Netflix's own architecture diagrams, they do have a local presence at colo facilities: https://openconnect.netflix.com/en/#sample-architectures https://openconnect.netflix.com/en/#sample-architectures If one of these appliances installed in an ISP's cabinet dies, content to end users will get served directly by an appliance within Netflix's infrastructure.
- maximus-decimus 4y agoI imagine their compression looking like complete trash compared to the competition helps with cost saving. Seriously, I find it funny when youtubers talk about youtube 4k looks like trash compared to the real thing, because it looks orders magnitude better than Netflix. Seriously, 720p on Google Play looks better than 4k on Netflix. Note: this mostly applies to dark scenes. bright scenes are not nearly as bad, but still don't look as good as youtube.
- Scaevolus 4y agoHere's a theory that it's all because they're using a deeply flawed quality metric (VMAF) to evaluate how much bitrate is needed. https://twitter.com/jonsneyers/status/1573371624132419585 https://twitter.com/jonsneyers/status/1573371624132419585 The VMAF-approved banding seems particularly telling!
- VirusNewbie 4y agoI find Netflix 4k looks way better on my new TV compared to Amazon and HBO. Haven't compared to youtube, I'll have to check it out.
- javier2 4y agovp9 on youtube actually looks very good, especially at lower bitrates.
- maximus-decimus 4y agoOh I agree it looks great, at least to me. I imagine those content producers like LTT and SavageGeese hate it only because they're seeing the original files in something like uncompressed 8k on color accurate monitors. But stuff like this intro just look incredibly good to me : https://www.youtube.com/watch?v=Yeh4rsXhsUg https://www.youtube.com/watch?v=Yeh4rsXhsUg
- ec109685 4y agoThis is a content marketing from acceldataio summarizing of the original blog post, which should be linked to instead: https://netflixtechblog.com/byte-down-making-netflixs-data-infrastructure-cost-effective-fee7b3235032 https://netflixtechblog.com/byte-down-making-netflixs-data-i...