37 ms·
> We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute. Most do
by bob1029 5y ago
> We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute.
Most don't. 1mm requests per minute is very pedestrian for a single vm in virtually all cases. 1mm per second is totally reasonable too if you are careful with a few things...
I genuinely believe you could put the literal public Netflix biz experience on a single VM. Account management, billing, preferences, view history, etc. The only pieces that need cloud scale are ddos mitigation, 4k video streams and media-dense static web content. Most businesses do not have strong need for these things.
- recuter 5y agoCorrectish in a sense: https://openconnect.netflix.com/en/appliances/#the-hardware https://openconnect.netflix.com/en/appliances/#the-hardware They throw one of these boxes at an ISP and interconnect. 4-6 year no touch reliability, couple hundred TB storage. Modern hardware is quite something. They will saturate 2x100GE. That's in the thousands of concurrent streams per box.
- Nextgrid 5y ago> Modern hardware is quite something. The one weird trick that cloud providers hate.
- charcircuit 5y agoWhat about Netflix's recommendation system and analytics?
- bob1029 5y agoCertainly not going to fit on the real-time production instance, but two computers isn't a whole lot more than one considering how many extra features we just added. There is a lot of elegance with this type of setup too. You can have your analytics system receive a synchronous replicated log from the production system (what else is it gonna analyze?), so it can also double as a manual failover site.
- karmakaze 5y agoHere's more of that context that paints a better picture. > First, it’s important to note that Enzymit’s use of cloud computing mainly entailed computationally intensive calculations for protein design. We do not have (yet) any public-facing applications that need to scale across multiple geographical zones and handle millions of requests per minute. Our primary use case is running CPU and GPU heavy analyses, and for that use case, we have found the IaaS/public cloud solution to be far from cost-effective in the long term.
- rhplus 5y ago1 million/sec is basically line speed on a 10Gbps link if each request is coming in at MTU of 1500 bytes. Sure, you might be able to push that much data through a VM on a test bench with well-behaved local clients, but you ain’t gonna be doing that rate once you add TLS, authz, logging, throttling, non-trivial serialization, non-trivial database access, A/B tests, metrics, fraud detection, recommendations, and everything else that makes an API like Netflix tick. Whether you do that all on one VM or split across service roles, you’re gonna be much more realistically in the range of 1000 rps per CPU core.
- bob1029 5y ago1500 bytes is a pretty big average payload size when you consider information theory and what actually must be communicated for this kind of business (on average). A user clicking "Watch later" on a video could theoretically be communicated in something as small as 64-bit integer for the user/session id, one for the command type, and another for the identity of the actual video. With serialization, padding, etc., you are still probably well under 50 bytes for this one event. Being sloppy with data throughout is certainly a good reason to need more pipes and servers. With enough discipline, you can process events at rates far exceeding 1 million per second with a single box and non-exotic network stack.
- TheCoelacanth 5y ago1500 bytes isn't even enough to send a list of video titles and thumbnail images for a single page of videos. If you imagine that the client already has a full database of all the available videos and metadata about them, then you could get by with tiny amounts of data, but that's not even close to the actual circumstances that Netflix operates under.