4 ms·

As an aside (and a noob question), how many requests can a single server handle? I know frameworks such as Node can do handle many more connections when compare

As an aside (and a noob question), how many requests can a single server handle? I know frameworks such as Node can do handle many more connections when compared to something like Django but what would be a reasonable ballpark? As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms. Any good article/book I can read to get more insight regarding this?

falcolas 9y ago
I've personally handled hundreds of requests per second using Django behind Nginx on a m3.medium EC2 instance. The trick to scaling like this is to take full advantage of caching, and the cores available to you. With Django (and Node), this means spinning up multiple instances on a single server (one per available core). With even moderate caching (say, 1m), you can saturate your VM's bandwidth before running out of CPU or memory. Ultimately, you will always have a bottleneck. All you can do is identify the bottleneck (via load testing) and address it. If it's network (and you're in Amazon), you have to scale vertically until they remove that limit, or horizontally if you're already on unrestricted instances. If it's the CPU, you'll need to be more aggressive at caching completed computations, or work in a language with less overhead. If it's memory, start evaluating the tradeoffs between CPU and memory. If it's your DB, start investigating why it's slow (it's probably a poor index or configuration; DBs on decent hardware can handle millions of requests per second before degrading). I don't have any book recommendations; I just recommend creating something and find its limits with a simple load test tool. Then figure out your bottleneck and fix that. Rinse and repeat, and learn.
fweespeech 9y ago
Its too application centric to be honest to give good generics unless you've built something similar before. > As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms. An ecommerce site can be nearly 100% cached except for cart/session/search data. This gives it a massive advantage over everything else (since you can load the non-cached data via js and its relatively tiny). I think the last time I benchmarked one of our front end servers it was ~500 r/s on 2 cores for our search page. You really aren't going to know until you try it and benchmark it yourself tho.

falcolas 9y ago

I've personally handled hundreds of requests per second using Django behind Nginx on a m3.medium EC2 instance. The trick to scaling like this is to take full advantage of caching, and the cores available to you. With Django (and Node), this means spinning up multiple instances on a single server (one per available core). With even moderate caching (say, 1m), you can saturate your VM's bandwidth before running out of CPU or memory. Ultimately, you will always have a bottleneck. All you can do is identify the bottleneck (via load testing) and address it. If it's network (and you're in Amazon), you have to scale vertically until they remove that limit, or horizontally if you're already on unrestricted instances. If it's the CPU, you'll need to be more aggressive at caching completed computations, or work in a language with less overhead. If it's memory, start evaluating the tradeoffs between CPU and memory. If it's your DB, start investigating why it's slow (it's probably a poor index or configuration; DBs on decent hardware can handle millions of requests per second before degrading). I don't have any book recommendations; I just recommend creating something and find its limits with a simple load test tool. Then figure out your bottleneck and fix that. Rinse and repeat, and learn.

fweespeech 9y ago

Its too application centric to be honest to give good generics unless you've built something similar before. > As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms. An ecommerce site can be nearly 100% cached except for cart/session/search data. This gives it a massive advantage over everything else (since you can load the non-cached data via js and its relatively tiny). I think the last time I benchmarked one of our front end servers it was ~500 r/s on 2 cores for our search page. You really aren't going to know until you try it and benchmark it yourself tho.