5 ms·
So this is what, a 2000 request/sec peak? Over 11 servers, that's like 200 requests/sec peak per frontend? The problem with scale-up is if you actually have to
by MichaelGG 12y ago
So this is what, a 2000 request/sec peak? Over 11 servers, that's like 200 requests/sec peak per frontend?
The problem with scale-up is if you actually have to get a few times larger, it becomes super expensive. But fortunately hardware is increasing so much that you can probably just get away with it now. There's probably a crossover point we're rapidly approaching where even global-scale sites can just do all their transactions in RAM and keep it there (replicated). I know that's what VoltDB is counting on.
- sklivvz1971 12y agoYes, the idea is to scale up as much as economically makes sense and only then scale out, but due to Moore's law we're still scaling up. Not everyone has google-like problems that are betters solved by a battery of cheap boxes.
- threeseed 12y agoActually most of the websites today are better solved by a battery of cheap boxes. You simply put a load balancer (Nginx/ELB/HAProxy etc) in front of a fleet of smaller web/application servers that dynamically scale depending on traffic. That way it is cost effective, far more reliable, easier to scale and you can tolerate DC outages better.
- mwsherman 12y agoAs with most systems, the data store is the hard thing to scale out. That requires intelligence at the application level, and that might include CAP issues. So for us, the SQL data store is the real “up” part of the equation. We have a fair amount of headroom there, so if we can keep sharding (and other data strategies) out of the codebase, so much the better. (Load-balancing HTTP requests “out” is not a big deal and we are doing that.)
- sklivvz1971 12y agoMarco/Stack Overflow here. Simply put, this is not true at all. Most of the websites today run happily on a single box and the choice of architecture is frankly irrelevant because of the abysmal amount of traffic they get. Large, and by large I mean top 100 sites, are large and typically complex systems. Each one is different and bespoke. Making broad generalizations doesn't really work, because typically they have parts that need to be scaled up, parts that need to be scaled out, etc. Finally, if you read the article you would know that we already use a load balancer with multiple web servers. What we are discussing here is whether it's better to have 100s of cheap/cloud boxes versus a few big ones. In our use case (we want fast response times) the second solution is demonstrably better.
- johansch 12y agoNot quite sure how you derive a peak figure from a monthly average; but on average it's more like 560000000/(30 * 24 * 3600) = 216 req/s
- MichaelGG 12y agoI took 4 hours at 20 days to get the peak, figuring it probably won't be worse than that. The average over the month isn't a number you'll see in practice. Most perf engineering work seems to go to keep things working nicely while at peak. (And making sure you have good 95th percentile times.)
- Nick-Craver 12y agoPeak is more like 2600-3000 requests/sec on most weekdays. Remember that programming, being a profession, means our weekdays are significantly busier than weekends (as you can see here: https://www.quantcast.com/p-c1rF4kxgLUzNc https://www.quantcast.com/p-c1rF4kxgLUzNc). It's almost all over 9 servers, because 10 and 11 are only for meta.stackexchange.com, meta.stackoverflow.com, and the development tier. Those servers also run around 10-20% CPU which means we have quite a bit of headroom available. Here's a screenshot of our dashboard taken just now: http://i.stack.imgur.com/HPdtl.png http://i.stack.imgur.com/HPdtl.png We can currently handle the full load of all sites (including Stack Overflow) on 2 servers...not 1 though, that ends badly with thread exhaustion. We could add web servers pretty cheaply; these servers are approaching 4 years old and weren't even close to top-of-the-line back them. Even current generation replacements would be several times more powerful, if we needed to go that route. Honestly the only scale-up problem we have is SSD space on the SQL boxes due to the growth pattern of reliability vs. space in the non-consumer space. By that I mean drives that have capacitors for power loss and such. I actually just wrote a lengthy email about what we're planning for storage on one of our SQL clusters...perhaps I should echo it verbatim as a blog post? I'm not sure how many people care about that sort of stuff outside our teams. Nick Craver - Stack Exchange Sysadmin & Developer
- qntmfred 12y agoyes please
- voltagex_ 12y ago>I actually just wrote a lengthy email about what we're planning for storage on one of our SQL clusters...perhaps I should echo it verbatim as a blog post? I'm not sure how many people care about that sort of stuff outside our teams. I'm sure some DBAs and devs here would find it interesting.
- bagels 12y agoThis is definitely the case. Every writeup from those in the trenches I read and share with coworkers.
- 12y ago