3 ms·

Well, the system is kind of over-dimensioned, but this is just good engineering, isn't it? If they are capable of rendering any question page in 33ms, it stand

Well, the system is kind of over-dimensioned, but this is just good engineering, isn't it? If they are capable of rendering any question page in 33ms, it stands to reason they could use just 3 web servers and still serve content in roughly 100ms (which is fast enough that most people wouldn't notice). My point is, without knowing what is the bottleneck for the web servers (we only know it's not CPU - I suspect it should be I/O) it's hard to make this kind of point. Aside from that, I think I remember the original development team was already familiar with C#/.NET, and this drove the decision for the original stack.

organsnyder 12y ago
> If they are capable of rendering any question page in 33ms, it stands to reason they could use just 3 web servers and still serve content in roughly 100ms (which is fast enough that most people wouldn't notice). It doesn't work that way. With less servers, requests would still take 33ms (until the servers are overloaded, at which point they may start to take longer unless they have rate-limiting to reject requests above their capacity), but their overall capacity would be lower.
- andrea_s 12y ago
  It definitely doesn't work that way, of course... That was meant to provide a reasonable upper bound for discussion. We know from the provided data that their architecture can compile and serve a page in about 33ms. This includes everything that is not web servers, so we are certain that the actual time spent in the webserver will be lower. Aside from that, we know that their webservers are nowhere near full CPU usage (they claim 15-20%). This also goes in the direction of not invalidating the use of 33ms as an upper bound. We would need to know more about why they have 9 webservers in the first place to get a better idea of what's going on (mostly how the I/O is behaving, I'd say). Putting this assumptions together, we can safely assume that by shrinking the number of webservers from 9 to 3 wouldn't cause the page loading time to increase to more than 100ms (and it's likely going to be much smaller in most conditions). Of course, this may be overlooking some piece of information we may be missing (the prime example still being I/O-related). edit: I see now that in the original post I stated "roughly" 100ms, while it would have been better to say "at maximum".
- barrkel 12y ago
  With fewer servers, requests would not take 33ms, they would, on average, take longer. As load increases, the probability that requests are queued while CPUs are busy with other requests increases. The number of requests being processed and not blocked on I/O can't exceed the physical parallelism available. You don't need to be CPU-bound before this starts happening because requests don't arrive in evenly spaced intervals. Decreasing CPU load, even on a moderately loaded, I/O bound system, can still reduce latency.

organsnyder 12y ago

> If they are capable of rendering any question page in 33ms, it stands to reason they could use just 3 web servers and still serve content in roughly 100ms (which is fast enough that most people wouldn't notice). It doesn't work that way. With less servers, requests would still take 33ms (until the servers are overloaded, at which point they may start to take longer unless they have rate-limiting to reject requests above their capacity), but their overall capacity would be lower.

andrea_s 12y ago
It definitely doesn't work that way, of course... That was meant to provide a reasonable upper bound for discussion. We know from the provided data that their architecture can compile and serve a page in about 33ms. This includes everything that is not web servers, so we are certain that the actual time spent in the webserver will be lower. Aside from that, we know that their webservers are nowhere near full CPU usage (they claim 15-20%). This also goes in the direction of not invalidating the use of 33ms as an upper bound. We would need to know more about why they have 9 webservers in the first place to get a better idea of what's going on (mostly how the I/O is behaving, I'd say). Putting this assumptions together, we can safely assume that by shrinking the number of webservers from 9 to 3 wouldn't cause the page loading time to increase to more than 100ms (and it's likely going to be much smaller in most conditions). Of course, this may be overlooking some piece of information we may be missing (the prime example still being I/O-related). edit: I see now that in the original post I stated "roughly" 100ms, while it would have been better to say "at maximum".
barrkel 12y ago
With fewer servers, requests would not take 33ms, they would, on average, take longer. As load increases, the probability that requests are queued while CPUs are busy with other requests increases. The number of requests being processed and not blocked on I/O can't exceed the physical parallelism available. You don't need to be CPU-bound before this starts happening because requests don't arrive in evenly spaced intervals. Decreasing CPU load, even on a moderately loaded, I/O bound system, can still reduce latency.