3 ms·

Your mistake is characterising all overloads as the same. Adding a queue to a system that is consistently overloaded won't solve the overload, but many overload

Your mistake is characterising all overloads as the same. Adding a queue to a system that is consistently overloaded won't solve the overload, but many overloads are temporary/bursty, like the Slashdot/HN effect. Queues absolutely do solve these kinds of overloads simply by increasing latency, assuming increased latency is an acceptable choice in your context, of course.

10000truths 3y ago
Maximum acceptable response times for websites is a few seconds. The HN/Slashdot effect lasts for minutes to hours. That time scale is way too long for a queue to be effective.
- naasking 3y ago
  You're missing the point. We're talking about general systems here, not websites specifically, and the Slashdot effect is a perfect example that everyone is familiar with where queues do solve maintain availability if longer latency is acceptable.
  - emn13 3y ago
    Furthermore, even as a website aiming to survive some momentary spike in performance - slowing everything down for everyone _can_ be part of a solution to serve more people but fewer things each. People might not normally be willing to wait for more than a second or two for a load; but when they expect or have a sign that things are slower than normal, they might have a little more patience (or just come back to that tab later). I think people are a little too negative on queues. Sure, really naively implemented with entirely unbounded capacity they're an accident waiting to happen... but you don't have to do that.
    - lanstin 3y ago
      The distinction is really not queues or not but unbounded in theory or not. There are queues everywhere, connection pools, unparsed messages, packets in flight, memory for calls sent out and not yet replied or timed out, etc. but one unbounded (in theory; they are never unbounded in practice) queue can make your system go from robust to fragile.
marcosdumay 3y ago
After the overload fixes itself somehow, a queue absolutely solves the availability problem. Yep. If the overload doesn't last for too long. What isn't happening on your example is a queue improving anything on a overloaded system. Queues are immensely helpful for all kinds of problems. Just not for overload.
- naasking 3y ago
  Overload is only an issue because availability is critical. When it isn't critical, then avoiding overload is obvious and practically trivial.
- lanstin 3y ago
  An unbounded queue means you are allocating a lot of stuff during the time you want to cap end all resources handling traffic. It means you slow down when busier. Depending on the framework it often means the system can't recover automatically but will crash if the queue is in memory or need DBAs to flush the queues or whatever.

10000truths 3y ago

Maximum acceptable response times for websites is a few seconds. The HN/Slashdot effect lasts for minutes to hours. That time scale is way too long for a queue to be effective.

naasking 3y ago
You're missing the point. We're talking about general systems here, not websites specifically, and the Slashdot effect is a perfect example that everyone is familiar with where queues do solve maintain availability if longer latency is acceptable.
- emn13 3y ago
  Furthermore, even as a website aiming to survive some momentary spike in performance - slowing everything down for everyone _can_ be part of a solution to serve more people but fewer things each. People might not normally be willing to wait for more than a second or two for a load; but when they expect or have a sign that things are slower than normal, they might have a little more patience (or just come back to that tab later). I think people are a little too negative on queues. Sure, really naively implemented with entirely unbounded capacity they're an accident waiting to happen... but you don't have to do that.
  - lanstin 3y ago
    The distinction is really not queues or not but unbounded in theory or not. There are queues everywhere, connection pools, unparsed messages, packets in flight, memory for calls sent out and not yet replied or timed out, etc. but one unbounded (in theory; they are never unbounded in practice) queue can make your system go from robust to fragile.

marcosdumay 3y ago

After the overload fixes itself somehow, a queue absolutely solves the availability problem. Yep. If the overload doesn't last for too long. What isn't happening on your example is a queue improving anything on a overloaded system. Queues are immensely helpful for all kinds of problems. Just not for overload.

naasking 3y ago
Overload is only an issue because availability is critical. When it isn't critical, then avoiding overload is obvious and practically trivial.
lanstin 3y ago
An unbounded queue means you are allocating a lot of stuff during the time you want to cap end all resources handling traffic. It means you slow down when busier. Depending on the framework it often means the system can't recover automatically but will crash if the queue is in memory or need DBAs to flush the queues or whatever.