3 ms·

I think it's the threshold of all people to truly understand that there's no solution to certain problems, only tradeoffs. Queues are great, but can lead to ca

by dijit 3y ago

I think it's the threshold of all people to truly understand that there's no solution to certain problems, only tradeoffs. Queues are great, but can lead to catastrophic failure if you don't have a good way of handling the queue, so making an active choice about how you handle overload is part of designing a robust and resilient system. Trading off new requests for current requests is, in my experience, a valid strategy for eCommerce for example. We called it "quenching".

kqr 3y ago
> Trading off new requests for current requests is, in my experience, a valid strategy for eCommerce for example. We called it "quenching". I'm not sure in which direction the trade happens but it sounds like you're dropping older requests in favour of newer. I agree, this has worked well for me also. Surprisingly often the oldest item in the queue is the one for which service will be least valuable.
- hinkley 3y ago
  I think the theory is that if half of your excess requests come from two actors, then half of the consequences of random dropping are felt by those two people. That’s not fair, but it’s better than dropping all requests after you get into trouble. Fairness is very difficult to achieve in a distributed system, due to coordination delays.
smaudet 3y ago
I think part of the article is just the author venting that people make design decisions without understanding the ramifications. Operating a queue as a buffer would be absolutely fine if there were also service level agreements implicit in every queue in operation - i.e. one queue reaches half capacity or is experiencing explosive growth, leading to reactive techniques such as spinning up larger queues/servers, alerting stakeholders to the performance issue(s), and possibly even dynamically rate-limiting end-users. But this is a whole-system centric view of the design. What they're specifically bemoaning is the asinine focus on component-centric design - your widget/service performs slowly? Throw a queue at it (and wonder why you broke everything 100 features and 10x customer base later)! For legitimately small problems, this is OK. But then, you accept that you aren't scaling, and you bake it into your design assumptions. And you sell that, explicitly, to the customer "hey this will only work for 1 thousand/1 million customers, if you want more we can talk".

by dijit 3y ago

kqr 3y ago

> Trading off new requests for current requests is, in my experience, a valid strategy for eCommerce for example. We called it "quenching". I'm not sure in which direction the trade happens but it sounds like you're dropping older requests in favour of newer. I agree, this has worked well for me also. Surprisingly often the oldest item in the queue is the one for which service will be least valuable.

hinkley 3y ago
I think the theory is that if half of your excess requests come from two actors, then half of the consequences of random dropping are felt by those two people. That’s not fair, but it’s better than dropping all requests after you get into trouble. Fairness is very difficult to achieve in a distributed system, due to coordination delays.

smaudet 3y ago

I think part of the article is just the author venting that people make design decisions without understanding the ramifications. Operating a queue as a buffer would be absolutely fine if there were also service level agreements implicit in every queue in operation - i.e. one queue reaches half capacity or is experiencing explosive growth, leading to reactive techniques such as spinning up larger queues/servers, alerting stakeholders to the performance issue(s), and possibly even dynamically rate-limiting end-users. But this is a whole-system centric view of the design. What they're specifically bemoaning is the asinine focus on component-centric design - your widget/service performs slowly? Throw a queue at it (and wonder why you broke everything 100 features and 10x customer base later)! For legitimately small problems, this is OK. But then, you accept that you aren't scaling, and you bake it into your design assumptions. And you sell that, explicitly, to the customer "hey this will only work for 1 thousand/1 million customers, if you want more we can talk".