5 ms·

If a queue isn't approaching empty, then its input is exceeding its output. Therefore it's size is trending to infinity. Since we don't have infinite space, we

• deleted 4y ago
  [deleted]
• beardedetim 4y ago
  I think this is a valid concern for Future Us to worry about. If you have alerts for when this is going to happen, you can punt the issue until it starts to become a concern and who knows if your product will be alive when that issue happens.
• londons_explore 4y ago
  M/M/1 Queue theory doesn't apply to real queues... Because the work input into the queue usually does depend on the latency of previous work completed by the worker. If there is a 1 hour queue for the checkout at a grocery store, you probably won't join the queue - you'll go to a different store...
  • lokar 4y ago
    In many systems you get the reverse. A client sends a retry without the ability to cancel the first request, now it has two requests in the queue.
    • cryptonector 4y ago
      Back pressure and circuit breakers matter.
• orf 4y ago
  > Since we don't have infinite space, we can expect eventually to lose some messages in this scenario. Sure, that's technically correct but applies to basically everything. It is very likely your users/orders/whatever table in your database is also "trending to infinity" over time, except this doesn't mean databases should always be empty or that we should expect to eventually lose some users/orders/whatever. Or, more succinctly, if your queue capacity is 10 million messages and your queue messages represent "houses purchased through our website", then in reality your queue capacity is infinite because nobody will ever purchase 10 million homes through your application per small unit of time.
  • scubbo 4y ago
    Your response is correct, but there's a more fundamental flaw in the preceding comment: although both paragraphs are true in isolation, the implication between them is false. "The queue is _currently_ trending to infinity" does not imply "The queue will _continue_ to monotonically trend to infinity". Periodicity is commonplace in real-life situations; and while parts of Queueing Theory produces some insightful and helpful insights by ignoring it, care should be taken when applying them.
• vhcr 4y ago
  The growth rate of a queue is not constant, it could get bigger during the day, but smaller at night, since customers are not using the service.
  • roflyear 4y ago
    If you're just making people wait 12 hours, or five minutes, or whatever - you aren't saving on compute. You're just adding a buffer. Maybe that is what you want, but it should be intentional.
    • happymellon 4y ago
      This is usually the approach of people with fixed compute. If I have a VPS then I'll let it churn through the backlog overnight. But this is also where the disagreement comes in with flexible serverless workers. They generally do not cost more, so you'll end up spending $24 in an hour rather than $1 an hour all day. If the serverless costs more then you are likely not working through your backlog. I agree it should be a business decision, so you catch up on weekends instead for example, but non-technical folks don't always grasp those numbers and that they are getting a lag with their data.
      • roflyear 4y ago
        It is a little counterintuitive, but "churn through overnight" still means you're decreasing the queue size. You're right that the cost really ends up being roughly the same, unless your service gives you a discount for compute overnight or something.
        • happymellon 4y ago
          Completely agree that the queue still goes down, I just see comments on here about folks preferring a VPS because they run it 24/7 for the same price. That is probably fine for their scenario but the non-technical members probably don't understand that they aren't saving money by doing it this method but are adding a delay. That is my only point.
• remram 4y ago
  > If a queue isn't approaching empty, then its input is exceeding its output. It depends what you mean by "approaching empty". If you mean "empty most of the time", then no, that's wasted resources. If you mean "its size is decreasing most of the time", then no, not possible. Even if you mean "it is empty some of the time", then maybe, but that is not a strong requirement either. Of course more average input than average output is bad, but stating it as "should be empty" or "should be approaching empty" seem to suggest the wrong thing entirely. It should be smaller than the target latency of the process.
  • Smaug123 4y ago
    Eh? It is possible for size to be decreasing most of the time. Just make traffic extremely bursty - add 60 on the hour every hour, and consume at one per minute.
    • remram 4y ago
      What does average mean there? Increasing and decreasing are punctual events. If you mean the 1-minute average can be negative most of the time, that is true, but it would not work for other time horizons, whether larger or shorter.
      • roflyear 4y ago
        the # of items in a queue is either growing or shrinking, there isn't the concept of "staying the same" and as you approach the line from shrinking to growing, performance (queue time) degrades after roughly 70-75% utilization.
        • remram 4y ago
          Of course it can stay the same? Where the hell do you get this 70-75% figure from? This makes no sense.
          • roflyear 4y ago
• User23 4y ago
  The Little’s law[1] offers considerable guidance here. In the shortish term it’s safe to model the system as being stationary and then you need to provision for seasonality[2]. Upward or downward trending demand after correcting for seasonality indicates that the system isn’t stationary and your choices are either to scale up or reject requests. Nevertheless the optimal queue size is not necessarily zero except in cases where extremely low latency is necessary. The optimal queue capacity for most web service type applications is one that trends toward almost but not quite full when seasonal requests are higher. [1] https://en.m.wikipedia.org/wiki/Little%27s_law https://en.m.wikipedia.org/wiki/Little%27s_law [2] broadly construed. For example a service that gets a mass of requests at the top of every hour displays seasonality.
• mecsred 4y ago
  This is so academic. Take any two samples; slope > 0 => we're going to be dropping messages!!! Slope < 0 our threads are idle!!!
• klyrs 4y ago
  > If a queue isn't approaching empty, then its input is exceeding its output. Therefore it's size is trending to infinity. This is literally the slippery slope fallacy. You aren't accounting for minor fluctuations or finite timescales. If you were driving a car, you might say "if you aren't steering into oncoming traffic then you're steering towards the ditch" and conclude that no cars will ever safely reach their destination. If the only valid state for a queue is empty, then why waste time implementing a queue?
  • teolandon 4y ago
    GP said "approaching empty", meaning it should be empty in the long term. It's not a slippery slope.
    • camdenreslink 4y ago
      It assumes the velocity of items being added to the queue never changes. In real life it speeds up (approaching infinite items), and it slows down (approaching empty). A more meaningful measurement would be something like average number of items in the queue and average time in the queue for any given item. If you feel like you have enough spare capacity and any given item isn’t taking too long to process then it doesn’t matter if you are ever empty.
      • Espressosaurus 4y ago
        You're on the right track, but I personally find averages to be not a useful measure...on average. Corner conditions are where I start to care: * What is the worst case latency? How long until the work can be done? How long until the work can finish from the point at which it enters the queue? * What is the worst case number of items in the queue? How large does the queue need to be? * What is the maximum utilization over some time period with some granularity? How spiky is the workload? A queue is a rate-matching tool. If your input rate exactly equals your output rate at all times, you don't need a queue. * What is the minimum utilization over some time period? Another view on how spiky the workload is. Minimums and maximums I find are much more illustrative than averages or even mean-squared-errors. Minimums and maximums bound performance; an average doesn't tell you where your boundary conditions are. In general you don't want your queue to fully fill up, but like the other poster said, it's some tradeoff between utilization and latency, and the two are diametrically opposed.
• derefr 4y ago
  What about a queue that is the ingress point for a fan-in between O(N) active continuous producers to a single active continuous consumer? In such a case, you'd expect at least N messages in the queue at all times — and often a small multiple of N. Not because the consumer can't consume those messages in a timely manner, but because, given a reliable queue, messages can't be consumed until they've been 2PCed into the queue, and that means that there are always going to be some messages sitting around in the "acknowledged to the producer, but not yet available to the consumer" state. Which still takes up resources of the queueing system, and so should still be modelled as the messages being in the queue.
• kqr 4y ago
  This is true only when you have relatively steady flows, or infinite multiprocessing. In that case, yes, the queue is in practise either "almost empty" or "infinite". I think the reason you get the responses you do is people are used to systems with high variation in flow rates that can only process a limited number of items at the same time, and for these, queues can soak up a lot of variability by growing long and then being worked off quickly again.
• deleted 4y ago
  [deleted]