4 ms·

A while back there was a post that said in essence: "queues grow to inifinte size when the consumer is approaching 100% utilisation" However it went deep into

A while back there was a post that said in essence: "queues grow to inifinte size when the consumer is approaching 100% utilisation" However it went deep into lots of theory, which made it all sound very sciency and clever. However it failed to get across the simplicity, which even a 9 year old can grasp: Queues are like sinks, if you put water in at close to the rate that the drain lets it out, the water level of the sink doesn't drop, any slight imperfection makes the level rise. animated gif, and be done. Instead it dressed up queues as some complex and difficult to predict beast, that only big companies with very clever people can use.

kqr 4y ago
Also known as "queueing theory is just discrete fluid dynamics".
whatshisface 4y ago
I don't think your analogy is actually right. If the water pouring into the sink delivered more water in a second than the drain could handle, that'd be more than 100% utilization. It's actually kind of surprising that you can end up with an queue going to infinity when you can process N events per second, and fewer than N events per second, on average, are occurring. I think it has something to do with gambler's ruin and the fact that continuing to bet $1 on a fair 50/50 game will eventually bankrupt you, every time, even though the game's "fair."
- KaiserPro 4y ago
  > If the water pouring into the sink delivered more water in a second than the drain could handle, that'd be more than 100% utilization. this is where the sink bit comes in. For small amounts, queues are great at absorbing load. for example, you can dump two or three who glasses at once into a sink, and the drain will be fully utilised. (try doing that by pouring the glasses concurrently into the drain pipe at once.) > fewer than N events per second, on average, are occurring average is doing a lot of work here! unless you are very lucky, an average can hide a lot of variance. If we go back to the sink analogy, pure water flows away more or less at a constant rate (if we ignore whatever the spinny whirlpool effect is) The variance in processing time is very little. However, if we start dumping washing up water in there, lumps of food will start to clog the drain. on average, assuming no blockages, processing time increases a little bit. but when we get close to peak capacity, a blockage will cause an overflow thats very difficult to clear. This is because there is no extra capacity to get through the backlog. the rule of thumb is that as you get close to 80-90% of your consuming capcity, your queues will begin to expand. when you are designing your stuff two things might save you: expiring keys, and bounded queues. The first one means dataloss that you need to handle yourself. the worst part is, you might not get informed that your message is lost. So that needs careful thought the second one means that things pushing into the queue need to check the state of it before pushing. Back pressure is always good. At least the client knows _before_ the data is sent.
- zmgsabst 4y ago
  I think it’s similar to that, with some critical point at which the behavior makes a discontinuous change — as seen from the examples: - the gambler goes broke - a sink carrying over 100% capacity can jam In general, if you have some kind of system where there’s a “critical threshold” and a discontinuous (or even non-linear) result for crossing it, then approaching that boundary will be increasingly dominated by that effect, as the distribution of events starts to cross into that regime. For the example of a queue, the odds of a packet failing due to delay suddenly spikes as you approach “full capacity” because the variation in packet timings causes it to “cross over” that boundary for brief periods, the closer the mean of the distribution moves to the critical threshold.

kqr 4y ago

Also known as "queueing theory is just discrete fluid dynamics".

whatshisface 4y ago

I don't think your analogy is actually right. If the water pouring into the sink delivered more water in a second than the drain could handle, that'd be more than 100% utilization. It's actually kind of surprising that you can end up with an queue going to infinity when you can process N events per second, and fewer than N events per second, on average, are occurring. I think it has something to do with gambler's ruin and the fact that continuing to bet $1 on a fair 50/50 game will eventually bankrupt you, every time, even though the game's "fair."

KaiserPro 4y ago
> If the water pouring into the sink delivered more water in a second than the drain could handle, that'd be more than 100% utilization. this is where the sink bit comes in. For small amounts, queues are great at absorbing load. for example, you can dump two or three who glasses at once into a sink, and the drain will be fully utilised. (try doing that by pouring the glasses concurrently into the drain pipe at once.) > fewer than N events per second, on average, are occurring average is doing a lot of work here! unless you are very lucky, an average can hide a lot of variance. If we go back to the sink analogy, pure water flows away more or less at a constant rate (if we ignore whatever the spinny whirlpool effect is) The variance in processing time is very little. However, if we start dumping washing up water in there, lumps of food will start to clog the drain. on average, assuming no blockages, processing time increases a little bit. but when we get close to peak capacity, a blockage will cause an overflow thats very difficult to clear. This is because there is no extra capacity to get through the backlog. the rule of thumb is that as you get close to 80-90% of your consuming capcity, your queues will begin to expand. when you are designing your stuff two things might save you: expiring keys, and bounded queues. The first one means dataloss that you need to handle yourself. the worst part is, you might not get informed that your message is lost. So that needs careful thought the second one means that things pushing into the queue need to check the state of it before pushing. Back pressure is always good. At least the client knows _before_ the data is sent.
zmgsabst 4y ago
I think it’s similar to that, with some critical point at which the behavior makes a discontinuous change — as seen from the examples: - the gambler goes broke - a sink carrying over 100% capacity can jam In general, if you have some kind of system where there’s a “critical threshold” and a discontinuous (or even non-linear) result for crossing it, then approaching that boundary will be increasingly dominated by that effect, as the distribution of events starts to cross into that regime. For the example of a queue, the odds of a packet failing due to delay suddenly spikes as you approach “full capacity” because the variation in packet timings causes it to “cross over” that boundary for brief periods, the closer the mean of the distribution moves to the critical threshold.