3 ms·

Can you please describe when you had to use formal queueing theory in your work? I have struggled - and failed - to find a use case relevant to daily SWE work,

Can you please describe when you had to use formal queueing theory in your work? I have struggled - and failed - to find a use case relevant to daily SWE work, since (a) the probability distribution for most customers waiting in a queue is not known and must be "guessed" or predicted ahead of time using historical data and (b) nearly all queue requirements are simple (i.e. minimize total waiting time) and the solutions just as simple (scale up your queue consumers!)

eindiran 6y ago
Mostly simple things: if you have a messaging queue used for logging, and you need to predict the change in its memory footprint given some change in a message format before rolling out/testing the format change, just use Little's law. Or you need to determine the smallest number of consumers that you can have online safely to maximize how quickly you can do maintenance without degrading logging.
dkarl 6y ago
You might think that something as simple as Little's Law is intuitive, but people who haven't learned it don't always find its conclusions obvious, and they are less likely to see when a problem can be framed as a queueing problem. The value is really in simple stuff like this, where having something in your mental toolkit makes the difference between a problem requiring a trivial application of something you know versus requiring thought and creativity. A particular example I remember was a poorly architected system that was essentially two systems connected like queues in sequence. The first one was fast and lightweight, basically a gateway/hydrator for the second one, and could handle a very large backup with no ill effects. The second one was very slow and fell over quickly if too many items accumulated in the queue during a load spike. An easy short-term fix to make the system stable in production was to introduce artificial slowness in the first queue to buffer load spikes and prevent backups in the second queue. Little's Law points straight to this solution, but people who didn't have queueing theory in their mental toolkit had not considered fixing the system by slowing part of it down. It looked like a big leap of imagination for them, even if they found it intuitive to understand after it was pointed out.

eindiran 6y ago

Mostly simple things: if you have a messaging queue used for logging, and you need to predict the change in its memory footprint given some change in a message format before rolling out/testing the format change, just use Little's law. Or you need to determine the smallest number of consumers that you can have online safely to maximize how quickly you can do maintenance without degrading logging.

dkarl 6y ago

You might think that something as simple as Little's Law is intuitive, but people who haven't learned it don't always find its conclusions obvious, and they are less likely to see when a problem can be framed as a queueing problem. The value is really in simple stuff like this, where having something in your mental toolkit makes the difference between a problem requiring a trivial application of something you know versus requiring thought and creativity. A particular example I remember was a poorly architected system that was essentially two systems connected like queues in sequence. The first one was fast and lightweight, basically a gateway/hydrator for the second one, and could handle a very large backup with no ill effects. The second one was very slow and fell over quickly if too many items accumulated in the queue during a load spike. An easy short-term fix to make the system stable in production was to introduce artificial slowness in the first queue to buffer load spikes and prevent backups in the second queue. Little's Law points straight to this solution, but people who didn't have queueing theory in their mental toolkit had not considered fixing the system by slowing part of it down. It looked like a big leap of imagination for them, even if they found it intuitive to understand after it was pointed out.