2 ms·
> If the water pouring into the sink delivered more water in a second than the drain could handle, that'd be more than 100% utilization. this is where the sink
by KaiserPro 4y ago
> If the water pouring into the sink delivered more water in a second than the drain could handle, that'd be more than 100% utilization.
this is where the sink bit comes in. For small amounts, queues are great at absorbing load. for example, you can dump two or three who glasses at once into a sink, and the drain will be fully utilised. (try doing that by pouring the glasses concurrently into the drain pipe at once.)
> fewer than N events per second, on average, are occurring
average is doing a lot of work here! unless you are very lucky, an average can hide a lot of variance. If we go back to the sink analogy, pure water flows away more or less at a constant rate (if we ignore whatever the spinny whirlpool effect is) The variance in processing time is very little.
However, if we start dumping washing up water in there, lumps of food will start to clog the drain. on average, assuming no blockages, processing time increases a little bit. but when we get close to peak capacity, a blockage will cause an overflow thats very difficult to clear. This is because there is no extra capacity to get through the backlog.
the rule of thumb is that as you get close to 80-90% of your consuming capcity, your queues will begin to expand.
when you are designing your stuff two things might save you: expiring keys, and bounded queues.
The first one means dataloss that you need to handle yourself. the worst part is, you might not get informed that your message is lost. So that needs careful thought
the second one means that things pushing into the queue need to check the state of it before pushing. Back pressure is always good. At least the client knows _before_ the data is sent.