We use queues for various jobs, some of which are time-sensitive (e.g. emails), so the queue should be empty most of the time, some of which are background or r

We use queues for various jobs, some of which are time-sensitive (e.g. emails), so the queue should be empty most of the time, some of which are background or re-processing (e.g. re-indexing), so queues can balloon and then take days to drain again. The unifying aspects of using queues for us is that it allows us to load-balance jobs across workers, and allows us to monitor throughput in a centralised fashion. We can set alerts on the time-sensitive queues and then use different thresholds for the background queues, but we're using the same metric data and same alerting system.