4 ms·
This is pretty fancy stuff! Sorry if I'm just not reading carefully enough, but does this approach account for tenants whose messages take longer to process, as
by ucarion 2y ago
This is pretty fancy stuff! Sorry if I'm just not reading carefully enough, but does this approach account for tenants whose messages take longer to process, as opposed to a tenant that sends a larger volume of messages?
- abelanger 2y agoNo, this assumes you've already split your tasks into quanta of approximately the same duration - so all tasks are weighted equally in terms of execution time. If each of the tasks have different weights you might be looking at something like deficit round-robin [1], which I've looked at implementations for in RabbitMQ and we're thinking about how to implement in PG as well [2]. [1] https://www.cs.bu.edu/fac/matta/Teaching/cs655-papers/DRR.pdf https://www.cs.bu.edu/fac/matta/Teaching/cs655-papers/DRR.pd... [2] https://nithril.github.io/amqp/2015/07/05/fair-consuming-with-rabbitmq/ https://nithril.github.io/amqp/2015/07/05/fair-consuming-wit...
- ucarion 2y agoMakes sense! My gut tells me that it would often make sense to jump straight to shuffle sharding, where you'd converge on fair solutions dynamically, in a lot of cases. I'm looking forward to that follow-on article!