4 ms·
Most of the companies I've worked for or consulted with had some sort of background worker system for , usually with a queue in postgres/redis/rabbitMQ/kafka/wh
by WJW 3y ago
Most of the companies I've worked for or consulted with had some sort of background worker system for , usually with a queue in postgres/redis/rabbitMQ/kafka/whatever in between. It usually gets used for non-latency sensitive work like sending emails instead of doing that from the main request handler, so that the user gets their response without having to wait for the other stuff to be completed. Bigger companies sometimes have multiple groups of background workers, one for each service and each with their own queues, servers and autoscaling policies.
As I understand the XFaaS system from the article it functions a bit like a consolidated background worker system that any service can submit their work to and it will (try to) make sure it gets done in whatever SLO is specified. This is usually cheaper because now you can do low-urgency work from service B when there are urgent jobs from service A and vice versa, leading to higher average server usage. For small/medium businesses this is probably not something you'd want, since developing and operating such a system costs more than you save. But if you have a multi-million dollar bill for your background workers alone then it might be worth it.