Are you choosing a tech stack to do this? Laravel has this built in with queues. https://laravel.com/docs/5.7/queues https://laravel.com/docs/5.7/queues You

Are you choosing a tech stack to do this? Laravel has this built in with queues. https://laravel.com/docs/5.7/queues https://laravel.com/docs/5.7/queues You can run multiple workers, it will intelligently distribute the jobs, and there is a Laravel Horizon package that can handle monitoring of the queues/jobs. I expect Rails would have something similar, but I haven't used queues in Rails.