There's almost no complexity on the scheduling side done properly. Just pop work off the queues in priority order. There are use cases for multiple latency tie

There's almost no complexity on the scheduling side done properly. Just pop work off the queues in priority order. There are use cases for multiple latency tiers 1. At the lower end you have realtime-ish responses for when the chat is via SMS/WhatsApp/tickets/email, i.e. the social convention is that responses may be delayed by a few minutes or even 10-20 minutes and that's reasonable, but not by 24 hours. 2. At the high end you have a database with 200 million documents, a prompt that works, and you just want to submit all of them in one go as quickly as your client can work, then get the results as they get processed. There is no latency requirement whatsoever, but you'd prefer not to have to chunk things up and track batch IDs and such yourself. What you DO really want though is to have exactly-once submission, so you don't accidentally lose your connection half way through a batch upload and then end up paying for the same batch twice. These are all basic problems of the type that occur all the time in the IT industry. The solutions are well known and mature. Your bank runs on message queues of various kinds. The tech really isn't that fancy.