AMQP isn't a custom messaging protocol and MQ libraries do all of that for you already (handling reconnects and retries etc). Connection interruptions aren't an

AMQP isn't a custom messaging protocol and MQ libraries do all of that for you already (handling reconnects and retries etc). Connection interruptions aren't an issue. You're actually worrying about low level details of the sort MQ systems are specifically designed to solve for you, but HTTP stacks do not. For example, someone below asks "how do I see my submitted jobs". MQ protocols solve that for you out of the box. You can browse the queues that you own. It's all already standardized and implemented. Because they rolled their own protocol, OpenAI lacks this function and will have to develop it themselves. Then because it's all oriented around batches (when the underlying problem does not actually need batching), the users will have to run a local DB to keep track of what prompts map to what batches or they'll find themselves redownloading the whole batch to find out what's in it. As for firewalls, you open the port. The only issue there is if you're behind a firewall you don't have any control over and where the people who do don't want you using AI, but that's not an issue in the datacenter deployments that are presumably generating inferencing batch workloads, and such firewalls or proxies can break HTTP APIs too anyway.