I built something like this using rabbitmq for automating delivery of online documents (taxes) for an accounting company. I had a queue per function, a DAG defi

I built something like this using rabbitmq for automating delivery of online documents (taxes) for an accounting company. I had a queue per function, a DAG defining the workflow, and the tasks themselves would know who to forward the result to. Different libraries of tasks ran in different environments with different permissions (filesystem access ran on a windows server, for example, to access the file shares), only browsers had Internet access (and only to certain domains). I eventually switched to a code monolith running in a segmented way with temporal.io, where I have libraries of activities which run in different nodes. You might be interested in their product.