I've done this for running long-lived subscriptions in a trading platform back-end service. The load is similar. Basically, you need a cluster software with sha

I've done this for running long-lived subscriptions in a trading platform back-end service. The load is similar. Basically, you need a cluster software with sharding option. I personally used Akka Cluster Sharding (a Scala library, which also happens to have Java bindings). It starts a network node in each instance of the service and binds all nodes into a cluster (discovery of other nodes is left for the service developer; simplest solution is to have a static list of node addresses). The sharding mechanism allows you to distribute arbitrary data objects among the nodes according to some rules (e.g. based on the value "object's hash modulo number of nodes in cluster", which produces a typical hashring). Data objects may originate on any node (e.g. on schedule or on some external event). They also need to be serializable to be passed between nodes and, obviously, the binary representation should not be too big. So, depending on the nature of jobs in your case, you may want to pass only the job ID as the data object and store the actual job definition and/or arguments in a separate place (e.g. a database). Now, Akka will guarantee that the job will be run only by 1 node and run once (until completion that is; the job may be started more than once, see tips below). But if there are many jobs coming in, you risk overwhelming the cluster (because passing messages between nodes takes time, jobs themselves take time, etc.) So to defend your cluster against load, which it might not be able to handle, you may put a message queue in front of it. This will let you set up a max number of jobs that concurrently run in the cluster, and nodes will take new jobs from the queue only when some of the currently running jobs complete. Most MQs have persistence, so the jobs will be safe even if they need to wait for a while sitting in the queue. Few tips: 1) Akka persists node's data, so the jobs that are taken from the queue are going to be safe in case the node they are located at fails 2) If a node leaves the cluster, all its' jobs will be moved to other nodes (according to the sharding rules mentioned above) and started over, so you may need to introduce some kind of transactions (same goes for cluster restart) 3) If the whole cluster is restarted, unfinished jobs will be distributed among the nodes according to the sharding rules, so it might be a good idea to make this rules "sticky", so that each individual job is always assigned to the same node (and each node will just have to load its' jobs from its' own persistent store). Otherwise there might be a lot of message passing which will slow down the startup.