I am in a researching of a similar problem and here is what I can tell you. Suppose a program which accepts some requests from Internet (server in client-serve

I am in a researching of a similar problem and here is what I can tell you. Suppose a program which accepts some requests from Internet (server in client-server terminology). One request needs 1 second to get done, but the client might send you either 1 request per 10 seconds in best case, because the server has 9 seconds to poke or 10 requests per 1 second in worst case, because the server needs 9 extra seconds. How to handle this? Use an MQ software (zeromq, rabbimq, apache kafka etc). It is a doubly-linked list with pointers to undone tasks, if any. If you have a 10 one-second requests in one second, you just do them as fast as you can and use MQ as a storage.