3 ms·
Your confusion is because you assume the requests have to come in via individual sockets, but several distributed designs have servers connect to a frontend and
by vidarh 7y ago
Your confusion is because you assume the requests have to come in via individual sockets, but several distributed designs have servers connect to a frontend and wait for requests instead of having the frontend connect. Many servers are also clients of another service.
E.g. Mongrel2 works that way.
It has the advantage that your reverse proxy does not need configuration changes to know how many processes are available - it only knows which backend servers have connected and not time out. Some such designs will also explicitly have the server do an RPC in to the frontend proxy requesting a request; doing that then also has the advantage that the backends effectively rate-limit themselves by placing themselves in the ready queue when they have capacity, and all you need to know to see if you need more backend capacity is to monitor how deep the ready queue is.