If your clients don't implement an increasing backoff retry with jitter, you can fake it by making your server begin to randomly and increasingly time-out conne

If your clients don't implement an increasing backoff retry with jitter, you can fake it by making your server begin to randomly and increasingly time-out connections or wait to accept them. You can do it a few different ways. For already-open connections: 1) Keep the connection open and respond with a tiny amount of data every once in a while, so the connection doesn't time out, but make it long enough that a complete request will take forever.. Applications may still re-connect if a successful request-response doesn't happen within a timeout. 2) Keep the connection open but don't respond. The application may time out the connection and re-connect if it receives no data. 3) Drop the connection but don't let the tcp/ip stack send a RST, FIN, or anything else. The applications' connection will be timed out pretty soon, either by the OS stack or an application timeout. 4) Respond to requests with HTTP response codes that will make the client retry. As long as this only happens at the load-balancer level, it will still remove pressure from your application layer. For new connections: 1) Play hard to get. During the 3-way handshake, respond with weird states that won't cause the connection to drop, but will make the client keep trying to connect, like the handshake is still trying to succeed but experiencing loss. 2) Do a normal 3-way handshake, but wait forever on each step to drag it out. All of these options are a terrible idea for user-facing applications. They can cause more issues on your server side due to connections in wait state, and your clients will just see requests hanging, stalling, etc. You could just pretend the problem is with their ISP rather than your end. Or you could throw a 429 or 503 to try to stop the retries, but then they really know it's your fault. The best option is to just add capacity. Back in the day we couldn't do that; you had the servers you had. So when all 400 machines in our colo were thrashing, we just had to shed load randomly, sometimes using the tricks above. Now with the cloud you can magically add capacity anytime, automatically. Much better option. What's often much better is to add another caching layer before the application layer and just return stale responses during pressure events. But sometimes you can't, so you resort to either a 429 or 503, or the dirty tricks above.