Specifically you want load shedding and in the servers and retry with back off in the clients. Clients should do their best to exponentially back off on retryi

Specifically you want load shedding and in the servers and retry with back off in the clients. Clients should do their best to exponentially back off on retrying failed requests and only try to contact healthy servers and maintain internal rate-limiting based on error rate, and servers should do their best to reply with failure quickly and cheaply to drive good client's backoffs/rate-limiting and just drop bad client's traffic, and the service discovery should try to detect and spread load across healthy servers (but this isn't always available at first in a cold start because the metrics or metadata probably aren't available yet), but in the end it's up to servers to reliably drop traffic they can't handle instead of building up giant queues and slowing to a crawl. Middleware is the hardest because it has to be a good client and also fail fast on overload as a server by correctly interpreting upstream behavior. Deadlines in RPCs that get passed across system boundaries can work pretty well for tall stacks of system layers where service health discovery or dependency discovery is hard, but require careful configuration to avoid failure or very slow starts under heavy load.