3 ms·
I would think the first resort in most of the non-flooding-related cases[1] is backpressure. If you've got too many requests coming in, be sure that internally
by jerf 3y ago
I would think the first resort in most of the non-flooding-related cases[1] is backpressure. If you've got too many requests coming in, be sure that internally you start parking the excess incoming requests in some low-resource-usage configuration (e.g., simply noting the incoming TCP connection but then not even read()ing from it until you are ready), and let the system push back naturally by the way its latency increases. This obviously naturally pairs with both client-side and server-side timeouts, giving both a vote in how long they're willing to wait.
This is generally what you want. Actively breaking the connections if I think I'm overloaded is only something I'd add if there is some additional constraint on the system that suggests it's a good idea due to some specific local concerns. Circuit breaking produces a much sharper (for lack of a better term) performance graph, where instead of degrading gracefully like backpressure will, you suddenly start getting active failures.
I've built a lot of backpressure in my systems; I'm yet to need a circuit breaker. YMMV, of course, but I see some people referring to it as a basic tool and I would disagree; the basic tool is backpressure.
[1]: Flooding is its own thing. In general, to weather DOS attacks but still have the service functional requires another system or likely set of systems in front of the service that can handle the DOS issues. I say this just to point out that circuit breakers aren't really for this case, because no system can on its own handle a full-on DOS flood, no matter what code you write, when the DOS flood may literally be coming in at a speed where it is faster than you can even reject connections, let alone do any work.
- vault_ 3y agoYeah, I don't think circuit breakers are really the appropriate choice in most of the situations the article is describing. Rate limiting and backpressure seem like better options most of the time. The way I see it, circuit breakers are safety devices. They're for when you need to keep a system in a safe control region and are wiling to sacrifice some reliability in order to achieve that. e.g. preventing customers from accidentally turning your globally distributed whatever into a DDOS platform or limiting the blast radius when infrastructure automation decides it should delete everything.
- rbanffy 3y agoOh yes! I used response times as a proxy for backend health a couple times. Still, it's better if the service tells you there is some degradation in a way the client can understand and handle. Sometimes, only some of the requests need to be slowed down or paused, leading to some limited functionality but not a complete loss of service.