3 ms·
Some additional things that can be done: * soft-shedding, where instead of dropping a request (which might just incur a retry storm), sometimes it is appropria
by drdrey 10y ago
Some additional things that can be done:
* soft-shedding, where instead of dropping a request (which might just incur a retry storm), sometimes it is appropriate to send back a cheap response so that the clients sees a successful response instead of an error
* route critical requests and non-critical requests to separate clusters that can be scaled and configured independently. The blog post mention doing that using DNS, but that also works for mid-tier services.
* build back-pressure into the client. Instead of a timeout or error, a well-conforming client can enter "polite" mode when it receives a signal that the backend is overwhelmed.
- euyyn 10y agoIs soft-shedding a good idea? You're still paying CPU time to come up with a response that is a 200, but probably won't fool the client or the user anyway. Without a signal that the server is having problems with load, the retries can be more aggressive.
- toomuchtodo 10y agoThat soft response should be a 429, with the client performing exponential backoff.
- dantiberian 10y agoImagine a Netflix type service which needed to calculate a bunch of different categories to show the user. If some of them are more expensive to compute than others, those categories could be dropped, while still returning an acceptable response to the user.
- drdrey 10y agoIt makes sense if CPU time is not your most constrained resource. For instance, you might use it to avoid doing expensive remote calls.