4 ms·
On backpressure: "Common versions include dropping new messages on the floor (and incrementing a metric) if the system’s resources are already over-scheduled."
by honkhonkpants 10y ago
On backpressure: "Common versions include dropping new messages on the floor (and incrementing a metric) if the system’s resources are already over-scheduled."
What I would add here is that "and incrementing a metric" must use fewer resources than would have been spent serving the request. I once worked on a system that would latch up into an overload state because in overload it would log errors at a severity high enough to force the log file to be flushed on every line. This was counterproductive to say they least. From that I learned that a system's flow control / pushback mechanism must be dirt cheap.
- susan_hall 10y agoOn dealing with backpressure, I would recommend what Zach Tellman writes about in his video "Everything will flow": https://www.youtube.com/watch?v=1bNOO3xxMc0 https://www.youtube.com/watch?v=1bNOO3xxMc0 Tellman also tells his own adventure about a system something like this: "a system that would latch up into an overload state because in overload it would log errors at a severity high enough to force the log file to be flushed on every line" He talks about some problems he faced when writing to Amazon S3, when S3 stopped accepting requests. It's a fascinating talk.