4 ms·

> and thus your system must handle the possibility that an event fails to be processed and must be retried/dropped Isn’t a retry just extending the queue to th

by konha 5y ago

> and thus your system must handle the possibility that an event fails to be processed and must be retried/dropped Isn’t a retry just extending the queue to the caller?

dcow 5y ago
That's why I said "retry/drop". Handling back-pressure means you must propagate it backwards to a point in your program where there is enough context to enable decision to be made on how to handle the failure. Also, a retry can wait some amount of time until the system is less busy (and this can be enforced by some other part if needed). Further, you may have many callers and one central thing processing requests so propagating back-pressure to callers also allows you to push the problem back to a point where there are potentially more resources. In a silly example, if someone spamming your queue starts to experience a sluggish UI or run out of RAM because back-pressure has forced the queue to grow on their machine an not your infrastructure, perhaps they'll get the message and cool it (ideally the program handles this gracefully too and tells the user a maximum has been hit, but even without that back-pressure is useful). So, yes. But that's the point.
ggrrhh_ta 5y ago
That's the best scenario; regardless of whether the event reaches the destination or not, in most cases (except continuous monitoring of some parameter which admits losses) the source still will keep the information or the way to generate that information, so the amount of state that the queue at the source requires is still being used. But moreover, the source can throttle the generation of events, or it can just decide that that event is stale and can be dropped, or can pack that event with the next events, or can signal other processes about what is happening, or it can choose other path, or maybe it has some out-of-band signaling capability, or....
im3w1l 5y ago
Yes but that queue is bounded too. And if you follow the systems backward eventually you end up with a real person. That is then forced to prioritize away less important tasks.
pornel 5y ago
Even worse, it multiplies the amount of requests hitting the overloaded processor. You can create a "metastable failure" situation that will prevent an overloaded service from ever recovering.
- sly010 5y ago
  "backpressure" is an unfortunate term because it's not something the queue does. "back-off" would probably be a better term, since it's the behavior of the producer. The producer has to be aware of it.
  - dcow 5y ago
    "backpressure" quite literally refers to a queue blocking when full. Exerting backpressure on a system is absolutely something a queue does. Backing off or "loadshedding" is something the producer does in response to backpressure from a queue.
    - sly010 5y ago
      The point I am trying to make is that backpressure is a contract between producer and consumer. It requires a shared understanding by the both sides. When you put a queue in your HTTP server, you are implementing both the producer (the code accepting connections) and the consumer (the code handling requests). TCP itself also works on a shared understanding that the producer will not send more data until it was ACK-ed. If a sender kept sending packets as fast as it can, that would be a DDOS attack, and you can't stop a network level DDOS attack by adding a queue, you can only be fast enough at discarding malicious packets. So back-pressure absolutely requires the producer to behave. You don't even need a queue for back-pressure. Java reactive streams create back-pressure without queues by making the consumers "request" data. In fact TCP would work "fine" without a queue (i.e. with a window size of 1) it would just be slow. In a sense queues are actually used to avoid back-pressure, not to create it.
      - dcow 5y ago
        Backpressure is not a contract. A protocol is a contract and there are plenty of examples of algorithms deployed that account for the existence of backpressure. But you do not need the consumer to do anything special for a system to have backpressure. I still don't understand your point about queues not creating backpressure. If we're getting abstract about it, everything has a queue size of 1. In the article and in this thread we're talking about infinite vs bounded queues. An ideal infinite queue has no backpressure because anything can be immediately queued. However, it comes at the unfortunate expense of infinite processing time. The difference between an infinite queue and a bounded queue is that a bounded queue, when it fills up, propagates backpressure. If you choose a queue size of 1 then whatever, it doesn't change anything conseptually. You're just trying to say a queue size of 1 is not a queue and you add a queue when you go from size 1 to > 1. And now we're just arguing semantics. Making a bounded queue bigger adds slack and relieves pressure caused by unpredictable producers. But it does not change the maximum throughput of your system. The only thing that can do that is adding more consumers or making the existing ones faster (either by improving their speed or allowing more sophistication in the handling of requests in the queue, e.g. batch processing). If you system is already at peak load, adding a queue does not relieve pressure.

by konha 5y ago

> and thus your system must handle the possibility that an event fails to be processed and must be retried/dropped Isn’t a retry just extending the queue to the caller?

dcow 5y ago

That's why I said "retry/drop". Handling back-pressure means you must propagate it backwards to a point in your program where there is enough context to enable decision to be made on how to handle the failure. Also, a retry can wait some amount of time until the system is less busy (and this can be enforced by some other part if needed). Further, you may have many callers and one central thing processing requests so propagating back-pressure to callers also allows you to push the problem back to a point where there are potentially more resources. In a silly example, if someone spamming your queue starts to experience a sluggish UI or run out of RAM because back-pressure has forced the queue to grow on their machine an not your infrastructure, perhaps they'll get the message and cool it (ideally the program handles this gracefully too and tells the user a maximum has been hit, but even without that back-pressure is useful). So, yes. But that's the point.

ggrrhh_ta 5y ago

That's the best scenario; regardless of whether the event reaches the destination or not, in most cases (except continuous monitoring of some parameter which admits losses) the source still will keep the information or the way to generate that information, so the amount of state that the queue at the source requires is still being used. But moreover, the source can throttle the generation of events, or it can just decide that that event is stale and can be dropped, or can pack that event with the next events, or can signal other processes about what is happening, or it can choose other path, or maybe it has some out-of-band signaling capability, or....

im3w1l 5y ago

Yes but that queue is bounded too. And if you follow the systems backward eventually you end up with a real person. That is then forced to prioritize away less important tasks.

pornel 5y ago

Even worse, it multiplies the amount of requests hitting the overloaded processor. You can create a "metastable failure" situation that will prevent an overloaded service from ever recovering.

sly010 5y ago
"backpressure" is an unfortunate term because it's not something the queue does. "back-off" would probably be a better term, since it's the behavior of the producer. The producer has to be aware of it.
- dcow 5y ago
  "backpressure" quite literally refers to a queue blocking when full. Exerting backpressure on a system is absolutely something a queue does. Backing off or "loadshedding" is something the producer does in response to backpressure from a queue.
  - sly010 5y ago
    The point I am trying to make is that backpressure is a contract between producer and consumer. It requires a shared understanding by the both sides. When you put a queue in your HTTP server, you are implementing both the producer (the code accepting connections) and the consumer (the code handling requests). TCP itself also works on a shared understanding that the producer will not send more data until it was ACK-ed. If a sender kept sending packets as fast as it can, that would be a DDOS attack, and you can't stop a network level DDOS attack by adding a queue, you can only be fast enough at discarding malicious packets. So back-pressure absolutely requires the producer to behave. You don't even need a queue for back-pressure. Java reactive streams create back-pressure without queues by making the consumers "request" data. In fact TCP would work "fine" without a queue (i.e. with a window size of 1) it would just be slow. In a sense queues are actually used to avoid back-pressure, not to create it.
    - dcow 5y ago
      Backpressure is not a contract. A protocol is a contract and there are plenty of examples of algorithms deployed that account for the existence of backpressure. But you do not need the consumer to do anything special for a system to have backpressure. I still don't understand your point about queues not creating backpressure. If we're getting abstract about it, everything has a queue size of 1. In the article and in this thread we're talking about infinite vs bounded queues. An ideal infinite queue has no backpressure because anything can be immediately queued. However, it comes at the unfortunate expense of infinite processing time. The difference between an infinite queue and a bounded queue is that a bounded queue, when it fills up, propagates backpressure. If you choose a queue size of 1 then whatever, it doesn't change anything conseptually. You're just trying to say a queue size of 1 is not a queue and you add a queue when you go from size 1 to > 1. And now we're just arguing semantics. Making a bounded queue bigger adds slack and relieves pressure caused by unpredictable producers. But it does not change the maximum throughput of your system. The only thing that can do that is adding more consumers or making the existing ones faster (either by improving their speed or allowing more sophistication in the handling of requests in the queue, e.g. batch processing). If you system is already at peak load, adding a queue does not relieve pressure.