4 ms·
Head-of-line blocking has two aspects: crashes and long processing times. If the first message for an ordered key can't be acked because it crashes subscribers,
by pradn 6y ago
Head-of-line blocking has two aspects: crashes and long processing times. If the first message for an ordered key can't be acked because it crashes subscribers, we can't make progress on the messages for that key. If the first message, likewise, takes too long, then it delays processing of subsequent messages for that key.
Even without ordering, we still have to deal with variations of these issues. A message may still crash subscribers, be redelivered, and crash yet more subscribers. A message may delay the processing of other messages on the subscription.
Here's one way to deal with crashing messages: keep a side-cache of processing attempts for every message ID and ack those that have been processed too many times before processing them. Cloud Pub/Sub has a dead-letter queues feature to do this automatically. (It is not yet implemented for subscriptions with ordering keys - one can't enable both features.) We can do something similar for long messages - acking the message on timeout or processing more messages in parallel.
So, with ordering keys, as it stands, you do have the extra problem of dealing with messages-of-death (crash-inducing messages) yourself, but the long-processing time issue still exists in a slightly different form. Hopefully, we can add DLQs for ordering keys if there's sufficient customer demand.
Disclaimer: I work on the Cloud Pub/Sub team, but this is my own explanation.