4 ms·
Most of these issues stem from the fact that queues provide no useful guarantees about how many times each entry will be processed (zero? one? many?), at least
by victorNicollet 3y ago
Most of these issues stem from the fact that queues provide no useful guarantees about how many times each entry will be processed (zero? one? many?), at least not without a large amount of work around the queues, usually enough to make them unrecognizable.
Despite the prose, the article is dripping with actual production experience, and many things I wish I had known earlier. To craft a good design, one must be aware of all the problems that the design should fix in order to be good (and accept that settling early into a design will prevent some problems from ever being fixed).
- barrkel 3y agoOne of the points of the article is that duplicate messages are a side-effect of error recovery, which is external to the queue itself. Even if your queue guaranteed exactly-once delivery, the thing which processes the delivery may fail, which requires re-enqueuing the message, and such error recovery re-enqueues may trivially race with normal enqueuing.
- victorNicollet 3y agoAgreed ! I would add that some queue solutions (such as Azure) implement a lease system where a dequeued message re-appears at the head of the queue after a while, unless its removal is explicitly confirmed.