4 ms·
AWS makes the same claim with FIFO SQS, and maybe I’m getting it wrong, but these claims 1) have a lot of caveats and 2) only work inside the boundaries of the
by sudhirj 6y ago
AWS makes the same claim with FIFO SQS, and maybe I’m getting it wrong, but these claims 1) have a lot of caveats and 2) only work inside the boundaries of the messaging system.
There’s a note in the next paragraph about how systems manage to say that if you pass in the same message ID / token for X minutes they won’t be duplicated, and my ensuring FIFO there’s a side effect of not giving out the next message until the current one is acknowledged.
This leads to a situation where there’s a guarantee of exactly once acknowledgement, but not necessarily exactly-once processing or delivery. Given that the semantics of at-most-once and at-least-once apply to processing and delivery, I personally don’t think the goalposts should move on exactly once.
Systems claiming exactly-once lull developers into not planning for multiple deliveries on the subscriber, or the need to do multiple publishes, both of which can still happen.
- vp8989 6y agoIt's better to use a SQS standard queue and have the consuming system provide the exactly-once processing guarantee for various reasons. You will need to introduce something like Redis, if you are not already using it, but I still think it's net superior to using an SQS FIFO queue if you want exactly-once processing.
- sudhirj 6y agoMight not even need to use Redis. If the message has a proper idempotent ID a transactional database is more than enough. If the consumer is running MySQL/Postgres/DynamoDB etc nothing else is needed.