4 ms·

The biggest gotcha in a design like this IMHO is that you can't post and delete atomically. You may post the new work into the queue and then a failure to delet

• nine_k 7y ago
  With a realistic, that is, not 100% reliable, queue you can have either "at most once" or "at least once" delivery anyway. "Exactly once" can't be guaranteed. So a duplicate message should be processed as normal anyway, e.g. by deduplication within a reasonable window, and/or by having idempotent operations.
  • Rapzid 7y ago
    Yes, it depends on the workload. Idempotency is typically always a good idea, but sometimes the operation itself is very expensive in terms of time, resources, and/or money. I have also seen people try to update the message when writing it back(with checkpoint information and etc) for long running processes. A slew of issues, including at least once delivery, can cause workflow bifurcation. Deduplication via FIFO _can_ help mitigate this, but it has a time window that needs to be accounted for. Once you start managing your own deduplication I'd say it has moved past trying to go databaseless.
• kondro 7y ago
  But you could adjust the message visibility timeout of the message you received so that it appears back later in the queue itself.
  • Rapzid 7y ago
    For this specific use case, as described, this sounds like a possible very good approach.
    • kondro 7y ago
      One thing to be aware of for this approach though is that you can only have 120k in-flight messages per queue.