4 ms·
> If we define 'processed' as fully committed, then messages ca be processed exactly once. If it can be processed once, it can be delivered exactly once if you
by BeeOnRope 2y ago
> If we define 'processed' as fully committed, then messages ca be processed exactly once. If it can be processed once, it can be delivered exactly once if you accept the simplest definition of that word.
I don't see how. Let's take a simple example of a the receiving end of a messaging system, where the messaging system can do whatever it wants to enforce idempotency or whatever semantics it wants, and calls, in-process, some `handler()` method of the application.
The useful processing happens somewhere in `handler()`. No matter what point you identify as the point where processing happens inside handler() it won't have delivered/processed-once semantics: it will potentially be called multiple times. The fact that the messaging system will internally have a commit step after handler() completes which is idempotent is irrelevant: as a user of the system you care how many times handler() is called.
- socketcluster 2y agoIf the sender is writing a message with a UUID and/or a unique incrementing sequence order (to maintain the order) to some kind of remote database using the UUID as the ID; they would be written to the DB in-order and unique. Then the receiver would receive the messages by reading them from the database sorted by their sequence number. The receiver would mark off messages in the DB as processed whenever it receives them. If it fails, it resumes from the last unprocessed one based on sequence number. If we have a chain of senders and receivers and absolutely wanted to avoid double processing instead of also maintaining idempotence down the line as described above. It is also technically possible (but I guess not worth the complexity); we could make it so that the receiver would add a flag 'about_to_process' on each record just before it starts processing the record and then it would change it to 'committed' once it finishes. If the receiver crashes before completing processing a message, it would see the 'about_to_process' flag and in that case it could ask the next process/receiver in the sequence if they already received a packet with that UUID and only send the output for that message if they have not. It's very ugly but it's possible.
- BeeOnRope 2y agoI guess we are probably in violent agreement on most of this.