I mean, you can do exactly-once delivery, it's just a more complex state machine with a specific design requirement. Here it is using mail: # Receive new mai

I mean, you can do exactly-once delivery, it's just a more complex state machine with a specific design requirement. Here it is using mail: # Receive new mail message, incoming id '1234'. # No other writer knows about this message. $ echo "hello world" > incoming/1.1234.txt # If files stay in incoming/ too long, it means they probably failed writing, # and are thus stale and can be removed. # When the message is done being written, the writer moves it to a new state. $ mv incoming/1.1234.txt new/1.1234.txt # The writer can then report to the sender that the message was accepted. # In mail, this is seen as optional, as the whole point is to deliver the message, # not to make sure you inform the sender that you will deliver the message. # Here's the exactly-once bit: # If you need assurance of exactly-once delivery, split out the delivery portion # from the confirmation portion. # # Have one actor talk to the sender, and another actor queue the message in another # temporary queue, like 'accepted/'. Once the message is queued there, you tell the # client it is accepted, wait for successful termination of the connection, and then # move the message to the 'new/' queue. # # This way, if there is an error between accepting the message and confirming with # the sender, you can detect that and leave the message in 'accepted/' where you can # later decide what to do with it. (But it won't get picked up for processing) # The message is now ready to be picked up for processing. # To process the message, a processor picks up the new message and moves it to a new state. # # The processor adds an identifier '5678', so the processor can identify which file it's working on. # It can have extra logic to re-attempt processing on this ID if it fails. $ mv new/1.1234.txt process/1.1234.5678.txt # If the processor dies or takes too long, you can identify that (stale file, processor tracking its work, etc) # and this can be moved back to 'new/' for another attempt. Moving it back to 'new/' is still atomic so # there is no danger of another processor continuing to work on it. # Once the processor is done processing, it will move to the complete state. $ mv process/1.1234.5678.txt done/1.txt # If the processing file no longer exists, it means it was either already done, # or previously died and was moved back to 'new/'. This process all depends on a database [filesystem] using synchronous atomic operations [mv]. If your distributed system can't handle that, yeah, you're gonna have a hard time.