4 ms·

> If you side effect outside of the exactly once system, and the receiver can’t suppress duplicate notifications then you’re screwed If you are getting duplica

> If you side effect outside of the exactly once system, and the receiver can’t suppress duplicate notifications then you’re screwed If you are getting duplicate notifications, then you don't have an "exactly once" system, but definition. Exactly once systems are not possible, bu your own explanation. You can store data in a durable way such that applying it multiple times is safe, but that is not "exactly once". Your options are at-least once or at-most once. Again, by your own statement: if you have duplicate notifications in your system, you do not have "exactly once". > As for your other points, with eventual consistency you get weird problems like “I wrote X, then Y, but then read X and the system converged to X, except a week later I read Y, but just from half my fleet, and only for a 30 minute window”. This is not data corruption, this is the semantics of an eventually consistent system. If you don't want those semantics, don't have an eventually consistent system, but that is entirely different than data corruption. Developers not understanding the system is different than corruption. I agree that eventual consistent is probably not a good idea for most problems.

hedora 3y ago
Many, many distributed systems provide exactly once semantics by giving up availability. They don’t compose well with systems that don’t provide exactly once semantics, but that doesn’t somehow make them stop existing or stop working. Also, you don’t need to arrange for data structures that can be modified multiple times safety. You just need the exactly once system to support transactions, compare and swap, atomic rename, etc. Such things are usually pretty easy to retrofit. For instance, Google and Microsoft S3 support conditional writes that check the etag for equality. That’s enough to let you layer consistency and exactly once on top. AWS S3 doesn’t support this, and people have been asking for it for a long time. These things are extremely well understood and done at scale all the time. Edit: I think they use etags for this. They might expose it as atomic rename or create if not exists instead. The expressive power of the two should be equivalent.
- sethammons 3y ago
  Exactly once is a mathematical impossibility. It has been proven. See the two generals problem. You can get similar end results if you have idempotent calls and allow duplicate calls to be made. This gives the illusion or impression of exactly once.

hedora 3y ago

Many, many distributed systems provide exactly once semantics by giving up availability. They don’t compose well with systems that don’t provide exactly once semantics, but that doesn’t somehow make them stop existing or stop working. Also, you don’t need to arrange for data structures that can be modified multiple times safety. You just need the exactly once system to support transactions, compare and swap, atomic rename, etc. Such things are usually pretty easy to retrofit. For instance, Google and Microsoft S3 support conditional writes that check the etag for equality. That’s enough to let you layer consistency and exactly once on top. AWS S3 doesn’t support this, and people have been asking for it for a long time. These things are extremely well understood and done at scale all the time. Edit: I think they use etags for this. They might expose it as atomic rename or create if not exists instead. The expressive power of the two should be equivalent.

sethammons 3y ago
Exactly once is a mathematical impossibility. It has been proven. See the two generals problem. You can get similar end results if you have idempotent calls and allow duplicate calls to be made. This gives the illusion or impression of exactly once.