4 ms·
Absolutely, we used store-and-forward semantics when I used Kafka at a previous company to guarantee zero message loss (for data getting into kafka). Now that k
by chicagobuss 9y ago
Absolutely, we used store-and-forward semantics when I used Kafka at a previous company to guarantee zero message loss (for data getting into kafka). Now that kafka streams provides exactly once semantics, you're in an even better spot to achieve this.
- zbentley 9y agoYou probably already know this, but obligatory reminder for others: Kafka's exactly-once delivery is not the same as exactly-once execution of an arbitrary workload. E1 mode is incredibly convenient, but it doesn't solve the tricky parts of, for example, hitting an external ACH to process a credit card payment. What if the external request times out? Fails silently? Fails loudly? Exactly-once delivery can help develop solutions in this area, but it doesn't eliminate this problem domain.
- chicagobuss 9y agoOf course, the key to any successful implementation of end-to-end exactly-once processing lies in the fact that you USE the kstreams APIs - including all semantics around "consumption is completed up to offset X"