5 ms·
Why not mention the architecture that comes closest to exactly-once delivery? If you store a Kafka offset along with your application state in a transactional d
by bjornsing 4y ago
Why not mention the architecture that comes closest to exactly-once delivery? If you store a Kafka offset along with your application state in a transactional datastore, then for all intents and purposes you have exactly-once delivery semantics. This is something I really like about Kafka’s design.
- infamouscow 4y agoI suspect it's not mentioned because in the real world there are ways to work around this limitation. And because those work arounds actually work, there's nothing interesting to say. The post is much ado about nothing.
- bjornsing 4y agoYes, it’s always possible to build a big pile of workarounds. I agree.
- kevincox 4y agoThis is again assuming that you have no side effects. Imagine that you want to email users based on a list in Kafka. You read the offset in a transaction and update it. But do you send the email inside the transaction or after closing it? You are back to picking between at-least-once and at-most-once.
- taspeotis 4y agoJust put the offset in the email to users and make them keep track of it??
- bjornsing 4y agoYou got some good jokes as answers already, but what you do if you care about correctness is to store a “send email job” in your transactional data store.
- morelisp 4y agoWhat transactional data store capable of sending email do you recommend?
- bjornsing 4y agoJust about any transactional datastore will do. Just google for “job queue in [insert name of datastore here]”.
- morelisp 4y agoYou've missed the point: How do you tie sending email to the transactional semantics? Your email server and your SQL server (or whatever) make just as much of a distributed system as Kafka and the email server without some other store in the middle. If you want to solve it really, your email server itself needs to participate in the consensus process in some way.