These are some pretty huge improvements to what was previously the weakest link in Kafka's API: the Producer. However, it's important to note that this can onl

These are some pretty huge improvements to what was previously the weakest link in Kafka's API: the Producer. However, it's important to note that this can only provide you with exactly-once semantics provided that the state/result/output of your consumer is itself stored in Kafka (as is the case with Kafka Streams). Once you have a consumer that, for example, makes non-idempotent updates to a database, there's the potential for duplication: if the consumer exits after updating the database, but before committing the Kafka offsets. Alternatively, it can lead to "message loss" if you use transactions on the database, and the application exits after the offsets were committed, but before the database transaction was committed. The traditional solution to this problem is to provide your own offset storage for this consumer within the database itself, and update these offsets in the same transaction as the database modifications. However, I'm not certain that, combined with the new Producer API, this would provide exactly-once semantics. Even if it doesn't, it's still a huge improvement, and significantly reduces the situations under which duplicate messages can be received.