Here's a good article from New Relic on the problem, if you'd like more detail: https://newrelic.com/blog/best-practices/kafka-consumer-config-auto-commit-data-

Here's a good article from New Relic on the problem, if you'd like more detail: https://newrelic.com/blog/best-practices/kafka-consumer-config-auto-commit-data-loss#kafka-consumer-auto-commit-mitigating-data-loss-and-duplication https://newrelic.com/blog/best-practices/kafka-consumer-conf... Or here, you can reproduce it yourself using the Bufstream or Redpanda/Kafka test suite. Here's a real quick run I just dashed off. You can watch it skip over writes: https://gist.github.com/aphyr/1af2c4eef9aacde7f08f158230490855 https://gist.github.com/aphyr/1af2c4eef9aacde7f08f1582304908... lein run test --enable-auto-commit --bin bufstream-0.1.3-rc.12 --time-limit 30 --txn --final-time-limit 1/10000