3 ms·

Does anyone know if Kafka has improved on their data loss issues since tested by Aphyr? https://aphyr.com/posts/293-jepsen-kafka https://aphyr.com/posts/293-jep

Does anyone know if Kafka has improved on their data loss issues since tested by Aphyr? https://aphyr.com/posts/293-jepsen-kafka https://aphyr.com/posts/293-jepsen-kafka A quote from the article: "At the end of the run, Kafka typically acknowledges 98–100% of writes. However, half of those writes (all those made during the partition) are lost."

lars_francke 11y ago
Yes, the suggestion discussed by Aphyr has been implemented. You can now set up a lower bound on the ISR size (min.insync.replicas). Together with required.acks=-1 you can wait for a message to be committed to at least min.insync.replicas nodes. https://issues.apache.org/jira/browse/KAFKA-1555 https://issues.apache.org/jira/browse/KAFKA-1555