4 ms·
Points 1 & 2. There is no direct communication between a producer and consumer. The producer writes to the broker, the consumer reads from the broker. There is
by apurvamehta 9y ago
Points 1 & 2. There is no direct communication between a producer and consumer. The producer writes to the broker, the consumer reads from the broker. There is a detailed flow diagram for the producer side operations in the article, and this deck has more of the details of how both the producer and consumer work: https://www.slideshare.net/apurva2/introducing-exactly-once-semantics-to-apache-kafka https://www.slideshare.net/apurva2/introducing-exactly-once-...
3. Yes, it has been tested empirically. Quoting from the article:
> We wrote over 15,000 LOC of tests, including distributed
> tests running with real failures under load and ran them
> every night for several weeks looking for problems. This
> uncovered all manner of issues, ranging from basic
> coding mistakes to esoteric NTP synchronization issues
> in our test harness. A subset of these were distributed
> chaos tests, where we bring up a full Kafka cluster with
> multiple transactional clients, produce message
> transactionally, read these messages concurrently, and
> hard kill clients and servers during the process to
> ensure that data is neither lost nor duplicated.