3 ms·
Author doesn't understand Kafka, doesn't have a good client for his language, therefore doesn't like Kafka. Jay Kreps responds to a few of his points -- the co
by Xorlev 10y ago
Author doesn't understand Kafka, doesn't have a good client for his language, therefore doesn't like Kafka.
Jay Kreps responds to a few of his points -- the complexity of the client is for scalability reasons.
> When you Produce a Message Set onto the bus, you don't directly get back a response telling you that the messages have successfully been persisted to one or more partitions.
At least in the Java client, this isn't true. True, if you used the async API before 0.9 you weren't able to get an ACK, but the sync producer would block until a message was published. In the new consumer, you're handed futures + the ability to provide a callback[1].
[1] http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html http://kafka.apache.org/082/javadoc/org/apache/kafka/clients...
- pfraze 10y agoThere's one specific thing that I'd point out to the author: Zookeeper and load-balancers accomplish different things. If leader-election is being used, then the cluster needs to have a single authority to enforce strict consistency in the logs. A load-balancer can only distribute load between separate nodes. If there were only a load-balancer, and no leader election, then Kafka would not be able to create logs with total orders, and you'd have a different system. (We can talk about alternative architectures, but I'm doubtful there's an alternative to leader-election that maintains the same properties.) He's complaining that Zookeeper takes 6 seconds to restore uptime in a failure, which may be improvable, but it's not really bad. If you want strict consistency, you either have a leader-election protocol, or you manually reconfigure your cluster when it fails. So, it's 6 automated seconds vs N manual minutes of labor, after your pager goes off.