4 ms·
Finally! Having had to deal with properly setting zookeeper up on multiple mesos clusters was so painful and recovering from outages via s3 backups so excruciat
by KenanSulayman 6y ago
Finally! Having had to deal with properly setting zookeeper up on multiple mesos clusters was so painful and recovering from outages via s3 backups so excruciating.. I feel somewhat ashamed that in two out of five cases we simply went with the managed Kafka by AWS because the team didn’t feel confident enough to maintain the cluster.
- gen220 6y agoHonestly this is probably the right thing to do. My cynical conspiracy is that the ZK dep was organized by cloud vendors to keep customers coming :) It’s been at least a year and a half since we’ve had severe data loss on our homebrew Kafka cluster, but after the first couple you never look at Kafka+Zk the same way... both iirc were due to leadership election bugs that had been reported many months ago with no progress on a solution. I have no idea how AWS internally puts up with this. I wouldn’t be surprised if they’d replaced the ZK dependency internally years ago.