4 ms·
AWS managed kafka is great but it's really just Kafka. Confluent offers stuff like topic monitoring, an http bridge, and other services that make kafka more pra
by EamonnMR 5y ago
AWS managed kafka is great but it's really just Kafka. Confluent offers stuff like topic monitoring, an http bridge, and other services that make kafka more practical to actually use (dare I say, missing features.)
My company uses Kafka extensively. We hosted it ourselves but after a few Kafka meltdowns we decided to go with a hosted solution. We compared AWS's offering to Confluent and at the time found that while Confluent had all of the nice tooling, we'd already built our own tooling to do the same thing, and rebuilding our entire infrastructure was a non-starter. So we went with AWS. It was a drop-in replacement for our own Kafka and it hasn't blown up on us since.
If we where starting fresh, Confluent would be more viable. I can't help but wish that the offerings that are part of Confluent where part of Kafka though. Kafka's lack of the tools that Confluent offers are imo what's holding it back from totally taking over the world.
- d_t_w 5y agoWe build a complete drop in replacement for Confluent's metrics/management tooling (https://kpow.io https://kpow.io) that fills a very large gap in the engineering experience for Kafka. I say replacement but tbh we offer a lot more in terms of features other than stopping/starting clusters. It feels obvious to me that AWS will roll-out S3 backed storage and managed Kafka Connect at some point, their recent IAM / ACL integration points to a pretty active MSK team. I completely agree in the past there were some pretty big gaps - that's why we built kPow - my feeling is those gaps are narrowing pretty significantly.
- EamonnMR 5y agoKafka S3 dumping would be amazing, could really clean up our tooling. I'll check out your product as well.
- d_t_w 5y agoHonestly I can't wait for perpetual topics backed by S3 integrated with MSK. If the AWS team aren't working on it I will eat my own shoes. Let me know how you go with kPow! If you want a guided tour just let me know. :) ps. we're releasing a major bump to the UI later today..