4 ms·

https://kpow.io https://kpow.io - we build kPow for Apache Kafka, a drop-in replacement for Confluent Control Center + much more. https://kpow.io/features http

by d_t_w 5y ago

https://kpow.io https://kpow.io - we build kPow for Apache Kafka, a drop-in replacement for Confluent Control Center + much more. https://kpow.io/features https://kpow.io/features - Multi-Cluster, Multi-Schema, Multi-Connect. - Multi-Tenancy, RBAC, all the Authz - LDAP, SAML, etc. - Very fast streaming multi-topic search (~1M messages / minute) - Built in JQ search filters (we implemented a subset of JQ in the browser and JVM) - Live Kafka Streams / Consumer Group topology visualisation (including RocksDB metrics for state stores, etc) - Offset management, resetting, scheduled mutations. - Consumer Group read and Topic write freshness / n-minutes inactivity metrics. - Heaps of admin functions like Staged Mutations, Temporary RBAC policies, Data Masking for PII, etc. - Prometheus / OpenMetrics endpoints. - kREPL (slightly mad programmatic interface to Kafka) - 7k commits, 100k docker pulls, 97.4% Clojure. - Unlimited users, unlimited features. - Commercially supported, annual licenses or available as hourly-metered (0.16c/hr) on the AWS Marketplace or Red Hat Marketplace.

d_t_w 5y ago
Multi-Tenancy is probably worth a link: https://kpow.io/how-to/manage-kafka-visibility-with-multi-tenancy/ https://kpow.io/how-to/manage-kafka-visibility-with-multi-te... kPow gives you the ability to restrict visibility of Kafka resources down to a group or topic level by user-role, then builds completely internally consistent synthetic cluster views for each tenant. We have a number of large enterprise customers who requested multi-tenancy support, it sounded tough at first but due to how kPow is built (we compute our own metrics / no JMX) and the fact that we use Clojure in both the front/back end it turned out to be fairly straight forward to implement.
dicka_j 5y ago
spoken to a few companies that have started using this internally - bit of a game changer apparently, kudos
- deleted 5y ago
  [deleted]
chadhq 5y ago
kRepl is mind blowing. Our engineers have used it very effectively to help diagnose issues in production.
tiew9Vii 5y ago
Is the monitoring functionality independent off the cluster it is monitoring being up to see monitoring? Control centre eats its own dog food. All its data is on topics. That means if your cluster is unhealthy you can see healthy stats until the control centre consumers have caught up. Which means if you have a lot of data and load, control centre can start lagging and in one case I had some very hard debugging wondering why control centre saying the cluster was unhealthy but the cluster was up. It turned out control centre was lagging by 24 hrs so I was seeing the previous days outage. Confluent recommended putting control centre on its own cluster for this very reason. One that isn’t cheap, a minimal prod cluster is 3 brokers+zookeepers and secondly what then monitors your monitoring cluster.
- d_t_w 5y ago
  Yes, in two ways. First - kPow also stores all telemetry in internal topics (and also keeps an audit-log in another topic for data governance). Setting up multi-cluster is fairly easy and kPow stores data in the first configured cluster - so like Confluent's suggestion you could have a dedicated kPow cluster. We really only recommend this if/when you get up to a large number of connected Kafka resources (Cluster, Schema, or Connect) and/or you have cluster/s that are very busy with other work. Second - from v85 (next week) our v3 compute engine make kPow monitoring functionally independent of topic load / performance of the underlying cluster. We do still persist to internal topics for recovery, etc, but live stats and control are only dependent on your Kafka cluster responding to AdminClient requests (a handful every minute). If you would like a trial license or guided tour, just say.
deleted 5y ago
[deleted]

by d_t_w 5y ago

d_t_w 5y ago

Multi-Tenancy is probably worth a link: https://kpow.io/how-to/manage-kafka-visibility-with-multi-tenancy/ https://kpow.io/how-to/manage-kafka-visibility-with-multi-te... kPow gives you the ability to restrict visibility of Kafka resources down to a group or topic level by user-role, then builds completely internally consistent synthetic cluster views for each tenant. We have a number of large enterprise customers who requested multi-tenancy support, it sounded tough at first but due to how kPow is built (we compute our own metrics / no JMX) and the fact that we use Clojure in both the front/back end it turned out to be fairly straight forward to implement.

dicka_j 5y ago

spoken to a few companies that have started using this internally - bit of a game changer apparently, kudos

deleted 5y ago
[deleted]

chadhq 5y ago

kRepl is mind blowing. Our engineers have used it very effectively to help diagnose issues in production.

tiew9Vii 5y ago

Is the monitoring functionality independent off the cluster it is monitoring being up to see monitoring? Control centre eats its own dog food. All its data is on topics. That means if your cluster is unhealthy you can see healthy stats until the control centre consumers have caught up. Which means if you have a lot of data and load, control centre can start lagging and in one case I had some very hard debugging wondering why control centre saying the cluster was unhealthy but the cluster was up. It turned out control centre was lagging by 24 hrs so I was seeing the previous days outage. Confluent recommended putting control centre on its own cluster for this very reason. One that isn’t cheap, a minimal prod cluster is 3 brokers+zookeepers and secondly what then monitors your monitoring cluster.

d_t_w 5y ago
Yes, in two ways. First - kPow also stores all telemetry in internal topics (and also keeps an audit-log in another topic for data governance). Setting up multi-cluster is fairly easy and kPow stores data in the first configured cluster - so like Confluent's suggestion you could have a dedicated kPow cluster. We really only recommend this if/when you get up to a large number of connected Kafka resources (Cluster, Schema, or Connect) and/or you have cluster/s that are very busy with other work. Second - from v85 (next week) our v3 compute engine make kPow monitoring functionally independent of topic load / performance of the underlying cluster. We do still persist to internal topics for recovery, etc, but live stats and control are only dependent on your Kafka cluster responding to AdminClient requests (a handful every minute). If you would like a trial license or guided tour, just say.

deleted 5y ago

[deleted]