4 ms·
Separates storage from brokers for better scaling and performance. Millions of topics without a problem and built-in tenant/namespace/topic hierarchy. Kubernete
by manigandham 7y ago
Separates storage from brokers for better scaling and performance. Millions of topics without a problem and built-in tenant/namespace/topic hierarchy. Kubernetes-native. Per-message acknowledgement instead of just an offset. Ephemeral pub/sub or persistent data. Built-in functions/lambda platform. Long-term/tiered storage into S3/object storage. Geo-replication across clusters.
- philote 7y agoI'd looked into Kafka before and really liked the partitioned topics and specifying partition keys. This makes sure only one consumer handles all messages for a specific key. Is there a way to do this in Pulsar? I see they have partitioned topics but it seems to only for performance reasons, but I don't see how you'd set up a partition key or make sure a specific key would always go to the same consumer. EDIT: Also wanted to note that, for Python, there is some support for async clients in Kafka (aiokafka, asynckafka) but I don't see any async libs for Pulsar yet. Maybe I'm missing something.
- manigandham 7y agoYes, you can tag messages with a 'key' for compaction (maintaining the latest message for a given key) and there's now a beta feature for key-ordered consumers on the same topic: https://pulsar.apache.org/docs/en/concepts-messaging/#key_shared https://pulsar.apache.org/docs/en/concepts-messaging/#key_sh... You can also just use separate topics since they're very lightweight and can be nested under the tenant/namespace hierarchy.
- staticassertion 7y agoWow, yeah that does sound way better than Kafka, and very useful for what I'm doing. Thanks.