4 ms·
I recommend reading the Pulsar architecture page: https://pulsar.apache.org/docs/latest/getting-started/ConceptsAndArchitecture/ https://pulsar.apache.org/docs/
by manigandham 9y ago
I recommend reading the Pulsar architecture page: https://pulsar.apache.org/docs/latest/getting-started/ConceptsAndArchitecture/ https://pulsar.apache.org/docs/latest/getting-started/Concep...
Biggest difference is that Pulsar splits compute from storage. Pulsar brokers are stateless services that manage topics and handle clients (publishers and consumers) while reading/writing data from a lower tier of bookies, which is the Apache BookKeeper project for low-latency real-time storage workloads.
Better performance and easier scaling compared to Kafka's partition rebalancing. It also natively supports data center replication and multiple clusters working together globally. Pulsar addressing is property/cluster/namespace/topic so multi-tenant isolation is built-in.
Anywhere you use Kafka can use Pulsar and also consolidate pub/sub only systems that need lower latency than Kafka.
- lmsp 9y agoI think it is beyond partition rebalancing. There is a fact that people didn't realize of making message broker `stateless`. It is actually much better on reacting to failures or shifting traffic, which is critical when running a messaging bus for online services. because it doesn't have to wait for copying the data of a whole partition when error occurs.
- manigandham 9y agoAgreed, the architecture page and other blog posts do a more thorough job of explaining the details. Having a stateless broker layer on top of a focused data layer makes all the operations much easier. I expect BookKeeper to further integrate with the various cloud storage APIs so it can also start to become stateless cache.