3 ms·
On the contrary, Ordering is something that is VERY clear in kafka, > By having a notion of parallelism—the partition—within the topics, Kafka is able to prov
by easytiger 12y ago
On the contrary, Ordering is something that is VERY clear in kafka,
> By having a notion of parallelism—the partition—within the topics, Kafka is able to provide both ordering guarantees and load balancing over a pool of consumer processes. This is achieved by assigning the partitions in the topic to the consumers in the consumer group so that each partition is consumed by exactly one consumer in the group. By doing this we ensure that the consumer is the only reader of that partition and consumes the data in order. Since there are many partitions this still balances the load over many consumer instances. Note however that there cannot be more consumer instances than partitions.
http://kafka.apache.org/documentation.html http://kafka.apache.org/documentation.html
- ewencp 12y agoIn particular, http://kafka.apache.org/documentation.html#intro_consumers http://kafka.apache.org/documentation.html#intro_consumers addresses the concept of consumer groups and what ordering is guaranteed. One thing that might be worth noting for the grandparent is that Kafka consumers have an offset commit API that gives some control over how failures are handled. If a consumer dies before committing an offset but after reading data from the broker, a fresh consumer that joins the consumer group can see the same data once the system determines the original has died; that ensures all data will be processed, even in the event of consumer failures. Kinesis provides the same ordering guarantees. They use different terminology (Kafka topics == Kinesis streams; Kafka partitions == Kinesis shards) but have the same system interface. The details of the APIs used for consumption differ, but they provide the same basic functionality of Kafka's "consumer groups".