I mean, they're kind of apples and oranges. nsq is a primarily in-memory (minus the fallback to disk if too many messages are in flight) pubsub service fundamen

I mean, they're kind of apples and oranges. nsq is a primarily in-memory (minus the fallback to disk if too many messages are in flight) pubsub service fundamentally designed for spreading load and availability. There's no delivery or ordering guarantees, no replication. Kafka is a distributed, partitioned log. Messages published to a partition in a given order will be stored in that order and replicated in that order, and consumed in that same order. Kafka writes all of your data to disk, and waits for replicas to acknowledge it (depending on your producer configuration). Kafka does considerably more work than nsq. I like nsq, it works well, but they aren't at all designed to solve the same problems. I mentioned the issues above because they'd happened, but in years of running Kafka it was never Kafka's fault. > "no leader for partition" is something I see frequently. Something is wrong with your cluster if it's constantly switching leadership. Leadership should be stable -- and furthermore, partitions have preferred leaders, so they should stay with specific nodes until that node goes down. Kafka has a process which shifts leadership back to preferred leaders automatically (unless it's off). Check your logs.