> BTW: Why do you think BookKeeper is better than Kafka for this purpose? :-) One of the reasons is that your storage is not tied to ingest. With Kafka, your b

> BTW: Why do you think BookKeeper is better than Kafka for this purpose? :-) One of the reasons is that your storage is not tied to ingest. With Kafka, your broker sits where the data sits. That has advantages and disadvantages. The disadvantages are that it's really difficult to do unlimited history topics / partitions in Kafka edit for significant volume topics. You can have only so much storage and only so many disks attached to your machine. It's not possible to add volumes at runtime. I could be wrong but I think Kafka still requires a broker restart after adding volumes to the server properties. With BookKeeper, you just add nodes with more storage. BookKeeper tracks who was the owner of that partition at the time of write. It's possible to spread partitions across multiple machines.