3 ms·

Can you explain what your issue is with Kafka? What makes it not reliable enough?

• capmorganbih 2y ago
  Kafka is good. The problem is we don’t have a dedicated person to manage it so sometimes we have kafka outages
  • jojohohanon 2y ago
    There’s a whole conference in Vegas next week that want to sell you something.
  • dikei 2y ago
    So basically, you want to build a write-ahead-log before writing data to Kafka, and I think you're underestimating the effort to implement a WAL. If you don't have a person that can manage Kafka, you almost definitely don't have the person to maintain a WAL.
  • lizhenqi 2y ago
    Just a suggestion, you can learn about automq (a message queue that has rewritten the storage layer based on Kafka), which has done a lot in automatic recovery and self-balancing, maybe it will help you. If you don't need it, you can ignore this comment.