3 ms·

"Really new" around November 2015. It was released in 2011 though. It's only 2 years younger than Redis, and is primarily an exercise in using Zookeeper. It's

"Really new" around November 2015. It was released in 2011 though. It's only 2 years younger than Redis, and is primarily an exercise in using Zookeeper. It's very surprising to hear people suggest Redis pubsub is a valid substitute for Kafka, when in fact it's not. It has a fundamentally different set of operating characteristics, a different sweet spot. Kafka isn't great from a consumer resumption point of view, but at least there ARE options. It's also untrue that Kafka gives no feedback on a successful message put. This is obviously a bug or design shortcoming in the post author's chosen toolchain which is correctable, and was part of the core java toolchain as of late 2015 AFAIK. I do agree that the Kafka system has an architecture that maximizes difficulty for new language bindings. Certainly C# has the tools to write an excellent implementation, assuming someone understands zookeeper well enough.

Sphax 10y ago
I don't understand what is difficult about thé kafka protocol. as a client you only ever need to talk to the broker. this uses a binary protocol that is really straightforward to implement. any language that can work with binary and can open a tcp socket is suitable to write a client.
- KirinDave 10y ago
  What threw me off is some disaster recovery scenarios are a lot easier if you can inspect and modify the broker data put into ZK.
  - Sphax 10y ago
    having your client modify the ZK data is not a good idea, you're bound to cause problems.
    - KirinDave 10y ago
      If you were in mid-block consumption in some cases it may make sense to advance your index to avoid replays. You can say, "You should have made the message consumer idempotent" and I reply, "I agree this was a consulting gig I was there to fix the problems." ;)
      - Sphax 10y ago
        oh you're talking about consumer offsets ! I exclusively use kafka to store offsets since I started using kafka with 0.8.0.2 so I can set my offsets by using the kafka protocol only

Sphax 10y ago

I don't understand what is difficult about thé kafka protocol. as a client you only ever need to talk to the broker. this uses a binary protocol that is really straightforward to implement. any language that can work with binary and can open a tcp socket is suitable to write a client.

KirinDave 10y ago
What threw me off is some disaster recovery scenarios are a lot easier if you can inspect and modify the broker data put into ZK.
- Sphax 10y ago
  having your client modify the ZK data is not a good idea, you're bound to cause problems.
  - KirinDave 10y ago
    If you were in mid-block consumption in some cases it may make sense to advance your index to avoid replays. You can say, "You should have made the message consumer idempotent" and I reply, "I agree this was a consulting gig I was there to fix the problems." ;)
    - Sphax 10y ago
      oh you're talking about consumer offsets ! I exclusively use kafka to store offsets since I started using kafka with 0.8.0.2 so I can set my offsets by using the kafka protocol only