4 ms·

Cool, would be nice to see more competition for Conduktor. I have worked with a few “Kafka is a database” people in my career and even if you agree with them, i

by thom 4y ago

Cool, would be nice to see more competition for Conduktor. I have worked with a few “Kafka is a database” people in my career and even if you agree with them, it’s still a terrible one from a tooling perspective. Having to debug Kafka issues on topics with almost any volume of data is slow and frustrating, either with a GUI or command line/REPL tools. Your only hope (if you absolutely have to wrangle Kafka in the core of a project) is heavy testing with the test driver from the start and hoping to avoid getting into too many knots. It’s interesting to note that Conduktor’s more enterprisey features are in this area now.

Bogdanp 4y ago
> Having to debug Kafka issues on topics with almost any volume of data is slow and frustrating, either with a GUI or command line/REPL tools. Agreed. The scripting feature in Franz is born out of my frustration with some of this stuff and seems like a promising route for quickly visualizing and interacting with what’s happening on a topic (though it’s early days and I’ve got lots of work to do until it becomes truly useful).
MuffinFlavored 4y ago
i’d love to hear some Kafka as a database success/horror stories. What’s the general consensus?
- beebmam 4y ago
  It works great with the right kind of data.
  - dotancohen 4y ago
    What are the wrong kinds of data? It would be nice to know before I fall into a trap.
- dtech 4y ago
  It works well with master data [1], i.e. semi-static data of limited volume that is widely needed throughout systems. Think physical store locations and such. Services can either store it in their own database and still get an update stream, or services can read the complete set on boot and keep it in memory. It can avoid the need for RPC/REST calls so centralized services that keep track off this kind of data, preventing cascading failures if those fail. [1] https://en.wikipedia.org/wiki/Master_data https://en.wikipedia.org/wiki/Master_data

by thom 4y ago

Bogdanp 4y ago

> Having to debug Kafka issues on topics with almost any volume of data is slow and frustrating, either with a GUI or command line/REPL tools. Agreed. The scripting feature in Franz is born out of my frustration with some of this stuff and seems like a promising route for quickly visualizing and interacting with what’s happening on a topic (though it’s early days and I’ve got lots of work to do until it becomes truly useful).

MuffinFlavored 4y ago

i’d love to hear some Kafka as a database success/horror stories. What’s the general consensus?

beebmam 4y ago
It works great with the right kind of data.
- dotancohen 4y ago
  What are the wrong kinds of data? It would be nice to know before I fall into a trap.
dtech 4y ago
It works well with master data [1], i.e. semi-static data of limited volume that is widely needed throughout systems. Think physical store locations and such. Services can either store it in their own database and still get an update stream, or services can read the complete set on boot and keep it in memory. It can avoid the need for RPC/REST calls so centralized services that keep track off this kind of data, preventing cascading failures if those fail. [1] https://en.wikipedia.org/wiki/Master_data https://en.wikipedia.org/wiki/Master_data