4 ms·
Anything can become the opposite of sensible, so that's not saying much. I'm saying there are logical, sensible ideas behind things such as Kafka -- "you could
by the_af 5y ago
Anything can become the opposite of sensible, so that's not saying much. I'm saying there are logical, sensible ideas behind things such as Kafka -- "you could have invented the distributed log" -- and that it's not snake oil. Of course it can be misused, not everything calls for it, and it comes with tradeoffs and pitfalls, but which tool doesn't?
- pram 5y agoThere is a logical idea behind it, of course! I didn't say it was snake oil anywhere. Kafka was even pretty novel 7 years ago when I started deploying it (although 0.9 was an enormous dumpster fire) Today you also have Pulsar, Redpanda, Kinesis, PubSub, and NATS off the top of my head. You can do the same streaming patterns with all of them, with the exception of log compaction probably. It's an architecture so amazing that the world needs 6 redundant implementations. I think my OP implied something regarding that.
- the_af 5y agoI agree with you that the slightly different implementations of the same idea are confusing and slightly embarrassing. No argument from me there!
- agallego 5y agofor the record, you can do compaction with redpanda - all the same compaction strategies you know w/ kafka work w/ redpanda. https://github.com/vectorizedio/redpanda/tree/dev/src/v/storage - see all 'compaction_*' for deets