6 ms·
Oh this reminds me a lot of a familiar kafka vs SQL battle. Yes it is possible to do it in kafka, but everything is obscured in the sense that you can't peek a
by fifilura 4y ago
Oh this reminds me a lot of a familiar kafka vs SQL battle.
Yes it is possible to do it in kafka, but everything is obscured in the sense that you can't peek at what you are doing, you spend your precious CPU on serializing/deserializing and things like backfill is a mess.
- lmm 4y agoI find the opposite; SQL databases work a lot like Kafka underneath, but hide it from you, burning all your CPU to generate this illusion of a globally consistent state of the world that you don't actually want or need and doing everything they can to avoid ever showing you what they're actually doing.
- ttfkam 4y agoHold up. I need some clarification here. SQL databases are opaque with regard to resource usage but Kafka is completely transparent in this regard? That is your personal experience and assertion?
- lmm 4y agoYes. I once saw an outage caused by an SQL database server collapsing because of a query that had been issued 23 days earlier. I saw another one gradually get slower and slower over a period of weeks because a data scientist had left a command prompt open and forgotten about it. Kafka brokers are a lot more consistent and predictable IME.