3 ms·
You don't need Kafka if you don't know (and I mean really know, because it's hurting) where you need it. But if you do know where you need it, you wish you had
by barrkel 6y ago
You don't need Kafka if you don't know (and I mean really know, because it's hurting) where you need it.
But if you do know where you need it, you wish you had it yesterday.
This is the way with scaling in startups. Product-market fit and agility and lean operations is what you want early on, and scaling should come later. Big architectural decisions are hard to reverse and committing to some pattern that needs something like Kafka, like event sourcing, is probably too much risk for most products. Most event-sourced CQRS implementations fail.
I say choose Kafka somewhere in the middle, when scaling needs are somewhat predictable and product market fit is good, but try and use Confluent's hosted service or something similarly light on ops. Do non-critical things with it first and ramp up. Don't build big from the get go - that rarely ends well.