3 ms·

I have not watched that clip, but as I said above, Confluent isn't a good resource for defining this term since Kafka cannot be used to do ES. I would suggest a

by bruth 5y ago

I have not watched that clip, but as I said above, Confluent isn't a good resource for defining this term since Kafka cannot be used to do ES. I would suggest an article like: https://domaincentric.net/blog/eventstoredb-vs-kafka https://domaincentric.net/blog/eventstoredb-vs-kafka Also refer to the list of great resources @oskar_dudycz posted in another comment: https://github.com/oskardudycz/EventSourcing.NetCore#1319-this-is-not-event-sourcing-but-event-streaming https://github.com/oskardudycz/EventSourcing.NetCore#1319-th...

claytonjy 5y ago
that first article finds only one issue with Kafka-for-ES: limited number of partitions. Use pulsar, problem solved? Doesn't have the support or ecosystem of Kafka, but surely more than eventstoreDB.
- bruth 5y ago
  They aren't solving the same problem. An event sourced entity (should) have a very small scope. It acts as a consistency boundary around some bit of state. Each event in "the stream" for that entity represents a state change. This is analogous to a database row which represents the _current_ state of the entity being modeled for that table. Like a table, there can be an arbitrary number of rows (>millions), thus you could have millions of event streams. You can't have millions of Kafka partitions. The scope/scale of a Kafka partition is not designed for that small granularity. Again, I have no issue with Kafka and I think Pulsar is superior in the event streaming/platform space. I also the Kafka ecosystem is impressive and I love listening Confluent Cloud podcast! But it is not designed for this use case.
- gunnarmorling 5y ago
  There's two main reasons why Kafka as of today isn't a good fit for implementing event sourcing: - Lacking support for key-based look-ups: when trying to materialize the current state of an entity, you cannot query Kafka efficiently for just those events (a partition-per-entity strategy doesn't scale due to the partion number limitation, which may change with KRaft in the future) - Lacking support for (optimistic) locking: you cannot make sure to reject events based on outdated state, unless you apply work-arounds like a single writer architecture

by bruth 5y ago

claytonjy 5y ago

that first article finds only one issue with Kafka-for-ES: limited number of partitions. Use pulsar, problem solved? Doesn't have the support or ecosystem of Kafka, but surely more than eventstoreDB.

bruth 5y ago
They aren't solving the same problem. An event sourced entity (should) have a very small scope. It acts as a consistency boundary around some bit of state. Each event in "the stream" for that entity represents a state change. This is analogous to a database row which represents the _current_ state of the entity being modeled for that table. Like a table, there can be an arbitrary number of rows (>millions), thus you could have millions of event streams. You can't have millions of Kafka partitions. The scope/scale of a Kafka partition is not designed for that small granularity. Again, I have no issue with Kafka and I think Pulsar is superior in the event streaming/platform space. I also the Kafka ecosystem is impressive and I love listening Confluent Cloud podcast! But it is not designed for this use case.
gunnarmorling 5y ago
There's two main reasons why Kafka as of today isn't a good fit for implementing event sourcing: - Lacking support for key-based look-ups: when trying to materialize the current state of an entity, you cannot query Kafka efficiently for just those events (a partition-per-entity strategy doesn't scale due to the partion number limitation, which may change with KRaft in the future) - Lacking support for (optimistic) locking: you cannot make sure to reject events based on outdated state, unless you apply work-arounds like a single writer architecture