3 ms·

Event sourcing is REALLY hard to figure out how to do “right.” A lot of getting it right is modeling knowledge/experience, understanding your domain. That said

Event sourcing is REALLY hard to figure out how to do “right.” A lot of getting it right is modeling knowledge/experience, understanding your domain. That said, you can succeed at building your entire arch around it and once you do, it’s glorious. Kafka Streams makes the technical aspects easy once you figure out how to model correctly.

the_duke 8y ago
Out of curiosity, how do you deal with consistency guarantees across aggregates? (which is much more relevant when your whole architecture is ES) I realize this is highly domain dependent. Some will be much less affected than others. But it's another drawback not mentioned, because now you start to need sagas/managers that coordinate across services with commit/rollback patterns , conflict resolution., etc.
- linkmotif 8y ago
  As zenpsycho said, all you get is eventual consistency across aggregates, if you’re talking about projection aggregates. As you say, for domain aggregates you can wire up transactions by writing your own 2PC on top of Kafka exactly once semantics. I would recommend only to people who are really committed to the idea or know what they are doing and modeling. It took me a long time because I went from zero knowledge of Java/DDD. First I had to learn Java, then Kafka, then KS, but still I was lost. Learning the most basic DDD was enough for me and did the trick though. Writing a 2PC to coordinate across aggregates wasn’t pleasant but also wasn’t hard with KS. The hard part was learning all the other stuff and the modeling. I think a well done ES framework based on Kafka streams would maybe be the first ES framework that would have a chance. The primitives in KS seem just right.
  - barbecue_sauce 8y ago
    DDD in this context = Domain Driven Design?
    - linkmotif 8y ago
      si
- ZenPsycho 8y ago
  The architecture obligates you to the "eventually consistent" model. If that's not good enough, you should probably look at other approaches. Disclaimer: I'm just an avid reader, not an experienced implementor.
- codebje 8y ago
  If you took event sourcing out of the picture, how would you solve it? This is a problem to do with distributed transactions, not event sourcing. If you don't need distributed transactions, don't use them. If you do need distributed transactions, redesign your system so you don't need distributed transactions. If you _still_ need distributed transactions, use one of the available mechanisms for making them occasionally work, like three-phase commit, or clocking, or blind luck, all of which work just as well with events as without. If you'd be perfectly happy using a single database server doing a more typical ORM solution with tons of database locking without event sourcing, you can use a single database server with tons of database locking with event sourcing: just serialise all (or maybe most) events. My domain has a lot of unpredictable coupling between events, such that there's not really a sensible aggregate model to divide up the information model (remember: an aggregate is the unit level of consistency in your system). But it's also low update volume, so we quite happily just serialise all events - which is what we did before event sourcing, using table locks.

the_duke 8y ago

Out of curiosity, how do you deal with consistency guarantees across aggregates? (which is much more relevant when your whole architecture is ES) I realize this is highly domain dependent. Some will be much less affected than others. But it's another drawback not mentioned, because now you start to need sagas/managers that coordinate across services with commit/rollback patterns , conflict resolution., etc.

linkmotif 8y ago
As zenpsycho said, all you get is eventual consistency across aggregates, if you’re talking about projection aggregates. As you say, for domain aggregates you can wire up transactions by writing your own 2PC on top of Kafka exactly once semantics. I would recommend only to people who are really committed to the idea or know what they are doing and modeling. It took me a long time because I went from zero knowledge of Java/DDD. First I had to learn Java, then Kafka, then KS, but still I was lost. Learning the most basic DDD was enough for me and did the trick though. Writing a 2PC to coordinate across aggregates wasn’t pleasant but also wasn’t hard with KS. The hard part was learning all the other stuff and the modeling. I think a well done ES framework based on Kafka streams would maybe be the first ES framework that would have a chance. The primitives in KS seem just right.
- barbecue_sauce 8y ago
  DDD in this context = Domain Driven Design?
  - linkmotif 8y ago
    si
ZenPsycho 8y ago
The architecture obligates you to the "eventually consistent" model. If that's not good enough, you should probably look at other approaches. Disclaimer: I'm just an avid reader, not an experienced implementor.
codebje 8y ago
If you took event sourcing out of the picture, how would you solve it? This is a problem to do with distributed transactions, not event sourcing. If you don't need distributed transactions, don't use them. If you do need distributed transactions, redesign your system so you don't need distributed transactions. If you _still_ need distributed transactions, use one of the available mechanisms for making them occasionally work, like three-phase commit, or clocking, or blind luck, all of which work just as well with events as without. If you'd be perfectly happy using a single database server doing a more typical ORM solution with tons of database locking without event sourcing, you can use a single database server with tons of database locking with event sourcing: just serialise all (or maybe most) events. My domain has a lot of unpredictable coupling between events, such that there's not really a sensible aggregate model to divide up the information model (remember: an aggregate is the unit level of consistency in your system). But it's also low update volume, so we quite happily just serialise all events - which is what we did before event sourcing, using table locks.