5 ms·

As this article pops out again, I'd like to the point that although it may have some valid points, those points are not about Event Sourcing. What's expressed i

As this article pops out again, I'd like to the point that although it may have some valid points, those points are not about Event Sourcing. What's expressed in the article is the Event Streaming or Event-Driven approach. So when events are not the source of truth etc. All of the event stores that I know supports strong consistency on appends, optimistic concurrency. Many guarantee global ordering. Some help in idempotency. All of that helps to reduce those issues. In Event Sourcing, events are the state. So the flow looks like that: 1. You get the events from the stream (that represents all the facts that happened for the entity), 2. You apply it one by one in the order of appearance to build the current state, 3. You run the business logic and, as a result, create the event, 4. You append a new event. You don't need to cache the write model state anywhere, as the state is in events. Event Sourcing has its issues and troubles, but the stuff described in the article apply to tools like Kafka, etc. They're not tools for Event Sourcing but Event Streaming. They're designed to move things from one place to another, not to be used as durable databases. I've made a list of articles like that: https://github.com/oskardudycz/EventSourcing.NetCore#this-is-not-event-sourcing https://github.com/oskardudycz/EventSourcing.NetCore#this-is... I'll repeat it again: this is not Event Sourcing.

kreetx 5y ago
BTW, does there happen to be a discord channel for this topic? ;)
- oskar_dudycz 5y ago
  There is a Slack channel: https://join.slack.com/t/ddd-cqrs-es/shared_invite/zt-m3vf3alt-S3L~YUoIV88wekj6wSNrUQ https://join.slack.com/t/ddd-cqrs-es/shared_invite/zt-m3vf3a...
nhumrich 5y ago
I re-read the article after this comment, and I would have to disagree. The article does never attempt to explain "what" recent sourcing means to them, so it's hard to know for sure. However, they do mention populating state from an event log which contains meaningless events, so I have to assume they _are_ talking about event sourcing. What leads you to believe they are taking about just streaming, and not sourcing?
- dm3 5y ago
  They were probably also sourcing their state from the events. However most of their problems come from sharing the events between the modules/services which is not a part of Event Sourcing.
  - oskar_dudycz 5y ago
    I think that they were just building the stale read models, and used them as the write model, which created the whole confusion. Regarding the sharing events between module, it's one of the most common and the most dangerous mistakes. It's a leaking abstraction that will eventually create a distributed monolith. It has only downsides of monolith and microservices, without the upsides. I wrote longer on the topic of internal and external events, and how to model them: https://event-driven.io/en/events_should_be_as_small_as_possible/ https://event-driven.io/en/events_should_be_as_small_as_poss...
    - nhumrich 5y ago
      What you are talking about is CQRS, which is a very valid pattern, and pairs well with event sourcing, but is not necessary part of event sourcing. You don't have to split your read and write models for events sourcing.
      - oskar_dudycz 5y ago
        Could you elaborate, I think that I'm missing your point.
  - nhumrich 5y ago
    I am confused when you say sharing events isn't part of event sourcing. How does a service populate it's state from other services event source if it can't access it's events?
    - oskar_dudycz 5y ago
      Did you read my article that I linked, or my initial explanation of the Event Sourcing flow?
goostavos 5y ago
Howdy! Author here ^_^ I'll respond to a few items because, even though I haven't touched the system in a few years, I could still rant endless about the mistakes I made building it. Deep scars were acquired! Firstly, to make sure we're talking about the same thing, where are you setting the bar for whether or not we can call something "Event Sourcing"? For instance, just to clarify, in our system events were indeed the source of truth. At the heart of everything sat a ledger, that ledger was stored in a database, and events were added to it as you describe in points 1-4 (hand waving away minor differences). You got a consistent, up-to-date state of an aggregate by replaying the ledger in order and applying all the events. So, to my understanding, I'd call that textbook. However, the problem may lay in our actual definition of the word. >You don't need to cache the write model state anywhere, as the state is in events I'd have to understand a bit more about where you're coming from with this one. While, yep, the state is in the events and you get the latest state by playing them back, that materialization is not free. If you're doing it for a single aggregate, then it's generally not a huge deal at small scales. However, once you need to make decisions against more than one thing, or compare things, or query things, that materialization cost becomes prohibitive, and you've got to set up materialized projections. With our tech stack, that had to be an eventually consistent process, and what got us into event streaming, and which ultimately caused most of my ugly crying.
- lolinder 5y ago
  As someone who's evaluating event sourcing for a real business use case, I really appreciated your article. I wish more people published their event sourcing war stories, rather than most resources being exclusively from consultants selling snake oil. I think the point OP is making is actually the same as what you conveyed in your article--several of the worst pain points you experienced aren't inherent to Event Sourcing, they're due to a particular implementation of Event Sourcing. The problem is that that implementation is pretty much the canonical one sold by Martin Fowler et al. I can think of a few specific ways in which event sourcing could be different than what you experienced: 1) Distributed computing is always hard. Event Sourcing is not a magical cure-all to distributed computing woes, but it's sold as that, and the result is that people go into a distributed event-driven architecture without being prepared for the engineering challenges that await. Eliminate distributed computing, and many problems simply go away. 2) Eventual consistency is always hard, but Event Sourcing doesn't inherently require eventual consistency, even for cached projections. If your projections are pure functions folded over the event stream, then caching and updating the materialized projections with strict consistency is straightforward. In the same transaction as inserting a new event, you run your projection function once on each of the materialized projections and the new event (`state = projection(oldState, event)`). This is only possible when you're not trying to create your projections in a distributed fashion, so see #1. 3) Language choice makes a big difference for reducing the cognitive overhead of updating projections. Your compiler should be able to tell you every place where you need to update a projection to account for a new event. Preferably, you should be able to define a hierarchy of events so that it only yells at you about places where the event is actually relevant.
oskar_dudycz 5y ago
I wrote the summary of my thoughts on this article: https://event-driven.io/en/event_streaming_is_not_event_sourcing/ https://event-driven.io/en/event_streaming_is_not_event_sour...

kreetx 5y ago

BTW, does there happen to be a discord channel for this topic? ;)

oskar_dudycz 5y ago
There is a Slack channel: https://join.slack.com/t/ddd-cqrs-es/shared_invite/zt-m3vf3alt-S3L~YUoIV88wekj6wSNrUQ https://join.slack.com/t/ddd-cqrs-es/shared_invite/zt-m3vf3a...

nhumrich 5y ago

I re-read the article after this comment, and I would have to disagree. The article does never attempt to explain "what" recent sourcing means to them, so it's hard to know for sure. However, they do mention populating state from an event log which contains meaningless events, so I have to assume they _are_ talking about event sourcing. What leads you to believe they are taking about just streaming, and not sourcing?

dm3 5y ago
They were probably also sourcing their state from the events. However most of their problems come from sharing the events between the modules/services which is not a part of Event Sourcing.
- oskar_dudycz 5y ago
  I think that they were just building the stale read models, and used them as the write model, which created the whole confusion. Regarding the sharing events between module, it's one of the most common and the most dangerous mistakes. It's a leaking abstraction that will eventually create a distributed monolith. It has only downsides of monolith and microservices, without the upsides. I wrote longer on the topic of internal and external events, and how to model them: https://event-driven.io/en/events_should_be_as_small_as_possible/ https://event-driven.io/en/events_should_be_as_small_as_poss...
  - nhumrich 5y ago
    What you are talking about is CQRS, which is a very valid pattern, and pairs well with event sourcing, but is not necessary part of event sourcing. You don't have to split your read and write models for events sourcing.
    - oskar_dudycz 5y ago
      Could you elaborate, I think that I'm missing your point.
- nhumrich 5y ago
  I am confused when you say sharing events isn't part of event sourcing. How does a service populate it's state from other services event source if it can't access it's events?
  - oskar_dudycz 5y ago
    Did you read my article that I linked, or my initial explanation of the Event Sourcing flow?

goostavos 5y ago

Howdy! Author here ^_^ I'll respond to a few items because, even though I haven't touched the system in a few years, I could still rant endless about the mistakes I made building it. Deep scars were acquired! Firstly, to make sure we're talking about the same thing, where are you setting the bar for whether or not we can call something "Event Sourcing"? For instance, just to clarify, in our system events were indeed the source of truth. At the heart of everything sat a ledger, that ledger was stored in a database, and events were added to it as you describe in points 1-4 (hand waving away minor differences). You got a consistent, up-to-date state of an aggregate by replaying the ledger in order and applying all the events. So, to my understanding, I'd call that textbook. However, the problem may lay in our actual definition of the word. >You don't need to cache the write model state anywhere, as the state is in events I'd have to understand a bit more about where you're coming from with this one. While, yep, the state is in the events and you get the latest state by playing them back, that materialization is not free. If you're doing it for a single aggregate, then it's generally not a huge deal at small scales. However, once you need to make decisions against more than one thing, or compare things, or query things, that materialization cost becomes prohibitive, and you've got to set up materialized projections. With our tech stack, that had to be an eventually consistent process, and what got us into event streaming, and which ultimately caused most of my ugly crying.

lolinder 5y ago
As someone who's evaluating event sourcing for a real business use case, I really appreciated your article. I wish more people published their event sourcing war stories, rather than most resources being exclusively from consultants selling snake oil. I think the point OP is making is actually the same as what you conveyed in your article--several of the worst pain points you experienced aren't inherent to Event Sourcing, they're due to a particular implementation of Event Sourcing. The problem is that that implementation is pretty much the canonical one sold by Martin Fowler et al. I can think of a few specific ways in which event sourcing could be different than what you experienced: 1) Distributed computing is always hard. Event Sourcing is not a magical cure-all to distributed computing woes, but it's sold as that, and the result is that people go into a distributed event-driven architecture without being prepared for the engineering challenges that await. Eliminate distributed computing, and many problems simply go away. 2) Eventual consistency is always hard, but Event Sourcing doesn't inherently require eventual consistency, even for cached projections. If your projections are pure functions folded over the event stream, then caching and updating the materialized projections with strict consistency is straightforward. In the same transaction as inserting a new event, you run your projection function once on each of the materialized projections and the new event (`state = projection(oldState, event)`). This is only possible when you're not trying to create your projections in a distributed fashion, so see #1. 3) Language choice makes a big difference for reducing the cognitive overhead of updating projections. Your compiler should be able to tell you every place where you need to update a projection to account for a new event. Preferably, you should be able to define a hierarchy of events so that it only yells at you about places where the event is actually relevant.

oskar_dudycz 5y ago

I wrote the summary of my thoughts on this article: https://event-driven.io/en/event_streaming_is_not_event_sourcing/ https://event-driven.io/en/event_streaming_is_not_event_sour...