5 ms·

Having been part of a project to rewrite a monolith e-commerce site into an event-sourced, domain driven, CQRS system, let me tell you in which situation that

Having been part of a project to rewrite a monolith e-commerce site into an event-sourced, domain driven, CQRS system, let me tell you in which situation that is not possible: when you already have data. Remember that in a DDD, ES, CQRS system, the event store is the single source of truth. If you already have data in a relational database, then the existing data is the source of truth. You can't have two sources of truth, that completely defeats the purpose. So it's not actually possible to migrate to an event sourced system, you can only create one from scratch, with no existing data.

UK-AL 10y ago
Well you can, you need to create some events for each entity to get into the existing state. You won't have any history though.
dragonwriter 10y ago
Conceptually, that's not really true: you just transform the pre-ES state into one or more events (in an basic accounting system, which is pretty much the simplest ES system, long-predating the name for the model, this is just creating "starting balance" entries as transactions.) In practice, that can be challenging, but it doesn't seem fundamentally more challenging than any other legacy data conversion effort.
- GundersenM 10y ago
  Sure, if the existing DB is simple, that is straight forward, but remember that likely this is a monolith that is so bad that even management have agreed that it needs to be rewritten. Likely there are lots of DB tables with foreign keys and relations (sometimes documented and enforced, most often not). This means you can't really convert the entire database into an event sourced system, as that means converting all of the tables in one single go, instead of a gradual change. And believe me, in a system like this you want slow gradual changes! Also, even of you got it into events, what happened to the domains? There are so many relations between the different events sources (because you didn't put everything into just one event source, right? What happened to bounding contexts?) that you are no better off. And this means you have to prevent anything else from using the database anymore, and in a legacy system where you can just join across any two or three tables to extract whatever information you want, you can be certain there are some analysis engines that are just feeding directly on the sql data. And there might be other systems writing to the database too! So the first step is to disentangle all the data and encapsulate it, trying to prevent others from using it, so you have full control over it. This includes tracking down any other system using this data, and ensuring they too go through the database. And you have to do this for one subsystem at a time, often in several iterations.
  - GundersenM 10y ago
    Sorry, replied too quickly. My point is that if things are so bad that they warrant a major rewrite, then they are probably so bad that there is no simple way to map the existing data into events or starting conditions. It might be true if you have a simple well defined silo of a system that does one thing well, but not of you have what the author described, a monolith that does several things in the same codebase.
  - dragonwriter 10y ago
    > Sure, if the existing DB is simple, that is straight forward, but remember that likely this is a monolith that is so bad that even management have agreed that it needs to be rewritten. Yeah, but that's not a "converting legacy data to ES" problem, that's a "converting legacy data to any non-broken thing" problem. > This means you can't really convert the entire database into an event sourced system, as that means converting all of the tables in one single go, instead of a gradual change. Whether its ES or something else you are converting to, you either do a big-bang conversion and eat the pain of that (which can be tremendous, sure), or you instead eat the pain of taking the monolith and finding a way to break out components and do it incrementally, even though that takes not only building the new components, but reengineering parts of the old monolith to support that. Which, also, can be tremendous pain. But, again, this isn't really essentially tied to event sourcing, you face this dilemma even if you are going from a (broken for current needs, which is why it is being replaced) classically-designed "current state" RDBMS-backed system to a (meeting current needs, and hopefully more adaptable to future needS) classically-designed "current state" RDBMS-backed system.
    - GundersenM 10y ago
      Yup, agree, this is the problem of having a legacy monolith RDBMS that needs to be rewritten and split apart. It's tempting to throw every new fancy technology at the problem when that is suddenly an option, but it's better to focus on the goal of splitting it apart only. If you have split it out, and it's now simple to convert to ES CQRS, then you are probably in a situation where you don't need to do that, as it works quite well.
- karmajunkie 10y ago
  I'd say the data isn't the problem. Its all the code that WRITES data that's the problem. finding every nook and cranny in your apps that do this, and very frequently across logical boundaries, is a challenging exercise at a minimum, even if you build projections that reproduced the old database state.
adymitruk 10y ago
You treat that differently. It's explained here http://adaptechsolutions.net/using-events-to-leverage-legacy-system-investments/ http://adaptechsolutions.net/using-events-to-leverage-legacy...
ci5er 10y ago
This is almost certainly a naive question, but wouldn't you treat your DBMS data as a "snapshot" of that (zero) point in time and then all of the new events update from there until the next snapshot?

UK-AL 10y ago

Well you can, you need to create some events for each entity to get into the existing state. You won't have any history though.

dragonwriter 10y ago

Conceptually, that's not really true: you just transform the pre-ES state into one or more events (in an basic accounting system, which is pretty much the simplest ES system, long-predating the name for the model, this is just creating "starting balance" entries as transactions.) In practice, that can be challenging, but it doesn't seem fundamentally more challenging than any other legacy data conversion effort.

GundersenM 10y ago
Sure, if the existing DB is simple, that is straight forward, but remember that likely this is a monolith that is so bad that even management have agreed that it needs to be rewritten. Likely there are lots of DB tables with foreign keys and relations (sometimes documented and enforced, most often not). This means you can't really convert the entire database into an event sourced system, as that means converting all of the tables in one single go, instead of a gradual change. And believe me, in a system like this you want slow gradual changes! Also, even of you got it into events, what happened to the domains? There are so many relations between the different events sources (because you didn't put everything into just one event source, right? What happened to bounding contexts?) that you are no better off. And this means you have to prevent anything else from using the database anymore, and in a legacy system where you can just join across any two or three tables to extract whatever information you want, you can be certain there are some analysis engines that are just feeding directly on the sql data. And there might be other systems writing to the database too! So the first step is to disentangle all the data and encapsulate it, trying to prevent others from using it, so you have full control over it. This includes tracking down any other system using this data, and ensuring they too go through the database. And you have to do this for one subsystem at a time, often in several iterations.
- GundersenM 10y ago
  Sorry, replied too quickly. My point is that if things are so bad that they warrant a major rewrite, then they are probably so bad that there is no simple way to map the existing data into events or starting conditions. It might be true if you have a simple well defined silo of a system that does one thing well, but not of you have what the author described, a monolith that does several things in the same codebase.
- dragonwriter 10y ago
  > Sure, if the existing DB is simple, that is straight forward, but remember that likely this is a monolith that is so bad that even management have agreed that it needs to be rewritten. Yeah, but that's not a "converting legacy data to ES" problem, that's a "converting legacy data to any non-broken thing" problem. > This means you can't really convert the entire database into an event sourced system, as that means converting all of the tables in one single go, instead of a gradual change. Whether its ES or something else you are converting to, you either do a big-bang conversion and eat the pain of that (which can be tremendous, sure), or you instead eat the pain of taking the monolith and finding a way to break out components and do it incrementally, even though that takes not only building the new components, but reengineering parts of the old monolith to support that. Which, also, can be tremendous pain. But, again, this isn't really essentially tied to event sourcing, you face this dilemma even if you are going from a (broken for current needs, which is why it is being replaced) classically-designed "current state" RDBMS-backed system to a (meeting current needs, and hopefully more adaptable to future needS) classically-designed "current state" RDBMS-backed system.
  - GundersenM 10y ago
    Yup, agree, this is the problem of having a legacy monolith RDBMS that needs to be rewritten and split apart. It's tempting to throw every new fancy technology at the problem when that is suddenly an option, but it's better to focus on the goal of splitting it apart only. If you have split it out, and it's now simple to convert to ES CQRS, then you are probably in a situation where you don't need to do that, as it works quite well.
karmajunkie 10y ago
I'd say the data isn't the problem. Its all the code that WRITES data that's the problem. finding every nook and cranny in your apps that do this, and very frequently across logical boundaries, is a challenging exercise at a minimum, even if you build projections that reproduced the old database state.

adymitruk 10y ago

You treat that differently. It's explained here http://adaptechsolutions.net/using-events-to-leverage-legacy-system-investments/ http://adaptechsolutions.net/using-events-to-leverage-legacy...

ci5er 10y ago

This is almost certainly a naive question, but wouldn't you treat your DBMS data as a "snapshot" of that (zero) point in time and then all of the new events update from there until the next snapshot?