3 ms·

I've worked on a few event sourcing systems, and the only ones that have been truly successful are the ones where the system was written from scratch. Trying t

I've worked on a few event sourcing systems, and the only ones that have been truly successful are the ones where the system was written from scratch. Trying to retrofit systems never seemed to quite work as expected. Also, the "from scratch" systems, seemed conceptually easier to not only understand, but extend, because you were forced to write code in an idiomatic and consistent way. Everything was an event (albeit, a "command" and an "event" confirming 'execution' of the "command").

bob1029 5y ago
I am currently looking at a new iteration of our product that uses an event log as the principal data store. I agree that if we start from "zero" and don't even provide the option to build things the wrong way it would go smoothly. "everything is an event" is very powerful if its not just some add-on policy. For us, the entire working set would be able to live in memory. Larger objects would be left as pointers to the actual data in the log. Snapshots of working set would be taken on a daily basis in hopes of keeping recovery under ~5 minutes during service restarts or updates.
- DrBazza 5y ago
  > Snapshots of working set would be taken on a daily basis in hopes of keeping recovery under ~5 minutes during service restarts or updates. Exactly this. We took hourly snapshots (for a trading system), and could restart the system in a second or so. Another nice side effect of an event driven system is that you start to think about what other data you can put in there. And then you realise you don't necessarily need a database any more, which makes the architecture of the system simpler. There are so many benefits working with a data-driven system. Once data is in the log/ledger/whatever you call it, it's immutable, which leads to parallelism if you want it. Ensuring the system is idempotent also means you get (easier) failover as well.
  - bob1029 5y ago
    > And then you realise you don't necessarily need a database any more This is precisely where we want to be. We are already exceptionally minimal when it comes to database usage - SQLite is all we use today. If we are in a place where 100% of the business facts go to an immutable/append-only log, we can handle the entire persistence mechanism with a simple filestream write command. Not having to worry about mutating prior data solves a large portion of database engine problems automagically.

bob1029 5y ago

I am currently looking at a new iteration of our product that uses an event log as the principal data store. I agree that if we start from "zero" and don't even provide the option to build things the wrong way it would go smoothly. "everything is an event" is very powerful if its not just some add-on policy. For us, the entire working set would be able to live in memory. Larger objects would be left as pointers to the actual data in the log. Snapshots of working set would be taken on a daily basis in hopes of keeping recovery under ~5 minutes during service restarts or updates.

DrBazza 5y ago
> Snapshots of working set would be taken on a daily basis in hopes of keeping recovery under ~5 minutes during service restarts or updates. Exactly this. We took hourly snapshots (for a trading system), and could restart the system in a second or so. Another nice side effect of an event driven system is that you start to think about what other data you can put in there. And then you realise you don't necessarily need a database any more, which makes the architecture of the system simpler. There are so many benefits working with a data-driven system. Once data is in the log/ledger/whatever you call it, it's immutable, which leads to parallelism if you want it. Ensuring the system is idempotent also means you get (easier) failover as well.
- bob1029 5y ago
  > And then you realise you don't necessarily need a database any more This is precisely where we want to be. We are already exceptionally minimal when it comes to database usage - SQLite is all we use today. If we are in a place where 100% of the business facts go to an immutable/append-only log, we can handle the entire persistence mechanism with a simple filestream write command. Not having to worry about mutating prior data solves a large portion of database engine problems automagically.