4 ms·

My teams' event sourcing implementation stores and publishes events in a well defined order within a specific aggregate. There are scenarios where a two-phase

My teams' event sourcing implementation stores and publishes events in a well defined order within a specific aggregate. There are scenarios where a two-phase commit is used to ensure that invariants across aggregates are maintained. We use an ACID compliant database to store our domain events and we project our events into a relational schema. When projecting events, we use transactions to make sure the database updates from a specific event are either all applied or not at all.

victorNicollet 10y ago
Yes, this is a typical way of doing things in the CQRS/ES world, but there are many things left unsaid: - would you have a single aggregate for "all bank accounts" so that one financial transaction equals one event, or is there a good reason to have multiple aggregates ? - is command execution transactional, or is it possible for another thread to generate (possibly conflicting) events between the time you check a precondition and the time you write the intended events ? My team uses an ES implementation with very strong guarantees: one aggregate per macro-service, and command execution is transactional, but I'm fairly certain that this is not the way CQRS/ES is described or recommended.
- partisan 10y ago
  I would have one aggregate instance per bank account. Each has a separate state and business rules would need to be applied at the individual account instance (e.g. overdraft fees, interest, etc). If you have to enforce rules at the bank level based upon actions taken within the account then you can have a bank aggregate that you can interact with through a saga and confirm changes through a two phase commit. Command execution is not transactional, but writing to the domain event store is. That means that two writers cannot write the same domain event version number to the event store. Say an aggregate is at version 20. The next action would put it at 21. If two threads generate version 21 of the aggregate, only one of them can write it to the event store and the other gets an exception. Domain events are written in a transaction batch to the event store so we cannot get any writes in the middle. So, the first to write the next version wins. The other one loses. We also partition our command and event processors by aggregate id. The same aggregate cannot have its events or commands processed more than one at a time because the processors are synchronous per partition. If I understand you correctly then you have one instance of a very large aggregate per bounded context? If so, then no, it is not in the spirit of CQRS/ES. If you are handling commands in transactions, then the designers of your system may have chosen to favor consistency instead of the availability that CQRS provides.
  - victorNicollet 10y ago
    We do have the same constraint on event store writes, which we then use to implement command-level transactions : a command is a pure function that maps a command model to a set of events, and if the set of events cannot be written to the stream, the command model is updated and the function is called again. This does, indeed, have a very strong bias towards consistency, although the main objective is to allow multiple command processors per aggregate, which lets us embed the command processors in services that are inherently multi-instance (e.g. web applications). I am not sure I understand why having multiple aggregates would provide availability. Unless the idea is that the events from different aggregates are stored on separate servers, so that a single-machine outage would only take down some aggregates ? If so, I agree that is the case in theory, though with limited practical applicability in our situation.
    - partisan 10y ago
      Aggregates represent consistency boundaries. To enforce consistency you would theoretically need to have your entire state loaded when you go to process a command for a single bank aggregate. Since each command produces a new version of the bank and many commands are coming in at the same time, most of them will fail when writing to the event store. Do they keep retrying until they succeed? Either way, this is not efficient and could effect the availability of your servers. If you have multiple aggregates, the consistency boundaries are smaller and therefore you have a smaller state that you have to maintain consistency across. There are less operations on a single aggregate and less opportunities for contention that arise from simultaneous operations. Also, if you are using queues for your commands and events (we are, but I suspect you are not), then you can partition your queues such that you can process your workload N ways without worrying about things happening out of order. Each aggregate processes serially within a partition. If you have just one large aggregate then you would have to jump through a lot of hoops to be able to partition the work while maintaining ordering guarantees. I can really only guess at the details of your implementation, but I am guessing that your design has accounted for all of the above in some way. If there is one thing I've learned, it's that there are many, many ways to implement CQRS. If your software works correctly under load, then I am not sure it would matter if it fits squarely into the definition of CQRS. In fact, you may have something new altogether that presents a better way to achieve the same goals as CQRS without the same cognitive overhead.

victorNicollet 10y ago

Yes, this is a typical way of doing things in the CQRS/ES world, but there are many things left unsaid: - would you have a single aggregate for "all bank accounts" so that one financial transaction equals one event, or is there a good reason to have multiple aggregates ? - is command execution transactional, or is it possible for another thread to generate (possibly conflicting) events between the time you check a precondition and the time you write the intended events ? My team uses an ES implementation with very strong guarantees: one aggregate per macro-service, and command execution is transactional, but I'm fairly certain that this is not the way CQRS/ES is described or recommended.

partisan 10y ago
I would have one aggregate instance per bank account. Each has a separate state and business rules would need to be applied at the individual account instance (e.g. overdraft fees, interest, etc). If you have to enforce rules at the bank level based upon actions taken within the account then you can have a bank aggregate that you can interact with through a saga and confirm changes through a two phase commit. Command execution is not transactional, but writing to the domain event store is. That means that two writers cannot write the same domain event version number to the event store. Say an aggregate is at version 20. The next action would put it at 21. If two threads generate version 21 of the aggregate, only one of them can write it to the event store and the other gets an exception. Domain events are written in a transaction batch to the event store so we cannot get any writes in the middle. So, the first to write the next version wins. The other one loses. We also partition our command and event processors by aggregate id. The same aggregate cannot have its events or commands processed more than one at a time because the processors are synchronous per partition. If I understand you correctly then you have one instance of a very large aggregate per bounded context? If so, then no, it is not in the spirit of CQRS/ES. If you are handling commands in transactions, then the designers of your system may have chosen to favor consistency instead of the availability that CQRS provides.
- victorNicollet 10y ago
  We do have the same constraint on event store writes, which we then use to implement command-level transactions : a command is a pure function that maps a command model to a set of events, and if the set of events cannot be written to the stream, the command model is updated and the function is called again. This does, indeed, have a very strong bias towards consistency, although the main objective is to allow multiple command processors per aggregate, which lets us embed the command processors in services that are inherently multi-instance (e.g. web applications). I am not sure I understand why having multiple aggregates would provide availability. Unless the idea is that the events from different aggregates are stored on separate servers, so that a single-machine outage would only take down some aggregates ? If so, I agree that is the case in theory, though with limited practical applicability in our situation.
  - partisan 10y ago
    Aggregates represent consistency boundaries. To enforce consistency you would theoretically need to have your entire state loaded when you go to process a command for a single bank aggregate. Since each command produces a new version of the bank and many commands are coming in at the same time, most of them will fail when writing to the event store. Do they keep retrying until they succeed? Either way, this is not efficient and could effect the availability of your servers. If you have multiple aggregates, the consistency boundaries are smaller and therefore you have a smaller state that you have to maintain consistency across. There are less operations on a single aggregate and less opportunities for contention that arise from simultaneous operations. Also, if you are using queues for your commands and events (we are, but I suspect you are not), then you can partition your queues such that you can process your workload N ways without worrying about things happening out of order. Each aggregate processes serially within a partition. If you have just one large aggregate then you would have to jump through a lot of hoops to be able to partition the work while maintaining ordering guarantees. I can really only guess at the details of your implementation, but I am guessing that your design has accounted for all of the above in some way. If there is one thing I've learned, it's that there are many, many ways to implement CQRS. If your software works correctly under load, then I am not sure it would matter if it fits squarely into the definition of CQRS. In fact, you may have something new altogether that presents a better way to achieve the same goals as CQRS without the same cognitive overhead.