6 ms·

Regarding eventual consistency, a CQRS\ES system can also be synchronous, or partially. You could have listeners for events that need to supply a strongly consi

by 7sigma 8y ago

Regarding eventual consistency, a CQRS\ES system can also be synchronous, or partially. You could have listeners for events that need to supply a strongly consistent model and others events that feed parts of the system that don't need strong consistency. "However the events in a event store are immutable and can’t be deleted, to undo an action means sending the command with the opposite action" Well they don't have to be immutable. I don't see why you can't update/migrate events.

superqwert 8y ago
The author has explained the problem with eventual consistency and the CAP theorem, but he is trying to blame the problem at least partly on CQRS/ES - the problem will exist independently of what pattern you use if your system is distributed. CQRS/ES is a pattern and doesn't need to be shoe-horned into every solution. I work at Transport for London and we have a CQRS/ES system for managing the data driven design data. It is synchronous and is incredibly useful for ensuring it has both strong business logic and a fast readable side, while providing auditing for free. We still have problems with EC and CAP, and they are not due to CQRS/ES. Make the system distributed and you will have to handle all the new scenarios. Those are the trade-offs.
zihotki 8y ago
I also share same opinion based on my experience. Events can be modified and deleted but that must be an exceptional situation (GDPR and other compliances, etc.). But even if it's exceptional you have to provide a clear and easy way to do so and that increases complexity of the solution by a lot. Another thing is strongly consistent models, there may be valid requirements in some problem areas to have a strongly consistent and normalized model and use it for command validation. This helps especially well in the case when all requirements are not known upfront and/or business domain changes very frequently. A small change in business may require to completely redo the aggregate roots and logic if you follow standard approach, this is very expensive. A better decision could be to use a normalized SQL database instead of an aggregate root. Such approach may be more flexible in certain cases and have it's own benefits as well as cost and drawbacks.
- jwhitlark 8y ago
  Encryption + lose the key policies seem to satisfy the GDPR. So you don't have to actually delete an event, you can just give up your ability to read its payload.
- victorNicollet 8y ago
  If the decision to abandon strong consistency involved careful analysis of the performance/maintenance trade-offs, then by definition the lack of consistency is less expensive than keeping a consistent but low-performance model, and you're just paying the price of having to solve a Hard Problem. But if strong consistency was abandoned because someone wrote general statements in favor of eventual consistency...
vorpalhex 8y ago
> can also be synchronous Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. > Well they don't have to be immutable. I don't see why you can't update/migrate events. The events are your source of truth. When you "migrate" your source of truth, you're in somewhat dangerous waters (and if you're using CQRS for scale, a 1% failure to migrate data might be millions of records). You also now have the issue that your source of truth is being migrated, and probably can't accept writes (or you need to emit writes in both new format and old format for whenever your change over happens)
- victorNicollet 8y ago
  >> can also be synchronous > Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. Even without asynchronous read/write, there are still benefits worth the (arguably, small) additional complexity. For instance, the ability to add new functionality without having to migrate existing data is amazing.
  - vorpalhex 8y ago
    You still have migrations, but they exist on the read store, which means they can be done semi-transparently to clients (pause writes and let them queue up, migrate existing read store to new instance, point read calls and indexers to new store, resume writes). CQRS adds a lot of complexity. Sometimes it's absolutely worth it, especially if you've already invested in the expertise and tooling to support it. It drastically changes the scaling math, both on the low end (at the very least you need a write store, a read store, a queue, an indexer and an api) and on the high end (you can scale any part of the system as needed in relative isolation). You add in timing issues, rollbacks, asynchronous error handling, and delays between reads/writes.
    - jmull 8y ago
      I don't think you necessarily have to have all this in a useful CQRS system. E.g., here's a real-world example of a general pattern in which CQRS is pretty simple and useful: A walking/running app which tracks distance, time, and other relevant information over the course of a workout. It collects a series of events like "location changed at time X", "user started workout", "user paused workout" etc., over the course of the workout period (actually starting before the user officially starts the workout), and converts these to time, distance and other stats. This fits CQRS really well since the input is inherently a series of events and the output is information gleaned from processing those events. You get a full CQRS system by simply fully logging the events. The advantage is that you can go back and reprocess the sequence if you want to glean new/different information from the sequence. E.g., in the walking/running app you could, after the fact and at the user's discretion, detect and fix the case where the user forgets to start or stop a workout. Or recalc distance in the case where you detect a bug or misapplication of your smoothing algorithm, etc. Or draw a pace graph or whatever. In all these cases you can process the events synchronously. I put this all in terms of a workout app, but there is a general pattern of an event-driven session-based activity or process, where you may want/need to derive new information from the events and the cost is to log the events (in a high-fidelity form so they could be fully reproduced. Whether or not you need to use queues and distributed data stores is an independent decision.

by 7sigma 8y ago

superqwert 8y ago

The author has explained the problem with eventual consistency and the CAP theorem, but he is trying to blame the problem at least partly on CQRS/ES - the problem will exist independently of what pattern you use if your system is distributed. CQRS/ES is a pattern and doesn't need to be shoe-horned into every solution. I work at Transport for London and we have a CQRS/ES system for managing the data driven design data. It is synchronous and is incredibly useful for ensuring it has both strong business logic and a fast readable side, while providing auditing for free. We still have problems with EC and CAP, and they are not due to CQRS/ES. Make the system distributed and you will have to handle all the new scenarios. Those are the trade-offs.

zihotki 8y ago

I also share same opinion based on my experience. Events can be modified and deleted but that must be an exceptional situation (GDPR and other compliances, etc.). But even if it's exceptional you have to provide a clear and easy way to do so and that increases complexity of the solution by a lot. Another thing is strongly consistent models, there may be valid requirements in some problem areas to have a strongly consistent and normalized model and use it for command validation. This helps especially well in the case when all requirements are not known upfront and/or business domain changes very frequently. A small change in business may require to completely redo the aggregate roots and logic if you follow standard approach, this is very expensive. A better decision could be to use a normalized SQL database instead of an aggregate root. Such approach may be more flexible in certain cases and have it's own benefits as well as cost and drawbacks.

jwhitlark 8y ago
Encryption + lose the key policies seem to satisfy the GDPR. So you don't have to actually delete an event, you can just give up your ability to read its payload.
victorNicollet 8y ago
If the decision to abandon strong consistency involved careful analysis of the performance/maintenance trade-offs, then by definition the lack of consistency is less expensive than keeping a consistent but low-performance model, and you're just paying the price of having to solve a Hard Problem. But if strong consistency was abandoned because someone wrote general statements in favor of eventual consistency...

vorpalhex 8y ago

> can also be synchronous Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. > Well they don't have to be immutable. I don't see why you can't update/migrate events. The events are your source of truth. When you "migrate" your source of truth, you're in somewhat dangerous waters (and if you're using CQRS for scale, a 1% failure to migrate data might be millions of records). You also now have the issue that your source of truth is being migrated, and probably can't accept writes (or you need to emit writes in both new format and old format for whenever your change over happens)

victorNicollet 8y ago
>> can also be synchronous > Then you're giving up several of the benefits of CQRS, and might as well just not bother with the additional complexity. Even without asynchronous read/write, there are still benefits worth the (arguably, small) additional complexity. For instance, the ability to add new functionality without having to migrate existing data is amazing.
- vorpalhex 8y ago
  You still have migrations, but they exist on the read store, which means they can be done semi-transparently to clients (pause writes and let them queue up, migrate existing read store to new instance, point read calls and indexers to new store, resume writes). CQRS adds a lot of complexity. Sometimes it's absolutely worth it, especially if you've already invested in the expertise and tooling to support it. It drastically changes the scaling math, both on the low end (at the very least you need a write store, a read store, a queue, an indexer and an api) and on the high end (you can scale any part of the system as needed in relative isolation). You add in timing issues, rollbacks, asynchronous error handling, and delays between reads/writes.
  - jmull 8y ago
    I don't think you necessarily have to have all this in a useful CQRS system. E.g., here's a real-world example of a general pattern in which CQRS is pretty simple and useful: A walking/running app which tracks distance, time, and other relevant information over the course of a workout. It collects a series of events like "location changed at time X", "user started workout", "user paused workout" etc., over the course of the workout period (actually starting before the user officially starts the workout), and converts these to time, distance and other stats. This fits CQRS really well since the input is inherently a series of events and the output is information gleaned from processing those events. You get a full CQRS system by simply fully logging the events. The advantage is that you can go back and reprocess the sequence if you want to glean new/different information from the sequence. E.g., in the walking/running app you could, after the fact and at the user's discretion, detect and fix the case where the user forgets to start or stop a workout. Or recalc distance in the case where you detect a bug or misapplication of your smoothing algorithm, etc. Or draw a pace graph or whatever. In all these cases you can process the events synchronously. I put this all in terms of a workout app, but there is a general pattern of an event-driven session-based activity or process, where you may want/need to derive new information from the events and the cost is to log the events (in a high-fidelity form so they could be fully reproduced. Whether or not you need to use queues and distributed data stores is an independent decision.