5 ms·
I used to be really excited about event sourcing but yeah, its usually just over-engineering. It appeals to the nerd in me because its a powerful & clean model
by zaptheimpaler 5y ago
I used to be really excited about event sourcing but yeah, its usually just over-engineering. It appeals to the nerd in me because its a powerful & clean model - events are immutable, your entire database can theoretically be reconstructed at any point in time by just replaying an event log up to time X. In the ideal form its sort of the highest fidelity version of data storage, throwing nothing away, supporting all ways of querying as materialized views or dependent DBs built on the stream. Its beautiful.
But also we live in reality. DBs use checkpoints because storing/replaying an event log from time 0 would take ungodly amounts of space and time. You deleted or resized a column to save some space? Lol no, the event log lives forever. You wanted to use SQL, a battle tested language to query your data? Lol no, the database is "inside out" so tough luck buddy, you're building the database now. Sure you might have to rebuild compaction, joins, query languages, concurrency control and the other 100 things a DB gives you, but on the plus side that one audit log that you could have built with some glue and a few INSERT triggers in Postgres is now an elegant map/reduce on your 100TB dataset! Yay!
Mad props to the consultants though my man wanted a car you sold him a car factory. Take that money to the bank, get a Lamborghini, travel the world drinking and talking shit at conferences fuck yeah. Fuels getting expensive, throw in some machine learning sauce, get a yacht all I do is win baby.
- quicklime 5y agoTotally agree. The field of software engineering would be a lot better off if we stopped using weasel words like "clean" and "elegant", and discussed concrete things like performance or the ability to produce an audit log.
- dtech 5y agoFor non-trivial amounts of data you should combine event sourcing with snapshots - i.e. a somewhat up-to-date materialized view/DB table - so you don't have to start for 0. At that point you can delete older events or move them to cold storage. A valid question then is: "what do you gain over just using the table"? You gain a well-described model of your business domain, with very clear actions about what should happen on which real-world event, and an audit log. Whether that's worth it depend on your use case. Secondary benefits include easily being able to share events for new apps or use-cases, business analysts really like them, and making it unlikely things go wrong because data is in an inconsistent state.
- kitd 5y agoES is the C in CQRS though. If you're using it for the Q too, then it's no wonder you'll quickly get in a bad way. It's specifically not designed for that.
- deepanchor 5y agoAs I understand it, ES and CQRS are somewhat orthogonal concepts (although they are indeed often used together). It's perfectly legitimate to implement CQRS without using event sourcing at all. The only thing CQRS dictates is that a system's read models are separate from its write models.
- kitd 5y agoThat's very true, but the benefits of ES come on the write side of the equation. Expecting it to solve read-side problems will inevitably lead to disappointment.
- rswail 5y agoMaterialized views are what you should be using, not reconstructing the state from the event log every time. That gives you access to SQL and everything to do with it. And in reality, the event log does live forever in the real world outside of your system. Attributes of your aggregates from last year are still valid for events related to last year, even if they're now deprecated or no longer in use. CQRS/ES is about system design that evolves. It evolves in a much cleaner and easier way if you follow it. Is it perfect? No. There are some gnarly problems related to, for example, GDPR's "right to be forgotten", but that needs to be solved across database backups as well.