3 ms·
There are certain ideas which are virulent that people seem to catch, like an incurable virus. One of them I believe is Event Sourcing. I was hired into one $
by jpz 8y ago
There are certain ideas which are virulent that people seem to catch, like an incurable virus. One of them I believe is Event Sourcing.
I was hired into one $100m+ greenfield project (silly money by a silly investor) where the novice technical management wanted everything done "perfect" - no room for compromise.
Project failed, and 600 people that were hired over the 18 months were then all fired.
Every piece of data was to be event-sourcing, every data transaction on a global message queue (Kafka), the ultimate pure event sourcing model.
The big expenses with event sourcing are up-front design of protocols, backward compatibility of protocol changes (if you got it wrong), and problematic structuring of coordinated queries (using coordinators.)
Trivial stuff like serialisation starts to become 30%+ of your development cost.
The simplest example we had, for instance, is the creation of a new user on sign up.
But the Authentication module was not the same as the personalisation module.
So we had authentication creation, then personalisation module need to learn about the user and create the personalisation defaults, as a coordinated, distributed query.
All with Kafka. The next thing here is you discover you need RPC and not Pub/Sub. So you end up seeing confused developers bending the architecture out of shape, doing RPC over pub/sub - which looks like a dog's dinner.
My general view of event-sourcing is "do it in the small" for specific problems where you need it.
If you think it's the silver bullet, then please read Fred Brooks again, and stop pushing your golden solution to all things on us all, please.
BTW, you don't need a fancy framework to keep time-versioned histories in a relational database (I realise RDBMS doesn't scale for all systems, but it does for many.) You just need to design your schema well, and preferably, make your writes go through stored procedures which transactionally maintain history and current schemas.
- 2color 8y agoI've had a similar experience at a smaller scale with kafka and something of a CQRS architecture. I can vouch that at least 30% of development cost went into things that are otherwise very trivial. The risks of defining the wrong domain boundaries early on has culminating costs and besides specific use cases the benefits are not outweighed by the disadvantages. Even things such as business intelligence need to have knowledge of events and how they reduce to comprehensible state. For a startup this is can turn into a disaster of dealing with technology instead of focusing on solving problems. Lastly, GDPR really changes things when you need to be able to anonymise or delete data. It doesn't play too well with the event sourcing model.
- vithlani 8y agoOr use Datomic