4 ms·
Message buses, as a source of truth, pass events around without much knowledge of its content relative to what applications need to process. They maintain histo
by bamazizi 6y ago
Message buses, as a source of truth, pass events around without much knowledge of its content relative to what applications need to process. They maintain history and offset of events based on topics.
Sink/aggregators listen to events and, as per instructions, store payloads into db for efficient storage or optimized querying. The query service is only connecting to that db to execute queries. It's possible to have multiple dbs that store the same exact data but for different purposes. i.e. sql for normalized data or acid transactions and nosql for fast map reduce ...
A properly designed system, treats dbs as disposable. Meaning, delete any db at anytime and just play back events from the beginning or from the last snapshot and rebuild a new db.
Unfortunately making message buses the single source of truth is not that easy, and scary. Kafka is an expensive family relative, you better make sure you have the $$$ & the initial configuration is near perfect but still streaming requires some experts to help you get it right.
Nats Steaming is good alternative to Kafka, I use and recommend it personally, but not perfect at high scale, like billions of events/day. Jetstream, successor to Nats Streaming, is coming but it's barely in preview stage and would need another year or so to be qualified for prod usage.
- stingraycharles 6y agoI see, so it’s a way of effectively decoupling the state of the aggregate (store it inside the db) with the actual querying (just query the db). That way, at any time, you can just stop the aggregate/sink, and recover where you left off. We have implemented our event store on top of PostgreSQL. I’m not a big fan of using Kafka as an event store.