16 ms·

What do you mean by “Event-Driven”?

276 points by Garbage 10y ago

whyileft 10y ago
I would highly recommend spending an evening reading through Martin Fowler's blog. Has has a gift of breaking down concepts into very pragmatic and digestible explanations. A couple of my favorites: https://www.martinfowler.com/articles/injection.html https://www.martinfowler.com/articles/injection.html https://www.martinfowler.com/articles/refactoring-dependencies.html https://www.martinfowler.com/articles/refactoring-dependenci...
Terr_ 10y ago
> a potential confusion between event-sourcing and CQRS IMO this comes up a lot in HN/Reddit discussions about either term. Event-sourcing implies CQRS, but the reverse is not true. > I'd love to write some definitive treatise [...] Sadly I don't have the time to do it. The "Fermat's Last Theorem" approach to computer architecture :P
- to3m 10y ago
  I'm not sure... sounds like CQRS (though it's a term I don't think I've ever seen before...) is independent. Like, you can have one data structure for read/write, and you just don't get public access to the write interface directly, only through functions that add to the event list and then do the event. Along the lines of something vaguely like this, perhaps: void Stuff::AddThing(Thing *thing) { this->AddEvent(new AddThingEvent(thing)); } void Stuff::AddEvent(Event *event) { m_events.push_back(event); event->Execute(this); } void AddThingEvent::Execute(Stuff *stuff) { stuff->m_things.push_back(m_thing); } and maybe AddThingEvent is a friend of Stuff, or it's an inner class, or whatever you like. Maybe this is what he meant though.
  - Terr_ 10y ago
    I think that most of the time event-sourcing will impact your model/business-logic layer, because you often want to capture the intent of changes, rather than calculating a naive combined delta of all changes that occurred in memory. If so, that means that instead of a CRUD interface like: if(customer.getApprovalLevel() < 5){ customer.setApprovalLevel(5); customer.setApprover(currentUser); } saveChangesSomehow(customer); So you'll end up with an API that's a bit more CQS (no 'R' yet) with: if(customer.getApprovalLevel() < 5){ customer.promoteApprovalLevel(5, currentUser); } saveChangesSomehow(customer); That way you capture each change-event at the right granularity with the necessary data. Finally, CQRS comes into play because you want to do something with that rich event data, such as eagerly populating a table: recentApprovals = getRecentApprovals(); if(recentApprovals.length() > 0){ sendApprovalAuditReport(recentApprovals); }
- pc86 10y ago
  Isn't it the eventual consistency model that implies CQRS? If you have synchronous event-sourcing you can have a single model/service for reads and writes.
jknoepfler 10y ago
generally I mean interrupt-driven, but maybe I'm being obtuse.
- kasey_junk 10y ago
  Meh. All events are just an abstraction 1 level above polling^, interrupts included. ^may not be true for spooky actions
jph 10y ago
Event-Driven software comes up on many of my projects. What I find works well is using command-query separation (CQS, CQRS) to start because CQS is a lever into two complementary ideas: backend implementation and continuous improvement. The backend implementation ideas involve REST, caching, replication, eventual consistency, CAP, and especially PACELC. The continuous improvement ideas involve upgrading existing software projects from imperative styles to functional styles, phasing in publish-subscribe modules, growing codebases to be more event-driven, and planning for evolutionary architecture. In practice, I see these areas need management help, such as funding and time for training, and help for teams creating infrastructure as code (IaC) to deploy many event-oriented modules as well as event monitoring and management. Related links: https://en.wikipedia.org/wiki/Command-query_separation https://en.wikipedia.org/wiki/Command-query_separation https://www.wikipedia.org/wiki/REST https://www.wikipedia.org/wiki/REST https://www.wikipedia.org/wiki/PACELC_theorem https://www.wikipedia.org/wiki/PACELC_theorem https://www.wikipedia.org/wiki/Publish-subscribe_pattern https://www.wikipedia.org/wiki/Publish-subscribe_pattern https://www.wikipedia.org/wiki/Infrastructure_as_Code https://www.wikipedia.org/wiki/Infrastructure_as_Code
- sethammons 10y ago
  I had not heard of PACELC and was excited to hear something new, only to see it as a very natural extension of CAP. It reminds me of, back in math class, you would learn of a new theorem that was only different than another because they singled out a particular case, like for 1 or 0 :)
  - sundvor 10y ago
    I hadn't heard of either, but they both make perfect sense - so I for one say thanks for that great post. :-) Did study up on CQRS/ES year and found it very exciting.
OJFord 10y ago
Idk, all my code's results-driven. (And goal-oriented.)
btilly 10y ago
One common anti-pattern is that "event driven" can lead to a push architecture that works on paper, but is unreliable in practice. The idea looks deceptively simple. If you need something to happen, you just broadcast the right event, someone else acts on it and everything is fine. You code it, it works in the demo, and you deploy to production. And then you run across network failures. Missed events. Doubled events. Events that arrived while a database was down. Events that were missed due to a minor software bug. And so on. You layer on fixes, and the system develops more complex edge cases, and more subtle bugs. Then you rewrite from a push based system to a pull based system, and all of your complex edge cases disappear. :-)
- maxxxxx 10y ago
  Our IT architects love event driven stuff but it always ends in hellish systems where you have to add retries on the publisher side and pull on the client side.
- virmundi 10y ago
  I've used events successfully for an important fraud detection system. There are a set of known events. A fraud model could hook into the event chain at any point. It would then produce an alert event with its findings. Downstream another handler listened for it. Coupled with RabbitMQ, the system is a pleasure to expand. All the fraud models are micro services. Just a start up script and boom, hooked in. We planned for duplication of messages. Operations are idempotent. If the message already ran, we could rerun the calculation safely. With Rabbit the only duplicate messages were the ones humans sent to rerun a model that broke.
  - Floegipoky 10y ago
    Sounds like a system I built, but what I didn't account for is how often the RabbitMQ cluster would get tanked by my coworkers. The system fails when messages start getting dropped on the floor due to server reboots. Still haven't come up with a great solution, but what we have is "good enough" for that project's requirements. It was a great learning experience though, an uncomfortable reminder of the difference between reliability and durability, and it definitely helped inform subsequent architectural decisions.
urbit 10y ago
Blatant plug: Urbit (urbit.org) is a general-purpose OS built on the event-sourced model. Urbit's state is a small, frozen function of its event log. I don't know of any other generalized event-sourced OSes. But almost every principle Fowler describes in special-purpose event-sourced designs also seems to apply in the general-purpose case. For example, the connection between CQRS and event-sourcing seems very deep and natural. The connection to patches in a revision-control system is also completely apropos. When Fowler starts pointing at advanced problems like "we have to figure out how to deal with changes in the schema of events over time," you start to see a clear path from special-purpose event-sourced apps to general-purpose event-sourced system software. Broadly speaking, one category of answer is: abstract the semantics of your app, until it becomes a general-purpose interpreter whose source code is delivered and updated in the event stream. Then the new schema is just a source patch. And your app is not just an app, but actually sort of an OS.
- dozzie 10y ago
  > Blatant plug: Urbit (urbit.org) is a general-purpose OS built on the event-sourced model. I've looked through its documentation, and while I couldn't make heads or tails of it, Urbit seems to be everything but an operating system. Espeically that it needs unix system to run.
  - urbit 10y ago
    Linux on Xen would also fail this test. Urbit is an OS in that it defines the complete lifecycle of a general-purpose computer. It's not an OS in that it's the lowest layer above bare metal. Sorry you had a bad experience with the docs. Urbit is actually much stupider than it looks. Unfortunately, premature documentation is the second root of all evil, so some of the vagueness you sense is due to immature code. It's always a mistake to document a system into existence -- that is the path of vaporware. Better to run and have weak docs, than have good docs but not run.
    - fb03 10y ago
      I agree with you on the documentation thing but, what we see (and I experience also) is that most of the time, when things get serious enough (production), there isn't any available time to document the core of the software properly, let alone document it fully.... "lack of time" in this context meaning either (1) management won't let you stop implementing stuff they need "for yesterday" (everything is critical priority) and document for 10-20 hours or (2) your self run project needs more features and it's always cooler to study/code than to document puzzles you have already solved. I'm not criticizing, I am exposing that I am part of the problem and lost, and I'd like to know how people handle this on their lives. I want to document more but there's always this rush to implement implement implement and not once, during some catastrophic breakdown of something, management wants everything solved quickly but they expect you to remember instantly of everything you did 2+ years ago :-) PS: ... and they'll get mad if you say "you didn't let me document this damn thing"
mbesto 10y ago
For anyone who has implemented a system using CQRS patterns, do you recommend it? What are the pros and cons as the codebase and application grow?
jaequery 10y ago
i was expecting to read about node.js
TheAceOfHearts 10y ago
I find the topic of event sourcing / CQRS a little frustrating. Somewhat tangential, but I've slightly similar experiences with rxjs as well. For both, whenever I read about em or I hear someone discussing em, it makes so much sense in my head! But later when I try to execute on the ideas, I can't make it "come together". It's like some puzzle piece refuses to click into place. In the cases when I've tried hacking something together, I've always ended up with something that seemed more brittle than if I'd just gone with a safe relational database. With that said, I'm also completely open to the possibility that I've just been trying to apply the pattern in cases where it's not a good fit. I'd seriously love to poke around in a real-world application showcasing CQRS. I don't expect something perfect, but getting a chance to look at what tradeoffs and concessions a bunch of engineers made would surely be incredibly insightful.
- akiselev 10y ago
  > In the cases when I've tried hacking something together, I've always ended up with something that seemed more brittle than if I'd just gone with a safe relational database. With that said, I'm also completely open to the possibility that I've just been trying to apply the pattern in cases where it's not a good fit. There is a 95% chance that event sourcing and CQRS were entirely the wrong patterns for your use case. If you're not dealing with a very complex domain (like, say, automatically orchestrating a global logistics or drug discovery pipeline with audit logs for regulatory compliance), that chance becomes 99.9%. If you're working on something that only you or a few people will ever work on, it's 110%. Just the tooling for a reliable event sourcing system would be a herculean task for a single developer and that's without even writing a single line of business logic! I don't know of any open source real world examples of CQRS off the top of my head and the only successful ES/CQRS systems I've seen in the wild were projects that involved hundreds of domain experts and programmers, dealing with extremely complicated fields like biotechnology and electrical engineering where nonprogrammers spent about as much time looking at code as the programmers. Any good ES/CQRS project will contain a lot of the business's secret sauce so it's not something Google or Facebook would open source.
- ibgib 10y ago
  I experienced something very similar with rxjs (and before that just Rx in .Net). I've studied Rx for a couple years, and I was developing event-sourcing ideas since before it became a big thing back around 2000, and I've done my fair share of head scratching in these areas. (I arrived at the event-sourcing stuff via a very abstract, conceptual route largely investigating physics/information theory...) It has since morphed into its present-day incarnation in ibGib (github.com/ibgib/ibgib and www.ibgib.com). I don't know how real-world it is, since I'm still pretty much the only one who is using it, but you may enjoy checking it out for a slightly different view on ES. It doesn't strictly fit the definition of CQRS at the moment. This is because I see CQRS as an implementation detail orthogonal to ES, but which dovetails nicely as they both work well with distributed systems. ES has immutable events (monotonically increasing), and CQRS implies a distributed system and allows for more effecient retrieval, leveraging ES's immutability. But ibGib does provide what I see as an augmentation, or maybe an evolution, to ES. It has shifted over the years away from ontological "events" and now is more biological in approach. All of the data is in terms of ibGib, which is a flat database structure comprising `ib`, `data`, `rel8ns` and a `gib` which is a hash of the other three fields. The ES "event" analog specifically about what is replayed to hydrate domain aggregates can be seen in a specific `rel8n` called `dna`. This contains the code necessary to "rebuild" that ibGib immutable frame. This came largely from my dislike of the foundation of the entire river of the source of truth being necessary. With the DNA structure, it actually keeps a dependency graph, which ends up being a projection of the "entire" source of truth. This allows for nodes to "replicate" ("reproduce" would be a better term) as projections of entire graphs. These kinds of deep aspects are really neat! But anyway, I digress. The server side is written in Elixir, which runs on top of Erlang's vm (the BEAM(!)). And you can visually interact with the structure, as the web app runs on d3. I have some demos on YouTube on the website if you just want to look at it, though I don't go into the DNA/ES aspect of it. Anyway, I had to say something, since ES is a topic near and dear and so closely related to ibGib's current architecture. My design decisions could very well help you & others understand ES more deeply. Plus I've been enjoying getting to a point where ibGib is actually usable (and useful). NOTE: I'm somewhat arbitrarily responding to you in this thread, as I used to play halo with an AceOfHearts.
snissn 10y ago
I think of it as using epoll http://man7.org/linux/man-pages/man7/epoll.7.html http://man7.org/linux/man-pages/man7/epoll.7.html
aryehof 10y ago
A good summary. However too many choose to view it only as Event-Sourcing/CQRS, a silver-bullet applicable to a minority of applications. One particularly attractive in Microsoft projects, since it is the only alternative enterprise architecture "practice" they propose to data modeling. Object domain modeling continues to be an anathema, because I think of a wizard and code-generation driven historical perspective. In commercial systems, transactions as actions and events, recorded for business or legal reasons, are central. That includes significant "events" like purchases, sales, reservations etc. In that sense, they all are event driven. Event-Driven as an architecture that is based on recording changes to a baseline state, should be applied only where really suited.
grandalf 10y ago
I'd like to see a simple e-commerce cart written using all of the patterns. That's really the minimal "hello world" for comparing the patterns he discusses.
- akiselev 10y ago
  A minimal "hello world" project for CQRS and event sourcing would be a simple e-commerce cart and all the software running the supply chain. With a simple shopping cart, you'd write 10x more code just for the tooling to get ES/CQRS up and running reliably.
- kpmah 10y ago
  Not sure how much this will help, but a couple of years ago I wrote about using event sourcing for a simple blog application in PostgreSQL: http://kevinmahoney.co.uk/articles/immutable-data/ http://kevinmahoney.co.uk/articles/immutable-data/ I wasn't aware of the term 'event sourcing' at the time though.
Flenser 10y ago
Reading this it struck me how React/Flux uses Event-Sourcing whereas Angular uses Event Notification.
osullivj 10y ago
My first contact with event driven programming was coding for the original Windows C API, back in 1990. Nowadays we call that API Win32, but then it may have been win 16. Switching from the traditional imperative style to callbacks was a big deal: don't call us, we'll call you! [1] and [2] show the difference in coding style. Later in the 90s C++ wrappers like MFC and XVT simplified things, but the paradigm was still an inversion of control. And that's what event driven means to me. [1] http://www.paulgriffiths.net/program/c/srcs/hellosrc.html http://www.paulgriffiths.net/program/c/srcs/hellosrc.html [2] http://www.paulgriffiths.net/program/c/srcs/winhellosrc.html http://www.paulgriffiths.net/program/c/srcs/winhellosrc.html
- barrkel 10y ago
  Unless you do async operations, UI framework programming with callbacks doesn't really approach the full horror that can exist in some styles of event-based programming. UI events are typically of two kinds: user performed an action and you need to update the model / execute a command, or the view is asking for information and you need to translate it from the model. Both are fairly encapsulated; the larger operation doesn't extend beyond the scope of the callback, and if it does (e.g. showing a modal dialog), you can use a nested event loop to take care of event dispatching while showing the modal. The events in a UI are also naturally scoped to the screen being shown. When the screen goes away, you can rely on those events not being dispatched any more. Events get to party on a shared state (the model, perhaps attributes of the view) but scoped within the view being displayed. It's all pretty flat. You also don't need to worry much about concurrency, unless doing async.
barrkel 10y ago
It's very easy to create an event-driven program that is effectively a larger program implemented in terms of "come-from". The decoupling provided by eventing is often illusory; if there's a higher-level semantic that's supposed to be implemented in terms of the outcome of coordinated event handlers, then the dependencies still exist - they're just implicit. If event-handlers are genuinely decoupled from one another, and there isn't a higher-level state machine that's being driven by events, then it can be an excellent way to structure logic. Events-as-deltas that are durably stored and can be reliably replayed is another fine way to go for a system with checkpoints, auditing, history, rollback, time-series reporting and similar requirements. But like the article says, code will be much simpler if it deals with an eternal present and the calculation and application of the deltas is centralized in code that doesn't change often. There's an isomorphism between building a bigger program out of event handlers, and distributed programming. Distributed programming is known to be difficult, and doing it at the message send level is IMO too low a level - one should build more expressive and composable primitives and work at a higher level. That's if it's warranted at all.
- guptron 10y ago
  "The decoupling provided by eventing is often illusory; if there's a higher-level semantic that's supposed to be implemented in terms of the outcome of coordinated event handlers, then the dependencies still exist - they're just implicit." - I've experienced the same myself and converted my "events" into coupled calls. More expressive.

276 points by Garbage 10y ago

I would highly recommend spending an evening reading through Martin Fowler's blog. Has has a gift of breaking down concepts into very pragmatic and digestible explanations. A couple of my favorites: https://www.martinfowler.com/articles/injection.html https://www.martinfowler.com/articles/injection.html https://www.martinfowler.com/articles/refactoring-dependencies.html https://www.martinfowler.com/articles/refactoring-dependenci...

Terr_ 10y ago

> a potential confusion between event-sourcing and CQRS IMO this comes up a lot in HN/Reddit discussions about either term. Event-sourcing implies CQRS, but the reverse is not true. > I'd love to write some definitive treatise [...] Sadly I don't have the time to do it. The "Fermat's Last Theorem" approach to computer architecture :P

to3m 10y ago
I'm not sure... sounds like CQRS (though it's a term I don't think I've ever seen before...) is independent. Like, you can have one data structure for read/write, and you just don't get public access to the write interface directly, only through functions that add to the event list and then do the event. Along the lines of something vaguely like this, perhaps: void Stuff::AddThing(Thing *thing) { this->AddEvent(new AddThingEvent(thing)); } void Stuff::AddEvent(Event *event) { m_events.push_back(event); event->Execute(this); } void AddThingEvent::Execute(Stuff *stuff) { stuff->m_things.push_back(m_thing); } and maybe AddThingEvent is a friend of Stuff, or it's an inner class, or whatever you like. Maybe this is what he meant though.
- Terr_ 10y ago
  I think that most of the time event-sourcing will impact your model/business-logic layer, because you often want to capture the intent of changes, rather than calculating a naive combined delta of all changes that occurred in memory. If so, that means that instead of a CRUD interface like: if(customer.getApprovalLevel() < 5){ customer.setApprovalLevel(5); customer.setApprover(currentUser); } saveChangesSomehow(customer); So you'll end up with an API that's a bit more CQS (no 'R' yet) with: if(customer.getApprovalLevel() < 5){ customer.promoteApprovalLevel(5, currentUser); } saveChangesSomehow(customer); That way you capture each change-event at the right granularity with the necessary data. Finally, CQRS comes into play because you want to do something with that rich event data, such as eagerly populating a table: recentApprovals = getRecentApprovals(); if(recentApprovals.length() > 0){ sendApprovalAuditReport(recentApprovals); }
pc86 10y ago
Isn't it the eventual consistency model that implies CQRS? If you have synchronous event-sourcing you can have a single model/service for reads and writes.

jknoepfler 10y ago

generally I mean interrupt-driven, but maybe I'm being obtuse.

kasey_junk 10y ago
Meh. All events are just an abstraction 1 level above polling^, interrupts included. ^may not be true for spooky actions

jph 10y ago

Event-Driven software comes up on many of my projects. What I find works well is using command-query separation (CQS, CQRS) to start because CQS is a lever into two complementary ideas: backend implementation and continuous improvement. The backend implementation ideas involve REST, caching, replication, eventual consistency, CAP, and especially PACELC. The continuous improvement ideas involve upgrading existing software projects from imperative styles to functional styles, phasing in publish-subscribe modules, growing codebases to be more event-driven, and planning for evolutionary architecture. In practice, I see these areas need management help, such as funding and time for training, and help for teams creating infrastructure as code (IaC) to deploy many event-oriented modules as well as event monitoring and management. Related links: https://en.wikipedia.org/wiki/Command-query_separation https://en.wikipedia.org/wiki/Command-query_separation https://www.wikipedia.org/wiki/REST https://www.wikipedia.org/wiki/REST https://www.wikipedia.org/wiki/PACELC_theorem https://www.wikipedia.org/wiki/PACELC_theorem https://www.wikipedia.org/wiki/Publish-subscribe_pattern https://www.wikipedia.org/wiki/Publish-subscribe_pattern https://www.wikipedia.org/wiki/Infrastructure_as_Code https://www.wikipedia.org/wiki/Infrastructure_as_Code

sethammons 10y ago
I had not heard of PACELC and was excited to hear something new, only to see it as a very natural extension of CAP. It reminds me of, back in math class, you would learn of a new theorem that was only different than another because they singled out a particular case, like for 1 or 0 :)
- sundvor 10y ago
  I hadn't heard of either, but they both make perfect sense - so I for one say thanks for that great post. :-) Did study up on CQRS/ES year and found it very exciting.

OJFord 10y ago

Idk, all my code's results-driven. (And goal-oriented.)

btilly 10y ago

One common anti-pattern is that "event driven" can lead to a push architecture that works on paper, but is unreliable in practice. The idea looks deceptively simple. If you need something to happen, you just broadcast the right event, someone else acts on it and everything is fine. You code it, it works in the demo, and you deploy to production. And then you run across network failures. Missed events. Doubled events. Events that arrived while a database was down. Events that were missed due to a minor software bug. And so on. You layer on fixes, and the system develops more complex edge cases, and more subtle bugs. Then you rewrite from a push based system to a pull based system, and all of your complex edge cases disappear. :-)

maxxxxx 10y ago
Our IT architects love event driven stuff but it always ends in hellish systems where you have to add retries on the publisher side and pull on the client side.
virmundi 10y ago
I've used events successfully for an important fraud detection system. There are a set of known events. A fraud model could hook into the event chain at any point. It would then produce an alert event with its findings. Downstream another handler listened for it. Coupled with RabbitMQ, the system is a pleasure to expand. All the fraud models are micro services. Just a start up script and boom, hooked in. We planned for duplication of messages. Operations are idempotent. If the message already ran, we could rerun the calculation safely. With Rabbit the only duplicate messages were the ones humans sent to rerun a model that broke.
- Floegipoky 10y ago
  Sounds like a system I built, but what I didn't account for is how often the RabbitMQ cluster would get tanked by my coworkers. The system fails when messages start getting dropped on the floor due to server reboots. Still haven't come up with a great solution, but what we have is "good enough" for that project's requirements. It was a great learning experience though, an uncomfortable reminder of the difference between reliability and durability, and it definitely helped inform subsequent architectural decisions.

urbit 10y ago

Blatant plug: Urbit (urbit.org) is a general-purpose OS built on the event-sourced model. Urbit's state is a small, frozen function of its event log. I don't know of any other generalized event-sourced OSes. But almost every principle Fowler describes in special-purpose event-sourced designs also seems to apply in the general-purpose case. For example, the connection between CQRS and event-sourcing seems very deep and natural. The connection to patches in a revision-control system is also completely apropos. When Fowler starts pointing at advanced problems like "we have to figure out how to deal with changes in the schema of events over time," you start to see a clear path from special-purpose event-sourced apps to general-purpose event-sourced system software. Broadly speaking, one category of answer is: abstract the semantics of your app, until it becomes a general-purpose interpreter whose source code is delivered and updated in the event stream. Then the new schema is just a source patch. And your app is not just an app, but actually sort of an OS.

dozzie 10y ago
> Blatant plug: Urbit (urbit.org) is a general-purpose OS built on the event-sourced model. I've looked through its documentation, and while I couldn't make heads or tails of it, Urbit seems to be everything but an operating system. Espeically that it needs unix system to run.
- urbit 10y ago
  Linux on Xen would also fail this test. Urbit is an OS in that it defines the complete lifecycle of a general-purpose computer. It's not an OS in that it's the lowest layer above bare metal. Sorry you had a bad experience with the docs. Urbit is actually much stupider than it looks. Unfortunately, premature documentation is the second root of all evil, so some of the vagueness you sense is due to immature code. It's always a mistake to document a system into existence -- that is the path of vaporware. Better to run and have weak docs, than have good docs but not run.
  - fb03 10y ago
    I agree with you on the documentation thing but, what we see (and I experience also) is that most of the time, when things get serious enough (production), there isn't any available time to document the core of the software properly, let alone document it fully.... "lack of time" in this context meaning either (1) management won't let you stop implementing stuff they need "for yesterday" (everything is critical priority) and document for 10-20 hours or (2) your self run project needs more features and it's always cooler to study/code than to document puzzles you have already solved. I'm not criticizing, I am exposing that I am part of the problem and lost, and I'd like to know how people handle this on their lives. I want to document more but there's always this rush to implement implement implement and not once, during some catastrophic breakdown of something, management wants everything solved quickly but they expect you to remember instantly of everything you did 2+ years ago :-) PS: ... and they'll get mad if you say "you didn't let me document this damn thing"

mbesto 10y ago

For anyone who has implemented a system using CQRS patterns, do you recommend it? What are the pros and cons as the codebase and application grow?

jaequery 10y ago

i was expecting to read about node.js

TheAceOfHearts 10y ago

I find the topic of event sourcing / CQRS a little frustrating. Somewhat tangential, but I've slightly similar experiences with rxjs as well. For both, whenever I read about em or I hear someone discussing em, it makes so much sense in my head! But later when I try to execute on the ideas, I can't make it "come together". It's like some puzzle piece refuses to click into place. In the cases when I've tried hacking something together, I've always ended up with something that seemed more brittle than if I'd just gone with a safe relational database. With that said, I'm also completely open to the possibility that I've just been trying to apply the pattern in cases where it's not a good fit. I'd seriously love to poke around in a real-world application showcasing CQRS. I don't expect something perfect, but getting a chance to look at what tradeoffs and concessions a bunch of engineers made would surely be incredibly insightful.

akiselev 10y ago
> In the cases when I've tried hacking something together, I've always ended up with something that seemed more brittle than if I'd just gone with a safe relational database. With that said, I'm also completely open to the possibility that I've just been trying to apply the pattern in cases where it's not a good fit. There is a 95% chance that event sourcing and CQRS were entirely the wrong patterns for your use case. If you're not dealing with a very complex domain (like, say, automatically orchestrating a global logistics or drug discovery pipeline with audit logs for regulatory compliance), that chance becomes 99.9%. If you're working on something that only you or a few people will ever work on, it's 110%. Just the tooling for a reliable event sourcing system would be a herculean task for a single developer and that's without even writing a single line of business logic! I don't know of any open source real world examples of CQRS off the top of my head and the only successful ES/CQRS systems I've seen in the wild were projects that involved hundreds of domain experts and programmers, dealing with extremely complicated fields like biotechnology and electrical engineering where nonprogrammers spent about as much time looking at code as the programmers. Any good ES/CQRS project will contain a lot of the business's secret sauce so it's not something Google or Facebook would open source.
ibgib 10y ago
I experienced something very similar with rxjs (and before that just Rx in .Net). I've studied Rx for a couple years, and I was developing event-sourcing ideas since before it became a big thing back around 2000, and I've done my fair share of head scratching in these areas. (I arrived at the event-sourcing stuff via a very abstract, conceptual route largely investigating physics/information theory...) It has since morphed into its present-day incarnation in ibGib (github.com/ibgib/ibgib and www.ibgib.com). I don't know how real-world it is, since I'm still pretty much the only one who is using it, but you may enjoy checking it out for a slightly different view on ES. It doesn't strictly fit the definition of CQRS at the moment. This is because I see CQRS as an implementation detail orthogonal to ES, but which dovetails nicely as they both work well with distributed systems. ES has immutable events (monotonically increasing), and CQRS implies a distributed system and allows for more effecient retrieval, leveraging ES's immutability. But ibGib does provide what I see as an augmentation, or maybe an evolution, to ES. It has shifted over the years away from ontological "events" and now is more biological in approach. All of the data is in terms of ibGib, which is a flat database structure comprising `ib`, `data`, `rel8ns` and a `gib` which is a hash of the other three fields. The ES "event" analog specifically about what is replayed to hydrate domain aggregates can be seen in a specific `rel8n` called `dna`. This contains the code necessary to "rebuild" that ibGib immutable frame. This came largely from my dislike of the foundation of the entire river of the source of truth being necessary. With the DNA structure, it actually keeps a dependency graph, which ends up being a projection of the "entire" source of truth. This allows for nodes to "replicate" ("reproduce" would be a better term) as projections of entire graphs. These kinds of deep aspects are really neat! But anyway, I digress. The server side is written in Elixir, which runs on top of Erlang's vm (the BEAM(!)). And you can visually interact with the structure, as the web app runs on d3. I have some demos on YouTube on the website if you just want to look at it, though I don't go into the DNA/ES aspect of it. Anyway, I had to say something, since ES is a topic near and dear and so closely related to ibGib's current architecture. My design decisions could very well help you & others understand ES more deeply. Plus I've been enjoying getting to a point where ibGib is actually usable (and useful). NOTE: I'm somewhat arbitrarily responding to you in this thread, as I used to play halo with an AceOfHearts.

snissn 10y ago

I think of it as using epoll http://man7.org/linux/man-pages/man7/epoll.7.html http://man7.org/linux/man-pages/man7/epoll.7.html

aryehof 10y ago

A good summary. However too many choose to view it only as Event-Sourcing/CQRS, a silver-bullet applicable to a minority of applications. One particularly attractive in Microsoft projects, since it is the only alternative enterprise architecture "practice" they propose to data modeling. Object domain modeling continues to be an anathema, because I think of a wizard and code-generation driven historical perspective. In commercial systems, transactions as actions and events, recorded for business or legal reasons, are central. That includes significant "events" like purchases, sales, reservations etc. In that sense, they all are event driven. Event-Driven as an architecture that is based on recording changes to a baseline state, should be applied only where really suited.

grandalf 10y ago

I'd like to see a simple e-commerce cart written using all of the patterns. That's really the minimal "hello world" for comparing the patterns he discusses.

akiselev 10y ago
A minimal "hello world" project for CQRS and event sourcing would be a simple e-commerce cart and all the software running the supply chain. With a simple shopping cart, you'd write 10x more code just for the tooling to get ES/CQRS up and running reliably.
kpmah 10y ago
Not sure how much this will help, but a couple of years ago I wrote about using event sourcing for a simple blog application in PostgreSQL: http://kevinmahoney.co.uk/articles/immutable-data/ http://kevinmahoney.co.uk/articles/immutable-data/ I wasn't aware of the term 'event sourcing' at the time though.

Flenser 10y ago

Reading this it struck me how React/Flux uses Event-Sourcing whereas Angular uses Event Notification.

osullivj 10y ago

My first contact with event driven programming was coding for the original Windows C API, back in 1990. Nowadays we call that API Win32, but then it may have been win 16. Switching from the traditional imperative style to callbacks was a big deal: don't call us, we'll call you! [1] and [2] show the difference in coding style. Later in the 90s C++ wrappers like MFC and XVT simplified things, but the paradigm was still an inversion of control. And that's what event driven means to me. [1] http://www.paulgriffiths.net/program/c/srcs/hellosrc.html http://www.paulgriffiths.net/program/c/srcs/hellosrc.html [2] http://www.paulgriffiths.net/program/c/srcs/winhellosrc.html http://www.paulgriffiths.net/program/c/srcs/winhellosrc.html

barrkel 10y ago
Unless you do async operations, UI framework programming with callbacks doesn't really approach the full horror that can exist in some styles of event-based programming. UI events are typically of two kinds: user performed an action and you need to update the model / execute a command, or the view is asking for information and you need to translate it from the model. Both are fairly encapsulated; the larger operation doesn't extend beyond the scope of the callback, and if it does (e.g. showing a modal dialog), you can use a nested event loop to take care of event dispatching while showing the modal. The events in a UI are also naturally scoped to the screen being shown. When the screen goes away, you can rely on those events not being dispatched any more. Events get to party on a shared state (the model, perhaps attributes of the view) but scoped within the view being displayed. It's all pretty flat. You also don't need to worry much about concurrency, unless doing async.

barrkel 10y ago

It's very easy to create an event-driven program that is effectively a larger program implemented in terms of "come-from". The decoupling provided by eventing is often illusory; if there's a higher-level semantic that's supposed to be implemented in terms of the outcome of coordinated event handlers, then the dependencies still exist - they're just implicit. If event-handlers are genuinely decoupled from one another, and there isn't a higher-level state machine that's being driven by events, then it can be an excellent way to structure logic. Events-as-deltas that are durably stored and can be reliably replayed is another fine way to go for a system with checkpoints, auditing, history, rollback, time-series reporting and similar requirements. But like the article says, code will be much simpler if it deals with an eternal present and the calculation and application of the deltas is centralized in code that doesn't change often. There's an isomorphism between building a bigger program out of event handlers, and distributed programming. Distributed programming is known to be difficult, and doing it at the message send level is IMO too low a level - one should build more expressive and composable primitives and work at a higher level. That's if it's warranted at all.

guptron 10y ago
"The decoupling provided by eventing is often illusory; if there's a higher-level semantic that's supposed to be implemented in terms of the outcome of coordinated event handlers, then the dependencies still exist - they're just implicit." - I've experienced the same myself and converted my "events" into coupled calls. More expressive.