3 ms·

This article is fairly basic, and may elicit a "well, duh!" response, but I see the issues mentioned in the article coming up over and over again with people ne

This article is fairly basic, and may elicit a "well, duh!" response, but I see the issues mentioned in the article coming up over and over again with people new to event driven architectures. I can think of two recent platforms I've looked at that implements very complex micro services and event messaging schemes that could (and should) all be replaced with simple in-process pipelines that can achieve the same outcomes. Every generation of developer seems to need to learn anew that in-process communications are orders of magnitude faster than (and more reliable) than inter-process communications, and that your chatty OOP design will not fair so well as a chatty distributed networking application.

nerdponx 4y ago
Let's say I'm not using microservices and I'm not interested in using microservices, but the idea of event-driven architecture appeals to me. Let's also say that I'm using Python or C# or Ruby or Go. Should I be seeking out some kind of EDA framework? Or do I just need some kind of task queue/cache and a pool of workers looping over its contents processing events?
- bdavid21wnec 4y ago
  Elixir Commanded is one of the best and full featured libraries/frameworks I have ever seen for working with event driven architecture and cqrs. Add in something like Oban Pro and you have a robust system that handles failures and scales
- falcolas 4y ago
  It doesn't have to be that complicated. Set up a list of callbacks that need to occur when a specific event is emitted, and call those callbacks. You can also make it dynamic by having a "register_event_hook" function that puts a callback on the event hook list. Tune for performance as necessary. I generally include these in the controller which emits the event, or as an ancestor class for those controllers. And if your needs evolve to the point that it's simpler to use a framework later, you'll understand your needs better and you'll be able to pick one that will best work for you.
- saltcured 4y ago
  You should stop and first think about your big picture goals. What is the source and timescale of your events? What is the lifecycle of higher order processes you build using event cascades? What are your reliability, durability, and recovery concerns? Get to know your concerns before you start seriously evaluating any tools or design approach. But, before you go too far in this thought process, wander over to the "you cannot have exactly-once delivery" thread and think on that for a while too. Distributed reliability or consistency goals may lead you away from event-driven and into state-driven approaches. You start to model the actual long-term process you want managed, and the producers and consumers can inspect or update a metadata record representing that process as they play their parts in it. I.e. you are better off with some idempotent order-placement protocol to create a customer order record than some simplistic "customer clicked order button" event which could leave to ambiguous scenarios where orders are placed, lost, or duplicated. Similarly, you don't want abnormalities in your fulfillment process to introduce more ambiguity as to whether an order has been shipped or not. This is where "business process" or "document process" frameworks would be more appropriate than event-driven ones. Such systems may have event-driven characteristics in them, but that's more of a scheduling optimization and not the fundamental state model. On the other end of the spectrum, you can have sensor-based systems where measurement samples are streamed. You cannot have the monitored environment pause until a sample is registered. You can only observe or fail to observe the environment while it continued on its natural course. You really want some kind of capture, logging, and analysis platform. It would be inappropriate to treat the individual samples as events in an EDA, but you might synthesize events from an analysis, i.e. when a measurement value crosses some threshold or shows some time-dependent signature across multiple samples.

nerdponx 4y ago

Let's say I'm not using microservices and I'm not interested in using microservices, but the idea of event-driven architecture appeals to me. Let's also say that I'm using Python or C# or Ruby or Go. Should I be seeking out some kind of EDA framework? Or do I just need some kind of task queue/cache and a pool of workers looping over its contents processing events?

bdavid21wnec 4y ago
Elixir Commanded is one of the best and full featured libraries/frameworks I have ever seen for working with event driven architecture and cqrs. Add in something like Oban Pro and you have a robust system that handles failures and scales
falcolas 4y ago
It doesn't have to be that complicated. Set up a list of callbacks that need to occur when a specific event is emitted, and call those callbacks. You can also make it dynamic by having a "register_event_hook" function that puts a callback on the event hook list. Tune for performance as necessary. I generally include these in the controller which emits the event, or as an ancestor class for those controllers. And if your needs evolve to the point that it's simpler to use a framework later, you'll understand your needs better and you'll be able to pick one that will best work for you.
saltcured 4y ago
You should stop and first think about your big picture goals. What is the source and timescale of your events? What is the lifecycle of higher order processes you build using event cascades? What are your reliability, durability, and recovery concerns? Get to know your concerns before you start seriously evaluating any tools or design approach. But, before you go too far in this thought process, wander over to the "you cannot have exactly-once delivery" thread and think on that for a while too. Distributed reliability or consistency goals may lead you away from event-driven and into state-driven approaches. You start to model the actual long-term process you want managed, and the producers and consumers can inspect or update a metadata record representing that process as they play their parts in it. I.e. you are better off with some idempotent order-placement protocol to create a customer order record than some simplistic "customer clicked order button" event which could leave to ambiguous scenarios where orders are placed, lost, or duplicated. Similarly, you don't want abnormalities in your fulfillment process to introduce more ambiguity as to whether an order has been shipped or not. This is where "business process" or "document process" frameworks would be more appropriate than event-driven ones. Such systems may have event-driven characteristics in them, but that's more of a scheduling optimization and not the fundamental state model. On the other end of the spectrum, you can have sensor-based systems where measurement samples are streamed. You cannot have the monitored environment pause until a sample is registered. You can only observe or fail to observe the environment while it continued on its natural course. You really want some kind of capture, logging, and analysis platform. It would be inappropriate to treat the individual samples as events in an EDA, but you might synthesize events from an analysis, i.e. when a measurement value crosses some threshold or shows some time-dependent signature across multiple samples.