10 ms·
Don't let event-driven architecture buzzwords fool you
- justusw 4y agoI read this, and this article has a lot of words that I don’t know, and that make me feel intimidated for having missed out on something important for the last ten years of my professional career. The first time someone told me about EDA was when I told them I had problems optimizing a certain SQL query, to which their answer was whether I have considered using event sourcing or storming.
- dncornholio 4y agoNo reason to panic. As I see it, this person is just talking buzz.
- mrkeen 4y ago> missed out on something important for the last ten years * If you write down what happened ("events"), and don't throw away said events, then you can always compare the current state of the system to how you think it should be (according to the events). > I had problems optimizing a certain SQL query * If you don't mutate your events, you are allowed to share them. Perhaps the DB you were working with had an unsuitable structure, making it hard to write fast SQL. No worries! Create a second service/DB with a structure designed for fast querying. Slurp the events which you did not throw away into your new database, and enjoy fast queries (without needing to break or cause regressions in the first service/db).
- throwaway290 4y ago> you can always compare the current state of the system to how you think it should be Unless what happens in the system is different now and old events don't make sense anymore...
- ElevenLathe 4y agoVersioning your events can help with this, along with a test suite that will complain if you change the semantics of events with older versions.
- Joker_vD 4y agoOr the events do still make sense, it's just the processing rules for them have changed several times over the years and of course, the event "use the business rules ver. X.Y from now on" is not logged anywhere.
- bitcurious 4y ago> * If you write down what happened ("events"), and don't throw away said events, then you can always compare the current state of the system to how you think it should be (according to the events). This, by the way, is the fundamental principle of accounting. When you hear “reconciliation” this is what they’re doing - comparing some data reflecting real events against a log of intended events. For example, a nonprofit might reconcile their bank account (actual $ amounts) against GL files generated from their CRM (expected $ amounts.)
- motoboi 4y agoConsidering that object oriented programming began as a message passing and ended up as a request response paradigm (and the entangled mess it can be), I suppose event-driven will follow the same path.
- cdogl 4y agoI’m not very literate in CS; I don’t understand your characterisation of late OO as a request response paradigm. Isn’t calling a function or method to get a result just procedural programming 101?
- senorrib 4y agoYes, it is, and OOP was all about message passing, not synchronous calls. The current incantation in pretty much all OOP languages (excepted maybe Smalltalk) is basically procedural programming with namespaces.
- whstl 4y ago"All about message passing" was Alan Kay's vision of OOP, which was very distinct from what is considered OOP both before and after. He did coin the term "OOP" himself, though, so of course his definition carries a lot of weight, of course. Even Simula, the original OOP language, is much closer to the procedural-ish OOP than to Smalltalk.
- robertlagrant 4y agoGenuine ignorant question: is this a language problem or a coding problem? I'm thinking of "sending a message" as a synchronous function call. Could you not code in a style that passes messages around?
- motoboi 4y agoMessage passing has no response. You can simulate request-response with it, using a channel to go one way and another the other way, but it can get pretty convoluted.
- 01acheru 4y agoI don't know for sure but this reads like an article full of Event-Driven Architecture buzzwords trying to fool me.
- voxadam 4y agoEvent-driven reverse psychology.
- nerdponx 4y agoI don't know about trying to fool you, but it's quite a chaotic mess of an article.
- rwoerz 4y ago+1 although I wouldn't call them buzzwords. The whole architecture community tends to use vague metaphors, borrowed or overloaded terms (events, commands, nouns, verbs and the like) without ever defining them technically or formally. Due to some kind of Illusion of Transparency [1] architecture people often overestimate the degree of common understanding about such terms. [1] https://en.m.wikipedia.org/wiki/Illusion_of_transparency https://en.m.wikipedia.org/wiki/Illusion_of_transparency
- marcosdumay 4y agoDoes it actually say anything, besides diving events in two groups and naming them? I admit that I couldn't focus very well on the article, my eyes wouldn't stop rolling. But I found nothing there.
- candiddevmike 4y ago> Yet, the event producers shouldn’t assume getting a response. This is why we can't have nice things. The number of bugs I've seen caused by this statement/core tenant of EDA makes me question people's sanity when they chase EDA.
- mrkeen 4y ago> Events are facts; they can (and should) have multiple receivers. Yet, the event producers shouldn’t assume getting a response. What this means is: There may be many consumers listening to payment events - a PaymentHistory service, a CustomerFraudDetection service, a Settlement service, a Bookkeeping service. If I publish the fact that {Customer x paid $3.40}, that event has been persisted, and is considered to have happened. What's to be done if something goes wrong downstream? From my point of view, I can't do any more. It is up to the downstream services to restore themselves to a good state. If I 'retry' that message, that's exactly the same as if the customer had paid twice.
- aynyc 4y agoOne word: Idempotent.
- Izkata 4y agoCool, now I can by hundreds of things that cost $3.40 and only pay once.
- JustSomeNobody 4y ago> Cool, now I can by hundreds of things that cost $3.40 Why not? > and only pay once. Sounds like your rules engine is borked.
- Izkata 4y ago> Sounds like your rules engine is borked. Well, that is what they suggested - it needs more work to prevent that situation.
- Dave3of5 4y agoWorking on a system which I think came out of these types of discussion. It's a mess.
- CodexArcanum 4y agoI see we're rebranding SOA/microservices again. I did enjoy reading the preview book for EventStorming, seems like a decent format for getting stakeholders involved in the design process and capturing business details. I'm pretty sure you can also do this with an org-mode doc and emailing the right questions to the right people, but discovery is hard and having a big post-it note brain storm with 30 people does sound like more fun! I don't know, I feel like tech people are regularly amazed that other people can describe the events, processes, and data needs of their jobs if you just ask them. Likewise I grant that many people are poorly onboarded and end up "faking it to make it" in their careers. I've seen similar "people lighting up as they finally understand what the company does" moments through the use of literally any form of high-level modeling of the business. People are starved for context at most jobs.
- bradhe 4y ago“Getting stakeholders involved in the design process” is giving me BDD flashbacks. It’s Gherkin all over again.
- rswskg 4y agohaha, weren't they meant to be writing the tests by now?
- mrkeen 4y agoI was happy that we got rid of Cucumber at my workplace in 2018. Two weeks ago at my new workplace I sat through a colleague's tech talk about how to test with Cucumber.
- yosamino 4y agoA bit naviely using cucumber seems like a good idea. Could you share some advice or resources on why it might not be ?
- 4y ago
- hcarvalhoalves 4y agoI don’t see a mention to this, but in event-driven architecture it’s really really important to either strongly guarantee order (affecting availability) or you need to design it around commutative operations / convergent state machines (better availability, but harder to design). Without either of those, it’s asynchronous CRUD and it’s much harder to guarantee correctness under concurrency.
- kulu2002 4y agoGood article. Nothing new though. Those who work on RTOS, OOP know better.
- fredgrott 4y agoYes, but it is re-inventing what FP discovered at the dawn of internet; the founder of Y Combinator has covered this before although not directly.
- phkahler 4y agoAfter seeing GUIs done in Visual Basic 3-6, I decided that event driven anything is a terrible idea. Even a simple thing like keyboard controlled video games best use a keydown or keyup event to simply set/clear a key state, and everything that cares should monitor (poll) the key state. If your only input is events, convert them to state ASAP because YUK to event driven software. P.S. see Therac-25
- rco8786 4y agoSorry how are keyup/keydown events not event-driven software?
- mandarax8 4y agoI think the point is you're not doing anything with the event immediately, just setting the key state so it can be polled later. Ironically 5 minutes before I read this I changed a coworkers POC 3d app to use event polling instead of key events as it was horribly unresponsive.
- moremetadata 4y agoExcept when using a USB keyboard and can type faster than the keyboard can determine the keys pressed especially when when using modifier keys like Alt and Shift and the USB bus is being overwhelmed with other USB data. Stick with a PS/2 keyboard, it makes it easier to spot malware using the USB bus https://wiki.wireshark.org/CaptureSetup/USB https://wiki.wireshark.org/CaptureSetup/USB
- randomdata 4y ago> I think the point is you're not doing anything with the event immediately, just setting the key state so it can be polled later. Which is event-driven architecture. A component signals that something happened and other components take in that information and update their local state as appropriate.
- phkahler 4y ago
- Scubabear68 4y agoThis article is fairly basic, and may elicit a "well, duh!" response, but I see the issues mentioned in the article coming up over and over again with people new to event driven architectures. I can think of two recent platforms I've looked at that implements very complex micro services and event messaging schemes that could (and should) all be replaced with simple in-process pipelines that can achieve the same outcomes. Every generation of developer seems to need to learn anew that in-process communications are orders of magnitude faster than (and more reliable) than inter-process communications, and that your chatty OOP design will not fair so well as a chatty distributed networking application.
- nerdponx 4y agoLet's say I'm not using microservices and I'm not interested in using microservices, but the idea of event-driven architecture appeals to me. Let's also say that I'm using Python or C# or Ruby or Go. Should I be seeking out some kind of EDA framework? Or do I just need some kind of task queue/cache and a pool of workers looping over its contents processing events?
- bdavid21wnec 4y agoElixir Commanded is one of the best and full featured libraries/frameworks I have ever seen for working with event driven architecture and cqrs. Add in something like Oban Pro and you have a robust system that handles failures and scales
- falcolas 4y agoIt doesn't have to be that complicated. Set up a list of callbacks that need to occur when a specific event is emitted, and call those callbacks. You can also make it dynamic by having a "register_event_hook" function that puts a callback on the event hook list. Tune for performance as necessary. I generally include these in the controller which emits the event, or as an ancestor class for those controllers. And if your needs evolve to the point that it's simpler to use a framework later, you'll understand your needs better and you'll be able to pick one that will best work for you.
- 4y ago
- yourapostasy 4y agoWhile they cover how it should be done, I come across many EDA's neglect to account for operationally providing forecasting and predictability. Without gathering from the business what should be expected and implementing that (within messages, for example), we won't know when an event expected to arrive never arrived, nor when too many events arrived within a given time period. This is mainly a function of developers implementing only for the successful execution path and not for failure modes. Mature code bases that haul in the bags of cash for a business in my experience tend to have more code handling the failure modes and edge cases than the path handling the successful execution. It can be a 90-10 split in cases where the business function is considered fully-automated (though of course that is actually an asymptotic, Platonic ideal). IMHO, too little is taught about coding for graceful failure recovery; it should be in the logical journey from TDD all the way through into devops. More often, what happens is this property is ad hoc accreted over time in code bases useful enough to stick around in production for a long enough time.
- babbledabbler 4y agoI concur with this article. The main pitfall I've seen with EDA (or really any other design paradigm) is buzzwords. Buzzwords are pernicious because they induce a socially held belief instead of an intuitively grasped concept. This tends to lead to whatchamacallit architecture (aka distributed monolith casserole). To counteract this there needs to be a grokking process by the organization to dissolve the buzzwords into clear organizational understanding and buy-in. This exercise is often overlooked to ill effect.
- AngeloAnolin 4y agoAt times, I still fall for buzzwords in the technology space. Looks shiny and most of the very vocal devs are harping about it - maybe worth a try. At the end of the day, in the absence of buzzwords and hype, I sit back and come to the conclusion that tried and tested technology will still be able to do the work 99.9999% of the time. I hold the .00001% for those possibilities where you are doing something breakthrough that in itself merits to be associated as a buzzword. Until then, always proceed with caution and rationalize whether it is only hype or it is something worth adding to your arsenal.
- flerchin 4y agoWritten by Chatbot? Not a lot of specifics here.
- awill88 4y agoHard yawn
- namaria 4y agoTool and design pattern marketing is one of the weirdest traits of software development in my opinion. I don't know if other fields suffer these too but it seems to me we waste so much time and effort because of tooling and design decisions made almost out of taste instead of plain old problem solving.
- tabtab 4y agoI agree some kind of event-driven architecture is ideal for typical business & admin CRUD. However, our existing languages and toolsets seem a poor fit and I've yet to see a good one. They force things into hierarchies and make it hard to search and manage events outside the official hierarchy. Relational is more powerful than trees, so we should find a way to leverage that. I'd like to see languages & IDE's that assist some form of "Table Oriented Programming" (TOP) were either the event snippets are stored in an RDBMS, or at least the RDBMS is used by the IDE to search and manage the myriad events. Then can search and view by SQL and SQL-bases query tools. Developers wouldn't be forced to think in trees. Here's what a typical generic event handler table may resemble: Event (table) ---------------- ID Area // general app category Entity EventType // Search, Listing, Add, Edit, Delete, Process, other Stage // Ex: initialize, loadTemplate, adjustTemplate, preQuery, postQuery, submit, queryErr, failedValidation, passedValidation, preDisplay, postDisplay Tags Version SourceCode // Actual code or a pointer/link to it. In most cases the defaults or other attributes would be good enough such that explicit coding isn't needed, and thus most coding would be for specialized things. // Sample event function/method pseudo-code Public void AAA_EEE_TTT_SSS(EventState ev) { ... if(problemX) { ev.statusOk=false; return; } ... ev.statusOk = true; // final status of event } Where AAA is the area name, EEE the entity name, TTT the event type name, and SSS the stage name. These four columns would be indexed for quick searching and grouping. Under the hood it may still use the language's preferred file trees for generating or saving snippets, but developers wouldn't normally have to know or care about that tree. I have been experimenting with TOP, but the R&D job is too big for me alone. I see and smell the potential power to automate most CRUD grunt work using TOP & events, but gluing it all together in a nice way is a work in progress. (I don't claim it's ideal for high-volume apps, but rather for those with intricate business logic and a medium load.)