5 ms·

The triggering of the action is a direct consequence of the information an event contains. Whether or not an action is triggered should not be the responsibilit

The triggering of the action is a direct consequence of the information an event contains. Whether or not an action is triggered should not be the responsibility of the event. If you are writing events with the intention of having them invoke some specific actions, then you should prefer to invoke those things directly. You should be describing a space of things that have occurred, not commands to be carried out. By default I would only include business keys in my event data. This gets you out of traffic on having to make the event serve as an aggregate view for many consumers. If you provide the keys of the affected items, each consumer can perform their own targeted lookups as needed. Making assumptions about what views each will need is where things get super nasty in my experience (i.e. modifying events every time you add consumers).

withinboredom 2y ago
Events shouldn't carry any data in my opinion, except parameterized data. In the context of a booking, for example, it would be SeatBooked {41A} instead of 41ABooked, though the latter is a better event, but harder to program for. The entire flow might looked like this: SeatTimeLimitedReserved {41A, 15m} SeatAssignedTo {UserA} SeatBooked {41A} If a consumer needs more data, there should be a new event.
lutzh 2y ago
Thanks for your feedback, I appreciate it! > The triggering of the action is a direct consequence of the information an event contains. Whether or not an action is triggered should not be the responsibility of the event. I agree, but still for different consumers events will have different consequences - in some consumers it'll trigger an action that is part of a higher-level process (and possibly further events), in others it'll only lead to data being updated. > If you are writing events with the intention of having them invoke some specific actions, then you should prefer to invoke those things directly. You should be describing a space of things that have occurred, not commands to be carried out. With this I don't agree. I think that's the core of event-driven architecture that events drive the process, i.e. will trigger certain actions. That's not contradicting them describing what has occurred, and doesn't make them commands. > By default I would only include business keys in my event data. This gets you out of traffic on having to make the event serve as an aggregate view for many consumers. If you provide the keys of the affected items, each consumer can perform their own targeted lookups as needed. Making assumptions about what views each will need is where things get super nasty in my experience (i.e. modifying events every time you add consumers). This is feedback I got multiple times, the "notification plus callback" seems to be a popular pattern. It has its own problems though, both conceptual (event representing an immutable set of facts) and technical (high volume of events). I think digging into the pros and cons of that pattern will be one of my next blog posts! Stay tuned!
- BerislavLopac 2y ago
  > I think that's the core of event-driven architecture that events drive the process, i.e. will trigger certain actions. In an event-driven system, there is neither guarantee not expectation that an event will trigger an action; it might, but it might not. Events are simply a log [0] of "things" happening in various subsystems, published to various channels for other subsystems to ignore or act upon on their own terms. Let say that we have two subsystems - A and B. When something happens on A, it will emit a corresponding event (e.g. SomethingHappened) to a specific channel (e.g. EventsFromA); if B is listening to that channel, it can "recognise" that event and initiate (i.e. "trigger") some action of its own. However, if A explicitly wants B to do something, it's a command, i.e. a direct coupling by definition. As GP states, that is better handled as a direct request from A to B. Theoretically, there is a possible scenario where A "knows" that a certain action needs to happen in the system, but does not know which subsystem has that capability, i.e. has no knowledge that B can do that. In that case it can "request" something to happen, e.g. by submitting an event like "UserCreationRequested"; however, there is no guarantee that any service will "see" that event and act upon it. [0] https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying https://engineering.linkedin.com/distributed-systems/log-wha...
williamdclt 2y ago
> each consumer can perform their own targeted lookups as needed that puts you into tricky race condition territory, the data targeted by an event might have changed (or be deleted) between the time it was emitted and the time you're processing it. It's not always a problem, but you have to analyse if it could be every time. It also means that you're losing information on what this event actually represents: looking at an old event you wouldn't know what it actually did, as the data has changed since then. It also introduces a synchronous dependency between services: your consumer has to query the service that dispatched the event for additional information (which is complexity and extra load). Ideally you'd design your event so that downstream consumers don't need extra information, or at least the information they need is independent from the data described by the event: eg a consumer needs the user name to format an email in reaction to the "user_changed_password" event? No problem to query the service for the name, these are independent concepts, updates to these things (password & name) can happen concurrently but it doesn't really matter if a race condition happens
- candiddevmike 2y ago
  There should be some law that says strictly serialized process should never be broken into discreet services. Distributed locks and transactions are hell.
  - withinboredom 2y ago
    The best way to avoid distributed locks and transactions is to manually do the work. For example, instead of doing a distributed lock on two accounts when transferring funds, you might do this (which is the same as a distributed transaction, without the lock): 1. take money from account A 2. if failed, put money back into account A 3. put money into account B 4. if failed, put money back into account A In other words, perform compensating actions instead of doing transactions. This also requires that you have some kind of mechanism to handle an application crash between 2 and 3, but that is something else entirely. I've been working on this for a couple of years now and getting close to something really interesting ... but not quite there yet.
jwarden 2y ago
Another architecture might be that the service responsible for Seat Selection emits a `SeatSelected` event, and another service responsible for updating bookings emits a `BookingUpdated(Reason: SeatSelected)` "fat" event. Same for `PaymentReceived` and `TicketIssued`. Both events would "describe a space of things that occurred" as @bob1029 suggests. The seat selection process for an actual airline probably needs to be more involved. @withinboredom recommends: - SeatTimeLimitedReserved {41A, 15m} - SeatAssignedTo {UserA} - SeatBooked {41A} In which case, only SeatBooked would trigger a BookingUpdated event.
- lutzh 2y ago
  Thanks for your feedback. I realize I should have elaborated the example a bit more, it's too vague. So, as I wrote in some other reply as well, please don't over-interpret it. The point was only to say that in order to differentiate the events, we don't necessarily need distinct types (which would result in multiple schemas on a topic), but can instead encode it in one type/schema. Like mapping in ORM - instead of "table per subclass", you can use "table per class hierarchy".
monksy 2y ago
Say that again louder for people in the back. Message queues aren't a networking protocol. Anyone can subscribe to consume the events.

withinboredom 2y ago

Events shouldn't carry any data in my opinion, except parameterized data. In the context of a booking, for example, it would be SeatBooked {41A} instead of 41ABooked, though the latter is a better event, but harder to program for. The entire flow might looked like this: SeatTimeLimitedReserved {41A, 15m} SeatAssignedTo {UserA} SeatBooked {41A} If a consumer needs more data, there should be a new event.

lutzh 2y ago

Thanks for your feedback, I appreciate it! > The triggering of the action is a direct consequence of the information an event contains. Whether or not an action is triggered should not be the responsibility of the event. I agree, but still for different consumers events will have different consequences - in some consumers it'll trigger an action that is part of a higher-level process (and possibly further events), in others it'll only lead to data being updated. > If you are writing events with the intention of having them invoke some specific actions, then you should prefer to invoke those things directly. You should be describing a space of things that have occurred, not commands to be carried out. With this I don't agree. I think that's the core of event-driven architecture that events drive the process, i.e. will trigger certain actions. That's not contradicting them describing what has occurred, and doesn't make them commands. > By default I would only include business keys in my event data. This gets you out of traffic on having to make the event serve as an aggregate view for many consumers. If you provide the keys of the affected items, each consumer can perform their own targeted lookups as needed. Making assumptions about what views each will need is where things get super nasty in my experience (i.e. modifying events every time you add consumers). This is feedback I got multiple times, the "notification plus callback" seems to be a popular pattern. It has its own problems though, both conceptual (event representing an immutable set of facts) and technical (high volume of events). I think digging into the pros and cons of that pattern will be one of my next blog posts! Stay tuned!

BerislavLopac 2y ago
> I think that's the core of event-driven architecture that events drive the process, i.e. will trigger certain actions. In an event-driven system, there is neither guarantee not expectation that an event will trigger an action; it might, but it might not. Events are simply a log [0] of "things" happening in various subsystems, published to various channels for other subsystems to ignore or act upon on their own terms. Let say that we have two subsystems - A and B. When something happens on A, it will emit a corresponding event (e.g. SomethingHappened) to a specific channel (e.g. EventsFromA); if B is listening to that channel, it can "recognise" that event and initiate (i.e. "trigger") some action of its own. However, if A explicitly wants B to do something, it's a command, i.e. a direct coupling by definition. As GP states, that is better handled as a direct request from A to B. Theoretically, there is a possible scenario where A "knows" that a certain action needs to happen in the system, but does not know which subsystem has that capability, i.e. has no knowledge that B can do that. In that case it can "request" something to happen, e.g. by submitting an event like "UserCreationRequested"; however, there is no guarantee that any service will "see" that event and act upon it. [0] https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying https://engineering.linkedin.com/distributed-systems/log-wha...

williamdclt 2y ago

> each consumer can perform their own targeted lookups as needed that puts you into tricky race condition territory, the data targeted by an event might have changed (or be deleted) between the time it was emitted and the time you're processing it. It's not always a problem, but you have to analyse if it could be every time. It also means that you're losing information on what this event actually represents: looking at an old event you wouldn't know what it actually did, as the data has changed since then. It also introduces a synchronous dependency between services: your consumer has to query the service that dispatched the event for additional information (which is complexity and extra load). Ideally you'd design your event so that downstream consumers don't need extra information, or at least the information they need is independent from the data described by the event: eg a consumer needs the user name to format an email in reaction to the "user_changed_password" event? No problem to query the service for the name, these are independent concepts, updates to these things (password & name) can happen concurrently but it doesn't really matter if a race condition happens

candiddevmike 2y ago
There should be some law that says strictly serialized process should never be broken into discreet services. Distributed locks and transactions are hell.
- withinboredom 2y ago
  The best way to avoid distributed locks and transactions is to manually do the work. For example, instead of doing a distributed lock on two accounts when transferring funds, you might do this (which is the same as a distributed transaction, without the lock): 1. take money from account A 2. if failed, put money back into account A 3. put money into account B 4. if failed, put money back into account A In other words, perform compensating actions instead of doing transactions. This also requires that you have some kind of mechanism to handle an application crash between 2 and 3, but that is something else entirely. I've been working on this for a couple of years now and getting close to something really interesting ... but not quite there yet.

jwarden 2y ago

Another architecture might be that the service responsible for Seat Selection emits a `SeatSelected` event, and another service responsible for updating bookings emits a `BookingUpdated(Reason: SeatSelected)` "fat" event. Same for `PaymentReceived` and `TicketIssued`. Both events would "describe a space of things that occurred" as @bob1029 suggests. The seat selection process for an actual airline probably needs to be more involved. @withinboredom recommends: - SeatTimeLimitedReserved {41A, 15m} - SeatAssignedTo {UserA} - SeatBooked {41A} In which case, only SeatBooked would trigger a BookingUpdated event.

lutzh 2y ago
Thanks for your feedback. I realize I should have elaborated the example a bit more, it's too vague. So, as I wrote in some other reply as well, please don't over-interpret it. The point was only to say that in order to differentiate the events, we don't necessarily need distinct types (which would result in multiple schemas on a topic), but can instead encode it in one type/schema. Like mapping in ORM - instead of "table per subclass", you can use "table per class hierarchy".

monksy 2y ago

Say that again louder for people in the back. Message queues aren't a networking protocol. Anyone can subscribe to consume the events.