4 ms·

The biggest problems I've run into with event-driven (microservices) architectures is that there is no built-in opportunity for any kind of application-level fe

The biggest problems I've run into with event-driven (microservices) architectures is that there is no built-in opportunity for any kind of application-level feedback, or even minimal synchronous data validation. Your queue may be up and accepting messages, but if you've sent bad data, or if nobody's listening, there's no ability to find that out without essentially reinventing HTTP or creating their own lower-overhead error channels -- an approach which leaves a lot open for mistakes over something well-understood like HTTP.

dragonwriter 9y ago
Why not do synchronous validation at the boundary with event-driven async architecture within the boundary?
- Chris_Newton 9y ago
  That works OK up to a point, but if validity depends on any sort of state that might be changing concurrently then potentially it opens you up to problems with race conditions. Just like locking shared state for thread synchronisation or operating on a shared filesystem, there are usually only two completely safe strategies. Assuming your underlying operations are guaranteed to be atomic but might fail, either you can do everything synchronously and report any failure immediately, or you can go all-in on asynchronicity, so even reporting success or failure is done by firing a message back asynchronously later. If you’re going with anything more exotic, you’ll need a strategy for dealing with the intermediate states, for example something that looked like it was going to work but then didn’t. That might mean something like doing just enough synchronously to guarantee you’ve got a record of what needs to happen as a result of the event, then returning a success indication, and then asynchronously following up on the other required actions later, with some sort of auditing and reconciliation processes to make sure that everything that should happen eventually does. In this sort of situation, maybe you would also be able to do enough synchronous validation up-front to know that your request will be able to succeed, even if the goalposts move during the process.
sametmax 9y ago
Some protocoles like WAMP do give you feedback, propagating errors in an automatic manner.

dragonwriter 9y ago

Why not do synchronous validation at the boundary with event-driven async architecture within the boundary?

Chris_Newton 9y ago
That works OK up to a point, but if validity depends on any sort of state that might be changing concurrently then potentially it opens you up to problems with race conditions. Just like locking shared state for thread synchronisation or operating on a shared filesystem, there are usually only two completely safe strategies. Assuming your underlying operations are guaranteed to be atomic but might fail, either you can do everything synchronously and report any failure immediately, or you can go all-in on asynchronicity, so even reporting success or failure is done by firing a message back asynchronously later. If you’re going with anything more exotic, you’ll need a strategy for dealing with the intermediate states, for example something that looked like it was going to work but then didn’t. That might mean something like doing just enough synchronously to guarantee you’ve got a record of what needs to happen as a result of the event, then returning a success indication, and then asynchronously following up on the other required actions later, with some sort of auditing and reconciliation processes to make sure that everything that should happen eventually does. In this sort of situation, maybe you would also be able to do enough synchronous validation up-front to know that your request will be able to succeed, even if the goalposts move during the process.

sametmax 9y ago

Some protocoles like WAMP do give you feedback, propagating errors in an automatic manner.