5 ms·

Is this ACID? getting so used to databases being the only true failsafe rollback systems, that programming business critical components in things like Java seem

Is this ACID? getting so used to databases being the only true failsafe rollback systems, that programming business critical components in things like Java seems scary. We must need at least an event queue! anyone got some insight? is this just a toy?

lorendsr 3y ago
Sagas are for when you can't do an update in an ACID transaction, for example when updating state across different types of data stores. If you're asking whether the catch clause in a Temporal Workflow saga is guaranteed to execute, the answer is yes. The way it's able to guarantee this is by persisting each step the code takes and recovering program state if a process crashes, server loses power, etc. For an explanation of how this works, see: https://temporal.io/blog/building-reliable-distributed-systems-in-node-js-part-2 https://temporal.io/blog/building-reliable-distributed-syste...
- agumonkey 3y ago
  Seems like a distributed stack unwind
  - lorendsr 3y ago
    If I'm getting your point right, I agree! If you have the workflow / durable execution primitive to depend on (a durable function is guaranteed to complete executing), then there are a lot of pieces of distributed systems stacks that you no longer need to use. Your durable code is automatically backed by the event sourcing, timers, task queues, transfer queues, etc that Temporal internally uses to provide the guarantee, so that you don't need to build them yourself.
  - fortunaTemporal 3y ago
    Author of the article here. Yeah, @agumonkey I like that analogy a lot! Generally, that's a good way of thinking about it. The one additional bit of nuance is it's like a "safe" stack unwind while other processes could be still modifying databases at the same time, so it's not a complete "rollback" of the whole world if that makes sense.
    - agumonkey 3y ago
      Thanks, you're probably right, there's more to it. It's a thrilling topic, are there any other patterns, or abstractions into controlling "distributed state" (apologies if i'm twisting things too much again) between agents to keep things in a correct order ?
      - fortunaTemporal 3y ago
        One pattern is having a Workflow that runs as long as the lifetime of a domain object and holds a conceptual lock on that object—it receives requests to modify the object, and makes sure to only perform one operation at a time. (like the state pattern on a particular agent) Also related: Signals are events that you can send to Workflows and between Workflows, and they’re always delivered in the order they’re received. More generally, for a handy reference of Distributed Systems patterns, check out https://microservices.io/patterns/data/saga.html https://microservices.io/patterns/data/saga.html (though I personally find his diagrams a bit...overwhelming) and the MSN writeups: https://learn.microsoft.com/en-us/azure/architecture/patterns/ https://learn.microsoft.com/en-us/azure/architecture/pattern...
        agumonkey 3y ago
        oh that's cool, thanks a lot
azurelake 3y ago
Nope, it's not a toy. It's a fork of Uber's workflow engine, Cadence. You don't need to explicitly interact with an event queue because it's a higher level abstraction that sits on a queue. In fact, that’s the big value add. ACID doesn't help you once you're trying to coordinate actions across systems, like the example in the article.
- mrkeen 3y ago
  > You don't need to explicitly interact with an event queue because it's a higher level abstraction that sits on a queue. In fact, that’s the big value add. The event queue is an even bigger value add: * It's the audit log * A human reading the event queue has final say over the 'true state' of the system (insofar as such a thing exists in a distributed system)
  - fortunaTemporal 3y ago
    Temporal has an event queue under the covers, so you don't have to implement (and debug) that yourself.
nivertech 3y ago
Not sure about Temporal's implementation, but classic Saga pattern is ACD (without Isolation), so overlapping or concurrent Sagas may overwrite each other. There are several Saga-related application-specific patterns called countermeasures, which help to somewhat mitigate this problem. Also unlike RDBMS with their ACID transactions, Saga design forces you to understand your business requirements better. Which steps are more likely to fail? Which steps are pivotal (i.e. points of no return)? Which steps are riskier or more valuable for business, etc.
fortunaTemporal 3y ago
Agree with what other commenters have said. Also wanted to say that with Temporal you can code Sagas in Go, Java, TypeScript, and PHP, and they're working on expanding to other languages as well.

lorendsr 3y ago

Sagas are for when you can't do an update in an ACID transaction, for example when updating state across different types of data stores. If you're asking whether the catch clause in a Temporal Workflow saga is guaranteed to execute, the answer is yes. The way it's able to guarantee this is by persisting each step the code takes and recovering program state if a process crashes, server loses power, etc. For an explanation of how this works, see: https://temporal.io/blog/building-reliable-distributed-systems-in-node-js-part-2 https://temporal.io/blog/building-reliable-distributed-syste...

agumonkey 3y ago
Seems like a distributed stack unwind
- lorendsr 3y ago
  If I'm getting your point right, I agree! If you have the workflow / durable execution primitive to depend on (a durable function is guaranteed to complete executing), then there are a lot of pieces of distributed systems stacks that you no longer need to use. Your durable code is automatically backed by the event sourcing, timers, task queues, transfer queues, etc that Temporal internally uses to provide the guarantee, so that you don't need to build them yourself.
- fortunaTemporal 3y ago
  Author of the article here. Yeah, @agumonkey I like that analogy a lot! Generally, that's a good way of thinking about it. The one additional bit of nuance is it's like a "safe" stack unwind while other processes could be still modifying databases at the same time, so it's not a complete "rollback" of the whole world if that makes sense.
  - agumonkey 3y ago
    Thanks, you're probably right, there's more to it. It's a thrilling topic, are there any other patterns, or abstractions into controlling "distributed state" (apologies if i'm twisting things too much again) between agents to keep things in a correct order ?
    - fortunaTemporal 3y ago
      One pattern is having a Workflow that runs as long as the lifetime of a domain object and holds a conceptual lock on that object—it receives requests to modify the object, and makes sure to only perform one operation at a time. (like the state pattern on a particular agent) Also related: Signals are events that you can send to Workflows and between Workflows, and they’re always delivered in the order they’re received. More generally, for a handy reference of Distributed Systems patterns, check out https://microservices.io/patterns/data/saga.html https://microservices.io/patterns/data/saga.html (though I personally find his diagrams a bit...overwhelming) and the MSN writeups: https://learn.microsoft.com/en-us/azure/architecture/patterns/ https://learn.microsoft.com/en-us/azure/architecture/pattern...
      - agumonkey 3y ago
        oh that's cool, thanks a lot

azurelake 3y ago

Nope, it's not a toy. It's a fork of Uber's workflow engine, Cadence. You don't need to explicitly interact with an event queue because it's a higher level abstraction that sits on a queue. In fact, that’s the big value add. ACID doesn't help you once you're trying to coordinate actions across systems, like the example in the article.

mrkeen 3y ago
> You don't need to explicitly interact with an event queue because it's a higher level abstraction that sits on a queue. In fact, that’s the big value add. The event queue is an even bigger value add: * It's the audit log * A human reading the event queue has final say over the 'true state' of the system (insofar as such a thing exists in a distributed system)
- fortunaTemporal 3y ago
  Temporal has an event queue under the covers, so you don't have to implement (and debug) that yourself.

nivertech 3y ago

Not sure about Temporal's implementation, but classic Saga pattern is ACD (without Isolation), so overlapping or concurrent Sagas may overwrite each other. There are several Saga-related application-specific patterns called countermeasures, which help to somewhat mitigate this problem. Also unlike RDBMS with their ACID transactions, Saga design forces you to understand your business requirements better. Which steps are more likely to fail? Which steps are pivotal (i.e. points of no return)? Which steps are riskier or more valuable for business, etc.

fortunaTemporal 3y ago

Agree with what other commenters have said. Also wanted to say that with Temporal you can code Sagas in Go, Java, TypeScript, and PHP, and they're working on expanding to other languages as well.