4 ms·

Durable execution looks a bit like a buzzword in general. If your state is defined in the execution graph then it's just an umbrella term for a group of pre-exi

Durable execution looks a bit like a buzzword in general. If your state is defined in the execution graph then it's just an umbrella term for a group of pre-existing algorithms and patterns. If it's undefined then what are you resuming to? The snapshot just before the crash likely leads to the undefined state again, in which case you're durably automating the crash (or even worse, uncaught incorrect behavior).

locknitpicker 15d ago
> Durable execution looks a bit like a buzzword in general. If your state is defined in the execution graph then it's just an umbrella term for a group of pre-existing algorithms and patterns. If you first approach a tool because of buzzwords, don't be surprised that you think of the buzzwords instead of the tool. Durable executions greatly simplify how workflows can be implemented and audited, and they literally allow eternal workflow executions that are not tied to the lifetime of an instance assigned to execute them. Durable executions do not require fancy infrastructure, only a terribly simple database. If you understand continuations, you understand durable executions. Otherwise, you'll be stuck with the excuse that they are buzzwords.
- weitendorf 15d ago
  Well, you have to either capture/eliminate/persist side effects and control the environment tightly, or it's limited in what it can do. In a distributed or concurrent system, for full granularity, that can require specialized timing or virtualization techniques up to ensuring fully atomic snapshots and deterministic execution environments (and whether or not that properly models the SUT in real environments, or introduces bias/breaks the reproducibility in a way you care about) Otherwise if you're only running against fixed checkpoints you have something closer to traces that maybe you could re-run or test against, in some cases, if you put in the work to set it up. In distributed systems that can be a lot of work so it's a bit vague if left unspecified. Because it's not enough to merely replay something if things can drift or don't accurately model the real system
  - locknitpicker 15d ago
    > Well, you have to either capture/eliminate/persist side effects and control the environment tightly, or it's limited in what it can do. You really need to read up on durable executions before commenting. Your comment reads as if you are completely oblivious to them. Their whole point is benefiting from an execution model where your workflow is comprised of idempotent pure functions whose inputs and outputs are tracked by the durable task persistence.
    - weitendorf 15d ago
      It's a problem I've spent a lot of time on, just not through the products marketed like temporal. I commented on the article because it's about program replay with checkpointing which is similar to what I worked with/mentioned. I didn't see the article mention idempotency anywhere, and you didn't in the original response to the guy who said it just sounded like a buzzword; atomic snapshotting with deterministic execution is literally how you make a program continuation an idempotent function! And the article is about solving the problem at the language runtime level so it doesn't even do that. So it would be reasonable to assume it is a buzzword if it does not have the essential property you mentioned and I was referring to. I was not even intending to disagree with you but just add what would make it less of a buzzword in this kind of case.
  - mike_hearn 14d ago
    For the sort of architecture described in the blog post, you're persisting the program rather than a log, so you only need idempotency between any two checkpoints. For a lot of apps that's easy to achieve if write the snapshot into the same transaction the user is using for application data, and if you create/propagate idempotency tokens to other services. You don't need specialized timing or virtualization techniques.
hypervs 15d ago
Well checkpointing does not make an incorrect program correct. Durability just guarantees that execution can survive infrastructure failure, not that the resumed program will not encounter the same deterministic bug again. TCC checkpoints at explicit durable boundaries rather than at an arbitrary instruction immediately before a crash. So if the next operation repeatedly fails, ordinary retry limits or operator intervention are still required. My proposed model is to represent the recoverable program state as a committed continuation, rather than reconstructing state by replaying the completed execution history.

locknitpicker 15d ago

> Durable execution looks a bit like a buzzword in general. If your state is defined in the execution graph then it's just an umbrella term for a group of pre-existing algorithms and patterns. If you first approach a tool because of buzzwords, don't be surprised that you think of the buzzwords instead of the tool. Durable executions greatly simplify how workflows can be implemented and audited, and they literally allow eternal workflow executions that are not tied to the lifetime of an instance assigned to execute them. Durable executions do not require fancy infrastructure, only a terribly simple database. If you understand continuations, you understand durable executions. Otherwise, you'll be stuck with the excuse that they are buzzwords.

weitendorf 15d ago
Well, you have to either capture/eliminate/persist side effects and control the environment tightly, or it's limited in what it can do. In a distributed or concurrent system, for full granularity, that can require specialized timing or virtualization techniques up to ensuring fully atomic snapshots and deterministic execution environments (and whether or not that properly models the SUT in real environments, or introduces bias/breaks the reproducibility in a way you care about) Otherwise if you're only running against fixed checkpoints you have something closer to traces that maybe you could re-run or test against, in some cases, if you put in the work to set it up. In distributed systems that can be a lot of work so it's a bit vague if left unspecified. Because it's not enough to merely replay something if things can drift or don't accurately model the real system
- locknitpicker 15d ago
  > Well, you have to either capture/eliminate/persist side effects and control the environment tightly, or it's limited in what it can do. You really need to read up on durable executions before commenting. Your comment reads as if you are completely oblivious to them. Their whole point is benefiting from an execution model where your workflow is comprised of idempotent pure functions whose inputs and outputs are tracked by the durable task persistence.
  - weitendorf 15d ago
    It's a problem I've spent a lot of time on, just not through the products marketed like temporal. I commented on the article because it's about program replay with checkpointing which is similar to what I worked with/mentioned. I didn't see the article mention idempotency anywhere, and you didn't in the original response to the guy who said it just sounded like a buzzword; atomic snapshotting with deterministic execution is literally how you make a program continuation an idempotent function! And the article is about solving the problem at the language runtime level so it doesn't even do that. So it would be reasonable to assume it is a buzzword if it does not have the essential property you mentioned and I was referring to. I was not even intending to disagree with you but just add what would make it less of a buzzword in this kind of case.
- mike_hearn 14d ago
  For the sort of architecture described in the blog post, you're persisting the program rather than a log, so you only need idempotency between any two checkpoints. For a lot of apps that's easy to achieve if write the snapshot into the same transaction the user is using for application data, and if you create/propagate idempotency tokens to other services. You don't need specialized timing or virtualization techniques.

hypervs 15d ago

Well checkpointing does not make an incorrect program correct. Durability just guarantees that execution can survive infrastructure failure, not that the resumed program will not encounter the same deterministic bug again. TCC checkpoints at explicit durable boundaries rather than at an arbitrary instruction immediately before a crash. So if the next operation repeatedly fails, ordinary retry limits or operator intervention are still required. My proposed model is to represent the recoverable program state as a committed continuation, rather than reconstructing state by replaying the completed execution history.