3 ms·

"If something is a precondition then the app shouldn't act like it wasn't just to make Docker configuration easier. It needs to fail Fail fast doesn't mean "cr

"If something is a precondition then the app shouldn't act like it wasn't just to make Docker configuration easier. It needs to fail Fail fast doesn't mean "crash completely". It means "fallback to the next sensible approach". "But I dont' think application launch is the right time for it. Application launch is an opportunity to make sure that all dependencies were in place at some point. If things break later on, the odds of it being a temporary issue are much better." This is presuming you or some human has control over the lifecycle of an individual process. The trend in both mobile and cloud native is the process model, which says the opposite: your app process can and will be killed or relaunched any time by the underlying OS. It may do so out of sequence with backing service availability. This, retries (with a time or count bound, perhaps) are a sensible default.

fauigerzigerk 11y ago
>This is presuming you or some human has control over the lifecycle of an individual process. No, that doesn't matter at all. It's like with DBMS transactions. I want a defined point at which the system is in a known good state or fail in some detectable way. For long running processes that get bounced automatically, there needs to be some sort of monitoring anyway. Monitoring is easier if the application does not linger endlessly in an inconsistent state.
- parasubvert 11y ago
  "I want a defined point at which the system is in a known good state or fail in some detectable way." I think we're (as an industry) getting to scale and complexity of systems that warrants systems to heal themselves for a range of predictable and well-understood failure modes, in a way that doesn't require my manual interference.
  - fauigerzigerk 11y ago
    Agreed, but I don't think we can do that without having transactional boundaries. When a transaction fails, it doesn't necessarily mean that a human has to intervene. It just means that we have a reduced set of possible states that are known to be consistent. I don't see how we could ever hope to define correct self healing algorithms without reducing the number of possible states a system can be in.
    - parasubvert 11y ago
      Yes. Consistency guarantees in the face of distributed failure is a popular topic. (CAP theorem, etc). The whole "cloud native" (12 factor apps, microservices, immutable or disposable infrastructure) movement is also trying to describe ways to simplify most codebases so that you keep as much of the system as stateless/ephemeral as possible. What is interesting is that , for the stateful/persistent data processing, most large scale systems are rejecting transaction boundaries as we know them (fully serializable isolation and consistently) for relaxed consistency. There are some good articles and papers on how programming needs to change to enable better self healing / "recoverable to a known state" behaviour, such as CRDTs.
  - davidbanham 11y ago
    Absolutely! But that doesn't need to be the app's job. It's the role of an orthogonal process, the monitoring or init daemon, to say "Hey, this process bailed. I should restart it." That way all your app has to worry about is "Every time I get started, I should try and connect to my dependent services. If it doesn't work, bail." And the monitoring process gets to worry about things like "I should retry X times before giving up. There need to be at least Y instances of this process running."

fauigerzigerk 11y ago

>This is presuming you or some human has control over the lifecycle of an individual process. No, that doesn't matter at all. It's like with DBMS transactions. I want a defined point at which the system is in a known good state or fail in some detectable way. For long running processes that get bounced automatically, there needs to be some sort of monitoring anyway. Monitoring is easier if the application does not linger endlessly in an inconsistent state.

parasubvert 11y ago
"I want a defined point at which the system is in a known good state or fail in some detectable way." I think we're (as an industry) getting to scale and complexity of systems that warrants systems to heal themselves for a range of predictable and well-understood failure modes, in a way that doesn't require my manual interference.
- fauigerzigerk 11y ago
  Agreed, but I don't think we can do that without having transactional boundaries. When a transaction fails, it doesn't necessarily mean that a human has to intervene. It just means that we have a reduced set of possible states that are known to be consistent. I don't see how we could ever hope to define correct self healing algorithms without reducing the number of possible states a system can be in.
  - parasubvert 11y ago
    Yes. Consistency guarantees in the face of distributed failure is a popular topic. (CAP theorem, etc). The whole "cloud native" (12 factor apps, microservices, immutable or disposable infrastructure) movement is also trying to describe ways to simplify most codebases so that you keep as much of the system as stateless/ephemeral as possible. What is interesting is that , for the stateful/persistent data processing, most large scale systems are rejecting transaction boundaries as we know them (fully serializable isolation and consistently) for relaxed consistency. There are some good articles and papers on how programming needs to change to enable better self healing / "recoverable to a known state" behaviour, such as CRDTs.
- davidbanham 11y ago
  Absolutely! But that doesn't need to be the app's job. It's the role of an orthogonal process, the monitoring or init daemon, to say "Hey, this process bailed. I should restart it." That way all your app has to worry about is "Every time I get started, I should try and connect to my dependent services. If it doesn't work, bail." And the monitoring process gets to worry about things like "I should retry X times before giving up. There need to be at least Y instances of this process running."