3 ms·

I'm having trouble finding a distinction between crash-only software and software with good error handling. If the software (or bank or government or whatever)

I'm having trouble finding a distinction between crash-only software and software with good error handling. If the software (or bank or government or whatever) crashes and then restarts gracefully and automatically, I don't see how it could be said to have crashed.

robinsloan 16y ago
That's a fair point -- a crash handled gracefully & completely sorta ceases to be a crash at all. I think one distinction the authors of the original paper make is that an app's off switch ought to be external to the app. More like "kill -9" than "exit"; more like pulling the plug than pressing a "power down" button.
- damienkatz 16y ago
  Yes. CouchDB is designed as crash only software, it detects and restarts immediately on crash, and shutting it down is done with kill, not exit.
tpz 16y ago
The distinction is simply (but critically!) this: the standard operating procedure (and, in fact, the only available way!) to shut down a piece of crash-only software is to crash it forcibly. There is no quit button, no shutdown command, no remote API, there is just kill. Sounds simple enough, but where it really gets interesting is in how this mindset affects your design and implementation decisions. Imagaine for a moment what it would be like if during every second of work during every day of the week each and every thing you did was accompanied by a corresponding design exercise along the lines of "okay, but what if it crashes right then." Now that would sure produce some "software with good error handling", now wouldn't it. That is crash-only software.
- mkramlich 16y ago
  Yep, you'd be forced to think more about transactions. What activities needed to be carried out as a transaction, and for what activities it didn't matter. Because at any point the process could exit.
Hoff 16y ago
It's the distinction between attempting an automatic recovery under application control, and the philosophy of the badness happened; get out NOW failure design. In complex systems, there's always the assumed potential for untoward behavior from unanticipated recovery environments; the handling of recovery can be a bigger problem than a complete failure. In these environments, clean failures are preferable. And staging the recovery processing can be preferred. This goes as far as staging application start-up and sequencing the component server reboots manually. Yes, manually.

robinsloan 16y ago

That's a fair point -- a crash handled gracefully & completely sorta ceases to be a crash at all. I think one distinction the authors of the original paper make is that an app's off switch ought to be external to the app. More like "kill -9" than "exit"; more like pulling the plug than pressing a "power down" button.

damienkatz 16y ago
Yes. CouchDB is designed as crash only software, it detects and restarts immediately on crash, and shutting it down is done with kill, not exit.

tpz 16y ago

The distinction is simply (but critically!) this: the standard operating procedure (and, in fact, the only available way!) to shut down a piece of crash-only software is to crash it forcibly. There is no quit button, no shutdown command, no remote API, there is just kill. Sounds simple enough, but where it really gets interesting is in how this mindset affects your design and implementation decisions. Imagaine for a moment what it would be like if during every second of work during every day of the week each and every thing you did was accompanied by a corresponding design exercise along the lines of "okay, but what if it crashes right then." Now that would sure produce some "software with good error handling", now wouldn't it. That is crash-only software.

mkramlich 16y ago
Yep, you'd be forced to think more about transactions. What activities needed to be carried out as a transaction, and for what activities it didn't matter. Because at any point the process could exit.

Hoff 16y ago

It's the distinction between attempting an automatic recovery under application control, and the philosophy of the badness happened; get out NOW failure design. In complex systems, there's always the assumed potential for untoward behavior from unanticipated recovery environments; the handling of recovery can be a bigger problem than a complete failure. In these environments, clean failures are preferable. And staging the recovery processing can be preferred. This goes as far as staging application start-up and sequencing the component server reboots manually. Yes, manually.