4 ms·

I've recently been musing on a similar theory, but for safety regulations, redundancy, and preventable disasters, for example at nuclear power plants, or the re

I've recently been musing on a similar theory, but for safety regulations, redundancy, and preventable disasters, for example at nuclear power plants, or the recent example with the Washington DC subway system shutdown. Initially, there are lots of safety regulations and redundancy, and the power plant goes for a while with no accidents. Then, either the people working there fall into complacency and get lax on the safety regulations, and/or politicians or higher-ups looking to cut costs cut some of the redundancy, in either case thinking "it'll be fine, there's never been an accident". Then the inevitable happens, and suddenly there's a renewed interest in safety and redundancy. Rinse and repeat. The implication of this is that while it might be theoretically possible to operate a nuclear power plant safely for an indefinite period, the inevitability of relaxing safety regulations after a long period of no accidents virtually guarantees an accident after a certain amount of time. I'm not sure if there's any empirical (i.e. non-anecdotal/confirmation biased) evidence to support this theory, or what could be done about it if it were true. See also: The resurgence of vaccine-preventable diseases every few generations, because parents who never experienced the disease since their generation had the vaccine decide that their kids don't need the vaccine.

akkartik 10y ago
I've been thinking along similar lines for many years, though with the language of externalities. For example, here's a comment of mine from 2012: https://news.ycombinator.com/item?id=4361596 https://news.ycombinator.com/item?id=4361596 I haven't thought about safety like you have, but my bias is to wonder if part of the reason regulations get lax is that they first get complicated due to the influence of people motivated by their own selfish interest. Perhaps over time people add crap to regulations, which makes it hard to sift wheat from chaff, and this leads to backlash? With the final sin being to forget Chesterton's Fence? That would fit with my experience. One thought-experiment idea I had was a society governed not by rules but by unit tests for rules. That would allow any cop or citizen to globally change a rule if he found an exception that didn't violate existing tests. See the final section of http://www.ribbonfarm.com/2014/04/09/the-legibility-tradeoff http://www.ribbonfarm.com/2014/04/09/the-legibility-tradeoff. Of course then the problem is that people would start gaming the tests rather than the rules. That's as far as I got.. :) These days I try not to think about the hard instance of this problem, and instead focus on an easier problem: avoiding externalities in software projects using more thorough tests. That's going a bit better.

akkartik 10y ago

I've been thinking along similar lines for many years, though with the language of externalities. For example, here's a comment of mine from 2012: https://news.ycombinator.com/item?id=4361596 https://news.ycombinator.com/item?id=4361596 I haven't thought about safety like you have, but my bias is to wonder if part of the reason regulations get lax is that they first get complicated due to the influence of people motivated by their own selfish interest. Perhaps over time people add crap to regulations, which makes it hard to sift wheat from chaff, and this leads to backlash? With the final sin being to forget Chesterton's Fence? That would fit with my experience. One thought-experiment idea I had was a society governed not by rules but by unit tests for rules. That would allow any cop or citizen to globally change a rule if he found an exception that didn't violate existing tests. See the final section of http://www.ribbonfarm.com/2014/04/09/the-legibility-tradeoff http://www.ribbonfarm.com/2014/04/09/the-legibility-tradeoff. Of course then the problem is that people would start gaming the tests rather than the rules. That's as far as I got.. :) These days I try not to think about the hard instance of this problem, and instead focus on an easier problem: avoiding externalities in software projects using more thorough tests. That's going a bit better.