5 ms·

The problem is that everyone in the chain ends up buying into this. The police make a record of it 'just in case.' You end up on a no-fly list 'just in case.' E

by pyre 12y ago

The problem is that everyone in the chain ends up buying into this. The police make a record of it 'just in case.' You end up on a no-fly list 'just in case.' Everyone in the chain is covering their ass 'just in case' this person really is a terrorist. No one ends up willing to stick their neck out and say, "this person probably isn't a terrorist"... just in case.

samspot 12y ago
Fantastic point that cannot be understated. When thinking about this I assumed the authorities would run some quick sanity check and dismiss it. While that probably does happen, we often see cases where it has not worked that way.
Beltiras 12y ago
And it all becomes Kafkaesque and you can't get the bureaucracy to respond because '... it might reveal our methods and tactics in combating global terrorism.'
dragonwriter 12y ago
The extreme example of "everyone in the chain" buying into an extreme form of this thinking came in the form of the "1% doctrine" articulated by then-Vice President Dick Cheney: "If there's a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response. It's not about our analysis ... It's about our response."
- mikeash 12y ago
  Much of the post-9/11 response to terrorism fits it. "We captured a bunch of people. Some are harmless unfortunate bystanders. Some are dangerous people who are nevertheless not immune to handcuffs. Still, we'd better lock them up in harsh conditions without trial just in case." "Iraq probably has no nuclear weapons and no means to deliver one, but we'd better invade the country just in case. We don't want to be wrong and end up with a mushroom cloud over Manhattan."
  - dragonwriter 12y ago
    > Much of the post-9/11 response to terrorism fits it. Naturally; the "1% Doctrine" was formulated very shortly after 9/11 and, while the quote articulating it addresses Pakistani scientists aiding al-Qaeda, the significance is that it was a critical framing mindset from the highest levels for the Bush Administration policy on terrorism.
- Houshalter 12y ago
  http://lesswrong.com/lw/il/hindsight_bias/ http://lesswrong.com/lw/il/hindsight_bias/ >Shortly after September 11th 2001, I thought to myself, and now someone will turn up minor intelligence warnings of something-or-other, and then the hindsight will begin. Yes, I'm sure they had some minor warnings of an al Qaeda plot, but they probably also had minor warnings of mafia activity, nuclear material for sale, and an invasion from Mars. >Because we don't see the cost of a general policy, we learn overly specific lessons. After September 11th, the FAA prohibited box-cutters on airplanes—as if the problem had been the failure to take this particular "obvious" precaution. We don't learn the general lesson: the cost of effective caution is very high because you must attend to problems that are not as obvious now as past problems seem in hindsight.
rlpb 12y ago
This is exactly what happened in the Twitter Joke Trial: "The airport management considered the message to be "not credible" as a threat, but contacted the police anyway." They ended up finding something unrelated to actually protecting the public from a terrorist threat that they thought would stick after it got escalated to the point where they couldn't back down. This led to a conviction (eventually overturned on appeal). http://en.wikipedia.org/wiki/Twitter_Joke_Trial#Background http://en.wikipedia.org/wiki/Twitter_Joke_Trial#Background
deleted 12y ago
[deleted]
mikeash 12y ago
One big problem is that people take the idea of "better safe than sorry" as a way to avoid thought. Don't analyze the situation, just pick the option that appears safest after a glance! What it should mean is that you should take both probabilities and costs into account when deciding what to do, as it's often worth doing something easy to reduce the risk of something unlikely but catastrophic. But that doesn't mean you can just avoid thinking about it! "Often" doesn't mean "always". For example, it's worth checking the tire pressure in your car on a regular basis, because incorrect pressure could lead to a blowout at high speeds which can be most unpleasant, and checking it takes little time. But it's not worth checking the car for bombs every time you drive it, at least not for most of us, because the odds of being the victim of a car bomb for most of us are indistinguishable from zero. A little thought reveals that this case is clearly in the second category, and thus calling the police is just a waste of public resources. But if you're incorrectly using "better safe than sorry" to mean "pick the option that appears to be safe without any analysis", you're not going to apply a little thought.
- dragonwriter 12y ago
  > One big problem is that people take the idea of "better safe than sorry" as a way to avoid thought. Sure, and while things like responses to terrorism are rather dramatic examples of that, "zero tolerance policies" in general are also a place where this manifests.
  - trhway 12y ago
    >as a way to avoid thought. no. It is a way to avoid liability, an insurance of a kind. Property owner would get his property confiscated if he knew or had reason to know about an illegal activity happening on the property. Trying to apply "thought" would expose her/him even to larger liability in case of a mistake. Thus blanket reporting of anything is the best liability minimizing strategy.
    - dragonwriter 12y ago
      > >as a way to avoid thought. > no. It is a way to avoid liability Its both, and largely the liability involved is the responsibility to apply thought.

by pyre 12y ago

samspot 12y ago

Fantastic point that cannot be understated. When thinking about this I assumed the authorities would run some quick sanity check and dismiss it. While that probably does happen, we often see cases where it has not worked that way.

Beltiras 12y ago

And it all becomes Kafkaesque and you can't get the bureaucracy to respond because '... it might reveal our methods and tactics in combating global terrorism.'

dragonwriter 12y ago

The extreme example of "everyone in the chain" buying into an extreme form of this thinking came in the form of the "1% doctrine" articulated by then-Vice President Dick Cheney: "If there's a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response. It's not about our analysis ... It's about our response."

mikeash 12y ago
Much of the post-9/11 response to terrorism fits it. "We captured a bunch of people. Some are harmless unfortunate bystanders. Some are dangerous people who are nevertheless not immune to handcuffs. Still, we'd better lock them up in harsh conditions without trial just in case." "Iraq probably has no nuclear weapons and no means to deliver one, but we'd better invade the country just in case. We don't want to be wrong and end up with a mushroom cloud over Manhattan."
- dragonwriter 12y ago
  > Much of the post-9/11 response to terrorism fits it. Naturally; the "1% Doctrine" was formulated very shortly after 9/11 and, while the quote articulating it addresses Pakistani scientists aiding al-Qaeda, the significance is that it was a critical framing mindset from the highest levels for the Bush Administration policy on terrorism.
Houshalter 12y ago
http://lesswrong.com/lw/il/hindsight_bias/ http://lesswrong.com/lw/il/hindsight_bias/ >Shortly after September 11th 2001, I thought to myself, and now someone will turn up minor intelligence warnings of something-or-other, and then the hindsight will begin. Yes, I'm sure they had some minor warnings of an al Qaeda plot, but they probably also had minor warnings of mafia activity, nuclear material for sale, and an invasion from Mars. >Because we don't see the cost of a general policy, we learn overly specific lessons. After September 11th, the FAA prohibited box-cutters on airplanes—as if the problem had been the failure to take this particular "obvious" precaution. We don't learn the general lesson: the cost of effective caution is very high because you must attend to problems that are not as obvious now as past problems seem in hindsight.

rlpb 12y ago

This is exactly what happened in the Twitter Joke Trial: "The airport management considered the message to be "not credible" as a threat, but contacted the police anyway." They ended up finding something unrelated to actually protecting the public from a terrorist threat that they thought would stick after it got escalated to the point where they couldn't back down. This led to a conviction (eventually overturned on appeal). http://en.wikipedia.org/wiki/Twitter_Joke_Trial#Background http://en.wikipedia.org/wiki/Twitter_Joke_Trial#Background

deleted 12y ago

[deleted]

mikeash 12y ago

One big problem is that people take the idea of "better safe than sorry" as a way to avoid thought. Don't analyze the situation, just pick the option that appears safest after a glance! What it should mean is that you should take both probabilities and costs into account when deciding what to do, as it's often worth doing something easy to reduce the risk of something unlikely but catastrophic. But that doesn't mean you can just avoid thinking about it! "Often" doesn't mean "always". For example, it's worth checking the tire pressure in your car on a regular basis, because incorrect pressure could lead to a blowout at high speeds which can be most unpleasant, and checking it takes little time. But it's not worth checking the car for bombs every time you drive it, at least not for most of us, because the odds of being the victim of a car bomb for most of us are indistinguishable from zero. A little thought reveals that this case is clearly in the second category, and thus calling the police is just a waste of public resources. But if you're incorrectly using "better safe than sorry" to mean "pick the option that appears to be safe without any analysis", you're not going to apply a little thought.

dragonwriter 12y ago
> One big problem is that people take the idea of "better safe than sorry" as a way to avoid thought. Sure, and while things like responses to terrorism are rather dramatic examples of that, "zero tolerance policies" in general are also a place where this manifests.
- trhway 12y ago
  >as a way to avoid thought. no. It is a way to avoid liability, an insurance of a kind. Property owner would get his property confiscated if he knew or had reason to know about an illegal activity happening on the property. Trying to apply "thought" would expose her/him even to larger liability in case of a mistake. Thus blanket reporting of anything is the best liability minimizing strategy.
  - dragonwriter 12y ago
    > >as a way to avoid thought. > no. It is a way to avoid liability Its both, and largely the liability involved is the responsibility to apply thought.