5 ms·

Eliezer agrees with you! Roko's Basilisk in particular is wrong. As a matter of policy though, you do not just publicly publish information hazards that might b

Eliezer agrees with you! Roko's Basilisk in particular is wrong. As a matter of policy though, you do not just publicly publish information hazards that might be real; that was why it was a big deal.

nsxwolf 12y ago
You don't what? What policy? Information hazards? What?! I feel like I just stumbled into a Scientology auditing session or something reading your comment.
- TeMPOraL 12y ago
  It's a concept coined by Nick Bostrom. Basically, some information can be harmful. As a simple, non-controvertial information hazard, imagine you learned an identity of a covert assasin of a foreign government, and you start telling about it to people around you. Don't be surprised if not only you drop dead, but the people you interact with as well. http://www.nickbostrom.com/information-hazards.pdf http://www.nickbostrom.com/information-hazards.pdf http://wiki.lesswrong.com/wiki/Information_hazard http://wiki.lesswrong.com/wiki/Information_hazard
- convexfunction 12y ago
  You should familiarize yourself with that feeling so you can recognize it later; it's the feeling of falling into an inferential gap. (The thing might be worthless, true, like Scientology. I usually find that resolving confusion is worth my time, though.)
- Strilanc 12y ago
  "Information hazard" is just a term for "things you or others learn and then are worse off because of". Examples of information hazards: - Instructions on how to make a gigaton bomb with processes and materials anyone can acquire and assemble privately. - A reachable-by-a-country mechanism for blowing up the sun. - An image designed to temporarily or permanently mess with your vision. Sharing [1] on reddit as a "cool optical illusion" would be mean. - Probably whatever a computer spits out if you give it a scan of your brain and set it on the task of finding a paragraph/image/movie/whatever that maximizes dopamine production. - Some people would consider a devastating argument against free will to be an information hazard, predicting that people would then do whatever they wanted because they "couldn't be blamed". - If you're a hardcore atheist: religions. If you're a hardcore theist who thinks the devil is out to trick you: convincing arguments against belief. 1: http://en.wikipedia.org/wiki/McCollough_effect http://en.wikipedia.org/wiki/McCollough_effect
zyxley 12y ago
> you do not just publicly publish information hazards that might be real But... for "might be real" to be valid at all, you have to buy into the Roko's basilisk concept in the first place. If you don't, there's nothing to get upset about.
- kbenson 12y ago
  That's like saying you shouldn't be mad at someone who attempted to murder you because they failed. There's a reason we have a crime called "attempted murder". You may find the wikipedia article on "Inchoate offense" interesting, specifically the section on Impossibility[1]. 1: http://en.wikipedia.org/wiki/Inchoate_offense#Impossibility http://en.wikipedia.org/wiki/Inchoate_offense#Impossibility
- convexfunction 12y ago
  I meant "might be real" according to generic-you in that statement. Someone being wrong doesn't make information hazards in general disappear.
- ciphergoth 12y ago
  If I learned that a co-worker believed that nuts were invariably fatal to me and gone on to hide some in my food, the nuts would do me no harm but I would still take the whole incident seriously.

Eliezer agrees with you! Roko's Basilisk in particular is wrong. As a matter of policy though, you do not just publicly publish information hazards that might be real; that was why it was a big deal.

nsxwolf 12y ago

You don't what? What policy? Information hazards? What?! I feel like I just stumbled into a Scientology auditing session or something reading your comment.

TeMPOraL 12y ago
It's a concept coined by Nick Bostrom. Basically, some information can be harmful. As a simple, non-controvertial information hazard, imagine you learned an identity of a covert assasin of a foreign government, and you start telling about it to people around you. Don't be surprised if not only you drop dead, but the people you interact with as well. http://www.nickbostrom.com/information-hazards.pdf http://www.nickbostrom.com/information-hazards.pdf http://wiki.lesswrong.com/wiki/Information_hazard http://wiki.lesswrong.com/wiki/Information_hazard
convexfunction 12y ago
You should familiarize yourself with that feeling so you can recognize it later; it's the feeling of falling into an inferential gap. (The thing might be worthless, true, like Scientology. I usually find that resolving confusion is worth my time, though.)
Strilanc 12y ago
"Information hazard" is just a term for "things you or others learn and then are worse off because of". Examples of information hazards: - Instructions on how to make a gigaton bomb with processes and materials anyone can acquire and assemble privately. - A reachable-by-a-country mechanism for blowing up the sun. - An image designed to temporarily or permanently mess with your vision. Sharing [1] on reddit as a "cool optical illusion" would be mean. - Probably whatever a computer spits out if you give it a scan of your brain and set it on the task of finding a paragraph/image/movie/whatever that maximizes dopamine production. - Some people would consider a devastating argument against free will to be an information hazard, predicting that people would then do whatever they wanted because they "couldn't be blamed". - If you're a hardcore atheist: religions. If you're a hardcore theist who thinks the devil is out to trick you: convincing arguments against belief. 1: http://en.wikipedia.org/wiki/McCollough_effect http://en.wikipedia.org/wiki/McCollough_effect

zyxley 12y ago

> you do not just publicly publish information hazards that might be real But... for "might be real" to be valid at all, you have to buy into the Roko's basilisk concept in the first place. If you don't, there's nothing to get upset about.

kbenson 12y ago
That's like saying you shouldn't be mad at someone who attempted to murder you because they failed. There's a reason we have a crime called "attempted murder". You may find the wikipedia article on "Inchoate offense" interesting, specifically the section on Impossibility[1]. 1: http://en.wikipedia.org/wiki/Inchoate_offense#Impossibility http://en.wikipedia.org/wiki/Inchoate_offense#Impossibility
convexfunction 12y ago
I meant "might be real" according to generic-you in that statement. Someone being wrong doesn't make information hazards in general disappear.
ciphergoth 12y ago
If I learned that a co-worker believed that nuts were invariably fatal to me and gone on to hide some in my food, the nuts would do me no harm but I would still take the whole incident seriously.