6 ms·

Information Hazards: A Typology of Potential Harms from Knowledge [pdf]

83 points by jnakayama 5y ago

techdragon 5y ago
Just going to say it. The table on page 6 desperately needed just one extra blank line to pad the row break onto the page break to prevent that hideous mid row page break with the row title split between pages…. All for the lack of a single extra new line. Other than that, kind of interesting to see the terminology construction for a type of “higher order analysis” I’ve been thinking about lately.
- deleted 5y ago
  [deleted]
jl6 5y ago
I’ll check the comments here later and hopefully someone will have told me whether this is safe to read.
- h2odragon 5y ago
  Well its OK for you to read, of course. Information hygiene is something that implicitly applies only to others, those not enlightened enough to make their own decisions or take the blame for them.
- fastball 5y ago
  Doesn't even mention Roko's Basilisk, so pretty safe.
  - hoseja 5y ago
    Now you've damned me to eternal torment.
    - contravariant 5y ago
      It's okay, you'll be saved as long as you believe the whole thing to be silly.
- faeyanpiraat 5y ago
  What would you consider unsafe to read?
  - ASalazarMX 5y ago
    I think they were joking about a fictional cognitive risk that could crash your brain. There are real cases of information that glitches your brain, like the McCollough effect. Can something exist that glitches the cerebral cortex instead of the visual cortex? https://en.wikipedia.org/wiki/McCollough_effect https://en.wikipedia.org/wiki/McCollough_effect
- rhn_mk1 5y ago
  Don't worry, the antimemetics division is keeping us safe https://scp-wiki.wikidot.com/antimemetics-division-hub https://scp-wiki.wikidot.com/antimemetics-division-hub
- ASalazarMX 5y ago
  It's safe to read, you have to read it immediately.
deleted 5y ago
[deleted]
jkhdigital 5y ago
Published in 2011, so perhaps that should be added to the title.
- woliveirajr 5y ago
  > But aside from such special cases, we shall not count robot hazards and artificial intelligence hazards as information system hazards Yes, it needs a [2011] tag on it. Because: >There is, however, another way for robot- and AI-related risks to enter our information hazard taxonomy. They can enter it in the same ways as any risk relating to potentially dangerous technological development is already happening. Given those autonomous decision without humans in the loop (e.g., flag some file as copyrighted), it isn't a basic-potential-risk, it's reallity.
petercooper 5y ago
I'm not sure if it would be classed as a "neuropsychological hazard" but one form of information hazard I encountered in recent years is true information that requires a finessed understanding of both risk and psychology to not act rashly upon it. For example, let's say you get a brain scan out of curiosity only. A small aneurysm is detected with a confidence of 95%, and said aneurysm, if real, has an estimated annual risk of rupture of 0.5%. The operation to clip said aneurysm, you are told, has a mortality rate of 10%. Do you have the op? In the books I've read covering such medical ethics topics, a disproportionate number of patients do/would have the op because the knowledge of the aneurysm will "play on their mind" even if the odds are hugely in favor of leaving it be. For this reason, amongst others, unnecessary/preventative testing is discouraged by many medical professionals. (A similar dilemma is faced by folks in affected families who have to choose whether or not to have genetics testing for fatal familial insomnia risk – would you want to know if you're likely to face this usually inherited condition?)
- jnakayama 5y ago
  Interesting! I think neuropsychological hazard according to this typology sounds fitting. Reminds me a little bit of the scenario of what would happen if you knew the exact date and time of your death. Maybe to add to what might be the rationale to have an op like this: I've recently been on a call where someone brought up a metric called quality-adjusted life years (something like: "Would I rather live another year relatively painless or five more years, but in constant excruciating pain?"). As far as I know, this is mostly used in insurance calculations (which struck me as a bit distasteful, but I didn't look into it deeply enough to make a fair judgement), but it's a good mental model for what people might use in their reasoning in such a situation. Maybe for the patients that decided to have the op, the knowledge of their condition was bad enough to substantially reduce their quality of life and hence lead them to accept far higher risks to alleviate this. For the second dilemma: Maybe simply knowing about the condition running in your family is an information hazard too. For someone very prone to worrying, knowing for sure that they have the condition might also come as a strange kind of relief.
  - neuroma 5y ago
bunbunbun 5y ago
For those interested, this is safe to read
frazbin 5y ago
I'm gonna read the whole paper I swear, but I do hope they get around to treating the converse: loss of trust due to hiding true information also causes harm, by reducing the chances future true information will be believed. Most fascinatingly, because it's possible that an excessively cautious attitude towards dangerous information can cause harm, this paper is a potential example of dangerous information. I hope the authors considered that before publication!
bobthechef 5y ago
motohagiography 5y ago
Searched for "noble lie" and found...nothing? > Information hazard: A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm. I'm getting the sense there is an underlying loop in the reasoning to support this concept of hazard, where the direction of truth and information seems confused. There's also the question of the durability of a given state of truth. What is the difference between information and fear? (e.g. data about potential information) When and how is information realized into truth, and in turn, consequences? Given we know truth is not a necessary or sufficient condition to have consequences, what are the qualities of information that does? If there were no truth, and in turn, only narratives and struggles for power, is an information hazard just something that hinders your will to power? Admittedly I find Bostrom's writing mostly impenetrable because he seems to publish things he's still sounding out and I don't quite detect the hand of an editor, but while this idea of a hazard seems interesting, I don't see that he's testing for or discovering its existance so much as coining and supporting a meme. Perhaps I've just missed it.
- playdead 5y ago
  I think you need to read more carefully. For example, he explicitly anticipates and puts aside some questions you ask: "In this paper, we will not be concerned with postmodernist critiques of the idea of objective truth nor with skeptical doubts about the possibility of knowledge. I shall assume some broad commonsensical understanding according to which there are truths and we humans sometimes manage to know some of these truths."
LargoLasskhyfv 5y ago
Ignorance is bliss for the sheeple, the bostromian creed wails. To feel empowered, wagging their sheepdog tails. To please their masters, by protecting the herd, from the ever lurking, black sheep nerd. Fuck OFF ye lousy boffins! edit: /me howls Total transparency, or BUST!

83 points by jnakayama 5y ago

Just going to say it. The table on page 6 desperately needed just one extra blank line to pad the row break onto the page break to prevent that hideous mid row page break with the row title split between pages…. All for the lack of a single extra new line. Other than that, kind of interesting to see the terminology construction for a type of “higher order analysis” I’ve been thinking about lately.

deleted 5y ago
[deleted]

jl6 5y ago

I’ll check the comments here later and hopefully someone will have told me whether this is safe to read.

h2odragon 5y ago
Well its OK for you to read, of course. Information hygiene is something that implicitly applies only to others, those not enlightened enough to make their own decisions or take the blame for them.
fastball 5y ago
Doesn't even mention Roko's Basilisk, so pretty safe.
- hoseja 5y ago
  Now you've damned me to eternal torment.
  - contravariant 5y ago
    It's okay, you'll be saved as long as you believe the whole thing to be silly.
faeyanpiraat 5y ago
What would you consider unsafe to read?
- ASalazarMX 5y ago
  I think they were joking about a fictional cognitive risk that could crash your brain. There are real cases of information that glitches your brain, like the McCollough effect. Can something exist that glitches the cerebral cortex instead of the visual cortex? https://en.wikipedia.org/wiki/McCollough_effect https://en.wikipedia.org/wiki/McCollough_effect
rhn_mk1 5y ago
Don't worry, the antimemetics division is keeping us safe https://scp-wiki.wikidot.com/antimemetics-division-hub https://scp-wiki.wikidot.com/antimemetics-division-hub
ASalazarMX 5y ago
It's safe to read, you have to read it immediately.

deleted 5y ago

[deleted]

jkhdigital 5y ago

Published in 2011, so perhaps that should be added to the title.

woliveirajr 5y ago
> But aside from such special cases, we shall not count robot hazards and artificial intelligence hazards as information system hazards Yes, it needs a [2011] tag on it. Because: >There is, however, another way for robot- and AI-related risks to enter our information hazard taxonomy. They can enter it in the same ways as any risk relating to potentially dangerous technological development is already happening. Given those autonomous decision without humans in the loop (e.g., flag some file as copyrighted), it isn't a basic-potential-risk, it's reallity.

petercooper 5y ago

I'm not sure if it would be classed as a "neuropsychological hazard" but one form of information hazard I encountered in recent years is true information that requires a finessed understanding of both risk and psychology to not act rashly upon it. For example, let's say you get a brain scan out of curiosity only. A small aneurysm is detected with a confidence of 95%, and said aneurysm, if real, has an estimated annual risk of rupture of 0.5%. The operation to clip said aneurysm, you are told, has a mortality rate of 10%. Do you have the op? In the books I've read covering such medical ethics topics, a disproportionate number of patients do/would have the op because the knowledge of the aneurysm will "play on their mind" even if the odds are hugely in favor of leaving it be. For this reason, amongst others, unnecessary/preventative testing is discouraged by many medical professionals. (A similar dilemma is faced by folks in affected families who have to choose whether or not to have genetics testing for fatal familial insomnia risk – would you want to know if you're likely to face this usually inherited condition?)

jnakayama 5y ago
Interesting! I think neuropsychological hazard according to this typology sounds fitting. Reminds me a little bit of the scenario of what would happen if you knew the exact date and time of your death. Maybe to add to what might be the rationale to have an op like this: I've recently been on a call where someone brought up a metric called quality-adjusted life years (something like: "Would I rather live another year relatively painless or five more years, but in constant excruciating pain?"). As far as I know, this is mostly used in insurance calculations (which struck me as a bit distasteful, but I didn't look into it deeply enough to make a fair judgement), but it's a good mental model for what people might use in their reasoning in such a situation. Maybe for the patients that decided to have the op, the knowledge of their condition was bad enough to substantially reduce their quality of life and hence lead them to accept far higher risks to alleviate this. For the second dilemma: Maybe simply knowing about the condition running in your family is an information hazard too. For someone very prone to worrying, knowing for sure that they have the condition might also come as a strange kind of relief.
- neuroma 5y ago

bunbunbun 5y ago

For those interested, this is safe to read

frazbin 5y ago

I'm gonna read the whole paper I swear, but I do hope they get around to treating the converse: loss of trust due to hiding true information also causes harm, by reducing the chances future true information will be believed. Most fascinatingly, because it's possible that an excessively cautious attitude towards dangerous information can cause harm, this paper is a potential example of dangerous information. I hope the authors considered that before publication!

bobthechef 5y ago

motohagiography 5y ago

Searched for "noble lie" and found...nothing? > Information hazard: A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm. I'm getting the sense there is an underlying loop in the reasoning to support this concept of hazard, where the direction of truth and information seems confused. There's also the question of the durability of a given state of truth. What is the difference between information and fear? (e.g. data about potential information) When and how is information realized into truth, and in turn, consequences? Given we know truth is not a necessary or sufficient condition to have consequences, what are the qualities of information that does? If there were no truth, and in turn, only narratives and struggles for power, is an information hazard just something that hinders your will to power? Admittedly I find Bostrom's writing mostly impenetrable because he seems to publish things he's still sounding out and I don't quite detect the hand of an editor, but while this idea of a hazard seems interesting, I don't see that he's testing for or discovering its existance so much as coining and supporting a meme. Perhaps I've just missed it.

playdead 5y ago
I think you need to read more carefully. For example, he explicitly anticipates and puts aside some questions you ask: "In this paper, we will not be concerned with postmodernist critiques of the idea of objective truth nor with skeptical doubts about the possibility of knowledge. I shall assume some broad commonsensical understanding according to which there are truths and we humans sometimes manage to know some of these truths."

LargoLasskhyfv 5y ago

Ignorance is bliss for the sheeple, the bostromian creed wails. To feel empowered, wagging their sheepdog tails. To please their masters, by protecting the herd, from the ever lurking, black sheep nerd. Fuck OFF ye lousy boffins! edit: /me howls Total transparency, or BUST!