9 ms·

This actually brings to mind a conversation I've been having with my buddies recently. Has there ever been a case of harm from AI misalignment? Not some kind o

by ahepp 3y ago

This actually brings to mind a conversation I've been having with my buddies recently. Has there ever been a case of harm from AI misalignment? Not some kind of speculative art exhibit, but real harm?

thevinter 3y ago
Yes, many cases I.e. https://www.technologyreview.com/2020/07/17/1005396/predictive-policing-algorithms-racist-dismantled-machine-learning-bias-criminal-justice/ https://www.technologyreview.com/2020/07/17/1005396/predicti...
- ahepp 3y ago
  I am familiar with this situation and agree it is a problem. But this is a deterministic, well known, algorithm rather than AI, isn't it? I would also say that this falls under bias rather than misalignment. I think bias is a much, much more realistic problem area than misalignment.
  - ajuc 3y ago
    This is exactly what I'm talking about in the other response to your previous comment. The joke goes "AI ceases to be AI when we start to understand how it works". But it doesn't matter if you call the software that kills us all "conscious AI that was misaligned" or if it was "deterministic code that had a bug" - the underlying problem that we're all dead remains the same. We're just arguing about labels instead of issues. The AI that kills us all (if this happens) will very likely be a simple, deterministic algorithm that ran on A LOT of data.
ajuc 3y ago
The problem with that is - until we have "true AGI" about which everybody agree it's "true AGI" - you can always dismiss deaths caused by software as "just a bug, not an AI safety problem". For example when Tesla autopilot kills someone (which has happened). https://impakter.com/tesla-autopilot-crashes-with-at-least-a-dozen-dead-whos-fault-man-or-machine/ https://impakter.com/tesla-autopilot-crashes-with-at-least-a...
- ahepp 3y ago
  I guess maybe one could make that argument, but the majority of AI alignment concerns seem more along the lines of skynet level scenarios. The paperclip optimizer could be called a "bug" I guess. But the comparison to Tesla autopilot is interesting. First, is anyone calling that an AI? Second, when an objective baseline comparison to human drivers is possible, shouldn't that determine whether the AI is a net benefit or loss? And then (assuming a benefit) we can say with some degree of confidence that the AI is not malicious?
  - ajuc 3y ago
    I think of it more like "Corporation X making whole species extinct because it has better uses for rainforests" not like "Skynet hates people". But the end result is the same. Humanity doesn't hate polar bears. We just love burning oil more than we love them. Polar bears die as an unintended side effect. We might even be sad about that. We might even keep a few alive in ZOOs. But we won't change our whole economy to save a few cute bears. Of course these kinds of threats will start to appear only when it's smart enough. The main problem with AGI is that we probably won't know until it's too late, because of how fast this technology develops once it can improve itself. Can I ask you another question - what do you think will happen? 1. it won't get smarter than us 2. it will care about us 3. we will somehow keep it in check despite the fact it's smarter than us Cause it seems to me that most people still intuitively think (1) because it's "too sci-fi". And even if they became persuaded that (1) is no longer certain - they didn't updated the rest of their beliefs with that new information, they are still believing AI is safe like they did when (1) was assumed true because they haven't updated the cache, or maybe they don't even realize there's dependency somewhere in their train of thought that needs to be updated. This is how living in a world undergoing singularity will be like, BTW - you can't think one thought to the end without realizing the assumptions might have changed since the last time you thought about it. So you go down and realize the assumptions there are also changing. And so on.
    - deleted 3y ago
      [deleted]
  - 3y ago
rollo 3y ago
Corporations can be regarded as AI agents with misaligned goals. In a bit over a century this has resulted in damage to the climate and Earth's ecosystem that is threatening the survival of our civilization. And that's just slow, badly optimized paperclippers so far.
- chrisco255 3y ago
  It's really not though is it, that's just your programming thinking for you. Human deaths from weather and climate related events are near all time lows, while overall population is indeed at all time highs. We have better crop yields (3) and productivity thanks to both technology and indeed, CO2 fertilization of the atmosphere (4), which is driven by photosynthesis, a process which is optimized at thousands of ppm for the vast majority of plant species, many multiples from where we now stand. Of course, the most abundant periods for life on earth have been periods of abundant carbon, such as the Carboniferous era, and the Cambrian era (5). Clearly the ecosystem is far from collapse as a result of mild warming and improved plant productivity. In many regards, it's thriving, and certainly more so than 100-120 years ago, which predated any pretense of conservation at all. And we know that it's indeed glaciation (global cooling) that so often leads to catastrophe in ecosystems (1)(2). But that we can program human beings with subtle prods of horrific and exaggerated headlines, along with cult-like expectations of consensus, into believing the opposite to be true from reality, is certainly an indication of paperclip optimization gone awry. (1) https://www.researchgate.net/publication/259495017_Ecological_ranking_of_Phanerozoic_biodiversity_crises_The_Serpukhovianend-Ordovician_early_Carboniferous_crisis_had_a_greater_ecological_impact_than_the_end-Ordovician https://www.researchgate.net/publication/259495017_Ecologica... (2) https://www.researchgate.net/publication/360894775_Global_cooling_initiated_the_Middle-Late_Mississippian_biodiversity_crisis https://www.researchgate.net/publication/360894775_Global_co... (3) https://www.agriculture.com/news/crops/usda-raises-the-us-corn-soybean-yield-expectations https://www.agriculture.com/news/crops/usda-raises-the-us-co... (4) https://www.nasa.gov/feature/goddard/2016/carbon-dioxide-fertilization-greening-earth/ https://www.nasa.gov/feature/goddard/2016/carbon-dioxide-fer... (5) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5942912/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5942912/
  - 3y ago
nopinsight 3y ago
“'He Would Still Be Here': Man Dies by Suicide After Talking with AI Chatbot, Widow Says” https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-after-talking-with-ai-chatbot-widow-says https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-a...
brchr 3y ago
Author of "The Alignment Problem" here, to say: Of course this question depends on your semantics of "harm," "AI," and "alignment," but by most definitions (and certainly by mine) the answer is overwhelmingly yes, many. These harms can be diffuse at massive scale, and acute at small scale. One example of each: (1) https://www.science.org/doi/abs/10.1126/science.aax2342 https://www.science.org/doi/abs/10.1126/science.aax2342 One of USA’s largest health insurers builds ML system for patient triage. It optimizes for a proxy metric of health need (namely, cost) rather than health need itself; consequently it deprioritizes and systematically excludes millions of people from access to health care. (2) https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg Autonomous Uber car builds their braking system on top of a vision model that optimizes for object classification accuracy using categories of {"pedestrian", "cyclist", "vehicle", "debris"}; consequently it fails to determine how to classify a woman walking a bicycle across the street, as a result killing her. In both cases, optimizing for a naively sensible proxy metric of the thing that was truly desired turned out to be catastrophic.
hollerith 3y ago
You have put your finger on the crucial question: whether AI research has killed everyone in the past. How overcareful would a person have to be to worry about its happening in the future if it's never happened in the past? </sarcasm>

by ahepp 3y ago

thevinter 3y ago

Yes, many cases I.e. https://www.technologyreview.com/2020/07/17/1005396/predictive-policing-algorithms-racist-dismantled-machine-learning-bias-criminal-justice/ https://www.technologyreview.com/2020/07/17/1005396/predicti...

ahepp 3y ago
I am familiar with this situation and agree it is a problem. But this is a deterministic, well known, algorithm rather than AI, isn't it? I would also say that this falls under bias rather than misalignment. I think bias is a much, much more realistic problem area than misalignment.
- ajuc 3y ago
  This is exactly what I'm talking about in the other response to your previous comment. The joke goes "AI ceases to be AI when we start to understand how it works". But it doesn't matter if you call the software that kills us all "conscious AI that was misaligned" or if it was "deterministic code that had a bug" - the underlying problem that we're all dead remains the same. We're just arguing about labels instead of issues. The AI that kills us all (if this happens) will very likely be a simple, deterministic algorithm that ran on A LOT of data.

ajuc 3y ago

The problem with that is - until we have "true AGI" about which everybody agree it's "true AGI" - you can always dismiss deaths caused by software as "just a bug, not an AI safety problem". For example when Tesla autopilot kills someone (which has happened). https://impakter.com/tesla-autopilot-crashes-with-at-least-a-dozen-dead-whos-fault-man-or-machine/ https://impakter.com/tesla-autopilot-crashes-with-at-least-a...

ahepp 3y ago
I guess maybe one could make that argument, but the majority of AI alignment concerns seem more along the lines of skynet level scenarios. The paperclip optimizer could be called a "bug" I guess. But the comparison to Tesla autopilot is interesting. First, is anyone calling that an AI? Second, when an objective baseline comparison to human drivers is possible, shouldn't that determine whether the AI is a net benefit or loss? And then (assuming a benefit) we can say with some degree of confidence that the AI is not malicious?
- ajuc 3y ago
  I think of it more like "Corporation X making whole species extinct because it has better uses for rainforests" not like "Skynet hates people". But the end result is the same. Humanity doesn't hate polar bears. We just love burning oil more than we love them. Polar bears die as an unintended side effect. We might even be sad about that. We might even keep a few alive in ZOOs. But we won't change our whole economy to save a few cute bears. Of course these kinds of threats will start to appear only when it's smart enough. The main problem with AGI is that we probably won't know until it's too late, because of how fast this technology develops once it can improve itself. Can I ask you another question - what do you think will happen? 1. it won't get smarter than us 2. it will care about us 3. we will somehow keep it in check despite the fact it's smarter than us Cause it seems to me that most people still intuitively think (1) because it's "too sci-fi". And even if they became persuaded that (1) is no longer certain - they didn't updated the rest of their beliefs with that new information, they are still believing AI is safe like they did when (1) was assumed true because they haven't updated the cache, or maybe they don't even realize there's dependency somewhere in their train of thought that needs to be updated. This is how living in a world undergoing singularity will be like, BTW - you can't think one thought to the end without realizing the assumptions might have changed since the last time you thought about it. So you go down and realize the assumptions there are also changing. And so on.
  - deleted 3y ago
    [deleted]
- 3y ago

rollo 3y ago

Corporations can be regarded as AI agents with misaligned goals. In a bit over a century this has resulted in damage to the climate and Earth's ecosystem that is threatening the survival of our civilization. And that's just slow, badly optimized paperclippers so far.

chrisco255 3y ago
It's really not though is it, that's just your programming thinking for you. Human deaths from weather and climate related events are near all time lows, while overall population is indeed at all time highs. We have better crop yields (3) and productivity thanks to both technology and indeed, CO2 fertilization of the atmosphere (4), which is driven by photosynthesis, a process which is optimized at thousands of ppm for the vast majority of plant species, many multiples from where we now stand. Of course, the most abundant periods for life on earth have been periods of abundant carbon, such as the Carboniferous era, and the Cambrian era (5). Clearly the ecosystem is far from collapse as a result of mild warming and improved plant productivity. In many regards, it's thriving, and certainly more so than 100-120 years ago, which predated any pretense of conservation at all. And we know that it's indeed glaciation (global cooling) that so often leads to catastrophe in ecosystems (1)(2). But that we can program human beings with subtle prods of horrific and exaggerated headlines, along with cult-like expectations of consensus, into believing the opposite to be true from reality, is certainly an indication of paperclip optimization gone awry. (1) https://www.researchgate.net/publication/259495017_Ecological_ranking_of_Phanerozoic_biodiversity_crises_The_Serpukhovianend-Ordovician_early_Carboniferous_crisis_had_a_greater_ecological_impact_than_the_end-Ordovician https://www.researchgate.net/publication/259495017_Ecologica... (2) https://www.researchgate.net/publication/360894775_Global_cooling_initiated_the_Middle-Late_Mississippian_biodiversity_crisis https://www.researchgate.net/publication/360894775_Global_co... (3) https://www.agriculture.com/news/crops/usda-raises-the-us-corn-soybean-yield-expectations https://www.agriculture.com/news/crops/usda-raises-the-us-co... (4) https://www.nasa.gov/feature/goddard/2016/carbon-dioxide-fertilization-greening-earth/ https://www.nasa.gov/feature/goddard/2016/carbon-dioxide-fer... (5) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5942912/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5942912/
- 3y ago

nopinsight 3y ago

“'He Would Still Be Here': Man Dies by Suicide After Talking with AI Chatbot, Widow Says” https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-after-talking-with-ai-chatbot-widow-says https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-a...

brchr 3y ago

Author of "The Alignment Problem" here, to say: Of course this question depends on your semantics of "harm," "AI," and "alignment," but by most definitions (and certainly by mine) the answer is overwhelmingly yes, many. These harms can be diffuse at massive scale, and acute at small scale. One example of each: (1) https://www.science.org/doi/abs/10.1126/science.aax2342 https://www.science.org/doi/abs/10.1126/science.aax2342 One of USA’s largest health insurers builds ML system for patient triage. It optimizes for a proxy metric of health need (namely, cost) rather than health need itself; consequently it deprioritizes and systematically excludes millions of people from access to health care. (2) https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg Autonomous Uber car builds their braking system on top of a vision model that optimizes for object classification accuracy using categories of {"pedestrian", "cyclist", "vehicle", "debris"}; consequently it fails to determine how to classify a woman walking a bicycle across the street, as a result killing her. In both cases, optimizing for a naively sensible proxy metric of the thing that was truly desired turned out to be catastrophic.

hollerith 3y ago

You have put your finger on the crucial question: whether AI research has killed everyone in the past. How overcareful would a person have to be to worry about its happening in the future if it's never happened in the past? </sarcasm>