4 ms·

Right, the model didn't try to kill anyone. It simply suggested that the person follow a series of steps, the culmination of which could have been serious harm

Right, the model didn't try to kill anyone. It simply suggested that the person follow a series of steps, the culmination of which could have been serious harm or potentially death. If that difference is meaningful to you, then good for you

mindslight 2y ago
The model did not "suggest" anything. The model generated text. Google, or more specifically some Google employees (likely following incentives that Google management set up), presented the output of the model as if it might be authoritative answers to users' questions. Some users seemingly fall for the framing/hype and take the generated text as if it is authoritative information to be relied upon. This distinction is critical as we face an onslaught of "AI did <blank>" narratives, when the actual responsibility for such situations lies with specific humans relying on "AI" in such ways.
- alexey-salmin 2y ago
  I think this is typical for software engineers (including myself) to view the world like this: this can't be murder, this is just words, you fell for it yourself. After all words alone are gibberish, the meaning arises in the head of the listener depending on how he interprets them. However in most countries the justice system works differently. If your neighbor gives you this advice and it kills you, what will matter the most is not the authoritativeness of the advice (he's not a food expert, he was just talking) but the intent. If it also turns out that he had an affair with your wife and mentioned he wants to get rid of you, then he'll very likely go to jail. The "he fell for it and took my advice as if it was authoritative information to be relied upon" defense won't work. Here (hopefully) the intent is missing. However the whole reasoning that advising partly bears no responsibility for texts it "generates" doesn't work very well.
  - mindslight 2y ago
    You're not responding to the argument I'm making, but seemingly to other arguments that have been made over things like 4chan trolling that got people to cook their phones in microwaves. I'm not saying that users are the only responsible parties here. I'm saying that the humans, including both the users and Google ('s managers and employees), are the only parties that can possibly be responsible. The LLM is not responsible. The LLM is incapable of having intent, as far as we currently know. I've never used Gemini so I don't know how the user interface comes across, but it might be completely appropriate for Google to be facing liability lawsuits over things like this. What doesn't make sense is narratives that push any of this shared blame onto LLMs or "AI" itself, when the real world effects hinge entirely upon how humans choose to attach these algorithms to the real world.
    - alexey-salmin 2y ago
      Thanks for clarification, I agree this is a reasonable view of the situation. Though I also don't really have problems with headlines like "AI tried to kill a man". True LLM can't be held responsible, but so are e.g. animals. If an unleashed dog kills a man, the owner will be responsible but the headlines will be still about the dog. Then it can be argued that LLMs don't have intent while the dog had, and many more analogies, arguments and nuances will follow. My point is, this headline is still at an acceptable level of accuracy, unlike many other headlines that distort the reality completely.
      - mindslight 2y ago
        The key difference is that everyone mostly understands a dog's capabilities. A reader knows there must be a larger context. Even in your simple example you felt the need to elaborate that it was an "unleashed" dog. If I merely say "a dog killed a man", the reader is left grasping for that larger context and defaulting to thinking a human must ultimately be responsible (most dogs are human owned rather than in wild packs). If I say "a bear killed a man", a similar grasping occurs but defaults to assuming the deceased likely got too close to a wild bear. Society has no such general understanding for LLMs. If we did, we wouldn't have situations like this in the first place, because people wouldn't be expecting them to generate instructions that can be followed verbatim. Instead we get the marketing buzz of anthropomorphized "AI" as if they're super-human intelligent entities capable of conversing as humans do, but also better. And when they inevitably fail to live up to that, the narrative shifts to the need for "alignment" and "safety", which seems to mostly mean adding some words to the short priming prompt to make it avoid outputting some specific type of thing that was found to be problematic (with the definition of problematic generally being the creation a public relations shitstorm while also not creating profit). But us humans, our only benchmark for comparison, develop our sense of wisdom much slower through repeated learning and development stages. You can't simply read a child an essay of any length about how to drive safely and then hand them the keys to the family car.
lynx23 2y ago
https://darwinawards.com/ https://darwinawards.com/ -- And yes, there is a quite meaningful difference. I know that some people are safety freaks, but I for one, will not grief for a single person that managed to kill themselves by following LLM instructions. There is a point of low IQ where we as a society can't really help anymore.
- arthens 2y ago
  Moral considerations aside, I'm really skeptical of the assumption that following LLM instructions = low IQ. As always, HN is a very biased audience. Most of us have probably read about how LLMs are the best bullshitters in the world, and we've seen a lot examples like this that prove how little LLMs can be trusted. But in the general population? I'd bet there's plenty of intelligent people who haven't had the same exposure to LLM critiques and might trust their output out of ignorance rather than lack of intelligence.
  - hnfong 2y ago
    I agree, and I think there might be an argument to be made that people with higher IQ are more prone to believing in (gramatically) well-written texts from authoritative-looking sources. After all, people with higher IQ are used to reading books and believing them, whereas those with supposedly lower IQ tend to dismiss books and experts, instead believing in their own real-world experiences. I'll provide an example from a comment I wrote a while ago: https://news.ycombinator.com/item?id=35540356 https://news.ycombinator.com/item?id=35540356
- fullstick 2y ago
  My only issue with concepts such as the Darwin Awards is that they do not reflect upon whether or not the deceased has reproduced. To attribute darwinism to lethal mistakes is meaningless unless we know whether or not the deceased has a child. Some person with children who dies on an idiotic way has done more for evolution than the still alive and childless me.
  - yencabulator 2y ago

mindslight 2y ago

The model did not "suggest" anything. The model generated text. Google, or more specifically some Google employees (likely following incentives that Google management set up), presented the output of the model as if it might be authoritative answers to users' questions. Some users seemingly fall for the framing/hype and take the generated text as if it is authoritative information to be relied upon. This distinction is critical as we face an onslaught of "AI did <blank>" narratives, when the actual responsibility for such situations lies with specific humans relying on "AI" in such ways.

alexey-salmin 2y ago
I think this is typical for software engineers (including myself) to view the world like this: this can't be murder, this is just words, you fell for it yourself. After all words alone are gibberish, the meaning arises in the head of the listener depending on how he interprets them. However in most countries the justice system works differently. If your neighbor gives you this advice and it kills you, what will matter the most is not the authoritativeness of the advice (he's not a food expert, he was just talking) but the intent. If it also turns out that he had an affair with your wife and mentioned he wants to get rid of you, then he'll very likely go to jail. The "he fell for it and took my advice as if it was authoritative information to be relied upon" defense won't work. Here (hopefully) the intent is missing. However the whole reasoning that advising partly bears no responsibility for texts it "generates" doesn't work very well.
- mindslight 2y ago
  You're not responding to the argument I'm making, but seemingly to other arguments that have been made over things like 4chan trolling that got people to cook their phones in microwaves. I'm not saying that users are the only responsible parties here. I'm saying that the humans, including both the users and Google ('s managers and employees), are the only parties that can possibly be responsible. The LLM is not responsible. The LLM is incapable of having intent, as far as we currently know. I've never used Gemini so I don't know how the user interface comes across, but it might be completely appropriate for Google to be facing liability lawsuits over things like this. What doesn't make sense is narratives that push any of this shared blame onto LLMs or "AI" itself, when the real world effects hinge entirely upon how humans choose to attach these algorithms to the real world.
  - alexey-salmin 2y ago
    Thanks for clarification, I agree this is a reasonable view of the situation. Though I also don't really have problems with headlines like "AI tried to kill a man". True LLM can't be held responsible, but so are e.g. animals. If an unleashed dog kills a man, the owner will be responsible but the headlines will be still about the dog. Then it can be argued that LLMs don't have intent while the dog had, and many more analogies, arguments and nuances will follow. My point is, this headline is still at an acceptable level of accuracy, unlike many other headlines that distort the reality completely.
    - mindslight 2y ago
      The key difference is that everyone mostly understands a dog's capabilities. A reader knows there must be a larger context. Even in your simple example you felt the need to elaborate that it was an "unleashed" dog. If I merely say "a dog killed a man", the reader is left grasping for that larger context and defaulting to thinking a human must ultimately be responsible (most dogs are human owned rather than in wild packs). If I say "a bear killed a man", a similar grasping occurs but defaults to assuming the deceased likely got too close to a wild bear. Society has no such general understanding for LLMs. If we did, we wouldn't have situations like this in the first place, because people wouldn't be expecting them to generate instructions that can be followed verbatim. Instead we get the marketing buzz of anthropomorphized "AI" as if they're super-human intelligent entities capable of conversing as humans do, but also better. And when they inevitably fail to live up to that, the narrative shifts to the need for "alignment" and "safety", which seems to mostly mean adding some words to the short priming prompt to make it avoid outputting some specific type of thing that was found to be problematic (with the definition of problematic generally being the creation a public relations shitstorm while also not creating profit). But us humans, our only benchmark for comparison, develop our sense of wisdom much slower through repeated learning and development stages. You can't simply read a child an essay of any length about how to drive safely and then hand them the keys to the family car.

lynx23 2y ago

https://darwinawards.com/ https://darwinawards.com/ -- And yes, there is a quite meaningful difference. I know that some people are safety freaks, but I for one, will not grief for a single person that managed to kill themselves by following LLM instructions. There is a point of low IQ where we as a society can't really help anymore.

arthens 2y ago
Moral considerations aside, I'm really skeptical of the assumption that following LLM instructions = low IQ. As always, HN is a very biased audience. Most of us have probably read about how LLMs are the best bullshitters in the world, and we've seen a lot examples like this that prove how little LLMs can be trusted. But in the general population? I'd bet there's plenty of intelligent people who haven't had the same exposure to LLM critiques and might trust their output out of ignorance rather than lack of intelligence.
- hnfong 2y ago
  I agree, and I think there might be an argument to be made that people with higher IQ are more prone to believing in (gramatically) well-written texts from authoritative-looking sources. After all, people with higher IQ are used to reading books and believing them, whereas those with supposedly lower IQ tend to dismiss books and experts, instead believing in their own real-world experiences. I'll provide an example from a comment I wrote a while ago: https://news.ycombinator.com/item?id=35540356 https://news.ycombinator.com/item?id=35540356
fullstick 2y ago
My only issue with concepts such as the Darwin Awards is that they do not reflect upon whether or not the deceased has reproduced. To attribute darwinism to lethal mistakes is meaningless unless we know whether or not the deceased has a child. Some person with children who dies on an idiotic way has done more for evolution than the still alive and childless me.
- yencabulator 2y ago