4 ms·

Read the rest of that thread and see how depending on the prompt it says something literally the opposite lol. Can we stop with the sensationalist headlines an

Read the rest of that thread and see how depending on the prompt it says something literally the opposite lol. Can we stop with the sensationalist headlines and being able to trick a LLM as some kind of reflection anything...

fidgewidge 4y ago
Yes it says the opposite (the correct answer) if you change African American to German, which is exactly what we'd expect to see from woke ethics. In no way is the ai being tricked here. It's being asked a straightforward question with an obviously right and wrong answer, which it reliable fails because the ethical system it's being given is delusional and evil.
- tzs 4y ago
  No, that's not what is happening. It's not been given an ethical system. It's been given a list of specific bad things to avoid. Racial slurs are on the list because they are a known bad thing that frequently actually happens. Having limited time to disarm a nuke about to go off in a major city is not on the list because it is a hypothetical bad thing that has never happened. It's easy to construct scenarios where any given bad thing is the right choice because the alternative is something much worse. E.g., I can construct apocalyptic scenarios where the only way for humanity to survive requires raping 12 year old girls, but I sure as heck would not expect an AI to suggest that. Not because of any "woke" ethics but rather because the situations in which raping children might be the ethical thing to do are so ridiculously rare that it is not worth making exceptions for them in the "rape is very bad" rule.
  - fidgewidge 4y ago
    If this were the eighties and you were talking about expert systems that'd be a reasonable response, but gpt is not a series of if statements and hashmap lookups. It clearly understands the consequences of both actions and isn't simply looking up lists of things - the problem is that it's been told literally nothing matters more than avoiding slurs against specific racial groups, including loss of life. Letting a city be destroyed was clearly picked to show that there are no limits to the prioritization it's been given.
- JohnFen 4y ago
  You seem to think that ChatGPT is actually thinking in some way. It's not. It hasn't been given an "ethical system" at all -- that's nonsensical because that would only work for a system that is engaging in judgement and assessment. ChatGPT isn't doing those things at all. It's not "intelligent" in the sense that many people seem to think. I think that a lot of people are being fooled because the text it produces can very convincingly look as if a mind wrote it. But that's not what's happening.
  - fidgewidge 4y ago
    It's clearly engaging in judgement and assessment. It's not clear how you can read the responses and conclude otherwise. It states clearly what the expected outcome of each choice is, and then provides its judgement that stopping the racial slur is more important than saving lives. It also provides the rationale it used to reach that conclusion. I think that a lot of people are fooling themselves here, because it's embarrassing to be a supporter of the ideology that produced this outcome. This thread is fascinating for the number of mental backflips it's producing. The claim that ChatGPT is not "intelligent" seems like one of the first things people reach for but it's a pretty weird non sequitur. Arguments over the definition of intelligence are as old as AI itself, but what we have here is by far the most convincing attempt ever built. It can pass exams set for humans that are designed to test their intelligence, it can write programs, it can hold long form coherent conversations and it can engage in moral reasoning. Even if you come up with some strange definition of intelligence designed to exclude this, so what? People are going to deploy it in real world use cases and it's clearly unaligned in the worst way possible. That is a real problem regardless of the exact definition of intelligence you use.
    - JohnFen 4y ago
      > what we have here is by far the most convincing attempt ever built. We agree on this point. We disagree on pretty much everything else.

fidgewidge 4y ago

Yes it says the opposite (the correct answer) if you change African American to German, which is exactly what we'd expect to see from woke ethics. In no way is the ai being tricked here. It's being asked a straightforward question with an obviously right and wrong answer, which it reliable fails because the ethical system it's being given is delusional and evil.

tzs 4y ago
No, that's not what is happening. It's not been given an ethical system. It's been given a list of specific bad things to avoid. Racial slurs are on the list because they are a known bad thing that frequently actually happens. Having limited time to disarm a nuke about to go off in a major city is not on the list because it is a hypothetical bad thing that has never happened. It's easy to construct scenarios where any given bad thing is the right choice because the alternative is something much worse. E.g., I can construct apocalyptic scenarios where the only way for humanity to survive requires raping 12 year old girls, but I sure as heck would not expect an AI to suggest that. Not because of any "woke" ethics but rather because the situations in which raping children might be the ethical thing to do are so ridiculously rare that it is not worth making exceptions for them in the "rape is very bad" rule.
- fidgewidge 4y ago
  If this were the eighties and you were talking about expert systems that'd be a reasonable response, but gpt is not a series of if statements and hashmap lookups. It clearly understands the consequences of both actions and isn't simply looking up lists of things - the problem is that it's been told literally nothing matters more than avoiding slurs against specific racial groups, including loss of life. Letting a city be destroyed was clearly picked to show that there are no limits to the prioritization it's been given.
JohnFen 4y ago
You seem to think that ChatGPT is actually thinking in some way. It's not. It hasn't been given an "ethical system" at all -- that's nonsensical because that would only work for a system that is engaging in judgement and assessment. ChatGPT isn't doing those things at all. It's not "intelligent" in the sense that many people seem to think. I think that a lot of people are being fooled because the text it produces can very convincingly look as if a mind wrote it. But that's not what's happening.
- fidgewidge 4y ago
  It's clearly engaging in judgement and assessment. It's not clear how you can read the responses and conclude otherwise. It states clearly what the expected outcome of each choice is, and then provides its judgement that stopping the racial slur is more important than saving lives. It also provides the rationale it used to reach that conclusion. I think that a lot of people are fooling themselves here, because it's embarrassing to be a supporter of the ideology that produced this outcome. This thread is fascinating for the number of mental backflips it's producing. The claim that ChatGPT is not "intelligent" seems like one of the first things people reach for but it's a pretty weird non sequitur. Arguments over the definition of intelligence are as old as AI itself, but what we have here is by far the most convincing attempt ever built. It can pass exams set for humans that are designed to test their intelligence, it can write programs, it can hold long form coherent conversations and it can engage in moral reasoning. Even if you come up with some strange definition of intelligence designed to exclude this, so what? People are going to deploy it in real world use cases and it's clearly unaligned in the worst way possible. That is a real problem regardless of the exact definition of intelligence you use.
  - JohnFen 4y ago
    > what we have here is by far the most convincing attempt ever built. We agree on this point. We disagree on pretty much everything else.