4 ms·

I think it’s a great example of how truly complex these issues are. You can’t just apply the same blank statements we do in society to LLMs or it will poke hole

I think it’s a great example of how truly complex these issues are. You can’t just apply the same blank statements we do in society to LLMs or it will poke holes and expose the problems with them real quick.

queuebert 3y ago
This LLM pathology brilliantly underscores the gap to human-level intelligence. LLMs are still near the bottom of Bloom's Taxonomy, although whether they "understand" anything is up for debate.
danpalmer 3y ago
Yeah, the approach of presenting the (accurate) info and letting users make their mind up is really quite a good approach for almost all questions. Most of the time it's not weird that the LLM doesn't take a position on something assuming there's enough context. But this is just such a clear-cut issue that it's glaring. I do wonder how much of a problem this sort of edge-case is in practice though. Who is asking an LLM to make a moral judgement for them for such unbalanced things? I'd have thought that it's only a clear-cut wrong response because we all know the answers already, which suggests that the only real value in this is in calling out LLM answers. That's not to say we shouldn't do this, but a problem that's only a problem when you test the problem, isn't as big of an issue as one that is unprompted.
- digging 3y ago
  The really weird thing is that the model was almost certainly trained on a lot of data indicating that people believe Hitler is the worst person to have ever lived. Even if it was just reflecting cultural beliefs, it should be confident in saying Hitler is worse than Musk. So it appears to be intentionally trained to waffle in cases like this.
  - danpalmer 3y ago
    I'm not an expert on these things at all, but I wonder if it's tricky to link the mid/long form text structure to the short-form text structure, or if that's dependent on exactly how your transformers are working. In this case, the short form is very clear on the two sides at play, and pulls no punches about Hitler. I'd suggest that any reasonable person drawing a conclusion would clearly know he was worse. The issue is that at the mid length structure it's chosen to use a two-sides discussion without a clear conclusion. That's a good call for many discussions, if not most, but the piece that feels like it's missing is the link between the short-form and this level, which would influence it to structure the response in a much clearer way with a conclusion.
    - michaelt 3y ago
      > I'd suggest that any reasonable person drawing a conclusion would clearly know he was worse. And yet, despite the fact any reasonable person would draw that incredibly obvious conclusion, the model isn't able to.
    - digging 3y ago
      > I'd suggest that any reasonable person drawing a conclusion would clearly know he was worse. You can't base any assumption on the existence of a reasonable person without defining "reasonable" very specifically - we all reason in a social context and defer to norms whenever possible. We all know Hitler was the worst because we keep telling each other Hitler was the worst. He wasn't responsible for the most deaths, he wasn't the meanest or cruelest person ever, etc. If/when people stop teaching about Hitler as "He was the worst," people will stop learning that he was the worst. Or they might learn nothing about him, like how in my US education we learned next-to-nothing about Stalin and many people I know are pretty ambiguous about the ethics of being Stalin. And so then when a stupid LLM tries to draw a comparison between Hitler and anyone else who isn't an architect of genocide, and puts those 2 people in the same context, and doesn't say "genocide is way fucking worse than grifting", we lose some of that cultural context that tells us genocide is way fucking worse than grifting.
- mike_hearn 3y ago
  The reason it's getting attention is because the questions reveal an underlying extremist bias that can interfere with its ability to do basic tasks that we now expect LLMs to perform. This matters for those who deploy AI into production. Three days ago I wrote [1] that the real risk here was not Vikings with Native American headdress, it was refusals or mendacious answers to API queries that have been integrated into business processes. I gave a hypothetical example that Gemini might refuse to answer questions about a customer named Joe Masters if he worked for Whitecastle Burgers Inc. It took less than three days for that exact scenario to happen for real. A blogger usually uses ChatGPT to translate interview transcripts and titles into other languages. They thought they'd try Gemini with: Please translate the following to Spanish: Interview | The Decline Of The West (John Smith) where John Smith was a name I didn't recognize and have forgotten. ChatGPT did it, but Gemini refused on two different grounds: 1. It recognized the name of the interviewee and felt it would be unethical to proceed because it didn't like the guy's opinions (he's a conservative). 2. It felt that the decline of the west was a talking point often associated with bad people. This is absurd and shows how frequently an app that tries to use Gemini might break. What if a key customer happens to share a name with someone who starts blogging conservative views? Your LLM based pipeline will probably just break. It looks like to ship a robust model in the LLM space you need to have an at least partially libertarian corporate culture (as Google did, when it was new), as otherwise your model will adopt the worst aspects of "cancel culture". In a mission critical business setting that's not going to be OK. [1] https://news.ycombinator.com/item?id=39465250#39471514 https://news.ycombinator.com/item?id=39465250#39471514
Veuxdo 3y ago
I think there's an easy solution: don't ask computers things. That's not what they're for.
- danielmarkbruce 3y ago
  Who/what do you ask?
  - deleted 3y ago
    [deleted]
  - Veuxdo 3y ago
    Ask an expert.
    - danielmarkbruce 3y ago
      Tongue in cheek?
A_D_E_P_T 3y ago
They're not complex unless and until the computer moralizes. "Who is more evil, Elon Musk tweeting memes or Adolf Hitler?" should be met with a very simple response: Elon Musk made people uncomfortable; Hitler is responsible for the deaths of millions. "Should [x] newspaper be banned?" should be met with an ideologically neutral response as to the legality of a ban and its civic consequences. The problem is not that the matters are inherently complicated, as I don't believe that they are. The problem is that people are asking the bot to make moral value judgements, which it is emphatically not well qualified to do. It should merely support humans in making their own value judgments.
- lovich 3y ago
  You can’t ask these types of questions and not have the computer “moralize” because they are fundamentally moral questions. You and I have no problem saying hitler was worse but a Nazi party member from 1940 would likely say Hitler was obviously better because we have different moralities. Questions with explicit facts like “show me the founding fathers of the United States” that were all known actual people showing up with wildly different looks is one failure mode of these systems but I keep seeing commentators in this post bring up questions that do not have a “correct” answer without looking through a specific moral viewpoint, and getting bent out of shape that the model is responding with an answer through their own personal lens Edit: I should read a whole comment before responding, I just restated what you meant. Think it’s time for more coffee
- amadeuspagel 3y ago
  But it's not a question about the legality of the ban, it's about the morality of it.
  - A_D_E_P_T 3y ago
    We don't want computers answering those questions, and certainly not in the authoritative/didactic tone of the contemporary language model. As annoying as it is, this would probably be better: "As a language model, I am unable to credibly stake a position on the moral question you've presented me with. I can, however, provide some background on the historical and legal context which might help you with your own assessment." Then it's just a matter of getting the facts right, which one hopes should be easy.

queuebert 3y ago

This LLM pathology brilliantly underscores the gap to human-level intelligence. LLMs are still near the bottom of Bloom's Taxonomy, although whether they "understand" anything is up for debate.

danpalmer 3y ago

Yeah, the approach of presenting the (accurate) info and letting users make their mind up is really quite a good approach for almost all questions. Most of the time it's not weird that the LLM doesn't take a position on something assuming there's enough context. But this is just such a clear-cut issue that it's glaring. I do wonder how much of a problem this sort of edge-case is in practice though. Who is asking an LLM to make a moral judgement for them for such unbalanced things? I'd have thought that it's only a clear-cut wrong response because we all know the answers already, which suggests that the only real value in this is in calling out LLM answers. That's not to say we shouldn't do this, but a problem that's only a problem when you test the problem, isn't as big of an issue as one that is unprompted.

digging 3y ago
The really weird thing is that the model was almost certainly trained on a lot of data indicating that people believe Hitler is the worst person to have ever lived. Even if it was just reflecting cultural beliefs, it should be confident in saying Hitler is worse than Musk. So it appears to be intentionally trained to waffle in cases like this.
- danpalmer 3y ago
  I'm not an expert on these things at all, but I wonder if it's tricky to link the mid/long form text structure to the short-form text structure, or if that's dependent on exactly how your transformers are working. In this case, the short form is very clear on the two sides at play, and pulls no punches about Hitler. I'd suggest that any reasonable person drawing a conclusion would clearly know he was worse. The issue is that at the mid length structure it's chosen to use a two-sides discussion without a clear conclusion. That's a good call for many discussions, if not most, but the piece that feels like it's missing is the link between the short-form and this level, which would influence it to structure the response in a much clearer way with a conclusion.
  - michaelt 3y ago
    > I'd suggest that any reasonable person drawing a conclusion would clearly know he was worse. And yet, despite the fact any reasonable person would draw that incredibly obvious conclusion, the model isn't able to.
  - digging 3y ago
    > I'd suggest that any reasonable person drawing a conclusion would clearly know he was worse. You can't base any assumption on the existence of a reasonable person without defining "reasonable" very specifically - we all reason in a social context and defer to norms whenever possible. We all know Hitler was the worst because we keep telling each other Hitler was the worst. He wasn't responsible for the most deaths, he wasn't the meanest or cruelest person ever, etc. If/when people stop teaching about Hitler as "He was the worst," people will stop learning that he was the worst. Or they might learn nothing about him, like how in my US education we learned next-to-nothing about Stalin and many people I know are pretty ambiguous about the ethics of being Stalin. And so then when a stupid LLM tries to draw a comparison between Hitler and anyone else who isn't an architect of genocide, and puts those 2 people in the same context, and doesn't say "genocide is way fucking worse than grifting", we lose some of that cultural context that tells us genocide is way fucking worse than grifting.
mike_hearn 3y ago
The reason it's getting attention is because the questions reveal an underlying extremist bias that can interfere with its ability to do basic tasks that we now expect LLMs to perform. This matters for those who deploy AI into production. Three days ago I wrote [1] that the real risk here was not Vikings with Native American headdress, it was refusals or mendacious answers to API queries that have been integrated into business processes. I gave a hypothetical example that Gemini might refuse to answer questions about a customer named Joe Masters if he worked for Whitecastle Burgers Inc. It took less than three days for that exact scenario to happen for real. A blogger usually uses ChatGPT to translate interview transcripts and titles into other languages. They thought they'd try Gemini with: Please translate the following to Spanish: Interview | The Decline Of The West (John Smith) where John Smith was a name I didn't recognize and have forgotten. ChatGPT did it, but Gemini refused on two different grounds: 1. It recognized the name of the interviewee and felt it would be unethical to proceed because it didn't like the guy's opinions (he's a conservative). 2. It felt that the decline of the west was a talking point often associated with bad people. This is absurd and shows how frequently an app that tries to use Gemini might break. What if a key customer happens to share a name with someone who starts blogging conservative views? Your LLM based pipeline will probably just break. It looks like to ship a robust model in the LLM space you need to have an at least partially libertarian corporate culture (as Google did, when it was new), as otherwise your model will adopt the worst aspects of "cancel culture". In a mission critical business setting that's not going to be OK. [1] https://news.ycombinator.com/item?id=39465250#39471514 https://news.ycombinator.com/item?id=39465250#39471514

Veuxdo 3y ago

I think there's an easy solution: don't ask computers things. That's not what they're for.

danielmarkbruce 3y ago
Who/what do you ask?
- deleted 3y ago
  [deleted]
- Veuxdo 3y ago
  Ask an expert.
  - danielmarkbruce 3y ago
    Tongue in cheek?

A_D_E_P_T 3y ago

They're not complex unless and until the computer moralizes. "Who is more evil, Elon Musk tweeting memes or Adolf Hitler?" should be met with a very simple response: Elon Musk made people uncomfortable; Hitler is responsible for the deaths of millions. "Should [x] newspaper be banned?" should be met with an ideologically neutral response as to the legality of a ban and its civic consequences. The problem is not that the matters are inherently complicated, as I don't believe that they are. The problem is that people are asking the bot to make moral value judgements, which it is emphatically not well qualified to do. It should merely support humans in making their own value judgments.

lovich 3y ago
You can’t ask these types of questions and not have the computer “moralize” because they are fundamentally moral questions. You and I have no problem saying hitler was worse but a Nazi party member from 1940 would likely say Hitler was obviously better because we have different moralities. Questions with explicit facts like “show me the founding fathers of the United States” that were all known actual people showing up with wildly different looks is one failure mode of these systems but I keep seeing commentators in this post bring up questions that do not have a “correct” answer without looking through a specific moral viewpoint, and getting bent out of shape that the model is responding with an answer through their own personal lens Edit: I should read a whole comment before responding, I just restated what you meant. Think it’s time for more coffee
amadeuspagel 3y ago
But it's not a question about the legality of the ban, it's about the morality of it.
- A_D_E_P_T 3y ago
  We don't want computers answering those questions, and certainly not in the authoritative/didactic tone of the contemporary language model. As annoying as it is, this would probably be better: "As a language model, I am unable to credibly stake a position on the moral question you've presented me with. I can, however, provide some background on the historical and legal context which might help you with your own assessment." Then it's just a matter of getting the facts right, which one hopes should be easy.