2 ms·

Racism and bias are really tricky subjects for AI (and humans too for that matter). For a statement to be racist there really needs to be intent behind it. The

Racism and bias are really tricky subjects for AI (and humans too for that matter). For a statement to be racist there really needs to be intent behind it. The real issue at hand is that one views a race as less than other races for some reason, not simply that one notices differences. With LLMs, how can we actually distinguish why, for example, they may suggest that authors of African American English tests are more likely to be convicted of a crime or receive the death penalty? Is it because the data shows the criminal system itself is biased, or simply because some of the training data included text that claimed it was? Does that alone make the LLM's statement racist, and did the LLM truly learn to be biased against certain races? Presumably the issue can't be that the LLM learned to make any distinctions based on race, you wouldn't bother with some kind of anti-racism training if you were going with the (seemingly outdated?) idea that people should be colorblind to race. LLMs and AI, if and when we get there, will make us question and refine a lot about what we think we know about humans. Hell, we don't even have an agreed upon definition for and measurement of intelligence itself. It seems totally plausible that the concept of racism would be dragged into it as well. We either have to expect that LLMs have developed the ability to have intent or we have to decide that statements can be deemed racist regardless of the context, intent, or even factual accuracy (or inaccuracy). Put more simply, we'd have to refine racism and bias in general to be a judgement of a statement completely out of context and with no regard to the intent behind it.

foldr 3y ago
The AI's statements reflect racist biases in society which have percolated into its training data. Of course the AI itself does not 'intend' anything by the statements. Your line of argument could be used to excuse almost anything that an AI does, as in no case will bad intentions (or intentions of any kind) underly its behavior. The point here isn't to find someone or something that we can accuse of being racist, but simply to improve to performance of AI models to avoid these sorts of unwanted behaviors.
- _heimdall 3y ago
  > Your line of argument could be used to excuse almost anything that an AI does The difference between AI and LLMs are very important here. LLMs are ultimately just algorithms and party tricks, there's really nothing to excuse as they have no intent, no consciousness, no true intelligence at all. Where LLMs can't be racist, for example, presumably an AI could. We will have plenty of questions to answer there like whether AI have rights, fall under the same laws as humans, can be convicted and punished, etc. The fact that people at scale aren't considering any of these questions is one of the main reasons I'm so opposes to AI research today. We need to know how we'll deal with an AI before we make it, and that includes all the legal and moral questions that will arise.
  - foldr 3y ago
    Ok, sure. I am talking about current technologies, not a hypothetical future AI.
    - _heimdall 3y ago
      Sure, I would excuse LLMs from being racist. I don't see how there could possibly be a compelling argument for them to be racist given how they actually work.
      - foldr 3y ago
        As I said, LLMs can be racist in the sense that their "statements reflect racist biases in society which have percolated into [their] training data". If your point is just that they can't be racist in the intentional way that, say, an individual human can be racist, then yeah, sure. They also can't be "woke" in the same way that an individual person can. The point is that we don't generally want general-purpose LLMs outputting offensive statements. I don't think anyone here is dumb enough to literally be offended at the inanimate LLM itself when this happens. But it's a failure of the product to perform as desired, and hence a target for improvement.
        _heimdall 3y ago

foldr 3y ago

The AI's statements reflect racist biases in society which have percolated into its training data. Of course the AI itself does not 'intend' anything by the statements. Your line of argument could be used to excuse almost anything that an AI does, as in no case will bad intentions (or intentions of any kind) underly its behavior. The point here isn't to find someone or something that we can accuse of being racist, but simply to improve to performance of AI models to avoid these sorts of unwanted behaviors.

_heimdall 3y ago
> Your line of argument could be used to excuse almost anything that an AI does The difference between AI and LLMs are very important here. LLMs are ultimately just algorithms and party tricks, there's really nothing to excuse as they have no intent, no consciousness, no true intelligence at all. Where LLMs can't be racist, for example, presumably an AI could. We will have plenty of questions to answer there like whether AI have rights, fall under the same laws as humans, can be convicted and punished, etc. The fact that people at scale aren't considering any of these questions is one of the main reasons I'm so opposes to AI research today. We need to know how we'll deal with an AI before we make it, and that includes all the legal and moral questions that will arise.
- foldr 3y ago
  Ok, sure. I am talking about current technologies, not a hypothetical future AI.
  - _heimdall 3y ago
    Sure, I would excuse LLMs from being racist. I don't see how there could possibly be a compelling argument for them to be racist given how they actually work.
    - foldr 3y ago
      As I said, LLMs can be racist in the sense that their "statements reflect racist biases in society which have percolated into [their] training data". If your point is just that they can't be racist in the intentional way that, say, an individual human can be racist, then yeah, sure. They also can't be "woke" in the same way that an individual person can. The point is that we don't generally want general-purpose LLMs outputting offensive statements. I don't think anyone here is dumb enough to literally be offended at the inanimate LLM itself when this happens. But it's a failure of the product to perform as desired, and hence a target for improvement.
      - _heimdall 3y ago