3 ms·

not surprising I mean: 1. their data has a lot of racism, including a lot of subtle ones 2. some stereotypes do statistically exist in a relevant way. The rac

not surprising I mean: 1. their data has a lot of racism, including a lot of subtle ones 2. some stereotypes do statistically exist in a relevant way. The racist part is in that case often the interpretations/conclusions you take from it. (For example statistically the ratio of Asians which are good at STEM is higher but that isn't because they are magically good at science but because due to their history in the US including discrimination against them they developed a culture where they motivate and push their children to become successful in STEM topic. Similar examples can be made for many other topics e.g. poor minorities with racist laws and police and crime statistics). Another common problem is that even if the statistic isn't misleading and racist due to bad data sometimes going applying a "statistically more likely" to a individual case is still racist. Even if __hypothetically__ Asians had a genetic advantage which makes it more likely for them to have talent for STEM independent from how they where raised then expecting any specific Asian to be good at STEM still would be pretty racist and it's just a probability, so there are always (many) individuals for which it doesn't apply. 3. LLMs don't work by refining understanding of the world through self reflection, they learn a flawed textual representation of the world based on statistics. That makes it very hard to counter the first 2 points in a relevant reliable way. You only can counter specific well known examples by putting them as anti-examples into the training data and hope it generalizes the anti-examples good enough and similar.

_heimdall 3y ago
I don't think its really that simple. What you describe is that, effectively, racism is little more than inaccurate data or interpretation when the data is broken down by race. The historic problems with racism all boil down to intent as far as I can tell. The real problem has never been whether stereotypes are accurate and backed by data, but whether someone furthers a stereotype out of ill-intent against specific races. Data itself doesn't have intent, and judging an LLM response without understanding why it says what it says doesn't have enough context. We can't really distinguish whether LLMs have somehow independently developed racism (which would be a huge concern) or whether they are just interpreting training data and answering prompts in a way that the reader ultimately decides is racist for whatever reason. It almost feels like a quantum physics paradox honestly. An LLM response is just text until a human reads it and decides if, when isolated, the response itself is racist.
- dathinab 3y ago
  > What you describe is that, effectively, racism is little more than inaccurate data or interpretation when the data is broken down by race. that was not how I wanted it to be understood And yes the problem is that just data, missing a lot of implicit context in both it's training and the interactions, including intend, is not good enough to train a fully non problematic AI on. And the context is too subtle, multi layered and large to be able to fix it by "just adding" more data/information. As most training data simple can't have the full context as that would require known what people thought when writing it in the past etc. In addition the AI would need to know the context of the person currently interacting with it (and why they do so) and then depending on it adapt it's answer to make sure it's out-of-context answer isn't e.g. racist. (E.g. if someone ask about the likely hood of someone from neighborhood X being criminal it would need to spit out more then just some percentage but e.g. tell the user without being asked for it that while the likely hood might be higher it comes from discriminatory policing and discriminatory laws targeting poor people.) Because a statistical point without context can be very easily racist, sexist etc.
  - _heimdall 3y ago
    > Because a statistical point without context can be very easily racist, sexist etc. By the do you mean a person could use the stat in a racist way, or that the stat itself could be racist?
    - dathinab 3y ago
      A stat by itself without any context at all is just a number without meaning and hence can't be racist, but that is somewhat meaningless as you always have some context, even if it's the context of intentionally just presenting the state by itself without any other context related to it. And the moment you put it into natural language you likely will lose some context (if you don't write a large paper and only consider the paper as a whole and not single sentences by it in isolation) and there is some risk to (potentially unintentionally) imply some implicit context many readers will have. Depending on that the sentence it may always be racist, or may be racist when ignoring the surrounding sentences etc. To some degree you could say it most times depends on usage, but then with LLMs and other AI I wouldn't say it requires persons to be involved. Lastly, but that's a different discussion, a stat by itself could be racist if the statistics it is from where designed to promote racist belies or similar (could also happen accidental due to e.g. naivety or not understanding the topic).

_heimdall 3y ago

I don't think its really that simple. What you describe is that, effectively, racism is little more than inaccurate data or interpretation when the data is broken down by race. The historic problems with racism all boil down to intent as far as I can tell. The real problem has never been whether stereotypes are accurate and backed by data, but whether someone furthers a stereotype out of ill-intent against specific races. Data itself doesn't have intent, and judging an LLM response without understanding why it says what it says doesn't have enough context. We can't really distinguish whether LLMs have somehow independently developed racism (which would be a huge concern) or whether they are just interpreting training data and answering prompts in a way that the reader ultimately decides is racist for whatever reason. It almost feels like a quantum physics paradox honestly. An LLM response is just text until a human reads it and decides if, when isolated, the response itself is racist.

dathinab 3y ago
> What you describe is that, effectively, racism is little more than inaccurate data or interpretation when the data is broken down by race. that was not how I wanted it to be understood And yes the problem is that just data, missing a lot of implicit context in both it's training and the interactions, including intend, is not good enough to train a fully non problematic AI on. And the context is too subtle, multi layered and large to be able to fix it by "just adding" more data/information. As most training data simple can't have the full context as that would require known what people thought when writing it in the past etc. In addition the AI would need to know the context of the person currently interacting with it (and why they do so) and then depending on it adapt it's answer to make sure it's out-of-context answer isn't e.g. racist. (E.g. if someone ask about the likely hood of someone from neighborhood X being criminal it would need to spit out more then just some percentage but e.g. tell the user without being asked for it that while the likely hood might be higher it comes from discriminatory policing and discriminatory laws targeting poor people.) Because a statistical point without context can be very easily racist, sexist etc.
- _heimdall 3y ago
  > Because a statistical point without context can be very easily racist, sexist etc. By the do you mean a person could use the stat in a racist way, or that the stat itself could be racist?
  - dathinab 3y ago
    A stat by itself without any context at all is just a number without meaning and hence can't be racist, but that is somewhat meaningless as you always have some context, even if it's the context of intentionally just presenting the state by itself without any other context related to it. And the moment you put it into natural language you likely will lose some context (if you don't write a large paper and only consider the paper as a whole and not single sentences by it in isolation) and there is some risk to (potentially unintentionally) imply some implicit context many readers will have. Depending on that the sentence it may always be racist, or may be racist when ignoring the surrounding sentences etc. To some degree you could say it most times depends on usage, but then with LLMs and other AI I wouldn't say it requires persons to be involved. Lastly, but that's a different discussion, a stat by itself could be racist if the statistics it is from where designed to promote racist belies or similar (could also happen accidental due to e.g. naivety or not understanding the topic).