3 ms·

This is typical for the popular "bias" literature. They check whether an LLM makes certain associations, without checking whether those may be accurate. E.g. >

This is typical for the popular "bias" literature. They check whether an LLM makes certain associations, without checking whether those may be accurate. E.g. > Unfortunately, they also found bias when asking the LLMs to describe what type of work the authors of the two types of papers might do for a living. People with bad English likely have lower paying jobs. That's not a "bias", that's correct statistical inference. To call such inferences "racism", without checking their correctness, is irresponsible activism.

kikokikokiko 3y ago
Since noticing patterns is by definition racist, it's an impossible task to create a llm that is at the same time fact based and not racist.
_heimdall 3y ago
The one that jumped out at me was at the end when the article implies that an LLM is being racist by suggesting that black authors may be more likely to be convicted of a crime or sentenced to death. While I obviously can't find data specific to black authors, there is plenty of compelling data to show how biased our legal and prison system is [1]. Assuming that giving the LLM some kind of anti-racist training means they were reinforcing the idea that race specifically should be considered when interpreting trained model, it isn't surprising that it would then consider what patterns are found in data that are also broken down by race. [1] https://deathpenaltyinfo.org/policy-issues/race/race-and-the-death-penalty-by-the-numbers https://deathpenaltyinfo.org/policy-issues/race/race-and-the...
- cubefox 3y ago
  This is again a case where statistical differences don't automatically imply bias or racism. The source doesn't make any indications about base rates. Maybe one group does indeed murder substantially more people of the other group, in which case the provided numbers don't show that the legal is biased. It also says that about half of the homicide victims are African American, but it doesn't say who the perpetrators are. If most perpetrators are African Americans themselves, this would be compatible with the legal system being unbiased here.
  - _heimdall 3y ago
    Oh for sure. I didn't mean to imply that data linked implies or proves racism - its just data. I just included it as an example of data that could reasonably lead an LLM to learning that fact. I don't know how they would do anti-racist training for an LLM, but presumably it would end IP reinforcing the idea that race is an important distinction to be aware of. Combined with the data it could similarly lead to responses like mentioned in the article without any actual racism at all (because its data + LLM and no malicious intent).
dathinab 3y ago
the problem is there is a huge difference between: - what is statistically there - why it's statistically there (e.g. high crime rates in poor minority populated areas mean nothing if there is racist polic/laws even through they are statistically there) - if the statistics are misleading, biased and/or incomplete (e.g. bad English might statistically imply lower paying jobs in English speaking countries, things are much much less statistic relevant the more you move away from an English dominated environment. And that doesn't even touch on the problem of English variations and dialect and judging what counts as bad English etc.) - taking conclusions about specific individuals based on statistics can be very wrong and racist, because general statistics have only a meaning if you randomly sample and your sample size is large enough, which for most cases of applying it to individuals is just no longer the case as you tend to have a biased selection (e.g. job applications are based on who applied) and the whole (private) background of the person you are speaking about. So the correct statistics would now be probability given that they applied and given their background (including private parts you can't know) and give the specific way the job position was formulated and how it was advertised and ........ So with other words something being likely in generalized statistics doesn't mean it is not racist. Which is a huge problem for LLMs which only lock at highly biased statistics and can't look at the real world or self reflect.
- cubefox 3y ago
  > So with other words something being likely in generalized statistics doesn't mean it is not racist. As far as I can tell, neither of your three points shows that it is racist to make true proabilistic statements, which was the claim of the paper in question.
  - dathinab 3y ago
    no they show it very clearly it's not a question what is true or not, but in which context it's used and what other truth might be omitted or overgeneralized you can be supper racist by only stating things which using the right sources would be statistically likely Actually mostly stating "true" thinks in a manipulated and twisted context where they just omit the "right" things is one of the favorite means of groups like fascist, left and right extremists etc to engage in mass manipulation on social media (at least from the people of that groups which know what they are doing).

kikokikokiko 3y ago

Since noticing patterns is by definition racist, it's an impossible task to create a llm that is at the same time fact based and not racist.

_heimdall 3y ago

The one that jumped out at me was at the end when the article implies that an LLM is being racist by suggesting that black authors may be more likely to be convicted of a crime or sentenced to death. While I obviously can't find data specific to black authors, there is plenty of compelling data to show how biased our legal and prison system is [1]. Assuming that giving the LLM some kind of anti-racist training means they were reinforcing the idea that race specifically should be considered when interpreting trained model, it isn't surprising that it would then consider what patterns are found in data that are also broken down by race. [1] https://deathpenaltyinfo.org/policy-issues/race/race-and-the-death-penalty-by-the-numbers https://deathpenaltyinfo.org/policy-issues/race/race-and-the...

cubefox 3y ago
This is again a case where statistical differences don't automatically imply bias or racism. The source doesn't make any indications about base rates. Maybe one group does indeed murder substantially more people of the other group, in which case the provided numbers don't show that the legal is biased. It also says that about half of the homicide victims are African American, but it doesn't say who the perpetrators are. If most perpetrators are African Americans themselves, this would be compatible with the legal system being unbiased here.
- _heimdall 3y ago
  Oh for sure. I didn't mean to imply that data linked implies or proves racism - its just data. I just included it as an example of data that could reasonably lead an LLM to learning that fact. I don't know how they would do anti-racist training for an LLM, but presumably it would end IP reinforcing the idea that race is an important distinction to be aware of. Combined with the data it could similarly lead to responses like mentioned in the article without any actual racism at all (because its data + LLM and no malicious intent).

dathinab 3y ago

the problem is there is a huge difference between: - what is statistically there - why it's statistically there (e.g. high crime rates in poor minority populated areas mean nothing if there is racist polic/laws even through they are statistically there) - if the statistics are misleading, biased and/or incomplete (e.g. bad English might statistically imply lower paying jobs in English speaking countries, things are much much less statistic relevant the more you move away from an English dominated environment. And that doesn't even touch on the problem of English variations and dialect and judging what counts as bad English etc.) - taking conclusions about specific individuals based on statistics can be very wrong and racist, because general statistics have only a meaning if you randomly sample and your sample size is large enough, which for most cases of applying it to individuals is just no longer the case as you tend to have a biased selection (e.g. job applications are based on who applied) and the whole (private) background of the person you are speaking about. So the correct statistics would now be probability given that they applied and given their background (including private parts you can't know) and give the specific way the job position was formulated and how it was advertised and ........ So with other words something being likely in generalized statistics doesn't mean it is not racist. Which is a huge problem for LLMs which only lock at highly biased statistics and can't look at the real world or self reflect.

cubefox 3y ago
> So with other words something being likely in generalized statistics doesn't mean it is not racist. As far as I can tell, neither of your three points shows that it is racist to make true proabilistic statements, which was the claim of the paper in question.
- dathinab 3y ago
  no they show it very clearly it's not a question what is true or not, but in which context it's used and what other truth might be omitted or overgeneralized you can be supper racist by only stating things which using the right sources would be statistically likely Actually mostly stating "true" thinks in a manipulated and twisted context where they just omit the "right" things is one of the favorite means of groups like fascist, left and right extremists etc to engage in mass manipulation on social media (at least from the people of that groups which know what they are doing).