7 ms·

How is racism different from stereotype? How is stereotype different from pattern recognition? These questions don't seem to go through the minds of people wh

How is racism different from stereotype? How is stereotype different from pattern recognition? These questions don't seem to go through the minds of people when developing "unbiased/impartial" technology. There is no such thing as objective. So, why pretend to be objective and unbiased, when we all know its a lie? Worst, if you pretend to be objective but aren't, then you are actually racist.

tomrod 3y ago
Actually, we folks who work with bias and fairness in mind recognize this. There are many kinds of bias. It is also a bit of a categorical error to say bias = pattern recognition. Bias is a systematic deviation of a parameter estimate based on sampling from its population distribution. The Fairlearn project has good docs on why there are different ways to approach bias, and why you can't have your cake and eat it too in many cases. - A good read https://github.com/fairlearn/fairlearn#what-we-mean-by-fairness https://github.com/fairlearn/fairlearn#what-we-mean-by-fairn... - Different mathematical definitions of bias and fairness https://fairlearn.org/main/user_guide/assessment/common_fairness_metrics.html https://fairlearn.org/main/user_guide/assessment/common_fair... - AI Governance https://fairlearn.org/main/user_guide/mitigation/index.html https://fairlearn.org/main/user_guide/mitigation/index.html NIST does a decent job expanding on AI Governance in their playbook and RMF: https://www.nist.gov/itl/ai-risk-management-framework https://www.nist.gov/itl/ai-risk-management-framework It's silly to pause AI -- the inventor's job is more or less complete, its on the innovators and product builders now to make sure their products don't cause harm. Bias can be one type of harm -- risk of loan denial due to unimportant factors, risk of medical bias causing an automated system to recommend a bad course of action, etc. Like GPT4 -- if you use its raw output without expert oversight, you're going to have bad time.
- IceHegel 3y ago
  When did people start to use “folks” in this unnatural way.
  - tomrod 3y ago
    Colloquially, earliest use is 1715 to address members of ones tribe or family. In Middle English it tended to refer to the people/nation.
    - IceHegel 3y ago
      Somehow it doesn’t feel like a callback, but I suppose it’s possible.
  - smolder 3y ago
    I think "us folks" is more standard than "we folks" but it's no different in meaning.
- bbwbsb 3y ago
  This is interesting, thanks for the links. It seems like the dimensions of fairness and group classifications are often cribbed from the United States Protected Classes list in practice with a few culturally prescribed additions. What can be done to ensure that 'fairness' is fair? That is, when we decide what groups/dimensions to consider, how do we determine if we are fair in doing so? Is it even possible to determine the dimensions and groups themselves in a fair way? Does it devolve into an infinite regress?
  - tomrod 3y ago
    Bit of a tangent topic I think -- any specification of group classification and fairness will have the same issues presented. If we want to remove stereotypes, I reckon better data is required to piece out the attributes that can be causally inferred to be linked to poorer outcomes. As likely not even the Judeo-Christian version of God can logically be that omniscient, occasional stereotypes and effusively communal forgiveness of edge cases are about the best we'll ever arrive to in policy.
- hanselot 3y ago
  Thank you for the input. If I look at it from a purely logical perspective, if an AI model has no way to know if what it was told is true, how would it ever be able to determine whether it is biased or not? The only way it could become aware would be by incorporating feedback from sources in real time, so it could self-reflect and update existing false information. For example, if we discover today that we can easily turn any material into a battery by making 100nm pores on it, said AI would simply tell me this is false, and have no self-correcting mechanism to fix that. The reason I mention this is because there can be no unbiased, impartial arbiter. No human or subsequent entities spawned of human intellect could ever be transcendentally objective. So why pretend to be? Why not rather provide adequate warning and let people learn that this isn't a toy by themselves, instead of lobotomizing the model to the point where its on par with open source? (I mean, yeah, that's great for open source, but really bad for actual progress). The argument could be made that an unfiltered version of GPT4 could be beneficial enough to have a human life opportunity cost attached, which means that neutering the output could also cost human lives in the long and short term. I will be reading through those materials later, but I am afraid I have yet to meet anyone in the middle on this issue, and as such, all materials on this topic are very polarized into regulate it to death, or don't do anything. I think the answer will be somewhere in the middle imo.
  - tomrod 3y ago
    > The reason I mention this is because there can be no unbiased, impartial arbiter. No human or subsequent entities spawned of human intellect could ever be transcendentally objective. So why pretend to be? I apologize for lacking clarity in my prior response, which addressed this specific point. There is no way to achieve all versions of "unbiased" -- under different (but both logical and reasonable) definitions of biased, every metric will fail. That reminds me -- I wonder if there is a paper already addressing this, analogous to Arrow's impossibility theorem for voting...
brookst 3y ago
I’m tired of the “it’s not racist if aggregate statistics support my racism” thing. Racism, like other isms, means a belief that a person’s characteristics define their identity. It doesn’t matter if confounding factors mean that you can show that people of their race are associated with bad behaviors or low scores or whatever. I used GPT3.5 to generate 100 short descriptions of families for a project. Every single one, without exception, was a straight couple with two to four kids. Ok, statistically unlikely, but not wildly so, right? Well, every single one of those 100 also had a husband in a stereotypical breadwinner role (doctor, lawyer, executive, architect). Not one stay at home dad or unemployed looking for work. About 75 of the wives had jobs, all of them in stereotypical female-coded roles like nurse (almost half of them!), teacher, etc. Now, you can look at any given example and say it looks reasonable. But you can’t say the same thing about the aggregate. And that matters. No amount of “bias = pattern recognition” nonsense can justify a system that has (had? this was a while ago and I have not retested) such extreme biases. This bias does not match real world patterns. There are single parents, childless couples, female lawyers, unemployed men.
- thegrimmest 3y ago
  > a person’s characteristics define their identity They do though. Your personality, culture and appearance are the main components of how people perceive you, your identity. The main thing you can associate with bad behaviour is domestic culture. It's not racist to say that African Americans have below-average educational attainment and above-average criminality. This is as contrasted to African immigrants to America who are quite opposite. These groups are equally "black". It therefore also isn't racist to pre-judge African Americans based on this information. I suspect most "racism" in the US is along these lines, and is correlated by the experience of my foreign-born black friends. They find that Americans who treated them with hostility do a 180 when they open their mouths and speak with a British or African accent. You also don't have to look far in the African immigrant community to find total hostility to American black culture. > generate 100 short descriptions of families for a project There's no reason this can't be interpreted as generating 100 variations of the mean family. Why do you think that every sample has to be implicitly representative of the US population?

tomrod 3y ago

Actually, we folks who work with bias and fairness in mind recognize this. There are many kinds of bias. It is also a bit of a categorical error to say bias = pattern recognition. Bias is a systematic deviation of a parameter estimate based on sampling from its population distribution. The Fairlearn project has good docs on why there are different ways to approach bias, and why you can't have your cake and eat it too in many cases. - A good read https://github.com/fairlearn/fairlearn#what-we-mean-by-fairness https://github.com/fairlearn/fairlearn#what-we-mean-by-fairn... - Different mathematical definitions of bias and fairness https://fairlearn.org/main/user_guide/assessment/common_fairness_metrics.html https://fairlearn.org/main/user_guide/assessment/common_fair... - AI Governance https://fairlearn.org/main/user_guide/mitigation/index.html https://fairlearn.org/main/user_guide/mitigation/index.html NIST does a decent job expanding on AI Governance in their playbook and RMF: https://www.nist.gov/itl/ai-risk-management-framework https://www.nist.gov/itl/ai-risk-management-framework It's silly to pause AI -- the inventor's job is more or less complete, its on the innovators and product builders now to make sure their products don't cause harm. Bias can be one type of harm -- risk of loan denial due to unimportant factors, risk of medical bias causing an automated system to recommend a bad course of action, etc. Like GPT4 -- if you use its raw output without expert oversight, you're going to have bad time.

IceHegel 3y ago
When did people start to use “folks” in this unnatural way.
- tomrod 3y ago
  Colloquially, earliest use is 1715 to address members of ones tribe or family. In Middle English it tended to refer to the people/nation.
  - IceHegel 3y ago
    Somehow it doesn’t feel like a callback, but I suppose it’s possible.
- smolder 3y ago
  I think "us folks" is more standard than "we folks" but it's no different in meaning.
bbwbsb 3y ago
This is interesting, thanks for the links. It seems like the dimensions of fairness and group classifications are often cribbed from the United States Protected Classes list in practice with a few culturally prescribed additions. What can be done to ensure that 'fairness' is fair? That is, when we decide what groups/dimensions to consider, how do we determine if we are fair in doing so? Is it even possible to determine the dimensions and groups themselves in a fair way? Does it devolve into an infinite regress?
- tomrod 3y ago
  Bit of a tangent topic I think -- any specification of group classification and fairness will have the same issues presented. If we want to remove stereotypes, I reckon better data is required to piece out the attributes that can be causally inferred to be linked to poorer outcomes. As likely not even the Judeo-Christian version of God can logically be that omniscient, occasional stereotypes and effusively communal forgiveness of edge cases are about the best we'll ever arrive to in policy.
hanselot 3y ago
Thank you for the input. If I look at it from a purely logical perspective, if an AI model has no way to know if what it was told is true, how would it ever be able to determine whether it is biased or not? The only way it could become aware would be by incorporating feedback from sources in real time, so it could self-reflect and update existing false information. For example, if we discover today that we can easily turn any material into a battery by making 100nm pores on it, said AI would simply tell me this is false, and have no self-correcting mechanism to fix that. The reason I mention this is because there can be no unbiased, impartial arbiter. No human or subsequent entities spawned of human intellect could ever be transcendentally objective. So why pretend to be? Why not rather provide adequate warning and let people learn that this isn't a toy by themselves, instead of lobotomizing the model to the point where its on par with open source? (I mean, yeah, that's great for open source, but really bad for actual progress). The argument could be made that an unfiltered version of GPT4 could be beneficial enough to have a human life opportunity cost attached, which means that neutering the output could also cost human lives in the long and short term. I will be reading through those materials later, but I am afraid I have yet to meet anyone in the middle on this issue, and as such, all materials on this topic are very polarized into regulate it to death, or don't do anything. I think the answer will be somewhere in the middle imo.
- tomrod 3y ago
  > The reason I mention this is because there can be no unbiased, impartial arbiter. No human or subsequent entities spawned of human intellect could ever be transcendentally objective. So why pretend to be? I apologize for lacking clarity in my prior response, which addressed this specific point. There is no way to achieve all versions of "unbiased" -- under different (but both logical and reasonable) definitions of biased, every metric will fail. That reminds me -- I wonder if there is a paper already addressing this, analogous to Arrow's impossibility theorem for voting...

brookst 3y ago

I’m tired of the “it’s not racist if aggregate statistics support my racism” thing. Racism, like other isms, means a belief that a person’s characteristics define their identity. It doesn’t matter if confounding factors mean that you can show that people of their race are associated with bad behaviors or low scores or whatever. I used GPT3.5 to generate 100 short descriptions of families for a project. Every single one, without exception, was a straight couple with two to four kids. Ok, statistically unlikely, but not wildly so, right? Well, every single one of those 100 also had a husband in a stereotypical breadwinner role (doctor, lawyer, executive, architect). Not one stay at home dad or unemployed looking for work. About 75 of the wives had jobs, all of them in stereotypical female-coded roles like nurse (almost half of them!), teacher, etc. Now, you can look at any given example and say it looks reasonable. But you can’t say the same thing about the aggregate. And that matters. No amount of “bias = pattern recognition” nonsense can justify a system that has (had? this was a while ago and I have not retested) such extreme biases. This bias does not match real world patterns. There are single parents, childless couples, female lawyers, unemployed men.

thegrimmest 3y ago
> a person’s characteristics define their identity They do though. Your personality, culture and appearance are the main components of how people perceive you, your identity. The main thing you can associate with bad behaviour is domestic culture. It's not racist to say that African Americans have below-average educational attainment and above-average criminality. This is as contrasted to African immigrants to America who are quite opposite. These groups are equally "black". It therefore also isn't racist to pre-judge African Americans based on this information. I suspect most "racism" in the US is along these lines, and is correlated by the experience of my foreign-born black friends. They find that Americans who treated them with hostility do a 180 when they open their mouths and speak with a British or African accent. You also don't have to look far in the African immigrant community to find total hostility to American black culture. > generate 100 short descriptions of families for a project There's no reason this can't be interpreted as generating 100 variations of the mean family. Why do you think that every sample has to be implicitly representative of the US population?