4 ms·

The set of possible models may be regarded as infinite. However, selection of model may fail to account for biases and prejudice that may not even be present in

by loopz 5y ago

The set of possible models may be regarded as infinite. However, selection of model may fail to account for biases and prejudice that may not even be present in the data at all. Indirectly, the bias might be from the researchers themselves, ignorance or some silly thing like chance. When talking about hypothetical models, flaws probably lingers in any part of the chain. If not accounted for, you'd indeed expect biases, a need to clear the most obvious ones and adhering to laws and rules. First thing is to eradicate the poorly-defined word "racism", and find a more fitting term regarding the flaw in question: unfairness, discrimination, prejudice, bias, etc., and then make it concrete. Ie. instead of "structural racism", we could instead use the term "structural discrimination", to be more clear about what we're talking about. It is also more neutral to view these flaws as bugs. That only becomes more important as algorithms gain more power over people's lives. The sinister part of such algorithmic rules is the tolerance of a silent majority, benefitting unfairly from the outcomes.

darawk 5y ago
So I think you and I agree on all the things you just said. My point is really just that, linguistically, I don't think it makes sense to describe the models as being "biased" or "discriminatory" here. Statistical learning models are designed intentionally to act like mirrors. They reflect the data they're trained on. And I don't think it's descriptively useful to describe a mirror as biased because you don't like your reflection. Even if, in some sense, you could design a curved mirror that generates the reflection you wanted. The mirror is just a mirror. Now, that being said, I think it is fair to talk about structural equitably in the use of models that produce outcomes we believe are discriminatory. If ML engineers at some company produce a model, and fail to check it for these issues, or do check it but fail to correct them, we can certainly describe that behavior in negative terms, and shame them appropriately. At the end of the day, if we didn't live in a racist/sexist society, these ML models wouldn't produce discriminatory outcomes. And it is in that sense that the bias is "in the data". That being said, we may still choose to correct that bias at the model level, just like people fix cinematic issues in post-processing all the time.
- loopz 5y ago
  IT is only about 3 things with data: 1) Communication 2) Transformation 3) Application 1 being the ray of light/information. 2 being your "mirror". 3 being the usage. But these processes are general enough to fit any process in the universe (aka "simulate the universe"). Potential of IT may be regarded that powerful. Saying models are only about being a mirror may be too easily misinterpreted. This since application require accountability for 1, 2 and 3, not just one of them. Complex and powerful solutions aren't easily divided into clean responsibility areas. If one chooses models that favours some groups, such usage may introduce bias and discrimination in an otherwise non-discriminatory and non-racist society. What matters is the final outcomes, even when well-intentioned. As usage of complex algorithms may introduce chaotic side-effects, solutions to such issues may become dauntingly complex and hard to grapple with. Complex and powerful solutions become embedded in the world and culture of humans. Even the knowledge of usage of black box models may change people's perceptions for the worse, and introduce unwanted side-effects. Poor or lack of explanations for decision-making by the black box may compound issues even further. Misuse may introduce events that alters perception forever, etc. You may retroactively attempt to correct wrongs, but it's a hard sell you've made no further mistakes however unintentional. I have doubts people make such mistakes intentionally, though that's not impossible either.

by loopz 5y ago

darawk 5y ago

So I think you and I agree on all the things you just said. My point is really just that, linguistically, I don't think it makes sense to describe the models as being "biased" or "discriminatory" here. Statistical learning models are designed intentionally to act like mirrors. They reflect the data they're trained on. And I don't think it's descriptively useful to describe a mirror as biased because you don't like your reflection. Even if, in some sense, you could design a curved mirror that generates the reflection you wanted. The mirror is just a mirror. Now, that being said, I think it is fair to talk about structural equitably in the use of models that produce outcomes we believe are discriminatory. If ML engineers at some company produce a model, and fail to check it for these issues, or do check it but fail to correct them, we can certainly describe that behavior in negative terms, and shame them appropriately. At the end of the day, if we didn't live in a racist/sexist society, these ML models wouldn't produce discriminatory outcomes. And it is in that sense that the bias is "in the data". That being said, we may still choose to correct that bias at the model level, just like people fix cinematic issues in post-processing all the time.

loopz 5y ago
IT is only about 3 things with data: 1) Communication 2) Transformation 3) Application 1 being the ray of light/information. 2 being your "mirror". 3 being the usage. But these processes are general enough to fit any process in the universe (aka "simulate the universe"). Potential of IT may be regarded that powerful. Saying models are only about being a mirror may be too easily misinterpreted. This since application require accountability for 1, 2 and 3, not just one of them. Complex and powerful solutions aren't easily divided into clean responsibility areas. If one chooses models that favours some groups, such usage may introduce bias and discrimination in an otherwise non-discriminatory and non-racist society. What matters is the final outcomes, even when well-intentioned. As usage of complex algorithms may introduce chaotic side-effects, solutions to such issues may become dauntingly complex and hard to grapple with. Complex and powerful solutions become embedded in the world and culture of humans. Even the knowledge of usage of black box models may change people's perceptions for the worse, and introduce unwanted side-effects. Poor or lack of explanations for decision-making by the black box may compound issues even further. Misuse may introduce events that alters perception forever, etc. You may retroactively attempt to correct wrongs, but it's a hard sell you've made no further mistakes however unintentional. I have doubts people make such mistakes intentionally, though that's not impossible either.