3 ms·

So I think you and I agree on all the things you just said. My point is really just that, linguistically, I don't think it makes sense to describe the models as

by darawk 5y ago

So I think you and I agree on all the things you just said. My point is really just that, linguistically, I don't think it makes sense to describe the models as being "biased" or "discriminatory" here. Statistical learning models are designed intentionally to act like mirrors. They reflect the data they're trained on. And I don't think it's descriptively useful to describe a mirror as biased because you don't like your reflection. Even if, in some sense, you could design a curved mirror that generates the reflection you wanted. The mirror is just a mirror. Now, that being said, I think it is fair to talk about structural equitably in the use of models that produce outcomes we believe are discriminatory. If ML engineers at some company produce a model, and fail to check it for these issues, or do check it but fail to correct them, we can certainly describe that behavior in negative terms, and shame them appropriately. At the end of the day, if we didn't live in a racist/sexist society, these ML models wouldn't produce discriminatory outcomes. And it is in that sense that the bias is "in the data". That being said, we may still choose to correct that bias at the model level, just like people fix cinematic issues in post-processing all the time.

loopz 5y ago
IT is only about 3 things with data: 1) Communication 2) Transformation 3) Application 1 being the ray of light/information. 2 being your "mirror". 3 being the usage. But these processes are general enough to fit any process in the universe (aka "simulate the universe"). Potential of IT may be regarded that powerful. Saying models are only about being a mirror may be too easily misinterpreted. This since application require accountability for 1, 2 and 3, not just one of them. Complex and powerful solutions aren't easily divided into clean responsibility areas. If one chooses models that favours some groups, such usage may introduce bias and discrimination in an otherwise non-discriminatory and non-racist society. What matters is the final outcomes, even when well-intentioned. As usage of complex algorithms may introduce chaotic side-effects, solutions to such issues may become dauntingly complex and hard to grapple with. Complex and powerful solutions become embedded in the world and culture of humans. Even the knowledge of usage of black box models may change people's perceptions for the worse, and introduce unwanted side-effects. Poor or lack of explanations for decision-making by the black box may compound issues even further. Misuse may introduce events that alters perception forever, etc. You may retroactively attempt to correct wrongs, but it's a hard sell you've made no further mistakes however unintentional. I have doubts people make such mistakes intentionally, though that's not impossible either.

by darawk 5y ago

loopz 5y ago

IT is only about 3 things with data: 1) Communication 2) Transformation 3) Application 1 being the ray of light/information. 2 being your "mirror". 3 being the usage. But these processes are general enough to fit any process in the universe (aka "simulate the universe"). Potential of IT may be regarded that powerful. Saying models are only about being a mirror may be too easily misinterpreted. This since application require accountability for 1, 2 and 3, not just one of them. Complex and powerful solutions aren't easily divided into clean responsibility areas. If one chooses models that favours some groups, such usage may introduce bias and discrimination in an otherwise non-discriminatory and non-racist society. What matters is the final outcomes, even when well-intentioned. As usage of complex algorithms may introduce chaotic side-effects, solutions to such issues may become dauntingly complex and hard to grapple with. Complex and powerful solutions become embedded in the world and culture of humans. Even the knowledge of usage of black box models may change people's perceptions for the worse, and introduce unwanted side-effects. Poor or lack of explanations for decision-making by the black box may compound issues even further. Misuse may introduce events that alters perception forever, etc. You may retroactively attempt to correct wrongs, but it's a hard sell you've made no further mistakes however unintentional. I have doubts people make such mistakes intentionally, though that's not impossible either.