5 ms·

Well, yes. That's exactly why the debate around 'fairness' has begun. ML will easily infer gender/color/religion from any other features about an individual. I

Well, yes. That's exactly why the debate around 'fairness' has begun. ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV - usage of words like 'captured' and 'executed' which were used more frequently by one gender. ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly. These are the facts, the world is not evenly distributed. The question for society now is where do we go from here, and what exactly should 'fair' mean. Do we introduce reverse bias into processes to equalize the output distribution? Do we change the criteria considered by various processes to try to influence the output distribution? Do we look at the underlying social causes of the non-uniform distribution and try to address those? Do we just leave things as they are? Do we legislate to get ML out of critical social processes? Should we maybe first address much more obvious cases of unfairness that don't even require ML to identify, like massive and growing wealth disparity? Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G https://www.reuters.com/article/us-amazon-com-jobs-automatio... https://www.cnbc.com/2018/03/02/google-accused-in-lawsuit-of-excluding-white-and-asian-men-in-hiring-to-boost-diversity.html https://www.cnbc.com/2018/03/02/google-accused-in-lawsuit-of...

KirinDave 8y ago
> ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV. Ah yes, subtle clues like the word "women's" and "girl's". > ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly. It's not a problem that the network could identify that fact. It's a problem that the training data for Amazon contained this, because it's pretty clear there is an illegal and unfair bias in Amazon hiring. The data, from the stories we have, didn't include anything but successful hire data. No credible notion of "success" for post-hires has been indicated here. > Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue. Have you looked at the courseware? The entire courseware is about how these systems optimize to fit data, not to fit truth. They explore numerous ways in which that subtlety creates surprising or undesired outcomes. It's very difficult to imagine a world where more awareness of the limitations of mathmatical models will lead to an undesirable outcome. Which one of these core tenants from the syllabus are YOU opposed to? * Engage with a diverse set of users and use-case scenarios, and incorporate feedback before and throughout project development. This will build a rich variety of user perspectives into the project and increase the number of people who benefit from the technology. * Model potential adverse feedback early in the design process, followed by specific live testing and iteration for a small fraction of traffic before full deployment. * Consider augmentation and assistance: producing a single answer can be appropriate where there is a high probability that the answer satisfies a diversity of users and use cases. In other cases, it may be optimal for your system to suggest a few options to the user. Technically, it is much more difficult to achieve good precision at one answer (P@1) versus precision at a few answers (e.g., P@3). * Ensure that your metrics are appropriate for the context and goals of your system, e.g., a fire alarm system should have high recall, even if that means the occasional false alarm. Most are equally best practice. It's a course designed to supplement the depth and breadth of education in a world increasingly staffed by ML practitioners who learned the entirity of their courseware from 2 free Coursera courses and a bunch of Siraj Raval videos. You're talking about "fairness" like this course has ML-based policy decisions or even addresses that substantially. This is a category error.
- deleted 8y ago
  [deleted]

KirinDave 8y ago

> ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV. Ah yes, subtle clues like the word "women's" and "girl's". > ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly. It's not a problem that the network could identify that fact. It's a problem that the training data for Amazon contained this, because it's pretty clear there is an illegal and unfair bias in Amazon hiring. The data, from the stories we have, didn't include anything but successful hire data. No credible notion of "success" for post-hires has been indicated here. > Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue. Have you looked at the courseware? The entire courseware is about how these systems optimize to fit data, not to fit truth. They explore numerous ways in which that subtlety creates surprising or undesired outcomes. It's very difficult to imagine a world where more awareness of the limitations of mathmatical models will lead to an undesirable outcome. Which one of these core tenants from the syllabus are YOU opposed to? * Engage with a diverse set of users and use-case scenarios, and incorporate feedback before and throughout project development. This will build a rich variety of user perspectives into the project and increase the number of people who benefit from the technology. * Model potential adverse feedback early in the design process, followed by specific live testing and iteration for a small fraction of traffic before full deployment. * Consider augmentation and assistance: producing a single answer can be appropriate where there is a high probability that the answer satisfies a diversity of users and use cases. In other cases, it may be optimal for your system to suggest a few options to the user. Technically, it is much more difficult to achieve good precision at one answer (P@1) versus precision at a few answers (e.g., P@3). * Ensure that your metrics are appropriate for the context and goals of your system, e.g., a fire alarm system should have high recall, even if that means the occasional false alarm. Most are equally best practice. It's a course designed to supplement the depth and breadth of education in a world increasingly staffed by ML practitioners who learned the entirity of their courseware from 2 free Coursera courses and a bunch of Siraj Raval videos. You're talking about "fairness" like this course has ML-based policy decisions or even addresses that substantially. This is a category error.

deleted 8y ago
[deleted]