5 ms·

So what's happening lately is that tech giants have gotten into the business of dictating how the world should be. No one was consulted about it, it was not put

So what's happening lately is that tech giants have gotten into the business of dictating how the world should be. No one was consulted about it, it was not put to a vote, it simply began, and over the last few years it's increasing in amplitude. The problem with them using words like 'fairness' is anyone who disagrees with what they're doing is immediately put on the defensive and labelled as 'unfair'. This is far from the truth though, as what society today is actually doing, is trying to figure out exactly what 'fair' should mean. Google et al just took it upon themselves to start pushing their particular brand of fairness onto society and acting as if it's the most natural widely-accepted common sense. Google's idea of fairness is to build ML solutions that don't accurately predict the world, but rather inject Google biases into the predictions in order to bias some real-world process, and ultimately change the distribution of some factors in the real world according Google's image of how things should be. Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time. This is not inherently a bad mechanism, but it's an extremely powerful mechanism with far-reaching consequences, and it is currently controlled by a small number of people at the top of the US tech industry, with no checks or oversight. I worry that there's a strong possibility that the long term effects of messing around with these levers have not been considered. We have done serious unintentional damage with far smaller levers like the Australian Rabbit Plague. I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate.

sytelus 8y ago
Your viewpoint is not correct. Many class of decisions we make needs to be independent of race, color, religion and gender. This is not Google’s idea, it’s US Constitution + laws in many states. As ML is increasingly used for making decisions such as whom to give loans or who do you hire, many models may predispose a human based on exactly these attributes violating laws.
- bicubic 8y ago
  Well, yes. That's exactly why the debate around 'fairness' has begun. ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV - usage of words like 'captured' and 'executed' which were used more frequently by one gender. ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly. These are the facts, the world is not evenly distributed. The question for society now is where do we go from here, and what exactly should 'fair' mean. Do we introduce reverse bias into processes to equalize the output distribution? Do we change the criteria considered by various processes to try to influence the output distribution? Do we look at the underlying social causes of the non-uniform distribution and try to address those? Do we just leave things as they are? Do we legislate to get ML out of critical social processes? Should we maybe first address much more obvious cases of unfairness that don't even require ML to identify, like massive and growing wealth disparity? Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G https://www.reuters.com/article/us-amazon-com-jobs-automatio... https://www.cnbc.com/2018/03/02/google-accused-in-lawsuit-of-excluding-white-and-asian-men-in-hiring-to-boost-diversity.html https://www.cnbc.com/2018/03/02/google-accused-in-lawsuit-of...
  - KirinDave 8y ago
    > ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV. Ah yes, subtle clues like the word "women's" and "girl's". > ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly. It's not a problem that the network could identify that fact. It's a problem that the training data for Amazon contained this, because it's pretty clear there is an illegal and unfair bias in Amazon hiring. The data, from the stories we have, didn't include anything but successful hire data. No credible notion of "success" for post-hires has been indicated here. > Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue. Have you looked at the courseware? The entire courseware is about how these systems optimize to fit data, not to fit truth. They explore numerous ways in which that subtlety creates surprising or undesired outcomes. It's very difficult to imagine a world where more awareness of the limitations of mathmatical models will lead to an undesirable outcome. Which one of these core tenants from the syllabus are YOU opposed to? * Engage with a diverse set of users and use-case scenarios, and incorporate feedback before and throughout project development. This will build a rich variety of user perspectives into the project and increase the number of people who benefit from the technology. * Model potential adverse feedback early in the design process, followed by specific live testing and iteration for a small fraction of traffic before full deployment. * Consider augmentation and assistance: producing a single answer can be appropriate where there is a high probability that the answer satisfies a diversity of users and use cases. In other cases, it may be optimal for your system to suggest a few options to the user. Technically, it is much more difficult to achieve good precision at one answer (P@1) versus precision at a few answers (e.g., P@3). * Ensure that your metrics are appropriate for the context and goals of your system, e.g., a fire alarm system should have high recall, even if that means the occasional false alarm. Most are equally best practice. It's a course designed to supplement the depth and breadth of education in a world increasingly staffed by ML practitioners who learned the entirity of their courseware from 2 free Coursera courses and a bunch of Siraj Raval videos. You're talking about "fairness" like this course has ML-based policy decisions or even addresses that substantially. This is a category error.
KirinDave 8y ago
> The problem with them using words like 'fairness' is anyone who disagrees with what they're doing is immediately put on the defensive and labelled as 'unfair'. This is far from the truth though, as what society today is actually doing, is trying to figure out exactly what 'fair' should mean. Google et al just took it upon themselves to start pushing their particular brand of fairness onto society and acting as if it's the most natural widely-accepted common sense. I'm sorry, but this comment seems reactionary in the extreme. Could you identify aspects of the courseware itself that are "unfair" in the sense you're using? Or give a misleading definition of "fair"? > Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time. Further, followups like this suggest you're afraid that this is a play by Google to lead to the (presumably undesirable, from your tone) outcome of fairness. You're not-very-subtly suggesting it'd be unfair for black people to do less jail time for the same crimes, which has no relevance to the conversation at hand. Why include this? Is this a subtle signal? A tangent slide you fell down? A bad edit of a post? > I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate. It's also extremely important to note that cultural elements are using the same processes to normalize acute unfairness in the workplace and in policy, often under the cover of "the machine predicted it." Making sure that a broader class of people know that all machine learning does is fit models to pre-existing data (and therefore are quite limited in what kind of novel insight they can glean safely, requiring utmost care in the model design and data normalization) is an incredibly good idea, both for the industry itself and for society as a whole. It's in the nearly the exact same vein as, "I wish more people understood the implications of Bayes/Laplace's rule or the law of large numbers." The only downside is that fewer folks will be able to peddle complex-sounding ML methods that boil down to (from a practitioner's standpoint) a few lines of trivial Python as if they are the cutting edge of AI research.

sytelus 8y ago

Your viewpoint is not correct. Many class of decisions we make needs to be independent of race, color, religion and gender. This is not Google’s idea, it’s US Constitution + laws in many states. As ML is increasingly used for making decisions such as whom to give loans or who do you hire, many models may predispose a human based on exactly these attributes violating laws.

bicubic 8y ago
Well, yes. That's exactly why the debate around 'fairness' has begun. ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV - usage of words like 'captured' and 'executed' which were used more frequently by one gender. ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly. These are the facts, the world is not evenly distributed. The question for society now is where do we go from here, and what exactly should 'fair' mean. Do we introduce reverse bias into processes to equalize the output distribution? Do we change the criteria considered by various processes to try to influence the output distribution? Do we look at the underlying social causes of the non-uniform distribution and try to address those? Do we just leave things as they are? Do we legislate to get ML out of critical social processes? Should we maybe first address much more obvious cases of unfairness that don't even require ML to identify, like massive and growing wealth disparity? Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G https://www.reuters.com/article/us-amazon-com-jobs-automatio... https://www.cnbc.com/2018/03/02/google-accused-in-lawsuit-of-excluding-white-and-asian-men-in-hiring-to-boost-diversity.html https://www.cnbc.com/2018/03/02/google-accused-in-lawsuit-of...
- KirinDave 8y ago
  > ML will easily infer gender/color/religion from any other features about an individual. It should not be too surprising that those features play a big role in identity. Consider the recent news about Amazon throwing out a model which kept learning to discriminate on gender no matter how much they tried to blacklist gender-related features. They ended up with a model that could determine the gender of a job applicant just from subtle language cues in how they write their CV. Ah yes, subtle clues like the word "women's" and "girl's". > ML is worthless if blinded to anything that might give away a sensitive feature about an individual because that's virtually everything. And the current generation of ML is making it it really clear that the world as a whole discriminates on these features all the time, whether directly or indirectly. It's not a problem that the network could identify that fact. It's a problem that the training data for Amazon contained this, because it's pretty clear there is an illegal and unfair bias in Amazon hiring. The data, from the stories we have, didn't include anything but successful hire data. No credible notion of "success" for post-hires has been indicated here. > Google's idea of fairness is only one of these choices, and it does not represent a public consensus on the subject - the public was never consulted. And yet the public is increasingly being impacted by this issue. Have you looked at the courseware? The entire courseware is about how these systems optimize to fit data, not to fit truth. They explore numerous ways in which that subtlety creates surprising or undesired outcomes. It's very difficult to imagine a world where more awareness of the limitations of mathmatical models will lead to an undesirable outcome. Which one of these core tenants from the syllabus are YOU opposed to? * Engage with a diverse set of users and use-case scenarios, and incorporate feedback before and throughout project development. This will build a rich variety of user perspectives into the project and increase the number of people who benefit from the technology. * Model potential adverse feedback early in the design process, followed by specific live testing and iteration for a small fraction of traffic before full deployment. * Consider augmentation and assistance: producing a single answer can be appropriate where there is a high probability that the answer satisfies a diversity of users and use cases. In other cases, it may be optimal for your system to suggest a few options to the user. Technically, it is much more difficult to achieve good precision at one answer (P@1) versus precision at a few answers (e.g., P@3). * Ensure that your metrics are appropriate for the context and goals of your system, e.g., a fire alarm system should have high recall, even if that means the occasional false alarm. Most are equally best practice. It's a course designed to supplement the depth and breadth of education in a world increasingly staffed by ML practitioners who learned the entirity of their courseware from 2 free Coursera courses and a bunch of Siraj Raval videos. You're talking about "fairness" like this course has ML-based policy decisions or even addresses that substantially. This is a category error.

KirinDave 8y ago

> The problem with them using words like 'fairness' is anyone who disagrees with what they're doing is immediately put on the defensive and labelled as 'unfair'. This is far from the truth though, as what society today is actually doing, is trying to figure out exactly what 'fair' should mean. Google et al just took it upon themselves to start pushing their particular brand of fairness onto society and acting as if it's the most natural widely-accepted common sense. I'm sorry, but this comment seems reactionary in the extreme. Could you identify aspects of the courseware itself that are "unfair" in the sense you're using? Or give a misleading definition of "fair"? > Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time. Further, followups like this suggest you're afraid that this is a play by Google to lead to the (presumably undesirable, from your tone) outcome of fairness. You're not-very-subtly suggesting it'd be unfair for black people to do less jail time for the same crimes, which has no relevance to the conversation at hand. Why include this? Is this a subtle signal? A tangent slide you fell down? A bad edit of a post? > I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate. It's also extremely important to note that cultural elements are using the same processes to normalize acute unfairness in the workplace and in policy, often under the cover of "the machine predicted it." Making sure that a broader class of people know that all machine learning does is fit models to pre-existing data (and therefore are quite limited in what kind of novel insight they can glean safely, requiring utmost care in the model design and data normalization) is an incredibly good idea, both for the industry itself and for society as a whole. It's in the nearly the exact same vein as, "I wish more people understood the implications of Bayes/Laplace's rule or the law of large numbers." The only downside is that fewer folks will be able to peddle complex-sounding ML methods that boil down to (from a practitioner's standpoint) a few lines of trivial Python as if they are the cutting edge of AI research.