9 ms·
A new course to teach people about fairness in machine learning
- jf- 8y agoThe video lecture calls for validation of ML results with regard to “appropriate social behaviour”. Is this actually a call to hide results that are politically inconvenient? A concrete example would help here.
- NewEntryHN 8y agoA bias is simply a non uniform distribution among the values of a feature of what you're trying to predict. Since the data used by Google in training AI comes from users, it's likely to contain all sorts of human and social bias adding noise to what they're trying to predict.
- jf- 8y agoIndeed, I know what a bias is and what the common cognitive biases are, though they always bear repeating. I’m referring to a specific statement at the end of the video lecture, where the viewer is encouraged to validate ML results against “socially appropriate behaviour”. This is somewhat ambiguous, and can be interpreted as meaning that the viewer should suppress any result that may appear taboo. I’m wondering if this is the intention behind the statement. If so, this is not to be encouraged. It would be the equivalent of the sciences rejecting results when they don’t conform to current theory. Go to extra lengths to validate the results, sure, but don’t throw them away out of hand.
- thrower123 8y agoIt's becoming increasingly clear that in the current environment, algorithms trained on real-world data that produce politically inconvenient results will be decried as X-ist, and either disregarded or fiddled with until the results they produce are inoffensive.
- jf- 8y agoI think it’s important to bear in mind that most machine learning models still aren’t very good, and the main problem you’ll encounter with them isn’t bias, it’s whether or not they’re producing gibberish. In this thread we seem to have assumed that ML is generally trustworthy, and I doubt very much that it is. What we may be debating is whether gibberish is more or less morally good depending on whether it includes race and gender.
- nkozyra 8y ago-against our own biases- we are evaluating and validating models all the time. There are many, many models with acceptable accuracy that we rely on every day. I think this is a myopic view of the current state of ML.
- pdkl95 8y agoThis isn't about "appearing taboo", it's remembering that data is always the product of the environment that created it. If you are using e.g. housing data, naively applying the data directly bakes in, for example, the decades of redlining, blockbusting, restrictive covenants, etc. Many cities are still segregated today, decades after redlining/etc ended. Lets say you are writing ML related to loan applications or deciding if someone gets parole. Do you want to "accurately" look at the historical data? Or should the decision also include the context of that data with an eye towards making a decision that is more socially appropriate than perpetuating the racism of the past?
- jf- 8y agoI take your point about misleading historical data, but I’m not sure that was the sentiment behind the statement. I took it more to mean “if this feels transgressive to you, don’t publish it!”. Your explanation is a valid observation of a phenomenon, but the “socially appropriate” part feels shoehorned in.
- deleted 8y ago[deleted]
- 123throwaway456 8y agoThere was a time when tech companies were driven by data, inspired by the scientific method. There was a time when the scientific method was about uncovering the 'what is' truth, as opposed to 'what ought be' decrees, which are by definition political. Today it seems that we are more than happy to interject arbitrary political biases in the equation, in the name of elusive fairness. Food for thought, Harrison Bergeron [0]. Teaser: > THE YEAR WAS 2081, and everybody was finally equal. They weren't only equal before God and the law. They were equal every which way. Nobody was smarter than anybody else. Nobody was better looking than anybody else. Nobody was stronger or quicker than anybody else. All this equality was due to the 211th, 212th, and 213th Amendments to the Constitution, and to the unceasing vigilance of agents of the United States Handicapper General. Residential segregation is a good example of the difficulty of quantifying the causality of fairness. On one hand, residential segregation is a human universal. From the Papuan jungle to the London asphalt jungle, humans tend to stick with their own kin, for some definition of own kin. People have built simulators demonstrating that even a weak preference for own kin leads to significant residential segregation [1]. On the other hand, there is a history of forced segregation, well documented in many places [2][3] etc. Furthermore there are more than one forms of segregation, read literally as 'separation from others'. There is the well studied economic and racial residential segregation, the lack of easy access to more economically desirable living quarters. On the flip side, USA is already a very atomised and isolating society. Some would point to the addiction and homelesness crises as direct consequences of social isolation [4]. Living isolated from one's own kin is yet another factor increasing social isolation, and a different kind of segregation. At what point does enforced fairness morph from an attempt to fix the past ills into the tyranny of today, steering people toward actions that are contrary to their natural preferences and well being? Or is there no such point and we're willing to bludgeon our way into an all out harrisonbergeronian future? [0] http://tnellen.com/cybereng/harrison.html http://tnellen.com/cybereng/harrison.html [1] https://ncase.me/polygons https://ncase.me/polygons [2] https://en.wikipedia.org/wiki/Residential_segregation_in_the_United_States https://en.wikipedia.org/wiki/Residential_segregation_in_the... [3] https://en.wikipedia.org/wiki/Jim_Crow_laws https://en.wikipedia.org/wiki/Jim_Crow_laws [4] https://www.ted.com/talks/johann_hari_everything_you_think_you_know_about_addiction_is_wrong https://www.ted.com/talks/johann_hari_everything_you_think_y...
- mar77i 8y ago> non uniform distribution Is it really imperative to treat reality as if it was uniformly distributed? Why would you want to cast a non-linear environment into a linear shape? Are we trying to get rid of non-uniformity (which is realistically impossible, let's be honest) or are we just correcting towards the falsehoods of current thinking?
- 908087 8y agoAlso behavior that happens to be inconvenient to Google's bottom line, I'd imagine.
- shekelstien 8y agoThat’s part of it. Another part is denial. I had to take out latitude and longitude out of a model once... The area was not only indicative of income but of race. These are dialogues we need to have about “why is this occurring”, not talk about bias. You are white washing reality and blaming an algo.
- joshuamorton 8y agoMoreso, it's a reminder that "training an ml model" isn't a controlled experiment. It's a process for constructing a tool. You aren't hiding anything, training wasn't (and isn't ever really) rigorous. You're more than likely discovering issues in your data set. It's like polling. You don't cover up politically inconvenient polls, but there's often a level of post processing you need to do to account for biases in your sample. Same with ml, except often times the first data set, or three, are useless because no one has ever analyzed them before and doesn't know how to post process correctly.
- coralcross 8y agoIf taking race into account helps us catch criminals, we should. It's not "fair" to victims of crime if we hamstring investigations just because of political consideration.
- wrongthink_bot 8y agoYou have been automatically flagged for committing wrong think.
- pokemongoaway 8y agoExactly HN "moderation" is right out of Brave New World.
- Hendrikto 8y agoUnequal outcome != bias. Most SJWs don‘t seem to understand this. Of course it can be an indicator that there may be bias, but it is FAR from a proof thereof. I can easily write you a completely unbiased “model” predicting who comitted crimes: func probability_of_comitting_crime(): return 1/human_population This is 100% unbiased and treats everybody the same. It’s also 100% useless...
- nkozyra 8y agoThe problem is that people view pure statistical based crime probability as "unbiased" because... math. It's not! Most bias is deeply hidden, and our training data comes directly or indirectly from some form of human subjectivity. The more autonomy plays a role in our lives, the more aware we need to be of how internal bias from human source has affected things like predictions. And please, there is nothing more flippant and off-putting than railing against "SJWs," this is a topic that demands thoughtfulness.
- unstuckdev 8y agoI still see people call policing maps "crime maps" as though you could map out all the crime like in SimCity.
- tomp 8y ago> The problem is that people view pure statistical based crime probability as "unbiased" because... math. It's not! Can you explain what you mean? “Biased” usually means not using just statistics but also extra-statistical personal convictions. What does “unbiased” mean for you?
- nkozyra 8y agoWhere do the numbers come from? What is the context of collection of data? What is the context of the classification of training data? There are hundreds of questions like that which apply. It's not (necessarily, it certainly can be) the algorithm that carries bias, it's the data.
- 8y ago
- nobody271 8y agoIs this for anyone or is it just posturing? I mean anyone doing ML knows that if you train an algorithm on a biased dataset that it will, surprise, learn to be biased. It's not at all inherent or restricted to unfair treatment of any group. It's a more general problem. By focusing on "fairness" you are basically overfitting the larger problem of not selecting good datasets or not having good data.
- nkozyra 8y agoHuh? Humans make the conscious decision to override instinctual and observational impulses all the time in response to empathy and emotion. It is a huge foundational factor in society. Why would we NOT want autonomous agents to have the same capacity? There's a reason things like The Trolley Problem come up in AI courses: our agents have to play reasonably in a human world.
- sokoloff 8y agoI find the classic Trolley Problem interesting mostly because it exposes the extremely limited capacity that humans have to act rationally. I would expect a trivial ML/AI implementation to "correctly" solve the Trolley Problem and I would not want to consciously program our future AI systems to suffer from the same limitations that cause humans to struggle with the Trolley Problem. By programming them this way (avoiding programming in human frailties for the purpose of better "fitting in"), I think we can more quickly expose and correct certain areas where us bags of mostly salt water act suboptimally.
- nkozyra 8y agoThe whole point of the trolley problem is there is no correct solution. It's a negative sum game. It's an ethical problem that has no solution. There is also no guarantee that a machine would kill one in lieu of five. Why? Training data. The bigger point I'm making is people think AI acts in some form of posthuman reason. At this time no such thing exists: decisions are still bound to human bias, and we should at least consider optimizing with regard to the rules of a modern society.
- retox 8y agoDo a Google image search for "new actress" without the speech marks. Or "American inventor"
- jf- 8y agoThe “American Inventor” result is easily explained by being a substring of “African American Inventor”. No conspiracy necessary, it’s just a substring hit.
- antt 8y agoWhat does this even mean?
- jf- 8y agoIt’s a trope that’s supposed to show that google searches are biased in favour of black people. If you search for American inventor you get lots of image hits for black inventors. The real reason for this is that American inventor is a substring of African American inventor, so google shows results containing the superstring.
- mbrumlow 8y ago> (fairness) how to define it Not even a paragraph in and I see problems with this. Seems somebody bias has already gotten in their own way. This is the first step - redefine the meanings of words to fit your agenda. I could be wrong about the intent. But when you can redefine fairness the you can make it fair to take from one person and give to another. Or deny one person and Grant access to another based on things that don't matter. Also. The attack on human bias is totally silly. It's a key trait that had made us and other species successful. Without these biases we might pick the wrong fruit eat it and die. We might try to cuddel into a cave with bears in the winter. Our entire existence and reasoning skills is based off bases. Also. Never in my life have a seen ripe or unripe bannans and described them by their color. Is this a education problem? But even if I had there is nothing wrong with pointing out when something is not atypical. Besides if I plan on eating bananas in a few days I might want the "green" ones. Or if I was going to make a yummy smoothie the "brown" ones are the ones I want. Also bananas are a bad analogy for what they were getting at. Every banana has the potential to be green, then yellow, then brown. So none of these states are out of the ordinary. I don't need or want to be unconsciously retrained by ML. I think we are going to need laws against big cooperations using their power and tech to change socal constructs. It's not their place and we can never be sure they are doing it for good or even the outcome will be good for human kind. It's also a power that can be abused. We already have internal documents comming from Google suggesting swaying public opinion on on political canadates. And this type of research is exactly where you would start.
- majos 8y agoWhat? This isn't about redefining fairness, but defining it in the first place. Seriously, try to come up with a rigorous technical definition of what a fair decision process means. It's not easy. But we need such a definition (or definitions). It mattered less in the past because humans made all the decisions so we could fall back on fuzzy qualitative notions. We can't do that when algorithms make decisions.
- humanrebar 8y agoIt's worse than that. Often fairness is a process, not a formula or metric.
- pokemongoaway 8y agoHah! So their intro video for this idea states that we are basically biased because we assume bananas to be yellow... I wonder how their code is going to read... If white, then reduce privileges If black, then give reparation as structRep If binary gender, then increase non-binary sexual imagery; equalize all norms If young female wants to be a mother, then show videos of females "succeeding professionally" Again, at the top of HN with the most political article, but commenters like myself are censored from being able to even give our take on why they're extremely misleading. How is this not exactly like Brave New World?
- 908087 8y ago"Fairness" as defined by Google. Ethically bankrupt advertising corporations have no business teaching people about ethics.
- hashr8064 8y agoI think google should take a bit of its own advice here. Google "american scientists", "american mathematicians" and tell me how these are not hugely biased results. I work in NLP and there is no way you get this without forcing your data/algorithms to return these types of results. This isn't a normative statement, just descriptive. Whether or not google should bias its results is a completely different discussion.
- nkozyra 8y ago1) I'm trying to figure out what NLP has to do with this problem. This is a classic collaborative filtering "problem." 2) I think Google is acutely aware that their results are driven by human behavior and thus are biased. It's the nature of its design
- hashr8064 8y agoCan you explain how this is collaborative filtering as opposed to a classic IR ranking problem? CF would suggest they are somehow getting user ratings of these scientists, but either way its going to boil down to a similarity metric basically. So I guess for me, I can't imagine how user data is creating these rankings and I'm pretty confident using IR techniques on the datasets they have would not return these either, ergo, they are likely tweaking the factors themselves to return results that are "less biased" i.e. less representative of the underlying distribution and more normally distributed aka politically correct. But If you have a better theory of how the 10 of the first 20 "american scientists" are black and 5 are women, I'd be interested to hear it.
- yorwba 8y agoCheck Baidu: http://www.baidu.com/s?ie=utf-8&wd=%22american+scientists%22 http://www.baidu.com/s?ie=utf-8&wd=%22american+scientists%22 Result #6 is the list of African-American inventors and scientists on Wikipedia. Unless Baidu has the same ideological biases as Google (would be strange), the most likely explanation is that it's driven by n-gram frequencies.
- Rainymood 8y agoBefore Google starts lecturing people on fairness, maybe they should look in the mirror and lecture themselves on what's good and evil ...
- humanrebar 8y agoI'll point out that broad categories of fairness have nothing to do with AI/ML and everything to do with institutions ensuring fair outcomes. For instance, we would be appalled if a court found that the conviction rate for people like the defendant was extremely low, so the trial was going to start with the presumption of 80% guilt. What's the problem? It's not one fixed by better modelling or a better bias percentage. The problem is that fairness in many contexts is about processes and institutions. We expect courts to follow principles like due process and equal protection under the law. Companies like Google can't jump right to social science and social engineering without thinking about fairness and justice in these terms. Can a user confirm that the data about them is accurate? If not, what can they do about it? If a business has its account algorithmically cancelled, what is the appeals process? How is it fair? When companies like Google start getting into social science, the problem is extremely complicated fast. For example, it starts looking like a judicial system in parts, but one that isn't established and maintained by a legitimate government.
- k__ 8y agoCan't they just remove unfair data from the trainings set?
- KirinDave 8y agoNot if the unfair data is an integral part of the set, as it appears it was in Amazon's hiring process debacle.
- humanrebar 8y agoI was talking about how to even engage with Google about untrue or unhelpful data. If Google wants to be fair, it's not just a matter of data, demographics, and statistics. It needs to think about what philosophers like Kant and Rawls would say about treating even outliers and exceptions equitably.
- atlantic 8y agoWhich begs the question: how do you identify unfair data? If the data is not false or inaccurate, you shouldn't tamper with it. On the other hand, if you already know at the outset the result that you would like your algorithm to produce, then why bother with machine learning at all? Just hard-code your output.
- jlawson 8y agoI read HN with showdead on. There's some pretty unjustifiable censorship going on in this thread. I'll just call their attention to their own rule on this: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith." The weak interpretation of these dead comments is that they're just racists who hate dark-skinned people. There are much stronger interpretations and those should be discussed. Especially in the context of Google, where it's been demonstrated over and over through video leaks, court filings and the Damore firing, that a single and very heavy-handed ideology has power and will not allow discussion. From a centrist position is seems obvious Google's internal thought process will be itself heavily biased, and the possibility of certain factual hypothesis being true about the physical world (not normative statements on what to do) will be denied for ideological reasons. It's like a group of fundamentalist Christian scientists studying geology. This needs to be acknowledged in any discussion of Google, ML, and 'bias'.
- shekelstien 8y agoThis should be the top comment.
- pokemongoaway 8y agoExactly. And I've spoken with several people who have been emailing the admins about this - and they just continue censoring without providing reasonable explanations (incorrectly citing abuse). HN has become extremely political - and how many aren't posting anymore because they were banned? Probably a lot! I've met at least 3 people who have reacted like I said, "do you watch Fox news," when I had actually asked, "do you visit HN?" There's a leftist article like this one at #1 nearly daily.
- atlantic 8y agoHow do posts become dead? Is it because of downvotes, or due rather to some form of editorial intervention?
- kyleperik 8y agoSo why isn't there anything about teaching fairness in statistics? To sport the classic over simplification, ML is just glorified curve fitting. But in reality, it's not like we created a special new species with intelligence that we have to careful teach not to be racist. _We_ just have to be careful not to be racist with it.
- bicubic 8y agoSo what's happening lately is that tech giants have gotten into the business of dictating how the world should be. No one was consulted about it, it was not put to a vote, it simply began, and over the last few years it's increasing in amplitude. The problem with them using words like 'fairness' is anyone who disagrees with what they're doing is immediately put on the defensive and labelled as 'unfair'. This is far from the truth though, as what society today is actually doing, is trying to figure out exactly what 'fair' should mean. Google et al just took it upon themselves to start pushing their particular brand of fairness onto society and acting as if it's the most natural widely-accepted common sense. Google's idea of fairness is to build ML solutions that don't accurately predict the world, but rather inject Google biases into the predictions in order to bias some real-world process, and ultimately change the distribution of some factors in the real world according Google's image of how things should be. Not enough <demographic> represented in the workforce? Bias the candidate selection model to equalize it. Some <demographic> is over-represented in crime statistics? Bias jail sentencing model to be more lenient on them and hope it equalizes their representation in stats over time. This is not inherently a bad mechanism, but it's an extremely powerful mechanism with far-reaching consequences, and it is currently controlled by a small number of people at the top of the US tech industry, with no checks or oversight. I worry that there's a strong possibility that the long term effects of messing around with these levers have not been considered. We have done serious unintentional damage with far smaller levers like the Australian Rabbit Plague. I don't know what 'fairness' should actually mean in modern society, but we definitely need to be aware that the US tech industry is hijacking the term to fit their agenda, and need to be checked. And the exact nature of the biases they're injecting should definitely be made transparent and put to public debate.
- sytelus 8y agoYour viewpoint is not correct. Many class of decisions we make needs to be independent of race, color, religion and gender. This is not Google’s idea, it’s US Constitution + laws in many states. As ML is increasingly used for making decisions such as whom to give loans or who do you hire, many models may predispose a human based on exactly these attributes violating laws.
- 8y ago
- gaius 8y agoHow about fairness in taxation? What do Google know about that?
- sytelus 8y agoThere seems to be big confusion on what is bias and fairness and this course doesn’t seem to help. Fairness is ensuring that peaple are not deliberately punished for things that they have no control over. Typically this translates to making sure that decisions made by governments and employers do not depend on race, color and gender. US constitution adds religion in to the mix and some EU countries may also want to include sexual orientation, disability, age, political orientation but the minimal thing required by US laws in most states are those 4 attributes. Every ML model can have different bias on different features. That’s what makes model a model. The fairness means you should make sure that your model is not biased towards above specific attributes and explicitly test for it. This is not some liberal propaganda by tech but required by law in many instances even if ML was not involved. This only applies to models used for certain classes of decision making, for example, loan eligibility or hiring or FaceID login. It doesn’t apply to things like breast cancer predictions or shopping recommendation system.
- btown 8y agoIt's more than this, though. For instance, is the presence of the word "captured" (e.g. in a dataset of performance reviews) correlated with gender? This was certainly the case for Amazon: https://www.theguardian.com/technology/2018/oct/10/amazon-hiring-ai-gender-bias-recruiting-engine https://www.theguardian.com/technology/2018/oct/10/amazon-hi... . If so, even if you don't include gender as a feature itself, your outputs may end up being biased (in the technical sense) by gender. If you want to correct for this, then you might want to include gender in your analysis in a structured way in order to determine: are there better features that apply relatively equally to indicate a superstar of either gender? Personally, I don't know what the battle-tested approaches in the ML literature are to do this type of thing, so this course would be helpful to people like me! At the end of the day, it all boils down to this: you can't be "blind" to race, color, religion, and gender if information is leaked to your evaluator (human or otherwise) via a side channel. You might get away with this legally (especially if there's not intent to discriminate), but if you really want to do it right, you have to engineer systems that minimize those side channels.
- Pengy7 8y ago
- realpinkie 8y agoA theoretical ML paper came out recently that directly ties the notion of bias/fairness in machine learning to the tuning of the bias parameter in deep neural network architectures. The authors used stochastic backpropagation to classify an unbalanced dataset (85% class 0, 15% class 1) and show, through a series of topological cross-validation techniques, that very small amounts of perturbation in the bias parameter results in significant increase in the overall bias of the model towards the minority class (class 1).
- p1esk 8y agoWhich paper?
- itry2develop 8y agoTruth hurts.
- deepnotderp 8y agoFairness (tm) by Google.