6 ms·
The Code We Can't Control
- morgante 12y ago> unless you consider invasive personalized ads a benefit. I do. I'm much happier that my Facebook includes ads for New Relic and MailChimp than condoms and sports. More generally, I don't understand how algorithms or profiles can be prejudiced. If they're inaccurate, there is a strong economic incentive to improve accuracy over time. If they're accurate, I think that should be defense alone—if your history suggests you are likely to default on a loan, companies should have every right to not loan you money.
- dsfsdfd 12y agoThere is a distinction to be made here between brute utility and morality. If a person commits a crime and serves the time for it and lives beyond the point where that crime has to be reported when applying for a job - should that previous conviction alter their likelihood of getting hired? Society as a whole may decide that it should not, yet the processes which determine the chances of hiring will not reflect the decision made by society as a whole. There will be a very large number of conflicts of this type, on the one hand businesses want to maximise profits and on the other society as a whole wants to impose certain moral and economic values upon those businesses. This is part of a more general tension between different structures within a society. The difference here is that before the conflict was less obvious because the agents involved could hide their motivation, when we are using machine agents for decision making we can easily query them and observe their bias objectively. So rather than creating discord between the various motives of society, it is really only making them more visible. Which I would say is a good thing.
- blfr 12y agoIf people are fine with college records from 20+ years ago influencing the likelihood of getting hired but want companies to ignore criminal convictions, or ban IQ test, I'd rather go with a machine making the decisions. Current process seems purely political with massive rents extraction ($100k for a degree, loan guarantees), almost naked votes buying (subsidies, "free" plan for whatever). And moral arguments are used mostly for cover (non-dischargeable debts?). Perhaps we can sidestep those problems, or at least escape from them for a while.
- pron 12y ago> Perhaps we can sidestep those problems, or at least escape from them for a while. But we can't. Any decision we make -- even one based purely on data -- is a political one, because we can choose to change the data by changing society. If you say we should make future decisions (which affect lives and future data) based on current correlations only, you're executing a conservative political policy of maintaining the status quo. If you program computers to just take into account existing data (and not, say, desired future outcomes for society), you're programming conservative computers.
- blfr 12y agoI'm pretty sure we can sidestep degree requirements and the associated mess with a better candidate screening technique. Why not? Also, the computer programs take into account the desired outcomes and the information they can obtain experimentally (through A/B testing for example), not just existing data.
- eli_gottlieb 12y ago>I'm pretty sure we can sidestep degree requirements and the associated mess with a better candidate screening technique. Why not? Because feature-learning problems in the setting of nearly-unlimited sets of possible features are very hard.
- theorique 12y agoPart of knowing what college a person went to is a proxy for knowing their likely SAT scores which is a proxy for an (illegal) IQ test. So it's a moderately effective workaround. Of course, college can be gamed in a lot of different ways (family money, connections, hustle) that do not relate specifically to IQ or to job performance (and others that, of course, do).
- deleted 12y ago[deleted]
- zimbatm 12y agoThe algorithms are usually statistical which by definition makes them prejudiced. Targeted ads only mean targeted to some group that they believe you belong in. Maybe some day each person will be exactly profiled but it's not the case right now. The other issue is that humans are much more permeable to their environment than we would like to admit ourselves. We are influenceable. Ads invade our environment constantly for that reason, their goal is to influence us into buying something. They can take us to directions that we want to avoid. What if a fat person is trying to lose weight ? Ads have measured that this individual is likely to buy cake and present him with delicious assortiments everywhere he goes. It's beneficial for the companies but what about that individual ? The algorithms can only be accurate to who you are, not what you want to become.
- icebraining 12y agoWhat if a fat person is trying to lose weight ? Ads have measured that this individual is likely to buy cake and present him with delicious assortiments everywhere he goes. It's beneficial for the companies but what about that individual ? Conversely, what if the algorithms have decided that the person is a good lead for a local gym, and those ads push him or her into losing weight? We're all influencing each other, constantly. In a way, banner ads and such are some of the most honest ways of going about it, since they're clearly delineated as such. The algorithms can only be accurate to who you are, not what you want to become. If aforementioned fat person searches for "weight loss" and Google starts showing ads for that, haven't the algorithms been accurate to what (s)he wants to become?
- marketforlemmas 12y agoI don't think the issue is whether ads should prompt them to lose weight, or they shouldn't. I think the crux of the issue is that we (humans) are ceding this judgment (which have political and societal ramifications) to the ad matching algorithm. Some people (myself included) find that bothersome. Of course not every example of a targeted ad is going to be controversial (like advertising merchandise of a sports team that I like versus the one that is just popular in my area) but its problematic that we don't have any sort of machinery in place to control/modify the examples that are politically/societally important. At least if that machinery were in place, then we could transfer the agency of decisions to the humans that run google, rather than the algorithms that are just running in the background.
- prof_hobart 12y agoIt's not always about your history - it's about your profile, and the history of people who have similar profiles to you, as determined by the rules of the algorithm. For example, it's a fact that on average, women drivers (at least in the UK) are less of an insurance liability than male drivers. As a result, if a man and a woman both applied for the same insurance with identical driving/insurance history, and otherwise identical profiles, the woman would get cheaper insurance. This was ruled illegal by the European Court of Justice in 2011.
- luisfersom 12y agoI Think that the most important question in this problem is What about Freedom?. Yes, The personalized ads can help you to find solutions to problems that algorthms detect of your profile/information, but what is the cost? Freedom, you can say that you have the choice, but they take your information and control your choices.
- totony 12y agoPeople should stop enlarging the scope of freedom this much. If you want more choices of ads, you just have to look for it. Nithing prevents you from CHOOSING to see more ads. There is a limited number of ads a website can show you, if it is random, you will have as much choice as if it is not, but companies will make more profit.
- yummyfajitas 12y agoThis is an interesting illustration of correlation vs causality. By "causality", I mean links which remain even after you change the composition of your inputs. See the do calculus, which is one of the better attempts to codify it: http://arxiv.org/pdf/1305.5506v1.pdf http://arxiv.org/pdf/1305.5506v1.pdf http://ftp.cs.ucla.edu/pub/stat_ser/r402.pdf http://ftp.cs.ucla.edu/pub/stat_ser/r402.pdf The do calculus attempts to describe causality as the correlations which remain present when you change the underlying composition of the sample set. In a certain sense, that's equivalent to saying it's the correlations which are not caused by Simpson's paradox. https://en.wikipedia.org/wiki/Simpson%27s_paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox So an interesting (but more difficult) question is whether the effects described here are causal or merely correlative. Personally I consider a causal link to be problematic, while a correlative one is completely innocuous. But regardless, it's important to distinguish the two.
- pron 12y agoCausality here makes absolutely no difference. These aren't research programs -- these are programs designed to increase revenue or some other arbitrary composite variable. They're not interested in why the correlations are there, only whether they can be used to increase revenue. The problem here is one of agency. Even true causality in humans is subject to change -- these are not laws of nature here but snapshots of human society, which changes all the time through political action -- but computers inadvertently perpetuate the existing condition, because they have no agency or interest in pursuing change. Perhaps this should be programmed: we could program computers for, say, affirmative action so that changing the status quo is one of their goals. But whether we do or not, the important thing to understand here is that once computers make decisions that influence humans, they are carrying out some political agenda (even if that is maintaining the status quo), whether we consciously program them or not.
- yummyfajitas 12y agoIf a program has affirmative action built in, it is indeed serving a political agenda. AA is a causal relation: if (race == black) score += 10. But if it merely programmed to show ads for "Think Like a Man Too" to people who watched "Think Like a Man" based on a rule about sequels, there is no agenda there beyond profit. In both cases they are disproportionately targeting black people, but in the latter case it's only a composition effect. If you run the sequel targeting program on a group of asians who enjoyed "Think Like a Man", you'll get the same outcome. If you run the AA program on a group of asians, you won't. AA is a causal relationship on race, whereas the sequel rule is not. From previous interactions with you, I gather that you want statistical equality rather than equal treatment. That's great, but many people only care about equality of treatment. The do-calculus (or similar tools) helps with the latter problem, not the former.
- Rapzid 12y agoI'm not convinced the article established the computer program "definitely is" racially biased. The claim was thrown out there and then not substantiated. I guess the article needed to be racially charged..
- pron 12y agoThis is because you (like manny others) are unfamiliar with what racism means. Racism (when applied to processes rather than individuals) means practices -- intentional or not -- that result in an unequal distribution of power among races. If a computer program perpetuates or increases the unequal distribution of power among races, then that program is racist. The same definition, BTW, applies to sexism, where that refers to unequal distribution of power between the sexes.
- yellowapple 12y agoBut what about that program is actually racist? The first paragraph establishes that the program bases its decisions on indicators of financial responsibility. It then makes a decision based (apparently) on race alone. This would require race to be a recorded indicator of financial responsibility as decided by the programmer (or his/her employer). Had race not been recorded at all by the computer, then it would not have made a "racist" decision (for it would be based entirely on individual merit, rather than any particular connections with others of a specific demographic), and this article would therefore be turned over onto its head and rendered entirely moot.
- pron 12y agoThis is a bit of a philosophical question, and I don't think you should take it too literally, but here's what I think. If race is one of the variables, then due to correlations created by society, race might be learned to be a negative signal, which is racist. If it isn't one of the variables, it might be still inferred. For example, if your purchasing history is an input, there is probably a correlation between purchase of cosmetics for blacks and chances of incarceration, which, in turn affects your chances of loan repayment. The correlation between being black and buying, say, hair-care products for blacks is direct and causal (and completely non racist -- there is no effect on power), so this could be an almost perfect, and neutral, proxy for race, and, in fact, might be the best and simplest correlation the program could come up with.
- yellowapple 12y agoI stopped after the first paragraph, when the premise of the entire article seemed to suddenly base itself upon some random jump from lots of indicators of financial responsibility to race alone without any apparent connection. If the program was designed to base its decisions on financial responsibility indicators, then the only way for it to become "racist" is if race is also recorded as a financial responsibility indicator, which would thus indicate that the programmer, rather than the program, is racist. Basically, the first paragraph effectively creates a strawman; I can only assume that the rest of the article seeks to defeat that particular strawman.