14 ms·

There is no design of AI that can be simultaneously utilitarian, procedurally fair, and representatively fair. I'm going to repeat this again, since many peopl

• mehrdadn 8y ago
  This sounds an awful lot like voting impossibility theorems (Arrow's and its generalizations)... are they related?
  • natalyarostova 8y ago
    I don't think so. This is simpler, it's just stating that an optimization problem on a constrained information set must be weakly dominated by the full information set. Arrow's theorem is a more domain specific and elegant proof regarding political properties that can be simultaneously held.
  • hannasanarion 8y ago
    More like the orthogonality thesis, I think. These different value axes are totally unrelated, and building an algorithm that maximizes one does not necessarily mean that it maximizes the others as well, and this mismatch can have disastrous consequences, a la "The Paperclip Machine"
• sam0x17 8y ago
  But it is possible to make one that is representatively fair first and foremost, with procedurally fair and utilitarianism at 2nd and 3rd priorities.
  • kauffj 8y ago
    Yes, it is. But if you're designing an algorithm that assesses credit risk and you do this, you will lose money compared to a company that does not do this.
    • chrisco255 8y ago
      You might even cause a financial crisis along the lines of the 2008 disaster.
      • toomuchtodo 8y ago
        As long as you're too big to fail and get bailed out, it's someone else's problem. EDIT: I didn't say this was just or right, just the current state.
    • sangnoir 8y ago
      The cost-to-benefit analysis of wheelchair ramps doesn't look great either (compared to not building one). Focusing on profit alone leads to some pretty terrible outcomes for people.
      • swsieber 8y ago
        Taking credit when you can't pay it back doesn't just hurt the bank - it hurts you too. Getting loaned money is not a right, and if you are irresponsible, it's harmful. So if lending more to a certain segment of the population hurts the bank's profit, I'd also expect it to cause more problems for that population segment too.
    • sam0x17 8y ago
      Which is why we need regulation forcing you to do it
    • sam0x17 8y ago
      Frankly, private industry has been lucky up until this point that anything other than your financial history has been allowed to factor into your credit score or credit worthiness __at all__. I wouldn't be surprised if it gets limited to solely financial factors by regulation within the next 10-20 years. In terms of regulations, I think by regulation they shouldn't be allowed to give the algorithms anything but loan amounts, payment schedules, and payment history. Don't give them locations. Don't give them age, name, race, or gender. Don't even give them the names of banks, since this too can be used in a discriminatory way that can be used to imply geographic location and thus race etc. Yes, people with no credit history will get terrible scores, but that's the point of a credit system. It should be based solely off of your merits and what you've actually done, not off of what people in your area/situation tend to do based on statistics. Then it's just statistics creating statistics, and socioeconomic mobility freezes
• dvtrn 8y ago
  I was interested in the slide deck and wanted to hear the author expand on some of his thoughts, found a video of his presentation if anyone else is interested. https://www.youtube.com/watch?v=Zn7oWIhFffs https://www.youtube.com/watch?v=Zn7oWIhFffs
  • Balgair 8y ago
    Wow, this is really good. I hadn't thought of some of these things and he exposes some points that should have been obvious to me. For example: You aren't guaranteed to be able to maximize two functions at the same time.
• shadowbained 8y ago
  The problem here is, we don't even know what "fair" means. What do we mean by fair? Say we want to base fair on group outcomes, in which ways should we divide people into groups in order to perform some sort of balancing? Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loans? Will we need to adjust our credit rating model to bump the credit scores of red haired people slightly? Will we need to agonize over other features fed to our model to ensure that they can't be used as a proxy for red hair? Maybe???? But it seems very dishonest. I think, if anything, the models are probably too true for our tastes as social creatures who evolved to avoid social conflict (being 100% honest with your unruly neighbors can lead to a loss of harmony that is much more damaging than just putting up with some disturbances). Our idea of "unbiasing" the models is more likely actually making them biased in a way that pleases our monkey brains.
  • simion314 8y ago
    Maybe don't split groups on genetics/biology/birth. Also make sure the data you use is not a proxy for the above criteria
    • Iolaum 8y ago
      Easier said than done.
      • simion314 8y ago
        I know, this is why all this algorithms should not be hidden/proprietary. At least we have a chance to spot why/how the computer decided my fate.
        • hnaccount141 8y ago
          The problem is that in many cases (especially with deep learning), "the algorithm" is essentially a black box. There often isn't a straightforward way to determine exactly what the algorithm is actually learning.
• benj111 8y ago
  I wouldn't say its impossible. They may happen to align, you cant force it though.
  • EpicEng 8y ago
    If there exists a scenario where it fails then yes, it is impossible. Just because something works the way you intend it to in some specific case doesn't mean it works in the general case (and the general case is what matters here.)
    • benj111 8y ago
      If it works in a specific case it isn't impossible. I agree that its the general case that matters. Impossible is a very strong word though. How wide do you want to cast the 'general' net? You can still have limited scenarios that are useful. You don't have to model the entire universe.
      • EpicEng 8y ago
        The GP's claim is that yes, it is _literally_ impossible. He cites a source[1]. If you have a disagreement with that theorem then by all means, share. Right now you're just saying "Nah I don't think so", which isn't very convincing. [1]: https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/slides.pdf https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...
        • benj111 8y ago
          That article gives an example of the Scotsman and Englishman. Each are equal, 25% get 1200 on the sat, there is no allocative harm. It is therefore fair, no? If it is _literally impossible_ I must be misunderstanding something. Because I don't see the things he's mentioning as being mutually exclusive.
• tboyd47 8y ago
  > There is no design of AI that can be simultaneously utilitarian, procedurally fair, and representatively fair. AI is just tools of humans. Any system that can be achieved with humans can also be achieved with AI. These AIs take us backwards from our existing human systems in addition to being much harder to correct later. The supermarket theft example in the slides you linked to is a prime example. In what store in America would that kind of display be deemed acceptable? Yeah, you save a few dollars a day on theft, but what kind of goodwill are you spending? Now that AI is here, people think they can get away with things like that because numbers.
  • gambler 8y ago
    > Any system that can be achieved with humans can also be achieved with AI. I hope you meant it the other way around.
    • dragonwriter 8y ago
      I think it's a rejection of “AI can't be racist”—if humans can be racist, the AI systems they employ can also be racist.
    • tboyd47 8y ago
      What I'm trying to say is that inasmuch as you consider the current legal and cultural environment of America to be utilitarian, procedurally fair, and representatively fair, any system that could be built using AI can achieve the same. I don't believe that they will a priori deliver an improvement. But there's no excuse for AI systems to show any discriminatory biases that have already been minimized in human systems via legislation, training, etc. Don't misunderstand me; I'm not saying that discrimination doesn't exist or that we live in a society with no biases. I'm saying there is no valid reason why AI should take us backward in that regard. Computers are extensions of the will of man. Our AI researchers and developers should ask themselves what kind of society they want to create. Whether they choose to turn a blind eye to discrimination and bias, or choose to actively combat it, then that will determine the result. Civil rights didn't just happen; there were people fighting for it actively. The same has to occur in the digital world if the same result is desired.
• deleted 8y ago
  [deleted]
• tomp 8y ago
  More to the point, “procedural fairness” or “equality of opportunity” (if I understand it correctly) is directly opposed to “representative fairness” or “equality of outcome”. In many situations, prioritising one will compromise the other. As an example, Harvard admission of Asian students. Perspectives differ on which one is more important. Personally, I’m absolutely for the former, but it increasingly seems I’m a part of the minority (or the silent majority).
  • 50656E6973 8y ago
    Equality of outcome seems like a mechanistic concept of the industrial age of standardized mass production. Equality of opportunity seems much more organic/humanistic because it allows for the natural variation in people's values and personality
    • SolaceQuantum 8y ago
      I am also interested in equality of opportunity- but how do I measure that everyone has an equal opportunity to achieve something? How do I measure the opportunity to consume nutritious food? The opportunity for access to the same healthcare?
      • yourbandsucks 8y ago
        It goes deeper than that if you go into cultural biases prevalent in poor (regardless of race) communities. Cigarettes, scratch-offs, crappy food.. what counts as opportunity? Is being raised a certain way tantamount to having 'less opportunity' to make better micro-decisions?
    • rtkwe 8y ago
      The problem is equality of opportunity is it's invariably applied in a way that ignores some very racist and uneven starting conditions and the effects of the past. Consider as a small example how schools are funded in the US, which is largely by local property taxes, so poorer places have poorer schools. These areas are heavily skewed towards minority populations because of some extremely racist issues in our past like redlining which kept blacks and other minorities out of the housing boom which drove a lot of middle class wealth growth through the 60s to 00s. So we build on this cycle generation after generation where kids in poor areas go to poorly funded schools and experience poor educational outcomes. Even if we solve the funding problem it's still not actually equal opportunity because the poorer kids don't get the same support outside of school due to parents working longer hours at lower paying jobs because they got a bad education or living in a crummy house with lead paint (because that's all your parents could afford) or living in a town with contaminated water (Flint is just the one to grab headlines cities all over the country have abysmally high lead levels). Even if you want to blame community culture for some of these issues the question becomes where do you think that culture comes from, perhaps just maybe it comes from decades of lack of opportunity and systemic abuse? How do you possibly hope to close the gap in our system where wealth almost invariably builds on wealth? tl;dr: Equality of outcome doesn't mean everyone has to have the exact same exact life, it's just acknowledging our opportunities are directly tied to the outcomes of the generation before us.
• jgord 8y ago
  yes, and yet "perfect is the enemy of good" ... I think the article encompasses your objections, to paraphrase : "algorithms can never be perfectly 'fair', because logic itself can never be perfectly fair" Can we even define 'fair', perhaps we just know unfair when we see it. We seem to be seeing more of it - poverty, homelessness, racism, fact/science-denial, gerrymandering, police violence, gun violence, etc. Yet, we should still strive for fairness. There is a lot of ground between where we are now and 'perfect' - and this is AOCs point, essentially - our system of [self?]-rule is palpably unfair for most people, yet we are busily encoding more of that unfairness into algorithms with little oversight. AOC [ and Bernie, and others ] want us to seek a better implementation of society, and encode that in our algorithms and laws - fairer voting laws, fairer district maps, fairer pay, fairer penalties. Or the converse - encode more fairness, as a means to realize it. For example, it may be a hard math problem to design optimally fair district boundaries, but its noncontroversial to suggest we can reduce gerrymandering significantly. One problem seems to be that most people equate socialism with communism [ 'evil' ], when in fact you are always picking from mix of Democracy, Capitalism, Socialism, Oligarchy etc. After Bernies campaign, and a couple years into the Trump experiment, people are now starting to listen to these old but useful ideas. Ultimately if we don't share any wealth with the poor we will have a bloody revolution, and if we don't reduce Carbon emissions then no wall will keep out the displaced millions of climate refugees. A bit more Socialism in our Capitalist/Democratic/Oligarchic/Socialist mix, is sorely needed right about now.
• compsciphd 8y ago
  new CAP theorem? PUR? RUP?
• ForrestN 8y ago
  Thank you very much for sharing this perspective. I think it probably does a good job standing in for how a lot of people think about these things. This is a difficult area of discourse, and I know people feel strongly about it. Obviously, the lecture you linked to makes many more distinctions. I'm going to try to simplify one of the distinctions to make it easier to talk about: that there is a tension between "procedurally fair" outcomes and "representatively fair" outcomes. In my opinion, I think this idea is wrong, at least in the context of designing AI within contemporary American society. To frame "procedurally fair" and "representatively fair" as opposing value systems is a misunderstanding of the best arguments for a "representatively fair." Let's use a common analogy: a race. This distinction imagines a situation where one race puts a bunch of runners at the starting line, tells them to run, and declares a winner based on who crosses the finish line first. That race is "procedurally fair" because all the participants raced under the same conditions. Then, there's a "representationally fair" race, where runners from "protected groups" are allowed to start the race from the middle of the track. I think there are two ways that this isn't right that come from a common misperception, that the only relevant time to think about in the context of these decisions is the present. (I also want to mention that there's a lot of heavy-handed language use in this presentation that reveals the author's perspective. Algorithms tilted against black people "reveal" disparities, First, it's important to have a more accurate perspective of the past. In most cases where "representationally fair"-type solutions are used, there is a historic reason why the "protected group" can't run as fast as the other group. If people in Hyderabad are more likely to defraud a micro-lender, that's not a spontaneous result of inherent differences between Hyderabad-type humans and other humans. A quick google search tells me that as of 2017 Hyderabad has the second largest poor populace in India. So, in the race to a successful economic outcome from birth, the so-called "procedurally fair" solution actually means taking the Hyderabad runners as infants very far back from the starting line. All else being procedurally equal, when the starting gun of "applying for a micro-loan" goes off, the Mumbaikars and Delhiites are already far ahead of the Hyderabadis. Once we have a more accurate perspective that includes the past, in order to make the race procedurally fair we have to move everyone into place at a common starting line, which will inevitably mean helping the Hyderabadis forward. Second, it's important to acknowledge the effect of biased algorithms on the future. Not only will the unfair, pseudo-"procedurally fair" approach unjustly disadvantage some runners in this race, future races are calibrated according to achievement in past races. If you win one race, you get a head start in the next one. Micro-lenders who refuse to lend to Hyderabadis will exacerbate the relative poverty situation that the algorithm is picking up on. It's easier to pretend to yourself that you're making an algorithm that peeks into the world, makes an objective judgment, and then pops back out of the world. But in fact, people designing algorithms have a responsibility for the outcomes of their algorithms. If an unfair situation exists (for example, that just by being born in Hyderabad and not Delhi, any given person will start life with less economic power), your algorithm's consequences will either be helping to fix that unfair situation, or making it worse. So, if your algorithm punishes Hyderabadi applicants for being poor, it is both unfair in the simplest sense once you account for where the applicants started and will increase the unfairness of any future round of applications. Put another way, what we're talking about at a high level is values. In one libertarian version of society, the purpose of the algorithm is to maximize profit for Simpl. That society values maximizing profit, and enshrines "shareholder value" as the centerpiece of its ethics. In AOC's socialist version of society, justice, equality, and eliminating poverty are valued highest. This is the broader point she's making about algorithms, that they reflect the values of the people who make them. Given the power that these algorithms have, in our example to lift people out of poverty or to deepen economic inequality, society and not just technologists (not the most diverse group in countless ways) should lead the decisions about what should be valued. We're seeing more and more the consequences of a techno-libertarian approach, and I'm curious what we'll see as consequences if a more socialist approach wins for a while. I value fairness over shareholder value, so the prospect excites me. But I'm sure there will be lots of unintended consequences in such a system too (the over-cited "asians applying to elite colleges" example being a good case of an outcome that's not easily justified on its own. I think elite college admissions are more fundamentally broken, but that's another thread!)
• axiom92 8y ago
  Reminds me of: https://en.wikipedia.org/wiki/Liberal_paradox https://en.wikipedia.org/wiki/Liberal_paradox
• anon4justice 8y ago
  The author of these slides is a known racist. He was banned from HN for posting racist comments: https://news.ycombinator.com/item?id=14115498 https://news.ycombinator.com/item?id=14115498 His slides reveal his biases. Slide 41 cites a graph taken from a white supremacist twitter account (if it's not a Russian troll). It's possible that AOC made a couple of minor computer science mistakes. But it's very clear that she's morally right and all the dissembling and racism apologia here is pretty sickening.