11 ms·

What does it mean for an algorithm to be fair?

83 points by maroonblazer 11y ago

lmm 11y ago
I wrote something on the same lines a couple of years ago: http://m50d.github.io/2013/11/21/in-the-future-everyone.html http://m50d.github.io/2013/11/21/in-the-future-everyone.html . Interested to see where this is going.
theVirginian 11y ago
I've long said that an ethics class should be a required "related course" (or at least be encouraged) for various Engineering and Business majors, not because it teaches you how to be a good person but to enhance the student's ability to clarify and explain their reasoning about ethical decisions they will inevitably encounter.
- stcredzero 11y ago
  The discussed situations are analogous to the routing of the interstates through US cities. Often, they just so happened to separate black communities from white ones. (Off-topic, but your username is the name of an old western as well as the name of one of my favorite Irish reels.)
- qzcx 11y ago
  It was at my college. Both for EE/Comp. Eng. and CS.
- navait 11y ago
  Almost all engineering accreditation organizations require an ethics class. However, most students and universities do not treat the class seriously. I took an honors course run by a philosophy professor aimed at general STEM students. We learned first about the different ideas of ethics, such as deontology and utiltarianism. We then applied these systems to various ethical issues in engineering, discussed books about STEM ethical problems, and were required to write a 15 page paper about a current ethics debate, discussing what perspectives had been taken on it(mine, for example was autonomous military drones). However, I then TAd for a non-honors ethics class for CS students, run by a CS adjunct who had so much work teaching she could not focus on teaching any of her classes well. Most of it was reading from slides, multiple choice tests, a 2 page essay of what one thought about an ethical topic, and no discussion of what ethics is, or why something might be unethical. With the exception of the class I took, which required membership in a specific honors college, and only had 20 students a semester all STEM ethics classes were taught that way. It's not enough to have a course. You have to take it seriously.
murbard2 11y ago
Notice how the article jumps from: "these algorithms are trained on historical data, and poor uneducated people (often racial minorities) have a historical trend of being more likely to succumb to predatory loan advertisements" to "Even if this is the most common question being asked on Google [..] this shows that algorithms do in fact encode our prejudice" It's hard to make the case for prejudice when the example involves historical data, so the author weasels in an unrelated example to make the case that there is prejudice. So which is it? There are correlates to credit worthiness which are indeed illegal, allegedly to correct bias. But the legal arguments have moved away from the concept of bias and now squarely focus on impact. See the recent Supreme Court decision on disparate impact for instance. This seems to be the author's point of view to: "Why is this ignorant? Because of the well-known fact that removing explicit racial features from data does not eliminate an algorithm’s ability to learn race. If racial features disproportionately correlate with crime (as they do in the US), then an algorithm which learns race is actually doing exactly what it is designed to do!" So for the author, anything that affects one ethnic group differently is racist, whether or not it is born by the data. Prejudice or bias are irrelevant to his definition. But then why focus on race? No scoring algorithm is perfect, and some people are always going to be shortchanged. So why not just declare by fiat: let's ignore people's height, it's height-ist. Height correlates to income and to credit worthiness. Even if you remove height from the data, your algorithm will end up picking correlates which can become a proxy to height... and the same will be true with virtually any attribute you can think of. So why race?
- theseatoms 11y ago
  There are two separate issues being discussed at once: 1) baseless prejudice, expressed en masse, and therefore gaining algorithmic notability, and 2) empirically backed, statistically defensible stereotypes. This second issue seems harder to resolve, because it will require lawmakers to clearly articulate a "bright line" law in the language of the algorithm itself. As the article acknowledges, algorithms can "learn" race, even if race is not an explicitly defined explanatory variable. Looking forward to part two. Edit: Why race? Because it's an obvious and easy example. I believe the author would be on board with your height example.
infinity0 11y ago
Looking forward to the next post, and glad that the author is working on this problem from a technical perspective! These issues are becoming highly important to society, but vague sociological and political accusations can be made from any viewpoint. Although they can help guide and inspire our research, it's unsustainable to depend on such imprecise concepts in the long run. Objective and neutral precise mathematical knowledge, that directly addresses these social issues instead of waving it away as technically irrelevant, is the only way to make progress in this area.
- dllthomas 11y ago
  I don't entirely disagree, but I urge caution. When we're designing an algorithm, there are things we care about. We try to capture these intuitions with imprecise labels like "fair". Trying then to capture these labels in "precise mathematical knowledge" still leaves room for the mathematical formulation to disagree with the intuition as well as the intuition to itself be faulty in terms of tending to lead to the outcomes we want. In either case we could easily wind up unambiguously and effectively optimizing for the wrong thing.
cousin_it 11y ago
I think mentioning race makes the topic unnecessarily politicized. There are much simpler cases where we don't want to use machine learning algorithms for maximum efficiency gains. For example, take health insurance. No matter if it's voluntary or mandatory, at root it's a system where healthy people pay for sick people. If you're healthy today, you still pay because you might get sick tomorrow. But if sickness were 100% knowable in advance, then healthy people would have no incentive to participate, or even to set up such a system in the first place! The system only works due to a lack of knowledge, and advanced machine learning algorithms could easily break it. The root of the problem is that the market-efficient way to redistribute resources is not the way that leads to the most happiness. (The technical term is "diminishing marginal utility of money".) We need to give free stuff to those who are statistically less likely to win it in competition. If our machine learning algorithms disagree, that just means they're optimizing for the wrong criteria.
- blfr 11y ago
  If we "need to give free stuff to those who are statistically less likely to win in it in competition," we should do it openly, as a transfer, rather than rig every algorithm and, even worse, sabotage our medical knowledge. I certainly want to know about every illness in advance, even if it means abolishing the insurance system.
- bradleyjg 11y ago
  If there are intentional cross subsidies than it isn't insurance. The concept behind insurance is to spread risk, not to spread cost. There's value in insurance exactly because there are risks (i.e. the future isn't perfectly predictable). But improving our ability to predict the future is a good thing, not a bad thing. It is true that it would make insurance useless, but insurance is not the only mechanism for subsidizing those in need, in fact it is a pretty terrible mechanism for that, since it only works that way inasmuch as it is imperfect. The root of your argument seems to be that we need to trick people into helping other people. That seems to be an ultimately futile strategy. Better to really convince them.
  - cousin_it 11y ago
jart 11y ago
Impartial algorithms fail to confirm the biases of those with political agendas.
- te_chris 11y ago
  Reality is political though.
  - valarauca1 11y ago
    Reality is indifferent. We've made it political.
    - davidgerard 11y ago
      That ... doesn't describe anything about human interaction, which has been politics all the way down for the last million years. This particularly applies given the "reality" that we're talking about here is human interaction and questions as to which groups get what when, i.e. politics.
deleted 11y ago
[deleted]
yarrel 11y ago
Like the question of whether a submarine can swim, the question of whether algorithms can be prejudiced is moot. Algorithms can, without their developers ever intending it, perform in ways functionally equivalent to prejudice. It is reasonable to seek to avoid them doing so.
willis77 11y ago
This problem is tightly coupled with issues of correlation vs causation. In real data, correlation is mostly transitive (there are toy exceptions - https://terrytao.wordpress.com/2014/06/05/when-is-correlation-transitive/ https://terrytao.wordpress.com/2014/06/05/when-is-correlatio..., but they are just that). This means that if you want to predict something, and that something is correlated with some some uncomfortable association, trying to predict the something without the uncomfortable association can leave little residual behind. For example, if hair length is societally taboo for gender prediction and I make an algorithm that uses a "politically correct" determination using XY chromosomes, I have also made an algorithm that correlates with hair length. Moreover, if I try to statistically correct my algorithm so that it does not correlate with hair length, I end up with an algorithm that works on the tiny leftover residual created by people who buck the trend, i.e. one that's much more likely to be wrong. Algorithms find both correlational and causal factors. If 9/10 men are from Mars, and you tell me you're from Mars, it is often via correlation that the algorithm labels you a man. You are not allowed jump to the assumption that, say, the drinking water on Mars is turning people into Men.
- seanflyon 11y ago
  I don't see where causation entered the conversation. Correlation and causation have the same predictive power. If the water on Mars turns 90% of people into men or 90% of the people that drink said water are already men, either way if you tell me you drank the water on Mars I know with a 90% certainty you are a man. Algorithms find both correlation and causal factors which is fine because I want accuracy whether or not it is based on causality.
  - gohrt 11y ago
    > Correlation and causation have the same predictive power. Is only true in a system without feedback. If your response to observations can affect the subjets under study, then acting on the correlation can change the correlation. If you decide to offer everyone in Nairobi a $million to join your super-jumpers space-exploration program, because Nairobi correlates to high jump ability, you will find that low jumpers will flood into Kenya to collect on your offer. Whereas, no matter how much you abuse the fact that gravity make things fall, you aren't going to make gravity stop making things fall.
jbob2000 11y ago
It seems to me even humans don't get fairness right. How can we program a machine to be fair, when we can't even "be fair"?
mjpuser 11y ago
Unless there is some identifier in the data that specifies race and the algorithm discriminates on that, it's racist. If not, then it's not racist. AM I RIGHT?
- logfromblammo 11y ago
  A sufficiently advanced algorithm could assemble a synthetic proxy for race from the other available data if race data are not already included. If you know where someone has their hair cut and styled, and where they attend their religious services, if any, you can probably guess with good confidence how they would self-report their race.
- davidgerard 11y ago
  If a company is found to be interacting with a legally protected group in a discriminatory manner, it really doesn't matter if it's doing so because of an algorithm running on a bureaucracy made of humans, or an algorithm running entirely in a machine.
reader5000 11y ago
What does this have to do with algorithms that is different from age old issues in probability, economics?
netcan 11y ago
In a lot of ways (or by a lot of definitions) prejudice & bas is exactly what these algorithms and (or selection criteria for loans or insurance policies) do. When we ban prejudice we are saying (IMO) two things. (2)It is unfair to the point of being illegal to assume that because one is black (old, transexual..), one is unfit to do a job, attend a university or somesuch. (2) It is harmful to society when such prejudice is widespread. It alienates or devalues a group and perpetuates social features we don't like. I think for insurance, loans and such there is a sort of special allowance because these industries need to be biased and their bias, being cold and calculated is not unfair. They also don't specifically discriminate on racial/gender/etc lines and so they were never part of any particular anti discrimination effort. Insurance companies particularly can work around any specific restrictions by substituting banned variables for allowable ones. By hiring one person and not another, you are always discriminating. This kind is clearly considered in the "fair" range since it's directly relevant to the thing you are selecting for. Which kind of discrimination are insurance companies doing? Anyway, algorithims are far more commonly used today so the "problem" is bigger. Also, I think they are the sort of leak in the anti-discimiation logic. I mean there might not be any political campaigns dedicated to the right of the 22-30ag.1-2yrsedu.PSTC.QXRT statistical cluster, but that cluster may is discriminated against. The discrimination is impactful. It may mean they pay higher interest or insurance fees. They may be denied a lease. Non trivial impacts. Is it fair? As the title here points out, biases & prejudices seems like a nonsensical vocabulary to use when it is based on "facts." This is all tricky stuff. Paypal identified me at some point as "high risk." They have frozen €35 and they have reversed transactions made by me. Not uncommon. It happened because I happened to share some characteristics with some fraudsters. They aren't race, but they are not any different (from a common sense morality perspective) then being the same race as a fraudster. I'm not sure where I'm going with this. I don't really understand it, but I don't think the relationship to legally prohibited prejudice is nonsensical.
- tomwalker 11y ago
  so you would be happy with your race paying a higher interest rate for loans if it is calculated as optimal?
  - 11y ago
codys 11y ago
I clicked expecting an actual discussion of "algorithm fairness" in the sense of how a scheduler can be fair (what that means seems to elude me still). Instead of talking about "algorithm fairness", it appears to be talking about "algorithm equality", or alternately "algorithm legal-non-descrimination". I'd expect one could just avoid feeding an algorithm data about enumerated protected classes (for jurisdictions where such a thing exists) and be done with it. Avoiding non-protected-class data that is correlated with protected-classes seems almost contradictory: you'd need to feed the algorithm data about protected classes so it could determine the correlation, and then apply the inverse correlation to to the protected classes. I'm not even sure such a thing is possible to do correctly, though I suppose what I've described is simply a techno implementation of affirmative action.
thanatropism 11y ago
Very often these articles ask excellent questions -- but fully 80% of their value is in the prompt-question. On the one hand, this question explodes into the "general purpose AI" literature. Less Wrong actually originated from a smaller community of people asking what did it mean for AI to be "friendly" to humans, very abstractly. On the other hand, there's "good citizenship" desiderata. Maybe Hubert Dreyfus is correct and there can't be general AI, but the fact of the matter is that our actor-networks (as in Latour) are already crowded with algorithms and machines. What does it mean for a machine to be accountable, dependable, reliable, and yes, fair? An image classifier should be invertible -- if we're consistently getting that black men look like apes, we should know what the machine thinks an ultra-archetype ape is. Interestingly, this has been the focus of a lot of research since I took that NN class in college back in 2001 -- we're able to visualize hidden layers now. A credit scorer should be consistent with some wider sense of credit worthiness, sure. It should be possible to increment your credit score, at the margin, with marginal changes to (real, important) inputs, rather than have it place you in a brittle "unreliable" type. You should be able to dig yourself out of debt. All of these are kind of domain specific, but I haven't spent much thought on it. It seems to me that there are some general characteristics (invertibility, smoothness across key controls) that should be identifiable.
- davidgerard 11y ago
  >Less Wrong actually originated from a smaller community of people asking what did it mean for AI to be "friendly" to humans, very abstractly. Only tangentially. It was literally the next stage of Yudkowsky's blogging on Overcoming Bias, which substantially attracted as its initial commenters people from the SL4 mailing list, which dealt with many topics of transhumanist interest, including friendly AI.
karmacondon 11y ago
The bottom line is that a well made algorithm with a sufficiently large [1] data set cannot be fair or unfair. It can only optimize with an estimated accuracy. If the numbers show that poor people are more likely to respond to payday loan advertisements, then that's what the numbers show. If the businesses are truly predatory then they should be regulated. It doesn't make sense to show ads for them to people who are statistically unlikely to respond to them, nor does it make sense to show poor people ads for institutions that are statistically unlikely to grant them loans. Algorithms aren't the problem here, the nature of the businesses is. And in fairness, I notice that when I watch certain tv shows late at night I'm much more likely to see payday loan ads than when I watch during prime time. I've long suspected that this is because the kind of person who is watching "30 Rock" re-runs at 1am is less likely to be employed and more likely to need a payday loan. So if it's fair for a media buyer to select audiences that are most likely to purchase a product, why is it unfair for an algorithm to do it? The article makes the point that algorithms are opaque and people often can't explain exactly why a decision was made. That's fine, as long as the results can be shown to be accurate with out of sample testing. It's good to understand why, but it isn't always necessary. The point of using advanced statistical modelling is to expand our level of capability beyond what our brains can easily perceive. The ability to draw reliable inferences from tens of millions of data points is a very powerful and wonderful thing, quite a leap for evolved monkeys who use lobes of fat to perform calculations. It's new, it's different, and it's definitely worrying. But the sooner we can adapt, the better off we'll be as a society. [1] in this case "large" probably means two or three orders of magnitude larger than you were thinking
current_call 11y ago
If I charge a man extra for car insurance because men on average get in more wrecks, then I am discriminating based on sex. If I charge someone extra for car insurance because he shares dozens of innocuous traits with other people who get in wrecks, I am not discriminating based on sex. If some algorithm can accurately predict the sex of that person by the same data, I am still not discriminating based on sex. If I target blacks with predatory loan advertisements because that demographic is on average more susceptible to those ads, then I am discriminating based on race. If I target an individual because they share dozens of innocuous traits with other people who fall for predatory loan advertisements, I am not discriminating based on race. If some algorithm can accurately predict the race of the individual based on the same data, I am still not discriminating based on race. The software is less biased than a human because it has no concept of race. Even if the data can be used to accurately predict the race of an individual, it does not matter. The program will not spontaneously recognize the concept of race and then discriminate based off of it. If some trait correlates with race, that's because reality is biased, not the math, algorithm, company, or implementer. It is not the responsibility of programmers to make their algorithms less accurate so ideologues can live in a fantasy world.
- ectoplasm 11y ago
  But that's how sexism and racism and any kind of discrimination works in the human mind. There are innocuous traits, like sex and skin color, but also many other traits, and we use them to make predictions about behavior. This is unfair discrimination; fair discrimination looks directly at the behavior of an individual, not some innocuous proxy for the behavior, even if that proxy is right 80% of the time. For example, most people who have an account throwawayXXX on HN are using it temporarily to say something. However, I actually kept one such account for a long time. My username is an innocuous trait, but you're discriminating if you assume my motives and behavior will be like most of the other throwaway accounts, just based on my username. That said, I believe unfair discrimination is unavoidable in life because we can't always wait around to see what an individual's behavior will be before we discriminate fairly. We couldn't function without stereotypes and assumptions. And so, while still unfair because of the potential for unjustified penalties, it's much better to look at many variables than it is to look at one or two.

83 points by maroonblazer 11y ago

lmm 11y ago

I wrote something on the same lines a couple of years ago: http://m50d.github.io/2013/11/21/in-the-future-everyone.html http://m50d.github.io/2013/11/21/in-the-future-everyone.html . Interested to see where this is going.

theVirginian 11y ago

I've long said that an ethics class should be a required "related course" (or at least be encouraged) for various Engineering and Business majors, not because it teaches you how to be a good person but to enhance the student's ability to clarify and explain their reasoning about ethical decisions they will inevitably encounter.

stcredzero 11y ago
The discussed situations are analogous to the routing of the interstates through US cities. Often, they just so happened to separate black communities from white ones. (Off-topic, but your username is the name of an old western as well as the name of one of my favorite Irish reels.)
qzcx 11y ago
It was at my college. Both for EE/Comp. Eng. and CS.
navait 11y ago
Almost all engineering accreditation organizations require an ethics class. However, most students and universities do not treat the class seriously. I took an honors course run by a philosophy professor aimed at general STEM students. We learned first about the different ideas of ethics, such as deontology and utiltarianism. We then applied these systems to various ethical issues in engineering, discussed books about STEM ethical problems, and were required to write a 15 page paper about a current ethics debate, discussing what perspectives had been taken on it(mine, for example was autonomous military drones). However, I then TAd for a non-honors ethics class for CS students, run by a CS adjunct who had so much work teaching she could not focus on teaching any of her classes well. Most of it was reading from slides, multiple choice tests, a 2 page essay of what one thought about an ethical topic, and no discussion of what ethics is, or why something might be unethical. With the exception of the class I took, which required membership in a specific honors college, and only had 20 students a semester all STEM ethics classes were taught that way. It's not enough to have a course. You have to take it seriously.

murbard2 11y ago

Notice how the article jumps from: "these algorithms are trained on historical data, and poor uneducated people (often racial minorities) have a historical trend of being more likely to succumb to predatory loan advertisements" to "Even if this is the most common question being asked on Google [..] this shows that algorithms do in fact encode our prejudice" It's hard to make the case for prejudice when the example involves historical data, so the author weasels in an unrelated example to make the case that there is prejudice. So which is it? There are correlates to credit worthiness which are indeed illegal, allegedly to correct bias. But the legal arguments have moved away from the concept of bias and now squarely focus on impact. See the recent Supreme Court decision on disparate impact for instance. This seems to be the author's point of view to: "Why is this ignorant? Because of the well-known fact that removing explicit racial features from data does not eliminate an algorithm’s ability to learn race. If racial features disproportionately correlate with crime (as they do in the US), then an algorithm which learns race is actually doing exactly what it is designed to do!" So for the author, anything that affects one ethnic group differently is racist, whether or not it is born by the data. Prejudice or bias are irrelevant to his definition. But then why focus on race? No scoring algorithm is perfect, and some people are always going to be shortchanged. So why not just declare by fiat: let's ignore people's height, it's height-ist. Height correlates to income and to credit worthiness. Even if you remove height from the data, your algorithm will end up picking correlates which can become a proxy to height... and the same will be true with virtually any attribute you can think of. So why race?

theseatoms 11y ago
There are two separate issues being discussed at once: 1) baseless prejudice, expressed en masse, and therefore gaining algorithmic notability, and 2) empirically backed, statistically defensible stereotypes. This second issue seems harder to resolve, because it will require lawmakers to clearly articulate a "bright line" law in the language of the algorithm itself. As the article acknowledges, algorithms can "learn" race, even if race is not an explicitly defined explanatory variable. Looking forward to part two. Edit: Why race? Because it's an obvious and easy example. I believe the author would be on board with your height example.

infinity0 11y ago

Looking forward to the next post, and glad that the author is working on this problem from a technical perspective! These issues are becoming highly important to society, but vague sociological and political accusations can be made from any viewpoint. Although they can help guide and inspire our research, it's unsustainable to depend on such imprecise concepts in the long run. Objective and neutral precise mathematical knowledge, that directly addresses these social issues instead of waving it away as technically irrelevant, is the only way to make progress in this area.

dllthomas 11y ago
I don't entirely disagree, but I urge caution. When we're designing an algorithm, there are things we care about. We try to capture these intuitions with imprecise labels like "fair". Trying then to capture these labels in "precise mathematical knowledge" still leaves room for the mathematical formulation to disagree with the intuition as well as the intuition to itself be faulty in terms of tending to lead to the outcomes we want. In either case we could easily wind up unambiguously and effectively optimizing for the wrong thing.

cousin_it 11y ago

I think mentioning race makes the topic unnecessarily politicized. There are much simpler cases where we don't want to use machine learning algorithms for maximum efficiency gains. For example, take health insurance. No matter if it's voluntary or mandatory, at root it's a system where healthy people pay for sick people. If you're healthy today, you still pay because you might get sick tomorrow. But if sickness were 100% knowable in advance, then healthy people would have no incentive to participate, or even to set up such a system in the first place! The system only works due to a lack of knowledge, and advanced machine learning algorithms could easily break it. The root of the problem is that the market-efficient way to redistribute resources is not the way that leads to the most happiness. (The technical term is "diminishing marginal utility of money".) We need to give free stuff to those who are statistically less likely to win it in competition. If our machine learning algorithms disagree, that just means they're optimizing for the wrong criteria.

blfr 11y ago
If we "need to give free stuff to those who are statistically less likely to win in it in competition," we should do it openly, as a transfer, rather than rig every algorithm and, even worse, sabotage our medical knowledge. I certainly want to know about every illness in advance, even if it means abolishing the insurance system.
bradleyjg 11y ago
If there are intentional cross subsidies than it isn't insurance. The concept behind insurance is to spread risk, not to spread cost. There's value in insurance exactly because there are risks (i.e. the future isn't perfectly predictable). But improving our ability to predict the future is a good thing, not a bad thing. It is true that it would make insurance useless, but insurance is not the only mechanism for subsidizing those in need, in fact it is a pretty terrible mechanism for that, since it only works that way inasmuch as it is imperfect. The root of your argument seems to be that we need to trick people into helping other people. That seems to be an ultimately futile strategy. Better to really convince them.
- cousin_it 11y ago

jart 11y ago

Impartial algorithms fail to confirm the biases of those with political agendas.

te_chris 11y ago
Reality is political though.
- valarauca1 11y ago
  Reality is indifferent. We've made it political.
  - davidgerard 11y ago
    That ... doesn't describe anything about human interaction, which has been politics all the way down for the last million years. This particularly applies given the "reality" that we're talking about here is human interaction and questions as to which groups get what when, i.e. politics.

deleted 11y ago

[deleted]

yarrel 11y ago

Like the question of whether a submarine can swim, the question of whether algorithms can be prejudiced is moot. Algorithms can, without their developers ever intending it, perform in ways functionally equivalent to prejudice. It is reasonable to seek to avoid them doing so.

willis77 11y ago

This problem is tightly coupled with issues of correlation vs causation. In real data, correlation is mostly transitive (there are toy exceptions - https://terrytao.wordpress.com/2014/06/05/when-is-correlation-transitive/ https://terrytao.wordpress.com/2014/06/05/when-is-correlatio..., but they are just that). This means that if you want to predict something, and that something is correlated with some some uncomfortable association, trying to predict the something without the uncomfortable association can leave little residual behind. For example, if hair length is societally taboo for gender prediction and I make an algorithm that uses a "politically correct" determination using XY chromosomes, I have also made an algorithm that correlates with hair length. Moreover, if I try to statistically correct my algorithm so that it does not correlate with hair length, I end up with an algorithm that works on the tiny leftover residual created by people who buck the trend, i.e. one that's much more likely to be wrong. Algorithms find both correlational and causal factors. If 9/10 men are from Mars, and you tell me you're from Mars, it is often via correlation that the algorithm labels you a man. You are not allowed jump to the assumption that, say, the drinking water on Mars is turning people into Men.

seanflyon 11y ago
I don't see where causation entered the conversation. Correlation and causation have the same predictive power. If the water on Mars turns 90% of people into men or 90% of the people that drink said water are already men, either way if you tell me you drank the water on Mars I know with a 90% certainty you are a man. Algorithms find both correlation and causal factors which is fine because I want accuracy whether or not it is based on causality.
- gohrt 11y ago
  > Correlation and causation have the same predictive power. Is only true in a system without feedback. If your response to observations can affect the subjets under study, then acting on the correlation can change the correlation. If you decide to offer everyone in Nairobi a $million to join your super-jumpers space-exploration program, because Nairobi correlates to high jump ability, you will find that low jumpers will flood into Kenya to collect on your offer. Whereas, no matter how much you abuse the fact that gravity make things fall, you aren't going to make gravity stop making things fall.

jbob2000 11y ago

It seems to me even humans don't get fairness right. How can we program a machine to be fair, when we can't even "be fair"?

mjpuser 11y ago

Unless there is some identifier in the data that specifies race and the algorithm discriminates on that, it's racist. If not, then it's not racist. AM I RIGHT?

logfromblammo 11y ago
A sufficiently advanced algorithm could assemble a synthetic proxy for race from the other available data if race data are not already included. If you know where someone has their hair cut and styled, and where they attend their religious services, if any, you can probably guess with good confidence how they would self-report their race.
davidgerard 11y ago
If a company is found to be interacting with a legally protected group in a discriminatory manner, it really doesn't matter if it's doing so because of an algorithm running on a bureaucracy made of humans, or an algorithm running entirely in a machine.

reader5000 11y ago

What does this have to do with algorithms that is different from age old issues in probability, economics?

netcan 11y ago

In a lot of ways (or by a lot of definitions) prejudice & bas is exactly what these algorithms and (or selection criteria for loans or insurance policies) do. When we ban prejudice we are saying (IMO) two things. (2)It is unfair to the point of being illegal to assume that because one is black (old, transexual..), one is unfit to do a job, attend a university or somesuch. (2) It is harmful to society when such prejudice is widespread. It alienates or devalues a group and perpetuates social features we don't like. I think for insurance, loans and such there is a sort of special allowance because these industries need to be biased and their bias, being cold and calculated is not unfair. They also don't specifically discriminate on racial/gender/etc lines and so they were never part of any particular anti discrimination effort. Insurance companies particularly can work around any specific restrictions by substituting banned variables for allowable ones. By hiring one person and not another, you are always discriminating. This kind is clearly considered in the "fair" range since it's directly relevant to the thing you are selecting for. Which kind of discrimination are insurance companies doing? Anyway, algorithims are far more commonly used today so the "problem" is bigger. Also, I think they are the sort of leak in the anti-discimiation logic. I mean there might not be any political campaigns dedicated to the right of the 22-30ag.1-2yrsedu.PSTC.QXRT statistical cluster, but that cluster may is discriminated against. The discrimination is impactful. It may mean they pay higher interest or insurance fees. They may be denied a lease. Non trivial impacts. Is it fair? As the title here points out, biases & prejudices seems like a nonsensical vocabulary to use when it is based on "facts." This is all tricky stuff. Paypal identified me at some point as "high risk." They have frozen €35 and they have reversed transactions made by me. Not uncommon. It happened because I happened to share some characteristics with some fraudsters. They aren't race, but they are not any different (from a common sense morality perspective) then being the same race as a fraudster. I'm not sure where I'm going with this. I don't really understand it, but I don't think the relationship to legally prohibited prejudice is nonsensical.

tomwalker 11y ago
so you would be happy with your race paying a higher interest rate for loans if it is calculated as optimal?
- 11y ago

codys 11y ago

I clicked expecting an actual discussion of "algorithm fairness" in the sense of how a scheduler can be fair (what that means seems to elude me still). Instead of talking about "algorithm fairness", it appears to be talking about "algorithm equality", or alternately "algorithm legal-non-descrimination". I'd expect one could just avoid feeding an algorithm data about enumerated protected classes (for jurisdictions where such a thing exists) and be done with it. Avoiding non-protected-class data that is correlated with protected-classes seems almost contradictory: you'd need to feed the algorithm data about protected classes so it could determine the correlation, and then apply the inverse correlation to to the protected classes. I'm not even sure such a thing is possible to do correctly, though I suppose what I've described is simply a techno implementation of affirmative action.

thanatropism 11y ago

Very often these articles ask excellent questions -- but fully 80% of their value is in the prompt-question. On the one hand, this question explodes into the "general purpose AI" literature. Less Wrong actually originated from a smaller community of people asking what did it mean for AI to be "friendly" to humans, very abstractly. On the other hand, there's "good citizenship" desiderata. Maybe Hubert Dreyfus is correct and there can't be general AI, but the fact of the matter is that our actor-networks (as in Latour) are already crowded with algorithms and machines. What does it mean for a machine to be accountable, dependable, reliable, and yes, fair? An image classifier should be invertible -- if we're consistently getting that black men look like apes, we should know what the machine thinks an ultra-archetype ape is. Interestingly, this has been the focus of a lot of research since I took that NN class in college back in 2001 -- we're able to visualize hidden layers now. A credit scorer should be consistent with some wider sense of credit worthiness, sure. It should be possible to increment your credit score, at the margin, with marginal changes to (real, important) inputs, rather than have it place you in a brittle "unreliable" type. You should be able to dig yourself out of debt. All of these are kind of domain specific, but I haven't spent much thought on it. It seems to me that there are some general characteristics (invertibility, smoothness across key controls) that should be identifiable.

davidgerard 11y ago
>Less Wrong actually originated from a smaller community of people asking what did it mean for AI to be "friendly" to humans, very abstractly. Only tangentially. It was literally the next stage of Yudkowsky's blogging on Overcoming Bias, which substantially attracted as its initial commenters people from the SL4 mailing list, which dealt with many topics of transhumanist interest, including friendly AI.

karmacondon 11y ago

The bottom line is that a well made algorithm with a sufficiently large [1] data set cannot be fair or unfair. It can only optimize with an estimated accuracy. If the numbers show that poor people are more likely to respond to payday loan advertisements, then that's what the numbers show. If the businesses are truly predatory then they should be regulated. It doesn't make sense to show ads for them to people who are statistically unlikely to respond to them, nor does it make sense to show poor people ads for institutions that are statistically unlikely to grant them loans. Algorithms aren't the problem here, the nature of the businesses is. And in fairness, I notice that when I watch certain tv shows late at night I'm much more likely to see payday loan ads than when I watch during prime time. I've long suspected that this is because the kind of person who is watching "30 Rock" re-runs at 1am is less likely to be employed and more likely to need a payday loan. So if it's fair for a media buyer to select audiences that are most likely to purchase a product, why is it unfair for an algorithm to do it? The article makes the point that algorithms are opaque and people often can't explain exactly why a decision was made. That's fine, as long as the results can be shown to be accurate with out of sample testing. It's good to understand why, but it isn't always necessary. The point of using advanced statistical modelling is to expand our level of capability beyond what our brains can easily perceive. The ability to draw reliable inferences from tens of millions of data points is a very powerful and wonderful thing, quite a leap for evolved monkeys who use lobes of fat to perform calculations. It's new, it's different, and it's definitely worrying. But the sooner we can adapt, the better off we'll be as a society. [1] in this case "large" probably means two or three orders of magnitude larger than you were thinking

current_call 11y ago

If I charge a man extra for car insurance because men on average get in more wrecks, then I am discriminating based on sex. If I charge someone extra for car insurance because he shares dozens of innocuous traits with other people who get in wrecks, I am not discriminating based on sex. If some algorithm can accurately predict the sex of that person by the same data, I am still not discriminating based on sex. If I target blacks with predatory loan advertisements because that demographic is on average more susceptible to those ads, then I am discriminating based on race. If I target an individual because they share dozens of innocuous traits with other people who fall for predatory loan advertisements, I am not discriminating based on race. If some algorithm can accurately predict the race of the individual based on the same data, I am still not discriminating based on race. The software is less biased than a human because it has no concept of race. Even if the data can be used to accurately predict the race of an individual, it does not matter. The program will not spontaneously recognize the concept of race and then discriminate based off of it. If some trait correlates with race, that's because reality is biased, not the math, algorithm, company, or implementer. It is not the responsibility of programmers to make their algorithms less accurate so ideologues can live in a fantasy world.

ectoplasm 11y ago
But that's how sexism and racism and any kind of discrimination works in the human mind. There are innocuous traits, like sex and skin color, but also many other traits, and we use them to make predictions about behavior. This is unfair discrimination; fair discrimination looks directly at the behavior of an individual, not some innocuous proxy for the behavior, even if that proxy is right 80% of the time. For example, most people who have an account throwawayXXX on HN are using it temporarily to say something. However, I actually kept one such account for a long time. My username is an innocuous trait, but you're discriminating if you assume my motives and behavior will be like most of the other throwaway accounts, just based on my username. That said, I believe unfair discrimination is unavoidable in life because we can't always wait around to see what an individual's behavior will be before we discriminate fairly. We couldn't function without stereotypes and assumptions. And so, while still unfair because of the potential for unjustified penalties, it's much better to look at many variables than it is to look at one or two.