5 ms·

The whole point is that biases are inherent in most models, no matter who the builders are.

by loopz 6y ago

Guest42 6y ago
What types of models and what types of bias are you referring to?
- deleted 6y ago
  [deleted]
- slg 6y ago
  Any model created using biased data will inherently mirror that bias unless there are active steps made to counteract this effect. For example, basically any financial evaluations of US citizens will likely result in an inherent bias against Black people due to institutional biases such as redlining that have long lasting socioeconomic and demographic repercussions. This might mean that something as simple as incorporating the zip code of a home into a mortgage pricing AI can end up with a racial bias.
  - Guest42 6y ago
    I agree that the data will shift the results of the model and that some predictors can be proxies for others that are protected, but the claim was that the models themselves were biased (rather globally) and was curious whether that was shown and if so how.
    - slg 6y ago
      I'm not OP, but I don't think the claim was specifically that models themselves are biased. It is that models are inherently biased because the data they are based on is biased. That might sound the same, but there is a nuanced difference. If you are able to strip the bias from the data, the models will work fine. The problem is the data and not the models.
      - refenestrator 6y ago
        At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? Let's say you have a racially-neutral observation of lower income, maybe disability status or a criminal rap in the past. That looks like a bad bet for a loan regardless of color, it just so happens that our society's created a statistical imbalance in those metrics.
        klyrs 6y ago
        > At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? AI shouldn't take the blame. Blame the folks collecting biased data, or those making biased decisions encoded in the data. The data is known to be tainted. Blame those using that data to train models, and sell/rent/apply those models for profit. Blame the researchers who know, or should know, better but make breathless claims about how their AI can be used without regard for the impact if people follow their advice
        refenestrator 6y ago
        But the underlying situation is biased. The data could be both accurate and unfair. Is it, just don't do data, same interest rate for everyone, no denials and amortize defaults across higher rates for lower-risk borrowers? You'd need a law, the first bank to do that would be crushed by other banks that can attract the lower-risk borrowers with lower rates.
        klyrs 6y ago
        > You'd need a law, the first bank to do that would be crushed by other banks that can attract the lower-risk borrowers with lower rates. Curious. Is your answer to the title "we need laws to regulate AI?" I'm not specifically agreeing or disagreeing.
        refenestrator 6y ago
    - klyrs 6y ago
      Do you think it's reasonable to expect a model, trained on biased data, to not display a similar degree of bias?
      - deleted 6y ago
        [deleted]
  - gedy 6y ago
    "Any model created using biased data" from what I've seen from Ghebru, et al means: real day-to-day language that people actually use and vast majority have no issue with. Phrases like "woman doctor" "both genders", etc. To remove this discourse from models ironically means applying a strong bias imho.
  - AnHonestComment 6y ago
    Your post sounds really racist. You don’t like the data, so you accuse the model of being biased (even though it’s accurate) and insist that you get to apply your own personal biases based on how you think races should look and interact? Racial correlation isn’t racial bias. In what way is viewing the world through a lens of racism useful?
  - PeterisP 6y ago
    In many unbalanced situations you simply can't have an unbiased decision that's fair across all measures, no matter if the decision is done by a model, a human or a deity; you have to trade off between different types of unfairness. Equality of opportunity for individuals will result in unequal results for groups; equality of outcomes requires unequal opportunities if the historical circumstances have resulted in socioeconomic unequalities. In your financial evaluations example, many biases and disadvantages would remain even if you solely reduce the decision to relevant financial facts for a specific individual, because a poorer individual with lower socioeconomic status and less opportunities actually has a higher risk of non-payment, and a disadvantaged group will have disproportionally more such individuals. Should we accept that? Should we require the other groups to subsidize their non-payment? Both options are unfair in some aspect and fair in another, you can't have your cake and eat it too, and it's not the fault of the model you use - the only difference with a human is that they can better hide the factors they use, lie about the influences (perhaps also lie to themselves) and rationalize/invent factors to justify their decision. For this topic, perhaps this talk "AI Ethics, Impossibility Theorems and Tradeoffs" https://www.youtube.com/watch?v=Zn7oWIhFffs https://www.youtube.com/watch?v=Zn7oWIhFffs or its slides https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/slides.pdf https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl... might be interesting for you, it has some flaws but is a decent exploration of the problem space.
  - tryonenow 6y ago
    IMO the discussion is occurring under the wrong framing. The real question is what to do with discriminatory data that is reflective of reality. If certain populations are more likely to default on loans, for example, it is disingenuous to simply claim that the data is biased - the question is whether we as a society are willing to allow usage of that data. And then the question becomes where to draw the line, since the entire purpose of such data for e.g. insurance firms is to discriminate among risk sources. So you're asking insurers (and other industries that will depend on ML) to adopt suboptimal business practices in the name of egalitarianism. And you can't simply remove certain data points that correlate with these specific risks to alleviate bias - you're merely introducing your own bias which is more connected to the reality you wish to see than that which exists, which, again, is not operationally efficient. If a group of people is more likely to default on loans then any statistic which is correlated with said group of people will also be correlated with loan defaulting. If you want e.g. the black community to have better access to loans, attacking ML risk models as institutionally biased is simply unscientific and completely political if African Americans are more likely to default on loans.
dvt 6y ago
This kind of statement, by its very nature, is vacuous and means nothing (and is most likely made by a non-expert). But on the surface of it seems deep and insightful. Note that it uses weasel words like "biases" and "models" which can actually mean a zillion things.
- loopz 6y ago
  It means exactly what it means. No more, no less. sig above got it, so it's not impossible to imagine. No need to be an expert, that shouldn't be necessary. Sorry if I used difficult words. The point is to think it through for yourself anyways, and not depend blindly on authority.

The whole point is that biases are inherent in most models, no matter who the builders are.

by loopz 6y ago

The whole point is that biases are inherent in most models, no matter who the builders are.

Guest42 6y ago

What types of models and what types of bias are you referring to?

deleted 6y ago
[deleted]
slg 6y ago
Any model created using biased data will inherently mirror that bias unless there are active steps made to counteract this effect. For example, basically any financial evaluations of US citizens will likely result in an inherent bias against Black people due to institutional biases such as redlining that have long lasting socioeconomic and demographic repercussions. This might mean that something as simple as incorporating the zip code of a home into a mortgage pricing AI can end up with a racial bias.
- Guest42 6y ago
  I agree that the data will shift the results of the model and that some predictors can be proxies for others that are protected, but the claim was that the models themselves were biased (rather globally) and was curious whether that was shown and if so how.
  - slg 6y ago
    I'm not OP, but I don't think the claim was specifically that models themselves are biased. It is that models are inherently biased because the data they are based on is biased. That might sound the same, but there is a nuanced difference. If you are able to strip the bias from the data, the models will work fine. The problem is the data and not the models.
    - refenestrator 6y ago
      At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? Let's say you have a racially-neutral observation of lower income, maybe disability status or a criminal rap in the past. That looks like a bad bet for a loan regardless of color, it just so happens that our society's created a statistical imbalance in those metrics.
      - klyrs 6y ago
        > At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? AI shouldn't take the blame. Blame the folks collecting biased data, or those making biased decisions encoded in the data. The data is known to be tainted. Blame those using that data to train models, and sell/rent/apply those models for profit. Blame the researchers who know, or should know, better but make breathless claims about how their AI can be used without regard for the impact if people follow their advice
        refenestrator 6y ago
        But the underlying situation is biased. The data could be both accurate and unfair. Is it, just don't do data, same interest rate for everyone, no denials and amortize defaults across higher rates for lower-risk borrowers? You'd need a law, the first bank to do that would be crushed by other banks that can attract the lower-risk borrowers with lower rates.
        klyrs 6y ago
        > You'd need a law, the first bank to do that would be crushed by other banks that can attract the lower-risk borrowers with lower rates. Curious. Is your answer to the title "we need laws to regulate AI?" I'm not specifically agreeing or disagreeing.
        refenestrator 6y ago
  - klyrs 6y ago
    Do you think it's reasonable to expect a model, trained on biased data, to not display a similar degree of bias?
    - deleted 6y ago
      [deleted]
- gedy 6y ago
  "Any model created using biased data" from what I've seen from Ghebru, et al means: real day-to-day language that people actually use and vast majority have no issue with. Phrases like "woman doctor" "both genders", etc. To remove this discourse from models ironically means applying a strong bias imho.
- AnHonestComment 6y ago
  Your post sounds really racist. You don’t like the data, so you accuse the model of being biased (even though it’s accurate) and insist that you get to apply your own personal biases based on how you think races should look and interact? Racial correlation isn’t racial bias. In what way is viewing the world through a lens of racism useful?
- PeterisP 6y ago
  In many unbalanced situations you simply can't have an unbiased decision that's fair across all measures, no matter if the decision is done by a model, a human or a deity; you have to trade off between different types of unfairness. Equality of opportunity for individuals will result in unequal results for groups; equality of outcomes requires unequal opportunities if the historical circumstances have resulted in socioeconomic unequalities. In your financial evaluations example, many biases and disadvantages would remain even if you solely reduce the decision to relevant financial facts for a specific individual, because a poorer individual with lower socioeconomic status and less opportunities actually has a higher risk of non-payment, and a disadvantaged group will have disproportionally more such individuals. Should we accept that? Should we require the other groups to subsidize their non-payment? Both options are unfair in some aspect and fair in another, you can't have your cake and eat it too, and it's not the fault of the model you use - the only difference with a human is that they can better hide the factors they use, lie about the influences (perhaps also lie to themselves) and rationalize/invent factors to justify their decision. For this topic, perhaps this talk "AI Ethics, Impossibility Theorems and Tradeoffs" https://www.youtube.com/watch?v=Zn7oWIhFffs https://www.youtube.com/watch?v=Zn7oWIhFffs or its slides https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/slides.pdf https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl... might be interesting for you, it has some flaws but is a decent exploration of the problem space.
- tryonenow 6y ago
  IMO the discussion is occurring under the wrong framing. The real question is what to do with discriminatory data that is reflective of reality. If certain populations are more likely to default on loans, for example, it is disingenuous to simply claim that the data is biased - the question is whether we as a society are willing to allow usage of that data. And then the question becomes where to draw the line, since the entire purpose of such data for e.g. insurance firms is to discriminate among risk sources. So you're asking insurers (and other industries that will depend on ML) to adopt suboptimal business practices in the name of egalitarianism. And you can't simply remove certain data points that correlate with these specific risks to alleviate bias - you're merely introducing your own bias which is more connected to the reality you wish to see than that which exists, which, again, is not operationally efficient. If a group of people is more likely to default on loans then any statistic which is correlated with said group of people will also be correlated with loan defaulting. If you want e.g. the black community to have better access to loans, attacking ML risk models as institutionally biased is simply unscientific and completely political if African Americans are more likely to default on loans.

dvt 6y ago

This kind of statement, by its very nature, is vacuous and means nothing (and is most likely made by a non-expert). But on the surface of it seems deep and insightful. Note that it uses weasel words like "biases" and "models" which can actually mean a zillion things.

loopz 6y ago
It means exactly what it means. No more, no less. sig above got it, so it's not impossible to imagine. No need to be an expert, that shouldn't be necessary. Sorry if I used difficult words. The point is to think it through for yourself anyways, and not depend blindly on authority.