5 ms·

I really don't get it. Nobody at an AI conference is alarmed at what they see in terms of AI research progress. Most well-educated people are alarmed about soci

by timkam 6y ago

I really don't get it. Nobody at an AI conference is alarmed at what they see in terms of AI research progress. Most well-educated people are alarmed about societal issues that _are also relevant to the application of automated decision-making and machine learning_. It's an applied research opportunity to investigate the issues (and as such, it is hyped because it is easy to "sell" to get funding); but it's not that AI research brings "bias" that didn't exist in society before. It's the same as for a rule-based system, or let's say, even a programming language: for sure, it can automate and hence exacerbate "unfair" decisions, but the problem is not in the specification, but in the specifying person/human. However, having a clear specification makes it easier to test/audit/verify and hence we (as a society) should do so. We don't need to burn down academic prototypes just because they make some naive assumptions about the application domain, though.

loopz 6y ago
The whole point is that biases are inherent in most models, no matter who the builders are.
- Guest42 6y ago
  What types of models and what types of bias are you referring to?
  - deleted 6y ago
    [deleted]
  - slg 6y ago
    Any model created using biased data will inherently mirror that bias unless there are active steps made to counteract this effect. For example, basically any financial evaluations of US citizens will likely result in an inherent bias against Black people due to institutional biases such as redlining that have long lasting socioeconomic and demographic repercussions. This might mean that something as simple as incorporating the zip code of a home into a mortgage pricing AI can end up with a racial bias.
    - Guest42 6y ago
      I agree that the data will shift the results of the model and that some predictors can be proxies for others that are protected, but the claim was that the models themselves were biased (rather globally) and was curious whether that was shown and if so how.
      - slg 6y ago
        I'm not OP, but I don't think the claim was specifically that models themselves are biased. It is that models are inherently biased because the data they are based on is biased. That might sound the same, but there is a nuanced difference. If you are able to strip the bias from the data, the models will work fine. The problem is the data and not the models.
        refenestrator 6y ago
        At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? Let's say you have a racially-neutral observation of lower income, maybe disability status or a criminal rap in the past. That looks like a bad bet for a loan regardless of color, it just so happens that our society's created a statistical imbalance in those metrics.
        klyrs 6y ago
        > At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? AI shouldn't take the blame. Blame the folks collecting biased data, or those making biased decisions encoded in the data. The data is known to be tainted. Blame those using that data to train models, and sell/rent/apply those models for profit. Blame the researchers who know, or should know, better but make breathless claims about how their AI can be used without regard for the impact if people follow their advice
        refenestrator 6y ago
- dvt 6y ago
  This kind of statement, by its very nature, is vacuous and means nothing (and is most likely made by a non-expert). But on the surface of it seems deep and insightful. Note that it uses weasel words like "biases" and "models" which can actually mean a zillion things.
  - loopz 6y ago
    It means exactly what it means. No more, no less. sig above got it, so it's not impossible to imagine. No need to be an expert, that shouldn't be necessary. Sorry if I used difficult words. The point is to think it through for yourself anyways, and not depend blindly on authority.

by timkam 6y ago

loopz 6y ago

The whole point is that biases are inherent in most models, no matter who the builders are.

Guest42 6y ago
What types of models and what types of bias are you referring to?
- deleted 6y ago
  [deleted]
- slg 6y ago
  Any model created using biased data will inherently mirror that bias unless there are active steps made to counteract this effect. For example, basically any financial evaluations of US citizens will likely result in an inherent bias against Black people due to institutional biases such as redlining that have long lasting socioeconomic and demographic repercussions. This might mean that something as simple as incorporating the zip code of a home into a mortgage pricing AI can end up with a racial bias.
  - Guest42 6y ago
    I agree that the data will shift the results of the model and that some predictors can be proxies for others that are protected, but the claim was that the models themselves were biased (rather globally) and was curious whether that was shown and if so how.
    - slg 6y ago
      I'm not OP, but I don't think the claim was specifically that models themselves are biased. It is that models are inherently biased because the data they are based on is biased. That might sound the same, but there is a nuanced difference. If you are able to strip the bias from the data, the models will work fine. The problem is the data and not the models.
      - refenestrator 6y ago
        At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? Let's say you have a racially-neutral observation of lower income, maybe disability status or a criminal rap in the past. That looks like a bad bet for a loan regardless of color, it just so happens that our society's created a statistical imbalance in those metrics.
        klyrs 6y ago
        > At what point does this stop being AI's fault and start being an accurate observation of things that are society's fault? AI shouldn't take the blame. Blame the folks collecting biased data, or those making biased decisions encoded in the data. The data is known to be tainted. Blame those using that data to train models, and sell/rent/apply those models for profit. Blame the researchers who know, or should know, better but make breathless claims about how their AI can be used without regard for the impact if people follow their advice
        refenestrator 6y ago
dvt 6y ago
This kind of statement, by its very nature, is vacuous and means nothing (and is most likely made by a non-expert). But on the surface of it seems deep and insightful. Note that it uses weasel words like "biases" and "models" which can actually mean a zillion things.
- loopz 6y ago
  It means exactly what it means. No more, no less. sig above got it, so it's not impossible to imagine. No need to be an expert, that shouldn't be necessary. Sorry if I used difficult words. The point is to think it through for yourself anyways, and not depend blindly on authority.