5 ms·

> whitelist acceptable input variables A core lesson in ML: this is difficult or impossible. See the classic example with wolves, dogs, and snow. [1] For a re

• jncfhnb 3y ago
  That is not the same problem. That’s a sample bias problem. The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate. And yes, that likely reduces the power of the model. Edit: to clarify, your example had the model failing to classify something correctly because the training data did not generalize across real world examples. The problem with diversity bias is that the modes are trained on past data where the labels themselves tend to have some bias. Black candidates do worse. So recreatingthese will generally produce worse outcomes for minority candidates. Now for a lot of these cases the average minority individual is more likely to do worse. Not intrinsically because of race, but still worse. You can’t realistically expect the model to produce equal outcomes for all classes when all classes are not equal on the actual traits. But you do want to try and make it so that the model doesn’t use race indicators as a crutch that ignores actual good indicators. The common complaint is you hear about a man and a woman applying with the exact same credentials but different outcomes. Well obviously the model had to have additional information besides those credentials if it made different decisions. If this causes problems, you probably should not permit those additional pieces of information.
  • parpfish 3y ago
    > The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate. The problem is that variables where you can safely discriminate don't exist in the real world because everything is intertwined. For example, if you see that somebody went to school at {Howard, Bryn Mawr, BYU} you can make some pretty solid inferences about their {race, gender, religion} even if they don't explicitly provide them.
    • pclmulqdq 3y ago
      These schoools' associations are all correlations, not guarantees, though. Howard is a historically black college, which means their student body today is majority black, not 100% black. Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life. BYU is a Christian school, but also admits several non-religious people too, and people regularly become apostates to their respective religion.
      • Wowfunhappy 3y ago
        > Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life. Technically true of course, but >99% of the population will identify with a single gender throughout their life. The correlation is so strong that it's not even really a correlation anymore.
        • pclmulqdq 3y ago
          If you're going with a probabilistic argument, I think BYU students are more likely to stick with mormonism (let alone Christianity) than Bryn Mawr students with their gender. That doesn't mean you aren't stereotyping people, though.
          • Wowfunhappy 3y ago
            Irrespective of the AI discussion (computers don't care if you're stereotyping or not), I really don't think it's stereotyping to assume someone who went to a school which literally only admits women is probably a woman. There's a difference between assuming someone's gender because they're a cheerleader, and assuming because they won the women's cheerleading championship. I also assume that the person who drives a fire truck is a fire fighter.
      • michaelt 3y ago
        Right, but the fact the associations are only correlations won't stop the ML model from learning them, if they're present in the training data. If the training data shows a history of bias against Howard graduates, the model will learn to replicate.
• cudgy 3y ago
  Why not simply target the advertising of the job at traditionally black universities, publications, and social networks? Why all the ai mumbo jumbo to filter resumes from a general marketing strategy?