3 ms·

> Instead of confronting racial and socioeconomic biases in our hiring process, we've decided to deflect blame to a consulting firm who will in turn blame it on

> Instead of confronting racial and socioeconomic biases in our hiring process, we've decided to deflect blame to a consulting firm who will in turn blame it on a mysterious 'cultural fit' algorithm Culture encompasses a range of human behavior. Much of that, though it may not be tasteful to say, is tied to workplace performance. Pressure and desire to achieve, collective/individualist tendency, ethics and morals, conformity and deference to authority; if one is willing to acknowledge that cultures exist and vary, then one cannot deny that there will be strong correlations between culture and fit for particular roles. And as it happens culture correlates with nationality, race, and socioeconomic status - because parents and communities generally pass culture on to children. At some point in the near future neural nets trained to predict human performance will undoubtedly condition on priors like nationality, race, and socioeconomic status, even indirectly if they are not explicit data points. What then? Will we continue to bend over backwards to deny reality in favor of a false Utopia? What happens when these trends suggest different interventions for medical conditions? Different learning environments? Different reactions to authority and punishment? Edit: I don't understand the downvotes. Why use data analysis tools if you're just going to ignore the results you don't like? How do you expect to solve problems like inequality if you aren't willing to explore their actual causes? Is there something illogical or factually incorrect in my comment?

mreome 7y ago
Assuming that any individual can be judged, or their actions predicted, from generalizations based on their culture or socioeconomic status is quite literally the definition of racism. Using correlations with race and socioeconomic status as a filter by witch to deny people the chance to achieve, advanced, and change those correlations/generalizations, is the definition of systemic racism. If you act based on those kind of generalizations or correlations in a way that enforces them, you're not identifying a problem, you're creating/perpetuating it.
- Reelin 7y ago
  I agree with the sentiment you express, but I think there is some nuance of definition that should be clarified here. Deployed on a wide scale, the filters you describe would indeed constitute systemic racism. However, on an individual basis such practices would be more correctly termed profiling rather than racism since they lack intent. (This depends entirely on your definition of racism, of course, which it turns out can be quite difficult to pin down. [1]) I realize this may seem like splitting hairs since I ultimately agree with your overall point, but often such definitional issues result in a great deal of miscommunication and misunderstanding. [1] https://slatestarcodex.com/2017/06/21/against-murderism https://slatestarcodex.com/2017/06/21/against-murderism
  - wtvanhest 7y ago
    It also depends on your definition of intent. If you knowingly implement a system with flaws that make it systematically racist, you have intended to be racist.
    - ghettoimp 7y ago
      Hrmn. This makes it sound like you can only implement a systematically racist system if you are overtly/intentionally trying to be racist. I suspect in many cases, folks who implement racist systems simply do not care whether their system is racist or not. They want to optimize/automate whatever it is that they are about, and if the result is unfair, well, that's just someone else's problem.
      - wtvanhest 7y ago
        That is why I said ‘knowingly’
  - mreome 7y ago
    Note that while I use/discuses the term racism here, I intend for this argument to apply to prejudice and systemic discrimination more broadly. I would personally argue against any definition of racism that discounts acts based on intent. Your source seems to start out trying to sort-out a kind of binary (trinary) fallacy, where it's arguing that there must be only a single valid definition of what qualifies as racist that can be used in any/all contexts, rather than just acknowledging that there are multiple ways to be racist, with varying degrees of intent/malice/severity/subtly and contextual applicability. It seems to reach that conclusion in the end though with "Overall We Probably Use A Combination Of All Of These, Weighted In Favor Of Definition By Motives." Beyond that, I’d argue that a more limited definition is dangerous as it allows people to argue/believe that a lack of blatantly malicious intent absolves them of the consequences of their acts. Broadly speaking, yes this would be accurately termed profiling. That is in no way mutually exclusive with being racist, given either of our definitions of racism. The developer of an AI profiling algorithm must make choices about what data is provided to that system, and others must make a choice to act based on the outputs of that system. The choice to include racial or socioeconomic criteria (or criteria that are proxies for these) as training data for a given model, and act on its outputs in ways that could contribute to systemic racism, could be done without malice or could be driven by blatantly racists motives. We just end up back at the same argument about the definition of racism. And again, I’d argue that the more limited definition is dangerous as it allows exactly what wlesieutre meant with “we've decided to deflect blame.” Allowing an act that leads to systemic racism to not qualify as racism without a clear blatant intent only serves to excuse those developing/using such systems for the consequences of their choices. Additionally, the narrower definition could actually discourage consideration of such consequences, as acting without any thought or consideration at all is acting without intent. Thus, willful or feigned ignorance becomes a defense against the furtherance of systemic racism.
Reelin 7y ago
The trouble is that there is no singular, well delineated definition for the term "culture" in the context of such systems (and thus this discussion in general). Is team culture defined by general attitude? By demeanor when interacting with others? By willingness to work unpaid overtime? Or perhaps by how others generally feel about you? That last one in particular has serious implications relating to biases and socioeconomic factors. > Will we continue to bend over backwards to deny reality in favor of a false Utopia? That would be a rather extreme position. Why do we have to choose between ignoring things that are broken versus ignoring reality itself? Why not attempt to work towards an ideal, even if that ultimately costs our society some small amount of efficiency on the whole?
majormajor 7y ago
> At some point in the near future neural nets trained to predict human performance will undoubtedly condition on priors like nationality, race, and socioeconomic status, even indirectly if they are not explicit data points. What then? Will we continue to bend over backwards to deny reality in favor of a false Utopia? What happens when these trends suggest different interventions for medical conditions? Different learning environments? Different reactions to authority and punishment? This doesn't feel like an argument in good faith because you both seem to already have strong beliefs about what the "actual causes" are, and also place a lot of words in the mouths of those who might disagree with you based on an uncharitable interpretation of their motives. Consider how you lump in "nationality, race, and socioeconomic status" all of the same category. It seems you've already decided that those are all root causes, and anyone who disagrees is just denying reality. One might say "I want to be able to identify interventions to counteract negative effects of [socioeconomic status, home life, etc] in a specific and personalized way to improve opportunities." That would be an interesting use of computational power, beyond what we could've accomplished without fancy computers. Or you could design a model that would just mimic simple naive stats and reinforce your beliefs on race and culture, while not actually be digging into precise, specific, details. Yes, your simple model will label groups that have suffered historical and ongoing discrimination and disadvantages as less likely to succeed if you don't fully model the compounding and generational effects of that. That's neither interesting nor compelling. People are extremely complicated and yet you've concluded that it's all down to a few blunt factors. Ok.
anigbrowl 7y ago
Edit: I don't understand the downvotes. Yeah you do.

mreome 7y ago

Assuming that any individual can be judged, or their actions predicted, from generalizations based on their culture or socioeconomic status is quite literally the definition of racism. Using correlations with race and socioeconomic status as a filter by witch to deny people the chance to achieve, advanced, and change those correlations/generalizations, is the definition of systemic racism. If you act based on those kind of generalizations or correlations in a way that enforces them, you're not identifying a problem, you're creating/perpetuating it.

Reelin 7y ago
I agree with the sentiment you express, but I think there is some nuance of definition that should be clarified here. Deployed on a wide scale, the filters you describe would indeed constitute systemic racism. However, on an individual basis such practices would be more correctly termed profiling rather than racism since they lack intent. (This depends entirely on your definition of racism, of course, which it turns out can be quite difficult to pin down. [1]) I realize this may seem like splitting hairs since I ultimately agree with your overall point, but often such definitional issues result in a great deal of miscommunication and misunderstanding. [1] https://slatestarcodex.com/2017/06/21/against-murderism https://slatestarcodex.com/2017/06/21/against-murderism
- wtvanhest 7y ago
  It also depends on your definition of intent. If you knowingly implement a system with flaws that make it systematically racist, you have intended to be racist.
  - ghettoimp 7y ago
    Hrmn. This makes it sound like you can only implement a systematically racist system if you are overtly/intentionally trying to be racist. I suspect in many cases, folks who implement racist systems simply do not care whether their system is racist or not. They want to optimize/automate whatever it is that they are about, and if the result is unfair, well, that's just someone else's problem.
    - wtvanhest 7y ago
      That is why I said ‘knowingly’
- mreome 7y ago
  Note that while I use/discuses the term racism here, I intend for this argument to apply to prejudice and systemic discrimination more broadly. I would personally argue against any definition of racism that discounts acts based on intent. Your source seems to start out trying to sort-out a kind of binary (trinary) fallacy, where it's arguing that there must be only a single valid definition of what qualifies as racist that can be used in any/all contexts, rather than just acknowledging that there are multiple ways to be racist, with varying degrees of intent/malice/severity/subtly and contextual applicability. It seems to reach that conclusion in the end though with "Overall We Probably Use A Combination Of All Of These, Weighted In Favor Of Definition By Motives." Beyond that, I’d argue that a more limited definition is dangerous as it allows people to argue/believe that a lack of blatantly malicious intent absolves them of the consequences of their acts. Broadly speaking, yes this would be accurately termed profiling. That is in no way mutually exclusive with being racist, given either of our definitions of racism. The developer of an AI profiling algorithm must make choices about what data is provided to that system, and others must make a choice to act based on the outputs of that system. The choice to include racial or socioeconomic criteria (or criteria that are proxies for these) as training data for a given model, and act on its outputs in ways that could contribute to systemic racism, could be done without malice or could be driven by blatantly racists motives. We just end up back at the same argument about the definition of racism. And again, I’d argue that the more limited definition is dangerous as it allows exactly what wlesieutre meant with “we've decided to deflect blame.” Allowing an act that leads to systemic racism to not qualify as racism without a clear blatant intent only serves to excuse those developing/using such systems for the consequences of their choices. Additionally, the narrower definition could actually discourage consideration of such consequences, as acting without any thought or consideration at all is acting without intent. Thus, willful or feigned ignorance becomes a defense against the furtherance of systemic racism.

Reelin 7y ago

The trouble is that there is no singular, well delineated definition for the term "culture" in the context of such systems (and thus this discussion in general). Is team culture defined by general attitude? By demeanor when interacting with others? By willingness to work unpaid overtime? Or perhaps by how others generally feel about you? That last one in particular has serious implications relating to biases and socioeconomic factors. > Will we continue to bend over backwards to deny reality in favor of a false Utopia? That would be a rather extreme position. Why do we have to choose between ignoring things that are broken versus ignoring reality itself? Why not attempt to work towards an ideal, even if that ultimately costs our society some small amount of efficiency on the whole?

majormajor 7y ago

> At some point in the near future neural nets trained to predict human performance will undoubtedly condition on priors like nationality, race, and socioeconomic status, even indirectly if they are not explicit data points. What then? Will we continue to bend over backwards to deny reality in favor of a false Utopia? What happens when these trends suggest different interventions for medical conditions? Different learning environments? Different reactions to authority and punishment? This doesn't feel like an argument in good faith because you both seem to already have strong beliefs about what the "actual causes" are, and also place a lot of words in the mouths of those who might disagree with you based on an uncharitable interpretation of their motives. Consider how you lump in "nationality, race, and socioeconomic status" all of the same category. It seems you've already decided that those are all root causes, and anyone who disagrees is just denying reality. One might say "I want to be able to identify interventions to counteract negative effects of [socioeconomic status, home life, etc] in a specific and personalized way to improve opportunities." That would be an interesting use of computational power, beyond what we could've accomplished without fancy computers. Or you could design a model that would just mimic simple naive stats and reinforce your beliefs on race and culture, while not actually be digging into precise, specific, details. Yes, your simple model will label groups that have suffered historical and ongoing discrimination and disadvantages as less likely to succeed if you don't fully model the compounding and generational effects of that. That's neither interesting nor compelling. People are extremely complicated and yet you've concluded that it's all down to a few blunt factors. Ok.

anigbrowl 7y ago

Edit: I don't understand the downvotes. Yeah you do.