6 ms·

>With a human, you can’t really do an A/B test to determine if they would have prioritized a candidate if they hadn’t included some signal; it’s really easy to

by slg 4y ago

>With a human, you can’t really do an A/B test to determine if they would have prioritized a candidate if they hadn’t included some signal; it’s really easy to rationalize away discrimination at the margins. Which is part of the reason that discrimination doesn't have to be intentional for it to be punishable. This is a concept known as "disparate impact". The Supreme Court has issued decisions[1] that a policy which negatively impacts a protected class and has no justifiable business related reason for existing can be deemed discrimination regardless of the motivations behind that policy. [1] - https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.

Ferrotin 4y ago
Everything has a disparate impact, so now everything is illegal.
- pc86 4y ago
  Might I suggest: https://www.merriam-webster.com/dictionary/term%20of%20art https://www.merriam-webster.com/dictionary/term%20of%20art
  - Ferrotin 4y ago
    No, you’re wrong here, it’s not a term of art.
    - pc86 4y ago
      It sure looks like it is. > Disparate impact in United States labor law refers to ... https://en.wikipedia.org/wiki/Disparate_impact https://en.wikipedia.org/wiki/Disparate_impact
      - Ferrotin 4y ago
        Your link is evidence for my side. It uses the plain definition. The plain meaning of the words.
        pc86 4y ago
        My point is that by the plain meaning of words you're right, disparate impact means any two groups impacted differently, regardless of anything else. In law, it means that an employment, housing, etc. policy has a disproportionately adverse impact on members of a protected class compared to non-members of that same class. It's much more specific and narrowly defined.
        thaumasiotes 4y ago
        That is in fact not more narrow; Ferrotin's original claim that "everything has a disparate impact" is correct with "disparate impact" so defined.
        kelseyfrog 4y ago
        If you ever intent to study law, become involved in a situation dealing with disparate impact, or are at the receiving end of disparate impact, knowing the legal definition may be helpful too. The DoJ spells[1] out the legal definition of disparate impact as so: ELEMENTS TO ESTABLISH ADVERSE DISPARATE IMPACT UNDER TITLE VI Identify the specific policy or practice at issue; see Section C.3.a. Establish adversity/harm; see Section C.3.b. Establish disparity; see Section C.3.c. Establish causation; see Section C.3.d. 1. https://www.justice.gov/crt/fcs/T6Manual7#D https://www.justice.gov/crt/fcs/T6Manual7#D
TimPC 4y ago
Justifiable business reason is still a strong bar. For example, with no evidence in either direction for a claim there is no justifiable business reason even if the claim is somewhat intuitive. So if you want to require high-school diplomas because you think people who have them will do the job better you better track that data for years and be prepared to demonstrate it if sued. If you want to use IQ tests because you anticipate smarter people will do the job better you better have IQ tests done on your previous employee population demonstrating the correlation before imposing the requirement.
- cmeacham98 4y ago
  EDIT: my parent edited and replaced their entire comment, it originally said "you can't use IQ tests even if you prove they lead to better job performance". I leave my original comment below for posterity: This is not true, IQ tests in the mentioned Griggs v. Duke Power Co. (and similar cases) were rejected as disparate impact specifically because the company provided no evidence they lead to better performance. To quote the majority opinion of Griggs: > On the record before us, neither the high school completion requirement nor the general intelligence test is shown to bear a demonstrable relationship to successful performance of the jobs for which it was used. Both were adopted, as the Court of Appeals noted, without meaningful study of their relationship to job performance ability.
  - deleted 4y ago
    [deleted]
- tgsovlerkhgsel 4y ago
  Wouldn't that be trivial if you have your training data set?
darawk 4y ago
He didn't say anything about intention, though. He just talked about the counterfactual. Disparate impact is about the counterfactual scenario.
- slg 4y ago
  They said "it’s really easy to rationalize away discrimination at the margins." My reply was pointing out that there is little legal protection in rationalizing away discrimination at the margins because tests for disparate impact require the approach to also stand up holistically which can't easily be rationalized away.
  - darawk 4y ago
    Yes, but a holistic test requires a realistic counterfactual. That's the problem. There is no way to evaluate that counterfactual for a human interviewer. It is true that extreme bias/discrimination will be evident, but smaller bias/discrimination, particularly in an environment where the pool is small (say, black women for engineering roles) is extremely hard to prove for a human interviewer. Your sample size is just going to be too small. On the other hand, if you have an ML algorithm, you can feed it arbitrary amounts of synthetic data, and get precise loadings on protected attributes.
  - theptip 4y ago
    I think perhaps you are looking at a different part of the funnel; disparate impact seems to be around the sort of requirements you are allowed to put in a job description. Like “must have a college degree”. However the sort of insidious discrimination at the margin I was imagining are things like “equally-good resumes (meets all requirements), but one had a female/stereotypically-black name”. Interpreting resumes is not a science and humans apply judgement to pick which ones feel good, which leaves a lot of room for hidden bias to creep in. My point was that I think algorithmic processes are more testable for these sorts of bias; do you feel that existing disparate impact regulations are good at catching/preventing this kind of thing? (I’m aware of some large-scale research on name-bias on resumes but it seems hard to do in the context of a single company.)

by slg 4y ago

Ferrotin 4y ago

Everything has a disparate impact, so now everything is illegal.

pc86 4y ago
Might I suggest: https://www.merriam-webster.com/dictionary/term%20of%20art https://www.merriam-webster.com/dictionary/term%20of%20art
- Ferrotin 4y ago
  No, you’re wrong here, it’s not a term of art.
  - pc86 4y ago
    It sure looks like it is. > Disparate impact in United States labor law refers to ... https://en.wikipedia.org/wiki/Disparate_impact https://en.wikipedia.org/wiki/Disparate_impact
    - Ferrotin 4y ago
      Your link is evidence for my side. It uses the plain definition. The plain meaning of the words.
      - pc86 4y ago
        My point is that by the plain meaning of words you're right, disparate impact means any two groups impacted differently, regardless of anything else. In law, it means that an employment, housing, etc. policy has a disproportionately adverse impact on members of a protected class compared to non-members of that same class. It's much more specific and narrowly defined.
        thaumasiotes 4y ago
        That is in fact not more narrow; Ferrotin's original claim that "everything has a disparate impact" is correct with "disparate impact" so defined.
      - kelseyfrog 4y ago
        If you ever intent to study law, become involved in a situation dealing with disparate impact, or are at the receiving end of disparate impact, knowing the legal definition may be helpful too. The DoJ spells[1] out the legal definition of disparate impact as so: ELEMENTS TO ESTABLISH ADVERSE DISPARATE IMPACT UNDER TITLE VI Identify the specific policy or practice at issue; see Section C.3.a. Establish adversity/harm; see Section C.3.b. Establish disparity; see Section C.3.c. Establish causation; see Section C.3.d. 1. https://www.justice.gov/crt/fcs/T6Manual7#D https://www.justice.gov/crt/fcs/T6Manual7#D

TimPC 4y ago

Justifiable business reason is still a strong bar. For example, with no evidence in either direction for a claim there is no justifiable business reason even if the claim is somewhat intuitive. So if you want to require high-school diplomas because you think people who have them will do the job better you better track that data for years and be prepared to demonstrate it if sued. If you want to use IQ tests because you anticipate smarter people will do the job better you better have IQ tests done on your previous employee population demonstrating the correlation before imposing the requirement.

cmeacham98 4y ago
EDIT: my parent edited and replaced their entire comment, it originally said "you can't use IQ tests even if you prove they lead to better job performance". I leave my original comment below for posterity: This is not true, IQ tests in the mentioned Griggs v. Duke Power Co. (and similar cases) were rejected as disparate impact specifically because the company provided no evidence they lead to better performance. To quote the majority opinion of Griggs: > On the record before us, neither the high school completion requirement nor the general intelligence test is shown to bear a demonstrable relationship to successful performance of the jobs for which it was used. Both were adopted, as the Court of Appeals noted, without meaningful study of their relationship to job performance ability.
- deleted 4y ago
  [deleted]
tgsovlerkhgsel 4y ago
Wouldn't that be trivial if you have your training data set?

darawk 4y ago

He didn't say anything about intention, though. He just talked about the counterfactual. Disparate impact is about the counterfactual scenario.

slg 4y ago
They said "it’s really easy to rationalize away discrimination at the margins." My reply was pointing out that there is little legal protection in rationalizing away discrimination at the margins because tests for disparate impact require the approach to also stand up holistically which can't easily be rationalized away.
- darawk 4y ago
  Yes, but a holistic test requires a realistic counterfactual. That's the problem. There is no way to evaluate that counterfactual for a human interviewer. It is true that extreme bias/discrimination will be evident, but smaller bias/discrimination, particularly in an environment where the pool is small (say, black women for engineering roles) is extremely hard to prove for a human interviewer. Your sample size is just going to be too small. On the other hand, if you have an ML algorithm, you can feed it arbitrary amounts of synthetic data, and get precise loadings on protected attributes.
- theptip 4y ago
  I think perhaps you are looking at a different part of the funnel; disparate impact seems to be around the sort of requirements you are allowed to put in a job description. Like “must have a college degree”. However the sort of insidious discrimination at the margin I was imagining are things like “equally-good resumes (meets all requirements), but one had a female/stereotypically-black name”. Interpreting resumes is not a science and humans apply judgement to pick which ones feel good, which leaves a lot of room for hidden bias to creep in. My point was that I think algorithmic processes are more testable for these sorts of bias; do you feel that existing disparate impact regulations are good at catching/preventing this kind of thing? (I’m aware of some large-scale research on name-bias on resumes but it seems hard to do in the context of a single company.)