5 ms·

This doesn't matter until someone tries suing them for it, right? And as I understand it, you don't really have a case without evidence that the hiring algorit

This doesn't matter until someone tries suing them for it, right? And as I understand it, you don't really have a case without evidence that the hiring algorithm is discriminating against people with disabilities. How would an individual even begin to gather that evidence?

marian_ivanco 4y ago
I am not sure, but if I remember correctly employer must prove they are not discriminating. And just because they are using AI they are not immune to litigation.
- danarmak 4y ago
  How can the employer prove a negative? At most I imagine the plaintiff is allowed to do discovery, and then has to prove positive discrimination based on that.
  - radu_floricica 4y ago
  - vajrabum 4y ago
    If you read the document again (?) maybe you'll see it's not about proving a negative. Instead, it's a standard of due care. Did you check whether using some particular tool illegally discriminates and document that consideration? From the document itself: "Clarifies that, when designing or choosing technological tools, employers must consider how their tools could impact different disabilities; Explains employers’ obligations under the ADA when using algorithmic decision-making tools, including when an employer must provide a reasonable accommodation;"
  - HWR_14 4y ago
    If it's a civil case, it's just the preponderance of the evidence. The jury just has to decide who they think is more likely to be correct.
- rascul 4y ago
  > I am not sure, but if I remember correctly employer must prove they are not discriminating. That seems backwards, at least in the US.
dsr_ 4y ago
The process of gathering evidence after the suit has started is called discovery. There are three major kinds of evidence that would be useful here. Most useful but least likely: email inside the company in which someone says "make sure that this doesn't select too many people with disabilities" or "it's fine that the system isn't selecting people with disabilities, carry on". Useful and very likely: prima facie evidence that the software doesn't make necessary reasonable accomodations - a video captcha without an audio alternative, things like that. Fairly useful and of moderate likelihood: statistical evidence that whatever the company said or did, it has the effect of unfairly rejecting applicants with disabilities.
- pfdietz 4y ago
  And one could go a step further: run the software itself and show that it discriminates. One doesn't just have to look at past performance of the software; it can be fed inputs tailored to bring out discriminatory performance. In this way software is more dangerous to the defendant than manual hiring practices; you can't do the same thing to an employee making hiring decisions.
  - twofornone 4y ago
    How would you make sure that the supplied version has the same weights as the production version? And wouldn't the weights and architecture be refined over time anyway?
    - dogleash 4y ago
      > How would you make sure that the supplied version has the same weights as the production version? You just run the same software (with the same state database, if applicable). Oh wait, I forgot, nobody knows or cares what software they're running. As long as the website is pretty and we can outsource the sysop burden, well then, who needs representative testing or the ability to audit?
    - emiliobumachar 4y ago
      Perjury laws. Once a judge has commanded you to give the same AI, you either give the same AI, or truthfully explain that you can't. Any deviation from that and everyone complicit is risking jail time, not just money. "this is the June 2020 version, this is the current version, we have no back ups in between" is acceptable if true. Destroying or omitting an existing version is not.
jejones3141 4y ago
These days, disparate impact is taken as evidence of discrimination, so it's easy to find "discrimination".
- mkr-hn 4y ago
  What's the difference? Discrimination is an effect more than an intent. Most people are decent and well-intentioned and don't mean to discriminate, but it still happens. If there's a disparate impact, what do you imagine causes that if not discrimination? Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain.
  - twofornone 4y ago
    >If there's a disparate impact, what do you imagine causes that if not discrimination? 20+ years of environmental differences, especially culture? The disabilities themselves? Genes? Nothing about human nature suggests that all demographics are equally competent in all fields, regardless of whether you group people by race, gender, political preferences, geography, religion, etc. To believe otherwise is fundamentally unscientific, though it's socially unacceptable to acknowledge this truth. >Remembering that we all have implicit bias This doesn't tell you anything about the direction of this bias, but the zeitgeist is such that it is nearly always assumed to go in one direction, and that's deeply problematic. It's an overcorrection that looks an awful lot like institutional discrimination. >Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain. Except pushing back against unilateral accusations of bias if you belong to one, and only one, specific demographic, you effectively are treated like a mustache-twirling villain. No one is openly complaining about "too much diversity" and keeping their job at the moment. That's bias.
    - etchalon 4y ago
      There is no scientific literature which confirms that any specific demographic quality determine's an individuals capability at any job or task. What does exist is, at best, shows mild correlation over large populations, but nothing binary or deterministic at an individual level. To whit, even if your demographic group, on average, is slightly more or less successful in a specific metric, there is no scientific basis for individualized discrimination. It's "not socially unacceptable to acknowledge this truth", it's socially unacceptable to pretend discrimination is justified.
mkr-hn 4y ago
This is what that demographic survey at the end of job applications is for. It can reveal changes in hiring trends, especially in the demographics of who doesn't get hired. I don't know how well it works in practice.
- hallway_monitor 4y ago
  I am a person, not a statistic. I always decline to answer these surveys; I encourage others to do the same.
  - deleted 4y ago
    [deleted]
  - mkr-hn 4y ago
    Those are for persuading people who do see you as a statistic. You can unilaterally disarm if you like, but they're going to keep discriminating until they see data that proves they're discriminating. Far too few people are persuaded by other means.
  - ccooffee 4y ago
    I also do this. But given the context of this post ("AI" models filtering resumes prior to ever getting in front of a human), maybe "decline to answer" comes with a hidden negative score adjustment that can't be (legally) challenged. I think the Americans with Disabilities Act (ADA) requires notification. (i.e. I need to talk to HR/boss/whoever about any limitations and reasonable accommodations.) If I am correct, not-answering the question "Do you require accommodations according to the ADA? []yes []no []prefer not to answer" can legally come with a penalty, and the linked DoJ reasoning wouldn't stop it.
    - frumper 4y ago
      "Employers should have a process in place to provide reasonable accommodations when using algorithmic decision-making tools;" "Without proper safeguards, workers with disabilities may be “screened out” from consideration in a job or promotion even if they can do the job with or without a reasonable accommodation; and" "If the use of AI or algorithms results in applicants or employees having to provide information about disabilities or medical conditions, it may result in prohibited disability-related inquiries or medical exams." This makes it sound like the employer needs to ensure their AI is allowing for reasonable accommodations. If an AI can assume reasonable accommodations then what benefit would they ever have to assume not supplying the reasonable accommodations that they are legally required to?

marian_ivanco 4y ago

I am not sure, but if I remember correctly employer must prove they are not discriminating. And just because they are using AI they are not immune to litigation.

danarmak 4y ago
How can the employer prove a negative? At most I imagine the plaintiff is allowed to do discovery, and then has to prove positive discrimination based on that.
- radu_floricica 4y ago
- vajrabum 4y ago
  If you read the document again (?) maybe you'll see it's not about proving a negative. Instead, it's a standard of due care. Did you check whether using some particular tool illegally discriminates and document that consideration? From the document itself: "Clarifies that, when designing or choosing technological tools, employers must consider how their tools could impact different disabilities; Explains employers’ obligations under the ADA when using algorithmic decision-making tools, including when an employer must provide a reasonable accommodation;"
- HWR_14 4y ago
  If it's a civil case, it's just the preponderance of the evidence. The jury just has to decide who they think is more likely to be correct.
rascul 4y ago
> I am not sure, but if I remember correctly employer must prove they are not discriminating. That seems backwards, at least in the US.

dsr_ 4y ago

The process of gathering evidence after the suit has started is called discovery. There are three major kinds of evidence that would be useful here. Most useful but least likely: email inside the company in which someone says "make sure that this doesn't select too many people with disabilities" or "it's fine that the system isn't selecting people with disabilities, carry on". Useful and very likely: prima facie evidence that the software doesn't make necessary reasonable accomodations - a video captcha without an audio alternative, things like that. Fairly useful and of moderate likelihood: statistical evidence that whatever the company said or did, it has the effect of unfairly rejecting applicants with disabilities.

pfdietz 4y ago
And one could go a step further: run the software itself and show that it discriminates. One doesn't just have to look at past performance of the software; it can be fed inputs tailored to bring out discriminatory performance. In this way software is more dangerous to the defendant than manual hiring practices; you can't do the same thing to an employee making hiring decisions.
- twofornone 4y ago
  How would you make sure that the supplied version has the same weights as the production version? And wouldn't the weights and architecture be refined over time anyway?
  - dogleash 4y ago
    > How would you make sure that the supplied version has the same weights as the production version? You just run the same software (with the same state database, if applicable). Oh wait, I forgot, nobody knows or cares what software they're running. As long as the website is pretty and we can outsource the sysop burden, well then, who needs representative testing or the ability to audit?
  - emiliobumachar 4y ago
    Perjury laws. Once a judge has commanded you to give the same AI, you either give the same AI, or truthfully explain that you can't. Any deviation from that and everyone complicit is risking jail time, not just money. "this is the June 2020 version, this is the current version, we have no back ups in between" is acceptable if true. Destroying or omitting an existing version is not.

jejones3141 4y ago

These days, disparate impact is taken as evidence of discrimination, so it's easy to find "discrimination".

mkr-hn 4y ago
What's the difference? Discrimination is an effect more than an intent. Most people are decent and well-intentioned and don't mean to discriminate, but it still happens. If there's a disparate impact, what do you imagine causes that if not discrimination? Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain.
- twofornone 4y ago
  >If there's a disparate impact, what do you imagine causes that if not discrimination? 20+ years of environmental differences, especially culture? The disabilities themselves? Genes? Nothing about human nature suggests that all demographics are equally competent in all fields, regardless of whether you group people by race, gender, political preferences, geography, religion, etc. To believe otherwise is fundamentally unscientific, though it's socially unacceptable to acknowledge this truth. >Remembering that we all have implicit bias This doesn't tell you anything about the direction of this bias, but the zeitgeist is such that it is nearly always assumed to go in one direction, and that's deeply problematic. It's an overcorrection that looks an awful lot like institutional discrimination. >Remembering that we all have implicit bias and it doesn't make you a mustache-twirling villain. Except pushing back against unilateral accusations of bias if you belong to one, and only one, specific demographic, you effectively are treated like a mustache-twirling villain. No one is openly complaining about "too much diversity" and keeping their job at the moment. That's bias.
  - etchalon 4y ago
    There is no scientific literature which confirms that any specific demographic quality determine's an individuals capability at any job or task. What does exist is, at best, shows mild correlation over large populations, but nothing binary or deterministic at an individual level. To whit, even if your demographic group, on average, is slightly more or less successful in a specific metric, there is no scientific basis for individualized discrimination. It's "not socially unacceptable to acknowledge this truth", it's socially unacceptable to pretend discrimination is justified.

mkr-hn 4y ago

This is what that demographic survey at the end of job applications is for. It can reveal changes in hiring trends, especially in the demographics of who doesn't get hired. I don't know how well it works in practice.

hallway_monitor 4y ago
I am a person, not a statistic. I always decline to answer these surveys; I encourage others to do the same.
- deleted 4y ago
  [deleted]
- mkr-hn 4y ago
  Those are for persuading people who do see you as a statistic. You can unilaterally disarm if you like, but they're going to keep discriminating until they see data that proves they're discriminating. Far too few people are persuaded by other means.
- ccooffee 4y ago
  I also do this. But given the context of this post ("AI" models filtering resumes prior to ever getting in front of a human), maybe "decline to answer" comes with a hidden negative score adjustment that can't be (legally) challenged. I think the Americans with Disabilities Act (ADA) requires notification. (i.e. I need to talk to HR/boss/whoever about any limitations and reasonable accommodations.) If I am correct, not-answering the question "Do you require accommodations according to the ADA? []yes []no []prefer not to answer" can legally come with a penalty, and the linked DoJ reasoning wouldn't stop it.
  - frumper 4y ago
    "Employers should have a process in place to provide reasonable accommodations when using algorithmic decision-making tools;" "Without proper safeguards, workers with disabilities may be “screened out” from consideration in a job or promotion even if they can do the job with or without a reasonable accommodation; and" "If the use of AI or algorithms results in applicants or employees having to provide information about disabilities or medical conditions, it may result in prohibited disability-related inquiries or medical exams." This makes it sound like the employer needs to ensure their AI is allowing for reasonable accommodations. If an AI can assume reasonable accommodations then what benefit would they ever have to assume not supplying the reasonable accommodations that they are legally required to?