4 ms·

Of course they could be. A company that doesn't want a racial bias won't intentionally filter on names, but might accidentally deploy an LLM that can discern r

Of course they could be. A company that doesn't want a racial bias won't intentionally filter on names, but might accidentally deploy an LLM that can discern race from name.

lanstin 2y ago
The humans will do it unconsciously if they can see the names. It is like orchestras used to use auditions where the judges could see the people; when they went to auditions where they couldn't see the people, the number of women being hired went right up.
- thaumasiotes 2y ago
  See, this is the problem with just deciding to invent your own facts. There is a famous study on this topic, usually presented in much the way you describe. But usually people are more careful not to lie about the results. The rate of women being hired went down, not up. The "success" for women was contained to advancement through particular audition rounds. https://archive.is/xmvp2 https://archive.is/xmvp2 The original paper is not exactly a model of investigative integrity: >> Women are about 5 percentage points more likely to be hired than are men in a completely blind audition, although the effect is not statistically significant. The effect is nil, however, when there is a semifinal round, perhaps as a result of the unusual effects of the semifinal round.
  - ZeroGravitas 2y ago
    Are you citing a weird racist blogger on medium that disagrees with the stated conclusion of the paper? Is that how science works now? Abstract from the original paper: > A change in the audition procedures of symphony orchestras--adoption of "blind" auditions with a "screen" to conceal the candidate's identity from the jury--provides a test for sex-biased hiring. Using data from actual auditions, in an individual fixed-effects framework, we find that the screen increases the probability a woman will be advanced and hired. Although some of our estimates have large standard errors and there is one persistent effect in the opposite direction, the weight of the evidence suggests that the blind audition procedure fostered impartiality in hiring and increased the proportion women in symphony orchestras. https://www.aeaweb.org/articles?id=10.1257/aer.90.4.715 https://www.aeaweb.org/articles?id=10.1257/aer.90.4.715
    - thaumasiotes 2y ago
      You'll get more accurate results if you look at the paper's conclusion, not the abstract. The whole point is that the paper doesn't actually support any of the claims, which it's fairly open about -- that's those "large standard errors". They did not in fact find that the screen increases the probability of a woman being hired. They found that it increased the probability of a woman passing the final round of auditions. There's more than one round. It's up to you, I guess, whether you want to follow the original authors to their conclusion that the semifinal round is fundamentally unlike the final and quarterfinal audition rounds. I wouldn't. The paper is meritless, but even it's actual worthless findings aren't in the direction that people like to claim. Here's Andrew Gelman: https://statmodeling.stat.columbia.edu/2019/05/11/did-blind-orchestra-auditions-really-benefit-women/ https://statmodeling.stat.columbia.edu/2019/05/11/did-blind-... > First, some equivocal results: > This is not very impressive at all. Some fine words but the punchline seems to be that the data are too noisy to form any strong conclusions. > Huh? Nothing’s statistically significant but the estimates “show that the existence of any blind round makes a difference”? I might well be missing something here. In any case, you shouldn’t be running around making a big deal about point estimates when the standard errors are so large. I don’t hold it against the authors—this was 2000, after all, the stone age in our understanding of statistical errors. But from a modern perspective we can see the problem. > Anyway, where’s the damn “50 percent” and the “increases by severalfold”? I can’t find it. It’s gotta be somewhere in that paper, I just can’t figure out where. > Pallesen’s objections are strongly stated but they’re not new. Indeed, the authors of the original paper were pretty clear about its limitations. The evidence was all in plain sight. > For example, here’s a careful take posted by BS King in 2017: >> Okay, so first up, the most often reported findings: blind auditions appear to account for about 25% of the increase in women in major orchestras. . . . [But] One of the more interesting findings of the study that I have not often seen reported: overall, women did worse in the blinded auditions. . . . If you read the paper, you'd notice the problems. But you might not want to run the risk of, um, "racism" that seems to be inherent in reading a paper on the impact of blind auditions on women.
      - ZeroGravitas 2y ago
        The racism was more the guy having very strong opinions about black people's IQs and immigrants destroying western civilization on twitter (plus being against COVID vaccines and human rights and other stuff that suggest you are not a level-headed truth seeker). Quote from your second, more respectable, link: > Pallesen’s objections are strongly stated but they’re not new. Indeed, the authors of the original paper were pretty clear about its limitations. When I read the original papers, like your second link, I find a level headed paper working with limited data they had available. I would suggest that the annoyance is with pop science retelling at best, or just a quack selling anger online to fuel his weird obsessions at worst.
- dudinax 2y ago
  A human might, but a dumb filter wouldn't.

Of course they could be. A company that doesn't want a racial bias won't intentionally filter on names, but might accidentally deploy an LLM that can discern race from name.

lanstin 2y ago

The humans will do it unconsciously if they can see the names. It is like orchestras used to use auditions where the judges could see the people; when they went to auditions where they couldn't see the people, the number of women being hired went right up.

thaumasiotes 2y ago
See, this is the problem with just deciding to invent your own facts. There is a famous study on this topic, usually presented in much the way you describe. But usually people are more careful not to lie about the results. The rate of women being hired went down, not up. The "success" for women was contained to advancement through particular audition rounds. https://archive.is/xmvp2 https://archive.is/xmvp2 The original paper is not exactly a model of investigative integrity: >> Women are about 5 percentage points more likely to be hired than are men in a completely blind audition, although the effect is not statistically significant. The effect is nil, however, when there is a semifinal round, perhaps as a result of the unusual effects of the semifinal round.
- ZeroGravitas 2y ago
  Are you citing a weird racist blogger on medium that disagrees with the stated conclusion of the paper? Is that how science works now? Abstract from the original paper: > A change in the audition procedures of symphony orchestras--adoption of "blind" auditions with a "screen" to conceal the candidate's identity from the jury--provides a test for sex-biased hiring. Using data from actual auditions, in an individual fixed-effects framework, we find that the screen increases the probability a woman will be advanced and hired. Although some of our estimates have large standard errors and there is one persistent effect in the opposite direction, the weight of the evidence suggests that the blind audition procedure fostered impartiality in hiring and increased the proportion women in symphony orchestras. https://www.aeaweb.org/articles?id=10.1257/aer.90.4.715 https://www.aeaweb.org/articles?id=10.1257/aer.90.4.715
  - thaumasiotes 2y ago
    You'll get more accurate results if you look at the paper's conclusion, not the abstract. The whole point is that the paper doesn't actually support any of the claims, which it's fairly open about -- that's those "large standard errors". They did not in fact find that the screen increases the probability of a woman being hired. They found that it increased the probability of a woman passing the final round of auditions. There's more than one round. It's up to you, I guess, whether you want to follow the original authors to their conclusion that the semifinal round is fundamentally unlike the final and quarterfinal audition rounds. I wouldn't. The paper is meritless, but even it's actual worthless findings aren't in the direction that people like to claim. Here's Andrew Gelman: https://statmodeling.stat.columbia.edu/2019/05/11/did-blind-orchestra-auditions-really-benefit-women/ https://statmodeling.stat.columbia.edu/2019/05/11/did-blind-... > First, some equivocal results: > This is not very impressive at all. Some fine words but the punchline seems to be that the data are too noisy to form any strong conclusions. > Huh? Nothing’s statistically significant but the estimates “show that the existence of any blind round makes a difference”? I might well be missing something here. In any case, you shouldn’t be running around making a big deal about point estimates when the standard errors are so large. I don’t hold it against the authors—this was 2000, after all, the stone age in our understanding of statistical errors. But from a modern perspective we can see the problem. > Anyway, where’s the damn “50 percent” and the “increases by severalfold”? I can’t find it. It’s gotta be somewhere in that paper, I just can’t figure out where. > Pallesen’s objections are strongly stated but they’re not new. Indeed, the authors of the original paper were pretty clear about its limitations. The evidence was all in plain sight. > For example, here’s a careful take posted by BS King in 2017: >> Okay, so first up, the most often reported findings: blind auditions appear to account for about 25% of the increase in women in major orchestras. . . . [But] One of the more interesting findings of the study that I have not often seen reported: overall, women did worse in the blinded auditions. . . . If you read the paper, you'd notice the problems. But you might not want to run the risk of, um, "racism" that seems to be inherent in reading a paper on the impact of blind auditions on women.
    - ZeroGravitas 2y ago
      The racism was more the guy having very strong opinions about black people's IQs and immigrants destroying western civilization on twitter (plus being against COVID vaccines and human rights and other stuff that suggest you are not a level-headed truth seeker). Quote from your second, more respectable, link: > Pallesen’s objections are strongly stated but they’re not new. Indeed, the authors of the original paper were pretty clear about its limitations. When I read the original papers, like your second link, I find a level headed paper working with limited data they had available. I would suggest that the annoyance is with pop science retelling at best, or just a quack selling anger online to fuel his weird obsessions at worst.
dudinax 2y ago
A human might, but a dumb filter wouldn't.