5 ms·

There's evidence in the literature that shows that there are systemic biases against women in github pull requests, even when in the aggregate women may display

There's evidence in the literature that shows that there are systemic biases against women in github pull requests, even when in the aggregate women may display greater levels of competence. https://peerj.com/articles/cs-111/ https://peerj.com/articles/cs-111/ To suggest that pull requests are a good hiring signal (as the author does in explicitly endorsing the "screening" of open source maintainers) ignores this bias. That can be positive or negative, depending on your commitment to diversity in our field.

ubernostrum 8y ago
For those who haven't read the linked article before, it really is fascinating. The relevant bits are two factors: * Is the person sending the pull request someone already known to the project, or an outsider? * Is the gender of the pull request's author easily determinable? It would be unsurprising to find that "outsider" pull requests get merged at a lower rate. But "outsider" pull requests from women merge at different rates depending on the second factor: when it's easy to determine the gender of the author, a pull request from an "outsider" woman is significantly less likely to be accepted than a pull request from an "outsider" woman whose gender cannot easily be determined. Also, "insider" women and women whose gender is not easily determinable have higher rates of merged PRs than their male counterparts. There's no reason to suspect that whether someone's gender is easily determinable by a PR reviewer (usually, via username, profile picture, other clues like linked blogs or social-media profiles) has an effect on the quality of their code. So while this of course doesn't prove gender bias (we'd need telepathy for that), it does very very very strongly suggest it. This is also in line with a lot of prior research and reporting on blind interview and auditioning practices, so shouldn't be too surprising, but for some reason people love to insist that tech is somehow different from all those other fields where hiding someone's gender changed how they were evaluated.
janeroe 8y ago
> systemic biases against women in github pull requests Did you read your link? > The hypothesis is not only false, but it is in the opposite direction than expected; women tend to have their pull requests accepted at a higher rate than men! This difference is statistically significant
- detaro 8y ago
  ... unless they can be easily identified as women from their profile, at which point their acceptance rate drops.
  - tziki 8y ago
    The acceptance rate of both men and women dropped if they were identifiable as men or women. The study doesn't make any attempts to explain this.
    - detaro 8y ago
      Yes, and the one of women drops below the one of men, and that's clearly what the initial reference to the paper was talking about.
      - tziki 8y ago
        But only in the case of 'outsiders'. In the case of 'insiders' the acceptance rate of men drops more than that of women when their gender is known (no p-value provided so hard to tell if statistically significant).
        realSlavojTrump 8y ago
        A person trying to use github as a hiring signal is going to be an outsider at some point.
- realSlavojTrump 8y ago
  Did you read the abstract?

ubernostrum 8y ago

For those who haven't read the linked article before, it really is fascinating. The relevant bits are two factors: * Is the person sending the pull request someone already known to the project, or an outsider? * Is the gender of the pull request's author easily determinable? It would be unsurprising to find that "outsider" pull requests get merged at a lower rate. But "outsider" pull requests from women merge at different rates depending on the second factor: when it's easy to determine the gender of the author, a pull request from an "outsider" woman is significantly less likely to be accepted than a pull request from an "outsider" woman whose gender cannot easily be determined. Also, "insider" women and women whose gender is not easily determinable have higher rates of merged PRs than their male counterparts. There's no reason to suspect that whether someone's gender is easily determinable by a PR reviewer (usually, via username, profile picture, other clues like linked blogs or social-media profiles) has an effect on the quality of their code. So while this of course doesn't prove gender bias (we'd need telepathy for that), it does very very very strongly suggest it. This is also in line with a lot of prior research and reporting on blind interview and auditioning practices, so shouldn't be too surprising, but for some reason people love to insist that tech is somehow different from all those other fields where hiding someone's gender changed how they were evaluated.

janeroe 8y ago

> systemic biases against women in github pull requests Did you read your link? > The hypothesis is not only false, but it is in the opposite direction than expected; women tend to have their pull requests accepted at a higher rate than men! This difference is statistically significant

detaro 8y ago
... unless they can be easily identified as women from their profile, at which point their acceptance rate drops.
- tziki 8y ago
  The acceptance rate of both men and women dropped if they were identifiable as men or women. The study doesn't make any attempts to explain this.
  - detaro 8y ago
    Yes, and the one of women drops below the one of men, and that's clearly what the initial reference to the paper was talking about.
    - tziki 8y ago
      But only in the case of 'outsiders'. In the case of 'insiders' the acceptance rate of men drops more than that of women when their gender is known (no p-value provided so hard to tell if statistically significant).
      - realSlavojTrump 8y ago
        A person trying to use github as a hiring signal is going to be an outsider at some point.
realSlavojTrump 8y ago
Did you read the abstract?