6 ms·
Open Closed Merged Merge Rate 95% Confidence Interval Women 8,216 21,890 111,011 78.6% [78.45%, 78.87%] Men 150,248 591,78
by zamalek 11y ago
Open Closed Merged Merge Rate 95% Confidence Interval
Women 8,216 21,890 111,011 78.6% [78.45%, 78.87%]
Men 150,248 591,785 2,181,517 74.6% [74.56%, 74.67%]
> The hypothesis is not only false, but it is in the opposite direction than expected; women tend to have their pull requests accepted at a higher rate than men! This difference is statistically significant (p < .001).
Many hypothesis are discussed in an attempt to explain this and the paper is well worth the read. Thanks to a documented GitHub API this stuff gets quite scientifically correct - everything is backed up with real-world data.
More quotes:
> To summarize this paper’s observations:
> 1. Women are more likely to have pull requests accepted than men.
> 2. Women continue to have high acceptance rates as they gain experience.
> 3. Women’s pull requests are less likely to serve an immediate project need.
> 4. Women’s changes are larger.
> 5. Women’s acceptance rates are higher across programming languages.
> 6. Women have lower acceptance rates as outsiders when they are identifiable as women.
- xbryanx 11y ago[Edit] Retracting my comment below, after the parent comment was edited to provide more context. --- This quote doesn't seem representative of the whole story that is easily discoverable in the abstract. > Surprisingly, our results show that women's contributions tend to be accepted more often than men's. However, when a woman's gender is identifiable, they are rejected more often. Our results suggest that although women on GitHub may be more competent overall, bias against them exists nonetheless.
- jimrandomh 11y agoThis is indeed in their abstract, but it's misleading. Take a look at page 15 of the preprint. Their data shows that gendered profiles have lower acceptance rates for both genders; that is, pull requests from a gender-neutral profile are more likely to be accepted regardless of the gender of the person behind it. This paper gives the impression that the authors started out hoping to find evidence of discrimination in the form of a lower pull acceptance rate for women than men, they found the opposite, and then they data mined until they could find something they could use to argue there's an anti-woman bias.
- tyingq 11y agoExcept, later.... "For outsiders, we see evidence for gender bias: women’s acceptance rates are 71.8% when they use gender neutral profiles, but drop to 62.5% when their gender is identifiable."
- nightski 11y agoIgnoring the fact that this difference could easily be explained away by some other factor when it is considered. It may have nothing to do with it. Causal relationships are so tricky when dealing with statistics.
- jjcm 11y agoI'm confused - how does that add up to the average of 78.6% then?
- KON_Air 11y agoAnd then you discover in Questions Section; "Our analysis (not in this paper -- we've cut a lot out to keep it crisp) shows that women are harder on other women than they are on men. Men are harder on other men than they are on women." https://peerj.com/questions/2002-do-you-have-data-on-the-gender-of-the-users-that/#annotation-2002-replies https://peerj.com/questions/2002-do-you-have-data-on-the-gen...
- cfcef 11y agoI look forward to seeing this paper cited for decades to come as experimental proof of the misogyny of the tech community.
- voodoomagicman 11y ago> 6. Women have lower acceptance rates as outsiders when they are identifiable as women. This is awful - I think it would be good to have some discussion around what we can do as a community to improve this?
- fancy_pantser 11y agoBe careful implying causation here. We would want to investigate who identifies themselves as any sex vs who remains unidentified (or less obviously identified) and the skill levels represented in each group. Since GitHub does not request your gender for your profile, they used Google+ profiles, which I think would significantly slew the results; they did not sample from all pull requests, but from those that were linked to a G+ account AND whose owners decided to post their gender. > Specifically, we extract users’ email addresses from GHTorrent, look up that email address on the Google+ social network, then, if that user has a profile, extract gender information from these users’ profiles. Out of 4,037,953 GitHub user profiles with email addresses, we were able to identify 1,426,121 (35.3%) of them as men or women through their public Google+ profiles.
- deleted 11y ago[deleted]
- luso_brazilian 11y ago> This is awful - I think it would be good to have some discussion around what we can do as a community to improve this? I don't agree that the result is as strong as "awful" indicate because the paper concludes that > Women had higher acceptance of pull request than men > Identifiable women had higher acceptance of pull request than men > Insider identifiable women had higher acceptance of pull request than men > Both outsider identifiable women and outsider identifiable men had lower acceptance rates than their insider counterparts The study proposes the existence of gender bias because, in this last point, the drop for women was higher than the drop for men, regardless of all the other instances where the opposite was true. I don't believe the study is representative for the whole industry at all (GitHub being mainly remote, voluntary and due to the FLOSS philosophy mostly liberal) but this seems to be one to be celebrated because of the positive conclusion it showed for women.
- Zikes 11y agoPR closes can be for a variety of factors besides programmer competence or gender bias. For example, point #4 says that "women's changes are larger", whereas many large projects are particularly wary of large PRs with multiple changes. That's not to say that the large PR would break something, but many times I've seen a PR closed with a request to break it down into multiple smaller PRs that are more easily reviewed and integrated.
- Animats 11y ago"4. Women’s changes are larger." If you control for change size, does the effect still show? Large changes are less likely to be accepted on some projects. Also, all this is for people who have "google+" accounts. Does anybody still use that?
- glenra 11y agoalong with this: > 6. Women have lower acceptance rates as outsiders when they are identifiable as women. They should have added this: 7. Men have lower acceptance rates as outsiders when they are identifiable as men. Seeing those two together - equally justified by the text - suggests it might just be a small bias (~10%) against "outsiders" who use their real name in favor of "outsiders" who use hacker aliases...regardless of gender.