5 ms·

I speculate it's extremely unlikely the person tweeting this conducted a meaningful study. First off, they call it A/B testing, but unless they were sending the

by Sacho 10y ago

I speculate it's extremely unlikely the person tweeting this conducted a meaningful study. First off, they call it A/B testing, but unless they were sending the same patch with a different pseudonym, then they're incorrectly using the term. Second, how did they determine pseudonyms as "male" or "female" sounding? If you want to study unconscious bias, you'd need to know whether the things that supposedly trigger it actually do so. Furthermore, did they account for their own unconscious bias? I can imagine, using the same unconscious bias theory, that they sent self-assessed lower quality patches with "female" pseudonyms unconsciously, to prove their theory correct. Lastly, the 100% is very close to unbelievable, having an implicit assertion that identity politics are the primary driver of motivation in the developer team, way above any coding merit. To me, this is more likely to be projection from an identity politics activist(for whom, by definition, identity politics are the primary driver).

yummyfajitas 10y ago
First off, they call it A/B testing, but unless they were sending the same patch with a different pseudonym, then they're incorrectly using the term. This isn't true. As long as the patches were IID (e.g. she generated patches and randomly chose which name to submit them as), it's a legitimate A/B test. https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables https://en.wikipedia.org/wiki/Independent_and_identically_di...
isislovecruft 10y ago
First, you're correct: I never meant to conduct a meaningful study. It was purely of my own curiousity that I did this. However that does not preclude someone else, of any gender, doing so more scientifically. And to answer your accusation: yes, I did submit the same patch twice, under two differently gendered names (although never publicly, I would copy-paste the patch and privately email it). Second, my primary source for whether a name is "female- or male- sounding" is historical lists of most popular baby names for each biological sex, found through google searches. Furthermore, I certainly have bias. Unfortunately, I have so much bias! As someone who, in grade school, was named Charlie, and had the shit beaten out of me for it, I very strongly know how it feels to be a man/boy who doesn't fit the socially-prescribed norms. However, when I am writing patches for code which determines a user's security, I do not play games; I want that code accepted, no matter what. The code submitted as a "female" was usually either rewritten by a certain gnupg developer (often to include further vulnerabilities and UX pitfalls) or later accepted as a "male". Lastly, if you submit 10-30 patches over a decade, it's not very hard to make the acceptance/rejection rates come out to nice even numbers. I don't think there's anything as high-level as "identity politics" at play here; I'm merely suggesting that there is bias and that all of us, no matter what our gender or how equitable we are, may have it.
- Sacho 10y ago
  > However that does not preclude someone else, of any gender, doing so more scientifically. And to answer your accusation: yes, I did submit the same patch twice, under two differently gendered names (although never publicly, I would copy-paste the patch and privately email it). I would have never suspected this, but I don't think it really addresses the concerns of unconscious bias. The scheme yummyfajitas proposed seems to address both(although you would have no way to prove the randomness of your choice). I'm sorry if I sounded too harsh in my initial criticism, it was all hypothetical without any knowledge of you or your specific situation.
- petertodd 10y ago
  It'd be good if you published a list of those patches. Also if anyone else repeats this experiment, note that there exist crypto techniques to make it more difficult for the experimenter to do the experiment dishonestly: https://twitter.com/petertoddbtc/status/811559259590660097 https://twitter.com/petertoddbtc/status/811559259590660097 (I would have said "prove the experimenters honesty", but those techniques aren't a 100% thing - you could for instance do the whole commitment/random beacon process multiple times, and only publish the one set that gets the results you want.) edit: To be clear, submitting the same patch twice isn't a good way to do this - if I were a maintainer and learned that an identical patch had been received by two different pseudonyms, I'd certainly consider rejecting it purely on the basis that something fishy was going on; that test doesn't say much about maintainer bias in normal situations.
- pjc50 10y ago
  "Charlie" is now coded sufficiently female to result in beatings? Jesus. Sorry to hear that. School is a vicious pit of norm enforcement at fist-point. The process by which names get "cooties" like this is rather nasty. People seem to be uncomfortable with even the possibility of both-gender names like "Sam" and "Alex".
pjc50 10y ago
> identity politics are the primary driver of motivation in the developer team There almost certainly is someone in the development team for whom this is important, given the reports that identical patches would be accepted after being "laundered" through a male dev. People will go a long way to defend their biases. For example, I expect this entire discussion to be flagged off the front two pages of HN and into oblivion fairly quickly. > meaningful study mjg59's reply: "Remember kids: never share your lived experiences without presenting your experimental design first".
- 1maginary 10y ago
  >I expect this entire discussion to be flagged off the front two pages of HN and into oblivion fairly quickly. I guess you were right. As soon as I was done reading your comment I refreshed the page and noticed it got flagged. Not sure what that means exactly on HN but it's kinda worrying that you were able to predict it so easily
  - davidgerard 10y ago
    dang appears, behaviourally, to have very strong and highly predictable preferences.
    - pjc50 10y ago
      I don't think this is done personally by dang, I believe it's a number of "anti-SJW" HN users using the flag button. mjg59 has observed this happening a lot before on e.g. http://hnrankings.info/13204464/ http://hnrankings.info/13204464/

by Sacho 10y ago

yummyfajitas 10y ago

First off, they call it A/B testing, but unless they were sending the same patch with a different pseudonym, then they're incorrectly using the term. This isn't true. As long as the patches were IID (e.g. she generated patches and randomly chose which name to submit them as), it's a legitimate A/B test. https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables https://en.wikipedia.org/wiki/Independent_and_identically_di...

isislovecruft 10y ago

First, you're correct: I never meant to conduct a meaningful study. It was purely of my own curiousity that I did this. However that does not preclude someone else, of any gender, doing so more scientifically. And to answer your accusation: yes, I did submit the same patch twice, under two differently gendered names (although never publicly, I would copy-paste the patch and privately email it). Second, my primary source for whether a name is "female- or male- sounding" is historical lists of most popular baby names for each biological sex, found through google searches. Furthermore, I certainly have bias. Unfortunately, I have so much bias! As someone who, in grade school, was named Charlie, and had the shit beaten out of me for it, I very strongly know how it feels to be a man/boy who doesn't fit the socially-prescribed norms. However, when I am writing patches for code which determines a user's security, I do not play games; I want that code accepted, no matter what. The code submitted as a "female" was usually either rewritten by a certain gnupg developer (often to include further vulnerabilities and UX pitfalls) or later accepted as a "male". Lastly, if you submit 10-30 patches over a decade, it's not very hard to make the acceptance/rejection rates come out to nice even numbers. I don't think there's anything as high-level as "identity politics" at play here; I'm merely suggesting that there is bias and that all of us, no matter what our gender or how equitable we are, may have it.

Sacho 10y ago
> However that does not preclude someone else, of any gender, doing so more scientifically. And to answer your accusation: yes, I did submit the same patch twice, under two differently gendered names (although never publicly, I would copy-paste the patch and privately email it). I would have never suspected this, but I don't think it really addresses the concerns of unconscious bias. The scheme yummyfajitas proposed seems to address both(although you would have no way to prove the randomness of your choice). I'm sorry if I sounded too harsh in my initial criticism, it was all hypothetical without any knowledge of you or your specific situation.
petertodd 10y ago
It'd be good if you published a list of those patches. Also if anyone else repeats this experiment, note that there exist crypto techniques to make it more difficult for the experimenter to do the experiment dishonestly: https://twitter.com/petertoddbtc/status/811559259590660097 https://twitter.com/petertoddbtc/status/811559259590660097 (I would have said "prove the experimenters honesty", but those techniques aren't a 100% thing - you could for instance do the whole commitment/random beacon process multiple times, and only publish the one set that gets the results you want.) edit: To be clear, submitting the same patch twice isn't a good way to do this - if I were a maintainer and learned that an identical patch had been received by two different pseudonyms, I'd certainly consider rejecting it purely on the basis that something fishy was going on; that test doesn't say much about maintainer bias in normal situations.
pjc50 10y ago
"Charlie" is now coded sufficiently female to result in beatings? Jesus. Sorry to hear that. School is a vicious pit of norm enforcement at fist-point. The process by which names get "cooties" like this is rather nasty. People seem to be uncomfortable with even the possibility of both-gender names like "Sam" and "Alex".

pjc50 10y ago

> identity politics are the primary driver of motivation in the developer team There almost certainly is someone in the development team for whom this is important, given the reports that identical patches would be accepted after being "laundered" through a male dev. People will go a long way to defend their biases. For example, I expect this entire discussion to be flagged off the front two pages of HN and into oblivion fairly quickly. > meaningful study mjg59's reply: "Remember kids: never share your lived experiences without presenting your experimental design first".

1maginary 10y ago
>I expect this entire discussion to be flagged off the front two pages of HN and into oblivion fairly quickly. I guess you were right. As soon as I was done reading your comment I refreshed the page and noticed it got flagged. Not sure what that means exactly on HN but it's kinda worrying that you were able to predict it so easily
- davidgerard 10y ago
  dang appears, behaviourally, to have very strong and highly predictable preferences.
  - pjc50 10y ago
    I don't think this is done personally by dang, I believe it's a number of "anti-SJW" HN users using the flag button. mjg59 has observed this happening a lot before on e.g. http://hnrankings.info/13204464/ http://hnrankings.info/13204464/