16 ms·
We built voice modulation to mask gender in technical interviews
- nxzero 10y agoIf gender bias is the issue, focus on bias detect and mitigation; not masking gender, which to me sends the wrong message.
- alansmitheebk 10y agoThis is ridiculous. At some point we need to face up to the reality that most women in the US are just not interested in working in technology. There is no evil conspiracy to keep women out of tech. Maybe high school guidance counselors and parents don't encourage women to go into tech. Maybe there is someone to blame, but it's not the tech industry's fault.
- hasenj 10y agoor D. A female who is not in the top 30% in terms of facial attractiveness (or is socially impaired for other reasons).
- sdml 10y agoHow critically are you judging the facial attractiveness and "social impairement" of male developers? This is a completely useless comment, and exactly the sexist kind of attitude that drives women out of tech.
- hasenj 10y agoIt's a mere observation. Not a comment of what should be the case. And yea, the average male in the field tends to be the same.
- argonaut 10y agoDid you even read the article? Nowhere does the author blame the tech industry. In fact, at the end, she is basically claiming that women tend to have less self-esteem when it comes to evaluating their performance.
- balls187 10y agoStrawman argument. The article delved into whether or not there was systemic bias against women in tech interviews, given that interviewing.io's data showed that men continued onto next rounds 1.4x more often. The conclusion based on the experiment, that it wasn't due to bias, but because women were more likely to not continue to refine their skills by continuing to interview. > When you do meet the rare American born female developer she is either: A. a designer who realized that there are no design jobs so she became a front-end dev; B. a first-generation Asian-American (B/C Asian parents steer their children into STEM); or C. it's a woman with blue hair and a pierced face who is there to "shake up the patriarchy" and make a statement about gender. This is a sexist comment (and borderline racist).
- deleted 10y ago[deleted]
- etjossem 10y agoI'd like you to read this. https://medium.com/tech-diversity-files/if-you-think-women-in-tech-is-just-a-pipeline-problem-you-haven-t-been-paying-attention-cb7a2073b996 https://medium.com/tech-diversity-files/if-you-think-women-i... The gender gap in the workforce is wider than the gender gap in computer science as a major. Particularly at senior and management levels, we see a severe underrepresentation of women. To me, that suggests either a hiring bias or an attrition problem, not a pipeline problem.
- tomp 10y agoMentioning senior/management levels is stupid. Obviously they will lag both the increase in the number of women studying CS in college and the decrease in sexism by employers. Wait 10-20 years, if the gap is still as wide as it is today, then it becomes a valid complaint.
- onion2k 10y agoAt some point we need to face up to the reality that most women in the US are just not interested in working in technology. Let's assume for a moment that this is true. I'm not sure it is, but we'll go with it. Software companies don't hire women who aren't interested in technology because those women don't apply for the job. The aim of ideas like the one in the article is to make the hiring process fairer by removing a gender bias from the pool of candidates who do apply. That's all. It doesn't imply that the candidate pool will change. The point is that every equally qualified candidate in that pool should have the same chance of getting the job. Better candidate should have higher chances. Biases skew that probability downwards, so we should aim to remove them. Otherwise we aren't hiring the best people.
- elcapitan 10y agoI want applicant-side voice modulation to mask my insecurity and non-native language skills.
- martinko 10y agoI want them to mask my weak technical skills.
- hasenj 10y agoI hear 10,000 hours of deliberate practice does wonders.
- malcolmgreaves 10y agoBoth maturity and strong communication are important for any professional position.
- ubernostrum 10y agoBoth maturity and strong communication are important for any professional position. So, shall you start firing executives at VC firms, or shall I?
- fredo699 10y agoIt's a simple fact that if a potential employee cannot communicate with the rest of the team, that is a detriment. The employer should be able to decide whether or not that detriment is enough to justify not hiring this person.
- sluggg 10y agoPeter Thiel ... ubernostrum is coming for you
- mmastrac 10y agoI had a demo from Aline on this last week after seeing it on Twitter and reaching out. It was a really interesting experience - it masks the voice pitch but leaves the person's characteristic of speaking (is there a word for that?) intact. You can also access the recordings after-the-fact to hear how you did. I think this is valuable for candidates who want to improve their interviewing skills. FWIW, I heard it both ways - my voice as female and her voice as male. The pitch shifting was very convincing both ways.
- aylons 10y ago> person's characteristic of speaking (is there a word for that?) I think you're talking about prosody: https://en.wikipedia.org/wiki/Prosody_(linguistics) https://en.wikipedia.org/wiki/Prosody_(linguistics)
- anon2016 10y agoSo it masks gender by making everyone sound "like a dude" according to the videos. That's not masking gender. Make everyone sound like a robot and the blog post title will be more accurate.
- StavrosK 10y agoYeah, I cringed when they said they thought of "giving all females a bane mask". How does that do anything? Anyone getting a person wearing a bane mask would automatically realize they were female.
- vonmoltke 10y agoThe point of the experiment was for the interviewer to think the interviewer was a particular gender to see if they as it based on that impression. Making everyone sound like a robot does not accomplish that goal, so I don't see how it makes the title more accurate.
- StavrosK 10y agoIt makes everyone sound the same. Changing only women's voices is a tell-tale sign. If you get a guy who sounds "computery", or "off", it's a girl.
- lotharbot 10y agoThey didn't only change women's voices. They also changed some men to sound like women. And they made some voices sound processed without pitch changes.
- jarmitage 10y agoAmazing how the author believes they have proved that there is 'no systematic bias' with This One Simple Trick! Edit: sorry for the tone, but I just find this whole article bait. The headline is bait. The writer acknowledges the limitations of their study (basically that it doesn't prove much) but then makes a whole bunch of extrapolations anyway, and makes those guesses the meat of the article rather than investigating why their methods didn't work or how they could be improved or what would be required to produce conclusive evidence.
- StavrosK 10y agoYes, that's exactly what they say: > On the subject of sample size, we have no delusions that this is the be-all and end-all of pronouncements on the subject of gender and interview performance. But let's ignore science to fit our pre-existing biases!
- dang 10y agoThat comment was snarky, dismissive, and inaccurate, but please don't respond with snark of your own.
- minimaxir 10y ago> Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance The main issue with gender and hiring is unconscious bias on the interviewer's end, which voice modulation does make sense as a tool to avoid bias. Per the grading criteria, they are judging on skill, problem solving, and communication. I'm not sure how voice modulation would affect the interviewee in those three criteria, which makes the results of the experiment not surprising.
- StavrosK 10y agoThe interviewer is the one grading the interviewee.
- minimaxir 10y agoRight, which highlights another problem: interview.io's interviewers are presumably trained to be more objective and stringent on the criteria. This may introduce a selection bias. Untrained interviewers, like the ones at startups randomly pulled into interviews and also hiring for culture fit, may have a bias that is more conscious.
- Karunamon 10y ago>interview.io's interviewers are presumably trained to be more objective and stringent on the criteria. The common thought is that bias is an unconscious and unavoidable thing, hence the voice masking. If that's truly the case, all the training in the world doesn't help.
- benkuhn 10y agoI would really have liked to see them check whether the gender-blinding actually worked. Even if someone's voice is modulated, they may have gendered behavior patterns that could influence someone's performance ratings. I wouldn't be surprised if I could guess the gender of a voice-modulated person pretty accurately from other cues. To combat this, they could have asked the interviewer during the performance assessment whether they thought the applicant was definitely male/probably male/unsure/probably female/definitely female. Then you could use voice modulation as an instrument for perceived gender and get a better estimate of the true effect of perceived gender when controlling for actual gender. Hope they do something like that for the next round of experiments!
- killface 10y agoIf you're tracking the interviewer gender guess, that's a legal liability to the interviewer. I'd refuse to answer, or always put "unsure" even if I was 100% sure of the gender.
- Jemmeh 10y agoThis would be a fantastic additional thing to monitor for. It was my first thought after hearing the voice modulation clips-- even in a manly voice, someone saying, "Oh I totally sound like a dude" sounded quite feminine since most guys wouldn't say "totally." Two things I noticed from the clip on the FastCompany article-- it's modulated, but I can hear A.) Vocal Fry B.) Statements that sound like they're ending with a question (the voice goes up at the end.). These are both more common speaking patterns with women. The question thing in particular is common with people who are not as confident.
- jsprogrammer 10y agoWhat evidence do you have that some speaking patterns are more common with women? Can you link to some data? Edit: Lots of opinions here. Still no data.
- MasterScrat 10y agoI don't know about cues in spoken English, but you can find relevant articles about written text from this page: https://civic.mit.edu/blog/natematias/best-practices-for-ethical-gender-research-at-very-large-scales https://civic.mit.edu/blog/natematias/best-practices-for-eth... Read under "Inferring Gender from Content".
- sevensor 10y agoAs we keep learning to our dismay, you can't solve social problems with technology. I've got to give them credit for trying, and for reporting the result, but I'm not at all surprised.
- virgil_disgr4ce 10y agoI was excited about this until I realized the sample size is incredibly small and there are few controls :/
- gravypod 10y agoWhy not just use text? There are many speech characteristics of gender that will never be found and masked by modulation. Simple placements of words, parts of speech referring to the self, and other things are all much better clues to gender then voice frequency and pitch. For example, when speaking to some people who I know who are either naturally very high or low pitch, or are trans-gendered, there is a lot more information that can be contextually extracted from the content of their speech. Really, just do text-based interviews. It will allow people to revise what they are thinking and really allow them to mask their gender.
- BoysenberryPi 10y agoBecause your actual interview will not be text-based and if their goal is to simulate an actual interview scenario where you will be speaking out loud an expected to form coherent thoughts to questions on the fly this comes closer to doing so than text based.
- mwcampbell 10y agoAnd why shouldn't the actual interview be text-based? Yes, that would be a drastic change, and it would render on-site interviews pointless. But then again, it would eliminate a lot more biases, e.g.based on accent and disabilities.
- jacobr 10y agoMany jobs will use mostly written communication, then it doesn't matter much how you perform verbally, and a text-based interview would probably work well, or even better. If you're expected to cooperate closely and communicate verbally with colleagues, you would want to know how the candidate handles that before hiring them.
- BoysenberryPi 10y agoYou're basically putting up a straw-man and attacking it at this point. I never claimed the actual interviews should be verbal but the fact is they are and they are preparing you for what actual interviews are like not what ideal interviews are like.
- andrewclunn 10y agoI wonder if the voice modulation had any discernible impact on communication ability across the board.
- myohan 10y agocool but it's sad to know that this is required to maintain gender equality
- huherto 10y agoIt actually proves the opposite. That there was not systematic bias.
- myohan 10y agoi think it is pretty obvious there is no systematic bias against women it's more of reminiscent of an ancient culture of mistreating women...i think it is rare to find in a modern/western corporate world, group of folks who systematically and in organized way trying to oppress women
- rottyguy 10y agoI always felt they should do something like this for politicians during a race. No names, age, gender, political leanings, etc. Just interviews and debate.
- alphapapa 10y agoNot sure exactly what you mean. But some kind of anonymous/pseudonymous debate or interview process would not be a good idea, even though it would have some nice benefits in theory. For example, it would be easy for a malicious candidate to profess views opposite to what he believes in order to be elected. Then he could pursue his actual policies after taking office. Oh, wait, this happens now... :( But it would be even easier without true identity and non-verbal cues, etc. Words on a screen/page could be written by anyone. Real identity is crucial, because what matters more than words is actions, and only with a real name, face, and history can past actions be examined to determine whether the person lives up to their rhetoric. If political history teaches us anything, it should teach us to pay less attention to what politicians say and more attention to what they have done. So, as interesting as debates can be, maybe we need less of those and more examination of candidates' past. If we really want to know what a candidate believes and what he will do if elected, we should look at what he's done in the past.
- j0e1 10y ago> Maybe tying coding to sex is a bit tenuous, but, as they say, programming is like sex — one mistake and you have to support it for the rest of your life. Hilarious! But I wonder if it is true.
- sluggg 10y agoLess and less true as most companies move towards a more agile development cycle
- breitling 10y agoPlanet Money did a great podcast on this recently [1]. It's pretty cool how it works. [1] http://www.npr.org/sections/money/2016/04/22/475339930/episode-697-help-wanted http://www.npr.org/sections/money/2016/04/22/475339930/episo...
- awesomepantsm 10y agoSo their conclusion is that the people who do worse, use the product they happen to sell less? Sounds like bullshit to me.
- ranko 10y agoGender blinding in orchestral auditions (which usually involves screening the performers from sight and thus is pretty-much foolproof) has been shown to improve the percentage of women who are hired. See, for example, 'Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians', Claudia Goldin, Cecilia Rouse: http://www.nber.org/papers/w5903 http://www.nber.org/papers/w5903
- Kevin_S 10y agoThat's interesting, I would have thought being able to see actual finger movements to analyze mechanics is a part of the evaluation process. Could be totally off base though I suppose.
- qq66 10y agoGood mechanics are a given at that level of performance. If you can play like Yo-Yo Ma with bad mechanics, more power to you, an orchestra is not going to care.
- Bartweiss 10y agoIf anything, I would imagine it's like the old story about drafting athletes. Given two equally successful point guards, you draft the one with crappy technique because he has more room to grow. If you're hiring for some mid-level orchestra, bad mechanics on a sufficiently-skilled player might be enticing. And as you say, bad mechanics among top performers are basically nonexistent.
- yanilkr 10y agoSweet that some one is trying new things with ideas. But Oh boy, Life does not revolve around tech interviews. Make peace with it. An old school psychologist would observe us and say that the people obsessed with tech interviews faced an emotional toll of rejection during early career and are trying to get over it. Google, Facebook and other companies are like parents of the tech kids and the kids are seeking some kind of approval or validation from them via passing their technical interviews. You are not cool enough if you don't crack our interview. Its possible we are doing something very wrong here.
- kylestlb 10y agoMaybe interviewing is over-important because of the frequency in which tech workers/engineers change jobs.
- titanomachy 10y agoMaybe un(der)employed programmers, who are naturally more concerned about interviewing, have extra free time and are thus over-represented on hacker news.
- prodigal_erik 10y agoEven when I'm not the candidate, I'm asked to conduct a hell of a lot of interviews at work, and it's a huge time sink when the signal/noise is poor.
- sluggg 10y agoyes, I believe you are completely correct. It also seems like a common way for the engineering community as a whole to gain insight into a company's culture. It seems to me that these two ideas go hand in hand in a encouraging our fetishization of the tech interview
- Jemmeh 10y agoAnd tied into this, job changes are frequently said to be one of the best ways to get a raise.
- AndreyErmakov 10y agoSo we all realize technical interviews serve no practical purpose and should be quietly abandoned, but instead of looking into the future and discovering better ways of identifying talent, people keep inventing stuff that makes the torturing mechanism even more sophisticated so that the suffering can be prolonged. I'd like to see the people behind this project apply their technical skills to something more useful to the industry and the society in general. >> to get to pipeline parity, we actually have to increase the number of women studying computer science by an entire order of magnitude Any woman who's gone through that process will likely want to get another career, that where people are treated with more respect. And I suppose many men are having the exact same thoughts. The more you mock your talent pool, the more actively that talent is running in the opposite direction, just to get out of this mess.
- crusso 10y agoSo we all realize technical interviews serve no practical purpose and should be quietly abandoned We don't really know that at all. Technical interviews are an invaluable tool in finding people who can write software and be collaborative co-workers. A lot of software developers on the receiving end don't like them. Part of that is because a good technical interview pushes you out of your comfort zone and forces you to show that you can think through problems. Not all developers appreciate being taken out of their comfort zone. The other part of the problem is that some technical interviews are conducted clumsily. "Code FizzBuzz, go!" As with the rest of the interview, skill of the interviewer matters.
- AndreyErmakov 10y agoWhat is the point of pushing people out of their comfort zone? Practical software engineering doesn't happen when you're uncomfortable. Discovering quality solutions requires concentration and immersion into problems.
- crusso 10y agoYMMV, but I've always worked in service-oriented development organizations for startups or companies that behaved like startups on the development side. Pressure comes with the territory and I'd rather know how potential colleagues behave with a small amount of interview pressure before being stuck with a veritable basket case in a real-world high-pressure situation. I had a guy freak out walking through a simple algorithm in java at the white board in an interview. Maybe he was the best developer ever if he were put in a quiet dark room and never made eye contact with anyone. Given that we had other candidates who could actually interact calmly and intelligently in an interview/collaborative situation, we weren't going to go with the guy that freaked out.
- blisterpeanuts 10y ago"...it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women..." The author admits it's not statistically significant, but nonetheless it somewhat makes sense, if you take human nature into account. If there's any bias in the technology field, it's probably that males are assumed "smarter", but women are still liked and desired. So, if a female is not answering questions as sharply, but is masked as a male, the bias is to deduct extra points. If a male is answering Q's while masked as a female, the interviewer is pleased and awards extra points. Just my theory, but it seems to fit. Again, though... a very small sampling.
- smokeyj 10y agoI'd be more interested in quantifying productivity and seeing if there's a systematic discrepancy between productivity and gender. Either the market is accurately pricing talent or there's a delta to be exploited. I think if a paper was released tomorrow that said "female programmers 30% undervalued as observed by double blind coding challenge" head hunters would fix that problem overnight.
- cortesoft 10y agoI think that the problem with your theory is that it relies on the idea that success on a coding challenge translates directly to value as an employee. I have managed many developers...... some of the ones that are the best coders are NOT the most productive nor the most valuable employees. There are a lot of skills that are needed besides ability to code.
- ketzu 10y agoAre you sure that a single data point like this would overthrow the "Intuition" of recruiters? If I understand [1] correct, thats fairly unlikely. [1] http://onlinelibrary.wiley.com/doi/10.1111/j.1754-9434.2008.00058.x/abstract;jsessionid=811D0CA936B1BB858AF3A9C0E644C500.f01t03?userIsAuthenticated=false&deniedAccessCustomisedMessage= http://onlinelibrary.wiley.com/doi/10.1111/j.1754-9434.2008....
- yummyfajitas 10y agoThis was studied among lawyers. Female lawyers were significantly less productive. https://www.upf.edu/rs/_pdf/jornadesGenere/GenderGaps_Ferrer.pdf https://www.upf.edu/rs/_pdf/jornadesGenere/GenderGaps_Ferrer...
- patmcc 10y agoWhile interesting, that's a very particular definition of "productive" - male lawyers billed more hours. That doesn't necessarily correspond to more actual work done.
- lettergram 10y ago
- humbleMouse 10y agoThis is a nice idea but seems like political correctness taken to the extreme.
- grb423 10y agoIs anybody studying the reasons behind the huge gender disparity in roofing, welding or kindergarten teaching? I think there is a tremendous gender bias in those and other fields that is going unstudied, because nobody cares or because those fields aren't as cool or important.
- rglover 10y agoThank you for bringing this up.
- Kliment 10y agoSomeone I know worked as a welder at a very high level of experience and skill. She reported constant sexist comments, abuse and obstruction from male coworkers, particularly those whose work she was sent to inspect and/or repair.
- Frondo 10y agoI have an aunt who's a civil engineer. When she started her business, she didn't use her first name--only her first initial--for all her professional communications. She didn't want people who didn't know her to know she was a woman.
- rcpt 10y agoSurgery has a large gap to but I rarely hear it brought up in these discussions. http://www.ama-assn.org/ama/ama-wire/post/medical-specialties-vary-gender http://www.ama-assn.org/ama/ama-wire/post/medical-specialtie...
- Bartweiss 10y agoThat's a particularly interesting one, because it's not susceptible to the usual charges of "no one cares because it's low pay/status". If you look to radiology rather than surgery, the gender imbalance is enormous. Suddenly we're looking at gender skew in two highly technical, prestigious, lucrative fields, and seeing that they prompt totally different types and amounts of discussion.
- hasenj 10y agoHonestly the modulated voice sounds like a gay man, or a transgender, or someone who is physically male but is trying hard to speak like a female. It's probably not the pitch itself but the manner of speaking.
- mxfh 10y agoCan you pick your own filter? Would go for Laurie Anderson's Voice of Authority anytime. https://www.youtube.com/watch?v=YajQNIAY78k&feature=youtu.be&t=37 https://www.youtube.com/watch?v=YajQNIAY78k&feature=youtu.be... Used to great extent here: The Cultural Ambassador - Live https://open.spotify.com/track/3uM0L8r6lJGaU54v9RjwJK https://open.spotify.com/track/3uM0L8r6lJGaU54v9RjwJK
- dang 10y agoI didn't realize it had a name. How wonderful.
- hharnisch 10y agoThis is interesting because it starts to peel another layer of the problem. Yes there are still cues that can give away gender, but this seems another step closer to making the interview process a science. The next logical question from here is "why are women more likely to quit after 1-2 bad interviews".
- jbmorgado 10y agoI don't think you understood the article. The conclusion is that there is no problem. This bias against woman in IT interviews people keep talking about, doesn't actually exist according to the study.
- imh 10y agoThis misunderstanding is kind of the core breakdown in these conversations. People don't agree on what the problem is. Is the issue about equal pay, or about equal pay for equal work? In hiring terms, is it about equal hiring or equal hiring for equal ability? If we don't have equal pay for equal work (and equivalent skill, etc), I think we can agree that's unfair and a problem. But if we do have equal pay for equal work, but still don't have equal pay, then I'd say we may still have a problem. Why aren't women making the same work decisions as men? If social pressure is keeping women from doing equal work, that's bad. If social pressure is forcing men to make shittier sacrifices for pay, that's bad. In the article linked, women churn more. That's probably bad. If there's inequality without unequal pay or unequal hiring, then the problem isn't the fault of the firms, but it's still a problem.
- gmarx 10y agoThe result does not surprise me at all. I realize this was not the kind of study that would pass rigorous review. I am surprised the author thinks this is a surprising result. The kind of people who do technical interviews have been taught since childhood that the system is biased against women and that they need to look out for this bias. If anything I would think the occasional super tech woman would be enough of a surprise that the average interviewer would have an unconscious bias in favor of her
- Jemaclus 10y agoHi Aline! Great read, as always. I had a couple of thoughts while reading: 1) One of the things you mentioned was that men whose voices were modulated to sound like women tended to score better than women whose voices were modulated to sound like men. Do you think that's because the men actually performed better than women, or do you think it's possible that interviewers have a lower set of expectations for female interviewees? Or some other reason? My hypothesis at this point is that, all other things being equal, it seems like if an interviewer expects you to do poorly because of an unconscious bias against women, and you actually do perfectly average, the interviewer might rate you higher simply because you surprised him. Does that make sense? 2) How did your partners (Mattermark, Yelp, etc) react to your findings? Will this change how they hire in the future or how they use interviewing.io? 3) Given enough time and resources, do you think it's really possible to eliminate gender bias in interviews (through techniques like this, or otherwise), or do you think the best we can do is minimize it? 4) Piggybacking off #3, do you do follow-ups with the companies after you've placed people? I'd be interested to see whether women who did well in the interview actually do well at the companies long-term -- perhaps those biases extend beyond the interviewing process, as well! Keep up the good work! :)
- tekcor 10y ago> "1) One of the things you mentioned was that men whose voices were modulated to sound like women tended to score better than women whose voices were modulated to sound like men. Do you think that's because the men actually performed better than women, or do you think it's possible that interviewers have a lower set of expectations for female interviewees? Or some other reason? My hypothesis at this point is that, all other things being equal, it seems like if an interviewer expects you to do poorly because of an unconscious bias against women, and you actually do perfectly average, the interviewer might rate you higher simply because you surprised him. Does that make sense?" I was actually expecting this even before reading the results. I believe this is because most companies try to have gender balance which tend to result in females being hired with lesser "score" than males, simply because of disparity between in pool male and female candidates available (even if you consider they perform equally well). Also I would say this is a bias towards females not against.
- vonklaus 10y agoHiring is a complex process that is extremely subjective accross role, organization and sub-sector of the tech industry. To the extent this result is correct within the population of interviewing.io; it is in fact quite positive on many levels, if unfortunate on others. Specifically-- again accepting the methodology used here, interviewers are unbiased in their reviewing. If we consider simply this subset however, I suspect there are several factors which contributed to this result: * As indicated by others, it was unclear if this actually successfully modulated voice AND that other factors of influence did not leak gender to the interviewer. * interviewing.io may simply attract men who are better than women. * The current climate provides a lot of resources and support for women entering tech. Many organizations have made a large push to hire women and thus the top and mid tier women (who are significantly less numerous than male counterparts) are hired into organizations and thus not applying here. * places where candidates learn about interviewing.io could differ based on gender. A contrived example being males learning about it on HN, while female counterparts learning about it through a short part-time coding bootcamp. * there was an experience gap or significant skill gap between genders. This was alluded to above. * women on interview.io are generally worse programmers or perform worse in technical interviews. * not enough data for statistical significance. This is a pretty interesting result and could actually be a positive thing for interview.io. It is possible they are objectively evaluating candidates and it is simply a marketing problem which they can adjust for. There are also some non-trivial differences between men and women which likely matter even in this context. Amy Cuddy does an amazing TED talk (and a longer one as well) about poses and cues and their effects on perception. If the idea of this investigation was to make a larger observation about the industry, it would be interesting if they could correct for experience & skill level possibly by something completely objective like HackerRank for example. If seperated into 3 skill bands, it would be interesting to compare the actual interview results accross similarly skilled populations. To correct for bias it may be useful to tell interviewers the candidates will be anonymized modulating both voices to try and have both genders voices sound alike as a single neutral voice, ideally while still allowing for pitch and intonation. Be interested to see a follow up as the site receives more candidates and exposure and grows their organization, ect.
- vonklaus 10y agoHiring is a complex process that is extremely subjective accross role, organization and sub-sector of the tech industry. To the extent this result is correct within the population of interviewing.io; it is in fact quite positive on many levels, if unfortunate on others. Specifically-- again accepting the methodology used here, interviewers are unbiased in their reviewing. If we consider simply this subset however, I suspect there are several factors which contributed to this result: * As indicated by others, it was unclear if this actually successfully modulated voice AND that other factors of influence did not leak gender to the interviewer. * interviewing.io may simply attract men who are better than women. * The current climate provides a lot of resources and support for women entering tech. Many organizations have made a large push to hire women and thus the top and mid tier women (who are significantly less numerous than male counterparts) are hired into organizations and thus not applying here. * places where candidates learn about interviewing.io could differ based on gender. A contrived example being males learning about it on HN, while female counterparts learning about it through a short part-time coding bootcamp. * there was an experience gap or significant skill gap between genders. This was alluded to above. * women on interview.io are generally worse programmers or perform worse in technical interviews. * not enough data for statistical significance. This is a pretty interesting result and could actually be a positive thing for interview.io. It is possible they are objectively evaluating candidates and it is simply a marketing problem which they can adjust for. There are also some non-trivial differences between men and women which likely matter even in this context. Amy Cuddy does an amazing TED talk (and a longer one as well) about poses and cues and their effects on perception. If the idea of this investigation was to make a larger observation about the industry, it would be interesting if they could correct for experience & skill level possibly by something completely objective like HackerRank for example. If seperated into 3 skill bands, it would be interesting to compare the actual interview results accross similarly skilled populations. To correct for bias it may be useful to tell interviewers the candidates will be anonymized modulating both voices to try and have both genders voices sound alike as a single neutral voice, ideally while still allowing for pitch and intonation. Be interested to see a follow up as the site receives more candidates and exposure and grows their organization, ect.
- vegabook 10y agoSo the point here, hopefully, is to allow truly talented female coders to overcome the bias against women generally, which exists because of the fallacy of averages. Hopefully, the idea is not to push the erroneous view that women are on average equally good at coding than men. That is objectively untrue. I can, with enthusiasm, support a technology which helps the (smaller number of) women who are genuinely good coders to shine through the bias. I cannot accept any technology which seeks to mask the reality that in this particular domain of modern life (coding), men are usually better performers than women. Because that would be regressive.
- infinite8s 10y agoDo you have a reference or any data to support your assertion that women on average are not equally as good at coding as men?
- yummyfajitas 10y agoIt would be great if the author posted the raw numbers, so we could check the statistical analysis for ourselves. I'm concerned by the low number of females in the sample - was it high enough for the study to have significant statistical power?
- bitL 10y agoSo, is it like interviewing vocodered Smurfs?
- balls187 10y ago> it’s not about systemic bias against women or women being bad at computers or whatever. Rather, it’s about women being bad at dusting themselves off after failing, which, despite everything, is probably a lot easier to fix. This conclusion (imo) should have been bolded. I disagree that getting women to "dust themselves off" is an easier fix than if there were inherent bias against women in interviewing. My theory is that this is tied to (In the US) men being more tolerant of taking risks, which leads to being more tolerant of failure, which leads to being able to more easily "dust oneself off." I'd imagine if you looked at women in this study who played competitive sports, they would be less likely to leave the platform. Risk-aversion might also contribute to women being less likely to ask for a promotion, raise, or negotiate salary. I disagree that "dusting yourself off" as being an easy fix similarly for why I think "pick yourself up by your bootstraps" is not an effective way to elevate people out of poverty. There are more cultural biases to overcome than a simple behavior change.
- zeveb 10y ago> My theory is that this is tied to (In the US) men being more tolerant of taking risks, which leads to being more tolerant of failure, which leads to being able to more easily "dust oneself off." There are some pretty obvious evolutionary psychology reasons for men to take more risks. Whether evolutionary psychology is just a bunch of just-so stories or not, that's another discussion.
- deleted 10y ago[deleted]
- cauterized 10y agoThere are plenty of enculturation reasons too. Doesn't need to have an evolutionary or biological basis.
- generalledger 10y agoI don't know if this is true, but intuitively it seems that something is more likely to be ingrained in a culture if there is already a biological basis for the behavior.
- Jemmeh 10y ago>"Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance." I've seen quite a few studies pointing out this "confidence gap" now. They have shown women to be less confident, and it shows in their actions and speaking patterns. I still hear these speaking patterns even in the voice modulated versions-- statements sounding like questions with the upward inflection at the end. Vocal fry. Lots of "uhm, well, erm" flustered speak. So it doesn't surprise me that even with a modulated voice these women still sound unsure of themselves and thus they still didn't interview as well. As women there are some actions we can take. It's good to be aware of these speaking patterns and try to break them--but ultimately there are so many of these patterns that show that the only real solution is to try to internalize confidence. It's easier said than done. And we can't ignore the fact that there are reasons outside of ourselves for why women are not as confident as men. It's not just an internal issue. While we can take some actions for ourselves, we should still be looking at the external reasons that caused the issue in the first place. Why are large amounts of women less confident compared to men? Let's tackle those issues, too. When women try to be confident, it can get frowned upon. A man being assertive is a leader, a woman is bossy. Women are quicker to hit walls for just how confident they're allowed to be before they get the "bossy/bitchy" label thrown at them. The below article makes good counterpoints for the confidence-gap, and provides sources to some studies regarding women and confidence. And I've seen women call each other bitchy/bossy, too. This is something that society as a whole needs to work on. https://www.theguardian.com/commentisfree/2014/apr/23/female-confidence-gap-katty-kay-claire-shipman https://www.theguardian.com/commentisfree/2014/apr/23/female...
- deleted 10y ago[deleted]
- jomamaxx 10y ago"When women try to be confident, it can get frowned upon. A man being assertive is a leader, a woman is bossy. Women are quicker to hit walls for just how confident they're allowed to be before they get the "bossy/bitchy" label thrown at them." Confidence has nothing to being 'aggressive' or 'bitchy' or even necessarily 'assertive'. Confidence usually means a lack of fear or anxiety when dealing with tough situations, or the lack of enough hubris to stand up to the anxiety in such scenarios.
- kafkaesq 10y agoWhich (by design) would unfortunately mask character, warmth, and authenticity, also.
- koolba 10y agoInterviwer: "Hi, my name is Roger. I'll be interviewing you today." Interviewee (modulated voice): "Hi Roger. Nice to meet you. My name is Alice." Hmmm ... Speech hints aside, this may not work 100%.
- onetwotree 10y agoYeah, also any linguist can tell you that women use language differently than men. Even in written communication you can usually, consciously or not, tell unless someone is very deliberately trying to appear to be the other gender.
- deleted 10y ago[deleted]
- thaumasiotes 10y agoHow are they getting attrition numbers? I signed up with interviewing.io many months ago, and never heard from them again. Sure enough, their website still advertises "join the waiting list" rather than "join". Any significant attrition should leave them completely devoid of users... right?
- xienze 10y agoI honestly feel like this sort of thing is a waste of time. Let's pretend someone invents the perfect "gender bias-free" system: no names, no faces, voices are flawlessly transcribed on-the-fly so you can't pick up on speech patterns. What if it works perfectly and you still end up hiring more men than women? I think exercises like this may end up giving people answers they don't want to hear. So, stop wasting time with stuff like this. If you're bothered by the fact that there's more men in your company than women just hire more women until you get your desired ratio. You've already made your minds up ("there's too many men here, clearly we're biased against women during the hiring process") and I doubt anything will make you think differently, so just go ahead and fix the imbalance.
- CobrastanJorji 10y agoI'm no expert, but isn't "let's just hire only women for a while to balance out the numbers" going to violate a whole bunch of employment regulations?
- killface 10y agoNo. If you're a white male, that's bad. Anything else is diversity. They'll come up with some bullshit like "since women aren't the majority of hiring managers, this doesn't count as discrimination" or something. Like they have with racism (where black people in America literally can't be racist).
- balls187 10y agoI'll just point out that white males are very well represented in Tech.
- nikdaheratik 10y agoOf course, the corollary to what you're saying is: "If you're a white male and you get hired, that's on merit, everyone else is hired for diversity reasons."
- bionsuba 10y ago
- ajmurmann 10y agoAre there actually studies that show that the interview process rather than the pipeline is the issue? At my previous job we ended up making an offer to every female candidate who made it to the phone screen. This wasn't because we preferred them because of their gender but because they were really good. Many men never even made it past the phone screen. We still ended up with a majority of men on the team.
- teamhappy 10y agoNote to self: Write screen play about Equilibrium-esque future were everybody wears burkas with built-in voice decoders.
- dghughes 10y agoThere's more to a person than the tone of their voice men and women speak differently use different phrases, words and cadence.
- onetwotree 10y agoSo I don't think that hiding factors that (we think might) influence biased people is the right way to address bias. Let's say a woman uses this voice modulator in an interview and gets a job that she wouldn't have otherwise. Now she's going to work in the same shitty, sexist environment that would have denied her the job had they known she was a woman. What problem have we actually solved here? I think that the problem isn't so much that women can't get jobs in tech as that they don't want to work in an industry that is (correctly or not) perceived as being a sexist sausagefest. We have to change attitudes and culture to address bias, not simply put on blindfolds.
- NetTechM 10y agoI liked this article, well written and thought provoking. I am curious if this attrition effect plays a large part in graduate rates from STEM programs.
- randyrand 10y agoSomeone off-topic but: Laws forbidding sexual and racial discrimination in hiring are a waste for everyone's time involved. Do I really want to work alongside someone that "isn't racist" or sexist just because the law forbids it? No of course not. I'd rather work next to people that aren't racist and sexists regardless of if the law forbids it. I want to work at companies where people actually want me. How difficult is that to understand? =( Most people want less racism/sexism. But this is not the way to do it. I'm in favor of removing this law and letting discriminators be upfront about it. So that I can knowingly avoid them without wasting hours and potentially years of my life.
- xenadu02 10y agoYou are clearly completely incorrect... Like so obviously incorrect I'm almost shocked to see such a comment here on HN. Making discrimination illegal sends a social signal that the behavior is bad and shouldn't be tolerated. It emboldens those who are slightly anti-discrimination (but part of the majority group) to speak out against it instead of following the herd. The few cases where actual discrimination cases are prosecuted serve to warn others that there are negative consequences to bad behavior. Over time this change in attitude transfers to younger generations that see it as normal. That leads to fewer and fewer racist/biased people in positions of power. Often institutionalized racism and sexism are enforced by a small group of busybodies... Take Cory Booker's family story where the Real Estate agent tried to enforce a "white only" town but once they bought the house (by using a white stand-in couple) all the neighbors were totally fine with a black family moving in. It took only one bad actor to effectively segregate an entire town. I have a suspicion (but no proof) that this is generally the case - a vocal minority wants to enforce the status quo but the majority would be fine with change. We can't stop some people from discriminating, but we can declare that we as a society think that is bad behavior and that declaration alone has positive consequences.
- hasenj 10y ago> You are clearly completely incorrect... Like so obviously incorrect I'm almost shocked to see such a comment here on HN. Your first line is terrible. Remove it and your comment would be a valuable contribution to the discussion. > Making discrimination illegal sends a social signal that the behavior is bad and shouldn't be tolerated. Does it? It only signals that the government doesn't like said behavior.
- zw123456 10y agoI think this is a great effort and I applaud their efforts. There are also a lot of very good points made in all the comments below, summary: 1. Simply changing the frequency of the voice does not necessarily mask gender, other intonations are still present or make the person sound perhaps gay or otherwise still feminine sounding. 2. There are other socialization issues as well as biological/chemical that make men more aggressive than women. One of the key things I have learned over the years is an appreciation for the different way that women approach problem solving than men, both technical problem solving as well as organizational. Both have merits, but a blend can be very powerful, a diverse work group has intangible value. I think rather than attempting to create a male avatar for women this lesson needs to be embraced more widely.
- fractalsea 10y agoIn the intro you say that data from >1000 interviews shows: > men were getting advanced to the next round 1.4 times more than women > men [...] had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women The conclusion of your experiment was > masking gender had no effect on interview performance So the question is: why do women perform worse (by the above metrics) even if the interviewer does not know they are a women? Unfortunately you spend the remainder of the article showing that women are more likely to quit after bad interviews, and hypothesise that this is due to lower self confidence. This is interesting and all, but it does nothing towards explaining the discrepancy you described in the introduction!
- deleted 10y ago[deleted]
- alarge 10y agoI guess the results here would surprise you if you assumed that hiring bias was primarily the result of an explicit bias against {gender, race, etc.}. While there is certainly this class of bias, I personally haven't found as much of it in the technology field as there is purported to be in general society. What I have found, instead, is a more subtle bias that goes something like this: "I view myself (or that guy over there) as the model employee. I'm looking for someone who thinks and acts like we do -- not just on our level, but views problems the same way, is likely to arrive at the same solutions, has the same energy level, etc.". I believe this attitude forms the fundamental underpinning of both company culture and hiring bias. I've seen it work successfully over the short term (particularly in startups where you're trying to build a small team of really smart people that can build something quickly), but I think it is counter-productive over the longer term. When I'm hiring, I simultaneously try to answer three questions: (1) Is this person capable of doing the job I'm hiring them for? (or grow into being capable), (2) Is this someone I could work with and who could work with my team?, and (3) What unique life experiences or perspectives does this person have that they could bring to the team and improve our culture or decision making? What I've found is that some folks over-focus on (1) and sometimes (2) without adequately recognizing the value of (3). I personally believe that you don't remove bias by trying to legislate against it - you remove it by getting people to desire the alternative - diversity.
- NTDF9 10y agoHere's an alternate hypothesis. Maybe, just maybe, men and women are different? Maybe men and women have different strengths and aptitudes?
- auganov 10y agoCould you redo the classic study where you just replay the same performance to every interviewer but change up the name to be strongly associated with either gender? It's been widely replicated but still cool to see redone in a modern setting.
- scotty79 10y agoDid women that quit after one or two bad rewiews have same avg score as men who did the same thing?