8 ms·

The Impact of “Blind” Auditions on Female Musicians (1997)

52 points by qiqing 12y ago

teraflop 12y ago
> Female musicians in the top five symphony orchestras in the United States were less than 5% of all players in 1970 but are 25% today. The article was published in 1997; it would be interesting to see some more recent numbers. I searched briefly and couldn't find anything definitive.
tomlock 12y ago
This seems to suggest that when we know someone is a woman, we judge them more harshly.
pervycreeper 12y ago
The original study does not appear to have had any control groups, unfortunately. There are too many possible confounding factors present to be able to attribute any of the difference definitively to bias.
cottonseed 12y ago
Someone should build an job search/interview site that gives "blind" interviews: scrubs candidates (application materials, resumes, etc.) of gender, race, age, etc. You might not be able to blind the whole process, but you could blind the preliminary steps. Even that would probably effect the composition of the final selections, as it did here.
- adamwong246 12y ago
  I had a similar idea: "Meritaco- a double-blind git middleman that hides identifying information from pull requests."
- picks_at_nits 12y ago
  A similar thing has been done many times by submitting the same resumé with male and females names, or with names statistically more likely to be white and names statistically more likely to be a person of colour, with different ages, and so forth. Every time they run a test like that, they discover that who you appear to be matters more than your objective experience.
  - swatow 12y ago
    I know these studies are common (probably because they are easy to conduct) but they don't measure the intrinsic bias of the interviewer. What they measure is a combination of the intrinsic bias of the interviewer, and the statistical inference that the interviewer does based on race/gender (statistical discrimination). Because a resume is a noisy signal, it is still possible that race/gender contains extra information even after you have seen the resume. For example, suppose people tend to exaggerate, and this exaggeration introduces some randomness. Given the signal "lead some impressive project", there is some probability that the person didn't really lead the project. Now if the probability of exaggerating is the same, but a Black person or woman was less likely to have lead the project a priori, then even after observing the resume that claims to have lead the project, the a posteriori probability of having lead the project is lower for the Black person or woman.
    - airza 12y ago
      how do you not understand that "statistical discrimination" is a subtype of "intrinsic bias"?
      - swatow 12y ago
        Many people, perhaps not you, are interested in distinguishing between discrimination based on Bayesian inference, and discrimination that isn't, and is instead, for example, based on incorrect views or inherent dislike of certain groups. You might say that both are equally bad. I suspect that if you were given the opportunity to argue that a particular form of discrimination is not justified by Bayesian inference, you would do so. It's only because I'm suggesting that it's possible discrimination is justified by Bayesian inference, that you are (rudely) objecting to the distinction.
    - domdip 12y ago
      You say Bayesian inference, I say bigotry.
      - drpgq 12y ago
- phinkle 12y ago
  One problem I have with this is that technical interviews are not as straightforward as orchestra auditions. Technical interviews are already argued about frequently and some important qualities in a candidate (culture fit, how easy they are to work with, how they handle pressure, etc) are hard to determine through a blind interview. As you said, there would have to be non-blind interviews anyway. I know many companies already have preliminary online coding problems, but once the candidate gets to the meaty in-person or phone interviews, we are right back where we started. Additionally, many of the standard algorithmic questions are similar to UIL and ICPC questions which are dominated by males for various reasons. Even if we could have completely blind interviews, I don't think that a significantly higher percentage of women would get the job because of larger problems in the tech industry like convincing minorities to join clubs like UIL and ICPC or enter the tech industry in the first place. That being said, I still think this is an interesting idea. I would definitely like to go through a blind phone interview to see if the interviewer treats me differently. Bring on the voice changer.
  - DanBC 12y ago
    "Culture fit" is a bafflingly backward bit of bigotry hiding in plain site. "Culture fit" allows companies to discriminate against a wide range of characteristics. I'm amazed any company thinks it's acceptable to include culture fit as part of the recruitment process. (Although I'll grudgingly accept it might be part of non-retention).
    - learc83 12y ago
      It seems "Culture fit" often just means young and willing to put in extra hours in exchange for foosball, snacks, and team building at the local bar.
      - DanBC 12y ago
        Yes, and those exclude many people some of who have protected characteristics.
- patio11 12y ago
  Thomas, Erin, and I are working on it. (It's a collateral goal of Starfighter.)
- aptwebapps 12y ago
  I wonder if Starfighter can do something like that. At some point they're going to have to give real info to their clients but they could mask these details as long as possible.
dmritard96 12y ago
To me this kind of thing begs questions around affirmative action and other methods of non-'absolute-performance' admittance to jobs/schools/etc. I totally agree that for one person to get to the same 'absolute-performance' point can mean that different people had to come a lot farther than others, but this article implies that it doesn't(shouldn't?) matter. In college admissions though, its never colorblind, socioeconimically bling etc. I'm never really sure I know where I stand as on some level the absolute performance for a job is what matters to share holders, other employees and even end users - but as a society, the idea of whole groups of people being left behind seems shitty.
- ubernostrum 12y ago
  Affirmative action in college admissions serves a different purpose; it's explicitly not a pure merit-based evaluation, and really is poorly understood. A sort of basic understanding of how to implement it would be to assume you have a points-based system for admission, which takes into account things like previous grades, standardized test scores, etc., and normally has a minimum threshold of n points for admission. One way to think of affirmative action is that it says "people from this group, on average, will score k points lower not because of a lack of ability or intelligence, but because of a lack of opportunities earlier in life due to social inequalities", and then asks questions about whether a score that is within k points of n should be admitted. The real implementations are more complicated, of course, but this is the basic idea. The main problem with it is that starting to apply remedies for inequality after someone has already lived through 18 years of it -- and 18 important, formative years at that -- is nowhere near enough.
  - noonespecial 12y ago
    The best way I've heard it put is that the admission process seeks to measure the "velocity" of a potential student and not just position. If someone has come from much further behind than another, they may be a better candidate for the school despite being at a slightly lesser relative position at the time of application.
    - 12y ago
tomlock 12y ago
Also interestingly, this study perhaps shows that there isn't a bias towards hiring women in this case for "political correctness". The screen prevents this.
- mwill 12y ago
  This study was published in 1997 though. I'm not suggesting there is a bias problem for "political correctness" sake nowadays, but I think the people who would argue that there is wouldn't accept a study from 18 years ago as a counterargument. It'd be interesting to see a similar study done in 2015, for this reason, and also just general comparison over time.
  - tomlock 12y ago
    Sure! However, I feel like the grand irony of these kind of studies is that the same kind of unconscious selection against orchestra members is probably also occurring in evidence acceptance. It feels like people are overwhelmingly critical towards evidence that women are treated unfairly in a way they aren't to other evidence. But hey, that could just be a feeling.
    - swatow 12y ago
      Isn't the exact opposite also possible? That there are all kinds of biases in the world, but studies are being selected for (a) showing a bias against women and (b) showing that women are objectively as good as or better than men?
      - tomlock 12y ago
        Sure! But I'd think that based on history there's a fair chance that studies that show (a) are being selected for because they are factual. For example, zero women have been american presidents (although maybe that'll change soon) :). Also there is a lot of evidence that the historically assumed inverse of (b), that man are as good as or better than women, in almost every domain, has made the selection of (b) something that people immediately perceive as an anomaly, a bias. Really, it makes sense that women are better than men at some things on average, and that men are better than women at some things on average. The number of women in tech certainly shows that there's some things at least that women aren't being hired as much for. I'll have to see the data suggesting the selection of "facts" is now leaning entirely towards women, but I haven't.
beloch 12y ago
If you're responsible for choosing applicants to interview from a stack of resumes, you can do something similar by simply having someone else strip the names out before you see them. While you probably are not consciously racist or sexist, simply consuming mass media and living in the society we live in gives us aliefs (subconsciously held beliefs that we consciously know are wrong) that can affect our choices without our knowledge. It's substantially harder to remove your subconscious from the equation when conducting interviews, but at least choosing the best candidates to interview is a step in the right direction.
- deleted 12y ago
  [deleted]
- growupkids 12y ago
  Which is why I'm always puzzled as to why job applications ask for information that has no bearing on the process of qualifying an applicant, such as age, gender, nationality, race and so on. Unless you are specifically looking for someone of particular age, gender, nationality and race why ask for this information?
  - DanBC 12y ago
    In theory that information is used by the company to male sure they are not discriminating against any groups. Ideally that information is on a seperate sheet which is never seen by people in the recruitment process; it goes directly to HR to use for their stats.
    - aptwebapps 12y ago
      OT, but that's a funny slip: "... by the company to male sure ..."
  - lawnchair_larry 12y ago
    Job applications do not ask for that information. It's illegal.

52 points by qiqing 12y ago

> Female musicians in the top five symphony orchestras in the United States were less than 5% of all players in 1970 but are 25% today. The article was published in 1997; it would be interesting to see some more recent numbers. I searched briefly and couldn't find anything definitive.

tomlock 12y ago

This seems to suggest that when we know someone is a woman, we judge them more harshly.

pervycreeper 12y ago

The original study does not appear to have had any control groups, unfortunately. There are too many possible confounding factors present to be able to attribute any of the difference definitively to bias.

cottonseed 12y ago

Someone should build an job search/interview site that gives "blind" interviews: scrubs candidates (application materials, resumes, etc.) of gender, race, age, etc. You might not be able to blind the whole process, but you could blind the preliminary steps. Even that would probably effect the composition of the final selections, as it did here.

adamwong246 12y ago
I had a similar idea: "Meritaco- a double-blind git middleman that hides identifying information from pull requests."
picks_at_nits 12y ago
A similar thing has been done many times by submitting the same resumé with male and females names, or with names statistically more likely to be white and names statistically more likely to be a person of colour, with different ages, and so forth. Every time they run a test like that, they discover that who you appear to be matters more than your objective experience.
- swatow 12y ago
  I know these studies are common (probably because they are easy to conduct) but they don't measure the intrinsic bias of the interviewer. What they measure is a combination of the intrinsic bias of the interviewer, and the statistical inference that the interviewer does based on race/gender (statistical discrimination). Because a resume is a noisy signal, it is still possible that race/gender contains extra information even after you have seen the resume. For example, suppose people tend to exaggerate, and this exaggeration introduces some randomness. Given the signal "lead some impressive project", there is some probability that the person didn't really lead the project. Now if the probability of exaggerating is the same, but a Black person or woman was less likely to have lead the project a priori, then even after observing the resume that claims to have lead the project, the a posteriori probability of having lead the project is lower for the Black person or woman.
  - airza 12y ago
    how do you not understand that "statistical discrimination" is a subtype of "intrinsic bias"?
    - swatow 12y ago
      Many people, perhaps not you, are interested in distinguishing between discrimination based on Bayesian inference, and discrimination that isn't, and is instead, for example, based on incorrect views or inherent dislike of certain groups. You might say that both are equally bad. I suspect that if you were given the opportunity to argue that a particular form of discrimination is not justified by Bayesian inference, you would do so. It's only because I'm suggesting that it's possible discrimination is justified by Bayesian inference, that you are (rudely) objecting to the distinction.
  - domdip 12y ago
    You say Bayesian inference, I say bigotry.
    - drpgq 12y ago
phinkle 12y ago
One problem I have with this is that technical interviews are not as straightforward as orchestra auditions. Technical interviews are already argued about frequently and some important qualities in a candidate (culture fit, how easy they are to work with, how they handle pressure, etc) are hard to determine through a blind interview. As you said, there would have to be non-blind interviews anyway. I know many companies already have preliminary online coding problems, but once the candidate gets to the meaty in-person or phone interviews, we are right back where we started. Additionally, many of the standard algorithmic questions are similar to UIL and ICPC questions which are dominated by males for various reasons. Even if we could have completely blind interviews, I don't think that a significantly higher percentage of women would get the job because of larger problems in the tech industry like convincing minorities to join clubs like UIL and ICPC or enter the tech industry in the first place. That being said, I still think this is an interesting idea. I would definitely like to go through a blind phone interview to see if the interviewer treats me differently. Bring on the voice changer.
- DanBC 12y ago
  "Culture fit" is a bafflingly backward bit of bigotry hiding in plain site. "Culture fit" allows companies to discriminate against a wide range of characteristics. I'm amazed any company thinks it's acceptable to include culture fit as part of the recruitment process. (Although I'll grudgingly accept it might be part of non-retention).
  - learc83 12y ago
    It seems "Culture fit" often just means young and willing to put in extra hours in exchange for foosball, snacks, and team building at the local bar.
    - DanBC 12y ago
      Yes, and those exclude many people some of who have protected characteristics.
patio11 12y ago
Thomas, Erin, and I are working on it. (It's a collateral goal of Starfighter.)
aptwebapps 12y ago
I wonder if Starfighter can do something like that. At some point they're going to have to give real info to their clients but they could mask these details as long as possible.

dmritard96 12y ago

To me this kind of thing begs questions around affirmative action and other methods of non-'absolute-performance' admittance to jobs/schools/etc. I totally agree that for one person to get to the same 'absolute-performance' point can mean that different people had to come a lot farther than others, but this article implies that it doesn't(shouldn't?) matter. In college admissions though, its never colorblind, socioeconimically bling etc. I'm never really sure I know where I stand as on some level the absolute performance for a job is what matters to share holders, other employees and even end users - but as a society, the idea of whole groups of people being left behind seems shitty.

ubernostrum 12y ago
Affirmative action in college admissions serves a different purpose; it's explicitly not a pure merit-based evaluation, and really is poorly understood. A sort of basic understanding of how to implement it would be to assume you have a points-based system for admission, which takes into account things like previous grades, standardized test scores, etc., and normally has a minimum threshold of n points for admission. One way to think of affirmative action is that it says "people from this group, on average, will score k points lower not because of a lack of ability or intelligence, but because of a lack of opportunities earlier in life due to social inequalities", and then asks questions about whether a score that is within k points of n should be admitted. The real implementations are more complicated, of course, but this is the basic idea. The main problem with it is that starting to apply remedies for inequality after someone has already lived through 18 years of it -- and 18 important, formative years at that -- is nowhere near enough.
- noonespecial 12y ago
  The best way I've heard it put is that the admission process seeks to measure the "velocity" of a potential student and not just position. If someone has come from much further behind than another, they may be a better candidate for the school despite being at a slightly lesser relative position at the time of application.
  - 12y ago

tomlock 12y ago

Also interestingly, this study perhaps shows that there isn't a bias towards hiring women in this case for "political correctness". The screen prevents this.

mwill 12y ago
This study was published in 1997 though. I'm not suggesting there is a bias problem for "political correctness" sake nowadays, but I think the people who would argue that there is wouldn't accept a study from 18 years ago as a counterargument. It'd be interesting to see a similar study done in 2015, for this reason, and also just general comparison over time.
- tomlock 12y ago
  Sure! However, I feel like the grand irony of these kind of studies is that the same kind of unconscious selection against orchestra members is probably also occurring in evidence acceptance. It feels like people are overwhelmingly critical towards evidence that women are treated unfairly in a way they aren't to other evidence. But hey, that could just be a feeling.
  - swatow 12y ago
    Isn't the exact opposite also possible? That there are all kinds of biases in the world, but studies are being selected for (a) showing a bias against women and (b) showing that women are objectively as good as or better than men?
    - tomlock 12y ago
      Sure! But I'd think that based on history there's a fair chance that studies that show (a) are being selected for because they are factual. For example, zero women have been american presidents (although maybe that'll change soon) :). Also there is a lot of evidence that the historically assumed inverse of (b), that man are as good as or better than women, in almost every domain, has made the selection of (b) something that people immediately perceive as an anomaly, a bias. Really, it makes sense that women are better than men at some things on average, and that men are better than women at some things on average. The number of women in tech certainly shows that there's some things at least that women aren't being hired as much for. I'll have to see the data suggesting the selection of "facts" is now leaning entirely towards women, but I haven't.

beloch 12y ago

If you're responsible for choosing applicants to interview from a stack of resumes, you can do something similar by simply having someone else strip the names out before you see them. While you probably are not consciously racist or sexist, simply consuming mass media and living in the society we live in gives us aliefs (subconsciously held beliefs that we consciously know are wrong) that can affect our choices without our knowledge. It's substantially harder to remove your subconscious from the equation when conducting interviews, but at least choosing the best candidates to interview is a step in the right direction.

deleted 12y ago
[deleted]
growupkids 12y ago
Which is why I'm always puzzled as to why job applications ask for information that has no bearing on the process of qualifying an applicant, such as age, gender, nationality, race and so on. Unless you are specifically looking for someone of particular age, gender, nationality and race why ask for this information?
- DanBC 12y ago
  In theory that information is used by the company to male sure they are not discriminating against any groups. Ideally that information is on a seperate sheet which is never seen by people in the recruitment process; it goes directly to HR to use for their stats.
  - aptwebapps 12y ago
    OT, but that's a funny slip: "... by the company to male sure ..."
- lawnchair_larry 12y ago
  Job applications do not ask for that information. It's illegal.