6 ms·

The hiring spreadsheet and the clash at The Markup

56 points by jashkenas 7y ago

x0x0 7y ago
I read a description -- sorry, don't remember where -- that finally made me feel like what happened made sense. Though who knows if any outsider will ever understand what happened. Reading between the lines, Gardner basically claimed Angwin -- who was the EIC -- didn't do the job. Angwin wanted to do journalism, but that's not the EIC job. Her job was to hire people to do journalism, and to set up the (prioritization, themes, etc) in which journalism was to be done. Angwin did that poorly. [1] Her job also was to set up technical infra. It was claimed she didn't. etc. From another article, "Angwin acknowledged to CJR that 'meetings are not my favorite thing.'" [2] . That makes me believe the firing of Angwin was justified; the EIC job is literally meetings. This is a pattern that is repeated in startups: a technical founder often doesn't understand the job is not to be the best engineer on the team, but rather to hire managers, set priorities, coordinate with the rest of the business, etc. [1] > "Hiring was slow. Recruitment was slow. Even as of this month, we didn’t have stories banked. We didn’t have editorial processes in place to accept and develop pieces. We hadn’t developed areas of coverage. We still lacked an editorial..." — Jeff Larson https://link.medium.com/oPwISQSD9V https://link.medium.com/oPwISQSD9V https://medium.com/@jeff_larson/about-the-markup-6adc6a778100 https://medium.com/@jeff_larson/about-the-markup-6adc6a77810... [2] https://www.cjr.org/analysis/the-markup.php https://www.cjr.org/analysis/the-markup.php
mruts 7y ago
The spreadsheet seems pretty damning. I'm not a lawyer, but isn't it illegal to judge an employee on whether they are male/female or "non-white"?
- panarky 7y ago
  Not a lawyer either, but seems to me if you're hiring writers to pursue a particular perspective, then their backgrounds matter.
  - ANPEQ-1 7y ago
    I'm 100% positive that quality journalists can research quality stories irrespective of their background. It happens all the time.
  - tschwimmer 7y ago
    Hopefully you are hiring writers to report on the objective state of the world, not pursue a particular perspective.
    - throwawayjava 7y ago
      And who gets to decide what "the objective state of the world" is? There is real value in reading about the same topic from differing perspectives. Not all writing is or aspires to be purely fact-based journalism, and that's okay. It's reasonable that The Markup wants to hire group of people with diverse viewpoints on how "powerful institutions are using technology in ways that impact people and society". I would be surprised and underwhelmed if they had e.g. only software engineers or only consumer rights activists writing. The issue is their assumption that only non-white, non-males have interesting things to say about how "powerful institutions are using technology in ways that impact people and society".
      - ANPEQ-1 7y ago
        I mean, I'd argue that journalism as it should be is focused on truthfulness, accuracy, objectivity, impartiality, fairness and public accountability - all standards one would assume are important given The Markup's stated mission - but your point about not all writing aspiring to be fact-based is well taken.
      - jdietrich 7y ago
        Why has it become acceptable to assume so much about someone's view of the world based on their race, gender and sexual identity? Why has our definition of "diversity" become so narrow and stereotypical? If you want to hire people with diverse viewpoints, then look for people with diverse viewpoints. It's incredibly easy to fill a room with people who look like a Benetton poster but who all read the same books in the same colleges and all rigidly adhere to the same dogma. It's incredibly easy to use tokenism to justify your own prejudices. It's difficult to build a genuinely diverse collaboration based on profound and meaningful differences of perspective.
- ANPEQ-1 7y ago
  Seems pretty clear cut that Sue Gardner was making hiring decisions based on practices prohibited by the U.S. Equal Employment Opportunity Commission. The CJR article says she sent them the spreadsheet herself... https://www.eeoc.gov/laws/practices/index.cfm https://www.eeoc.gov/laws/practices/index.cfm EDIT: I guess EEOC doesn't apply to businesses smaller than 15 people. The 'ol Discrimination Loophole for Small Businesses!
- orpheline 7y ago
  One thing wasn't clear to me from this article: did being non-white or non-male improve or hurt a prospective employees' score? I'm assuming in this case being 'non' raised the score, but the article doesn't state one way or another...
  - ANPEQ-1 7y ago
    The CJR skirts this issue by simply ignoring it. Color me surprised...
ANPEQ-1 7y ago
"Non-white" "Non-male" How does this spreadsheet not represent practices prohibited by the EEOC? https://www.eeoc.gov/laws/practices/index.cfm https://www.eeoc.gov/laws/practices/index.cfm
- panarky 7y ago
  EEOC rules don't apply to private businesses with less than 15 employees. https://employment.law.tulane.edu/blog/what-is-an-equal-opportunity-employer https://employment.law.tulane.edu/blog/what-is-an-equal-oppo...
  - ANPEQ-1 7y ago
    Ahhh okay, interesting. I guess that is interesting news for all the racist and sexist small business owners out there barf
ww520 7y ago
Myers-Briggs personality test?! Isn't that like horoscope? The last time I took one in a company was administrated by a consultant paddling some psycho analysis things to HR. We all had a good laugh, like reading our palms to pigeonhole us into artificial compartments.
- barry-cotter 7y ago
  No, it’s nothing like a horoscope. It’s not the best we can do as far as empirically validated personality tests but it’s results are, to use the psychometric terms, reliable and valid. One test will agree with another and the scores are useful for prediction. Comparison with the best in class personality test. > The big five versus the big four: the relationship between the Myers-Briggs Type Indicator (MBTI) and NEO-PI five factor model of personality > Abstract This study sets out examine the relationship between two personality measures—most popularly used measure in the consultancy and training world (the Myers-Briggs Type Indicator) and one of the most heavily used measures in the academic research area on personality (the five factor NEO-PI). One hundred and sixty adults completed the NEO-PI and the MBTI. The NEO-PI Agreeableness score was correlated only with the thinking-feeling (T-F) dimension; the NEO-PI Conscientiousness score was correlated with both thinking-feeling and judging-perceiving (J-P) dimension; the NEO-PI Extraversion score was strongly correlated with the extraversion-introversion (E-I) dimensions, while the Neuroticism score from the NEO-PI was not related to any MBTI subscale score. The openness dimension was correlated with all four especially sensing-intuitive. These results were related to two other similar comparative studies. Results are discussed in terms of recent criticisms of the MBTI. https://www.sciencedirect.com/science/article/pii/0191886996000335 https://www.sciencedirect.com/science/article/pii/0191886996...
- gboudrias 7y ago
  Hi, psychology student here. It's not very good in a psychometric sense, but its validity problems aren't really a concern when it comes to idiographic data ie: "helpful tools for getting to know the people with whom she was creating a new company". It's not the devil we memed it into, there are better tools for categorizing and rejecting candidates based on objective-ish traits but this wasn't the purpose according to thr article.
jefftk 7y ago
This spreadsheet looks basically fine to me? They're taking the information they'd normally use to make decisions ("understands the tech industry" etc) and using a tool to be more consistent across candidates. With race/sex/class those are after the computation of the candidate's total score, which makes me think they're being used for affirmative action in a pretty normal way.
- prepend 7y ago
  This spreadsheet is what I dread serving on interview panels. It has column headers that are hard to understand but gives a score to that header. Hopefully there’s a companion doc describing the criteria and the meaning between a 1-5. Without such definitions, this is just a way to ouija board bias into results because you can sum factors and sort candidates. I’ve found that people like to put more importance on numbers like this than they should and will naturally try to create cutoffs. The class column is concerning as a hiring criteria, but also that it’s just based on superficial review of info and imputing. Someone going to Wesleyan and studying abroad might be extremely poor on scholarship. Probably not, but might be. Trying to guess class by reading someone’s LinkedIn and then using that guess for something like hiring is confusing to me. That doesn’t seem like good decision making since the “data” aren’t factual. There’s a term called scientism [0] which is when something is cosmetically like science but kind of bullshit. This smells like whatever the equivalent is for data where the data are all made up bullshit, but then a reasonable looking method is applied to it. Maybe it’s “data scientism.” [0] https://en.wikipedia.org/wiki/Scientism https://en.wikipedia.org/wiki/Scientism
thaumasiotes 7y ago
There's something about this spreadsheet that I don't get. The article gives examples of categories graded on a 1-to-5 scale: > The criteria [for the "social class" column] are: “1) very poor family background, 2) working class, 3) middle class, 4) slightly upper middle, and 5) super rich, super privileged.” > A key for the column marked “Famous” describes the criteria as: “1) has only published in small media, or hasn’t published very much 2) has published regularly in ordinary-sized media 3) has published a LOT in ordinary-sized or larger media, or has a social media presence exceeding 10K, 4) has published a lot in very popular media AND has a social media presence bigger than 10K 5) practically a household name.” These do not look like evenly spaced points to me. In particular, the gap between a 4 and a 5 appears to be much larger, in both cases, than the gaps between other adjacent numbers such as 2 and 3. And that is fine; there's nothing saying your scale needs to use uniform spacing. But if you look at the spreadsheet, it starts with a bunch of columns assessing various metrics. Then there's a column labeled "Total", the value of which is just the sum of each entry in the preceding columns. This makes no sense. If your numbers come from a non-uniform scale, adding two numbers together is an error. With the sharp dropoff from 4 to 5, it should be much easier to have a 3 in two categories than to have a 5 in one and a 1 (the minimum! You're guaranteed to be at least this good!) in another. The "famous" and "social class" columns aren't among the ones that get summed. (Presumably because higher social class is bad rather than good.) But I tend to suspect that the scales for the summed columns were assigned similarly. And the summed columns do show direct evidence of a very similar problem -- they are color-coded. Each cell is assigned a color ranging from deep red (bad) to white (neutral) to deep green (great). In most of these summed columns, a 3 is neutral, a 4 is light green, and so on, exactly what you'd expect. But in "Competent technology end user", a 5 is deep green, and a 4 is already light red. In "explanatory, helpful to readers", a 3 is deep red. This suggests again that the columns aren't being measured in similar scales, and that adding them makes no sense.
novok 7y ago
Well this spreadsheet reaffirms a bunch of recent stereotypes about journalists now.

56 points by jashkenas 7y ago

x0x0 7y ago

I read a description -- sorry, don't remember where -- that finally made me feel like what happened made sense. Though who knows if any outsider will ever understand what happened. Reading between the lines, Gardner basically claimed Angwin -- who was the EIC -- didn't do the job. Angwin wanted to do journalism, but that's not the EIC job. Her job was to hire people to do journalism, and to set up the (prioritization, themes, etc) in which journalism was to be done. Angwin did that poorly. [1] Her job also was to set up technical infra. It was claimed she didn't. etc. From another article, "Angwin acknowledged to CJR that 'meetings are not my favorite thing.'" [2] . That makes me believe the firing of Angwin was justified; the EIC job is literally meetings. This is a pattern that is repeated in startups: a technical founder often doesn't understand the job is not to be the best engineer on the team, but rather to hire managers, set priorities, coordinate with the rest of the business, etc. [1] > "Hiring was slow. Recruitment was slow. Even as of this month, we didn’t have stories banked. We didn’t have editorial processes in place to accept and develop pieces. We hadn’t developed areas of coverage. We still lacked an editorial..." — Jeff Larson https://link.medium.com/oPwISQSD9V https://link.medium.com/oPwISQSD9V https://medium.com/@jeff_larson/about-the-markup-6adc6a778100 https://medium.com/@jeff_larson/about-the-markup-6adc6a77810... [2] https://www.cjr.org/analysis/the-markup.php https://www.cjr.org/analysis/the-markup.php

mruts 7y ago

The spreadsheet seems pretty damning. I'm not a lawyer, but isn't it illegal to judge an employee on whether they are male/female or "non-white"?

panarky 7y ago
Not a lawyer either, but seems to me if you're hiring writers to pursue a particular perspective, then their backgrounds matter.
- ANPEQ-1 7y ago
  I'm 100% positive that quality journalists can research quality stories irrespective of their background. It happens all the time.
- tschwimmer 7y ago
  Hopefully you are hiring writers to report on the objective state of the world, not pursue a particular perspective.
  - throwawayjava 7y ago
    And who gets to decide what "the objective state of the world" is? There is real value in reading about the same topic from differing perspectives. Not all writing is or aspires to be purely fact-based journalism, and that's okay. It's reasonable that The Markup wants to hire group of people with diverse viewpoints on how "powerful institutions are using technology in ways that impact people and society". I would be surprised and underwhelmed if they had e.g. only software engineers or only consumer rights activists writing. The issue is their assumption that only non-white, non-males have interesting things to say about how "powerful institutions are using technology in ways that impact people and society".
    - ANPEQ-1 7y ago
      I mean, I'd argue that journalism as it should be is focused on truthfulness, accuracy, objectivity, impartiality, fairness and public accountability - all standards one would assume are important given The Markup's stated mission - but your point about not all writing aspiring to be fact-based is well taken.
    - jdietrich 7y ago
      Why has it become acceptable to assume so much about someone's view of the world based on their race, gender and sexual identity? Why has our definition of "diversity" become so narrow and stereotypical? If you want to hire people with diverse viewpoints, then look for people with diverse viewpoints. It's incredibly easy to fill a room with people who look like a Benetton poster but who all read the same books in the same colleges and all rigidly adhere to the same dogma. It's incredibly easy to use tokenism to justify your own prejudices. It's difficult to build a genuinely diverse collaboration based on profound and meaningful differences of perspective.
ANPEQ-1 7y ago
Seems pretty clear cut that Sue Gardner was making hiring decisions based on practices prohibited by the U.S. Equal Employment Opportunity Commission. The CJR article says she sent them the spreadsheet herself... https://www.eeoc.gov/laws/practices/index.cfm https://www.eeoc.gov/laws/practices/index.cfm EDIT: I guess EEOC doesn't apply to businesses smaller than 15 people. The 'ol Discrimination Loophole for Small Businesses!
orpheline 7y ago
One thing wasn't clear to me from this article: did being non-white or non-male improve or hurt a prospective employees' score? I'm assuming in this case being 'non' raised the score, but the article doesn't state one way or another...
- ANPEQ-1 7y ago
  The CJR skirts this issue by simply ignoring it. Color me surprised...

ANPEQ-1 7y ago

"Non-white" "Non-male" How does this spreadsheet not represent practices prohibited by the EEOC? https://www.eeoc.gov/laws/practices/index.cfm https://www.eeoc.gov/laws/practices/index.cfm

panarky 7y ago
EEOC rules don't apply to private businesses with less than 15 employees. https://employment.law.tulane.edu/blog/what-is-an-equal-opportunity-employer https://employment.law.tulane.edu/blog/what-is-an-equal-oppo...
- ANPEQ-1 7y ago
  Ahhh okay, interesting. I guess that is interesting news for all the racist and sexist small business owners out there barf

ww520 7y ago

Myers-Briggs personality test?! Isn't that like horoscope? The last time I took one in a company was administrated by a consultant paddling some psycho analysis things to HR. We all had a good laugh, like reading our palms to pigeonhole us into artificial compartments.

barry-cotter 7y ago
No, it’s nothing like a horoscope. It’s not the best we can do as far as empirically validated personality tests but it’s results are, to use the psychometric terms, reliable and valid. One test will agree with another and the scores are useful for prediction. Comparison with the best in class personality test. > The big five versus the big four: the relationship between the Myers-Briggs Type Indicator (MBTI) and NEO-PI five factor model of personality > Abstract This study sets out examine the relationship between two personality measures—most popularly used measure in the consultancy and training world (the Myers-Briggs Type Indicator) and one of the most heavily used measures in the academic research area on personality (the five factor NEO-PI). One hundred and sixty adults completed the NEO-PI and the MBTI. The NEO-PI Agreeableness score was correlated only with the thinking-feeling (T-F) dimension; the NEO-PI Conscientiousness score was correlated with both thinking-feeling and judging-perceiving (J-P) dimension; the NEO-PI Extraversion score was strongly correlated with the extraversion-introversion (E-I) dimensions, while the Neuroticism score from the NEO-PI was not related to any MBTI subscale score. The openness dimension was correlated with all four especially sensing-intuitive. These results were related to two other similar comparative studies. Results are discussed in terms of recent criticisms of the MBTI. https://www.sciencedirect.com/science/article/pii/0191886996000335 https://www.sciencedirect.com/science/article/pii/0191886996...
gboudrias 7y ago
Hi, psychology student here. It's not very good in a psychometric sense, but its validity problems aren't really a concern when it comes to idiographic data ie: "helpful tools for getting to know the people with whom she was creating a new company". It's not the devil we memed it into, there are better tools for categorizing and rejecting candidates based on objective-ish traits but this wasn't the purpose according to thr article.

jefftk 7y ago

This spreadsheet looks basically fine to me? They're taking the information they'd normally use to make decisions ("understands the tech industry" etc) and using a tool to be more consistent across candidates. With race/sex/class those are after the computation of the candidate's total score, which makes me think they're being used for affirmative action in a pretty normal way.

prepend 7y ago
This spreadsheet is what I dread serving on interview panels. It has column headers that are hard to understand but gives a score to that header. Hopefully there’s a companion doc describing the criteria and the meaning between a 1-5. Without such definitions, this is just a way to ouija board bias into results because you can sum factors and sort candidates. I’ve found that people like to put more importance on numbers like this than they should and will naturally try to create cutoffs. The class column is concerning as a hiring criteria, but also that it’s just based on superficial review of info and imputing. Someone going to Wesleyan and studying abroad might be extremely poor on scholarship. Probably not, but might be. Trying to guess class by reading someone’s LinkedIn and then using that guess for something like hiring is confusing to me. That doesn’t seem like good decision making since the “data” aren’t factual. There’s a term called scientism [0] which is when something is cosmetically like science but kind of bullshit. This smells like whatever the equivalent is for data where the data are all made up bullshit, but then a reasonable looking method is applied to it. Maybe it’s “data scientism.” [0] https://en.wikipedia.org/wiki/Scientism https://en.wikipedia.org/wiki/Scientism

thaumasiotes 7y ago

There's something about this spreadsheet that I don't get. The article gives examples of categories graded on a 1-to-5 scale: > The criteria [for the "social class" column] are: “1) very poor family background, 2) working class, 3) middle class, 4) slightly upper middle, and 5) super rich, super privileged.” > A key for the column marked “Famous” describes the criteria as: “1) has only published in small media, or hasn’t published very much 2) has published regularly in ordinary-sized media 3) has published a LOT in ordinary-sized or larger media, or has a social media presence exceeding 10K, 4) has published a lot in very popular media AND has a social media presence bigger than 10K 5) practically a household name.” These do not look like evenly spaced points to me. In particular, the gap between a 4 and a 5 appears to be much larger, in both cases, than the gaps between other adjacent numbers such as 2 and 3. And that is fine; there's nothing saying your scale needs to use uniform spacing. But if you look at the spreadsheet, it starts with a bunch of columns assessing various metrics. Then there's a column labeled "Total", the value of which is just the sum of each entry in the preceding columns. This makes no sense. If your numbers come from a non-uniform scale, adding two numbers together is an error. With the sharp dropoff from 4 to 5, it should be much easier to have a 3 in two categories than to have a 5 in one and a 1 (the minimum! You're guaranteed to be at least this good!) in another. The "famous" and "social class" columns aren't among the ones that get summed. (Presumably because higher social class is bad rather than good.) But I tend to suspect that the scales for the summed columns were assigned similarly. And the summed columns do show direct evidence of a very similar problem -- they are color-coded. Each cell is assigned a color ranging from deep red (bad) to white (neutral) to deep green (great). In most of these summed columns, a 3 is neutral, a 4 is light green, and so on, exactly what you'd expect. But in "Competent technology end user", a 5 is deep green, and a 4 is already light red. In "explanatory, helpful to readers", a 3 is deep red. This suggests again that the columns aren't being measured in similar scales, and that adding them makes no sense.

novok 7y ago

Well this spreadsheet reaffirms a bunch of recent stereotypes about journalists now.