5 ms·

"Given any mechanism for filtering people, I claim that there are smart, productive people who will be filtered out. These are called false negatives. It is imp

"Given any mechanism for filtering people, I claim that there are smart, productive people who will be filtered out. These are called false negatives. It is impossible to create a repeatable hiring process that does away with false negatives." While it is true that no test can avoid false negatives, it is also true that if the quantity measured by your test is uncorrelated with the phenomenon of interest then your test is neither sensitive nor specific and its PPV and NPV will be poor no matter where you set your decision threshold on the ROC curve. If you advocate such a test publicly, you should expect to be criticized for doing so. "I now understand that he was using a strategy for hiring people, not a metric for measuring me. His strategy for hiring people worked for his purposes. Why take that personally?" You could decide which programmer to hire based on skin color and write a blog advocating that "strategy" as well. (You'd get sued for doing so, but that's tangential to the point here.) If you did that, it would be unreasonable to expect the "it's just my strategy; it isn't about you, so don't take it personally" argument to be taken seriously. Whether you are aware of it or not, when you advocate a hiring practice publicly you are making an implicit claim that it has some validity and the community is right to spew vitriol in your direction if your hiring criteria are not well constrained within the realm of demonstrable merit and cross over into areas that are none of your damn business as an employer.

raganwald 16y ago
Your argument above intersperses merit with missing the mark. Merit: While it is true that no test can avoid false negatives, it is also true that if the quantity measured by your test is uncorrelated with the phenomenon of interest then your test is neither sensitive nor specific and its PPV and NPV will be poor no matter where you set your decision threshold on the ROC curve. Absolutely, and all strategies can be evaluated as strategies, on this basis. Missing the mark: "If you advocate X you should expect to be criticized." How about, if I advocate X, my argument should be criticized? Criticizing me for my words is the ultimate example of making something personal. I can deal with it, but I suggest you ask yourself whether making it about me is also a way of making it about you, getting emotionally invested in it in a way that is not good for you. JM2C, I am not laden down with emotional intelligence, so I can hardly advise you on how to be happy. Merit: "when you advocate a hiring practice publicly you are making an implicit claim that it has some validity" We agree. Missing the mark: The implied claim that when I advocate that a strategy has some validity as a strategy, I am also advocating it is a metric. Strategies are not metrics. Ultimate straw-man example: One strategy for hiring people on a small scale is to hire people you've worked with before in some capacity and have observed performing well. The ultimate reference check! I suggest that if you don't have a huge requirement for scaling, this can be very effective as a strategy. However, it is trivially easy to demonstrate that this is a terrible metric for evaluating someone's competence. And no, that isn't personal in any way, shape, or form. UPDATE: This is a really fruitful line of discussion, so I'm upmodding you. One thing that occurred to me as an example of a strategy that is not a metric is a University Degree. It's a certain kind of evidence of the ability to follow through on something for a few years, it is allegedly evidence that you have been exposed to basic computer science, and so on. I think it's reasonable to value a degree when hiring as part of a larger strategy. However, I don't think it's reasonable to use a degree as a metric for measuring programmers.
- blahblahblah 16y ago
  We're in agreement about the "If you advocate X you should expect to be criticized" remark. It was inarticulately phrased. I am only interested in criticizing behaviors and ideas, not people. The intended meaning was "you should expect to have your idea criticized". The key point that I'm making is that, "I won't hire a programmer who doesn't pass my unsubstantiated litmus test based on a stereotype about programmers," is a qualitatively different statement than, "All else being equal, if I have two job applicants with similar qualifications and demonstrated merit to choose from and one of these applicants programs as a hobby in addition to performing their professional duties and the other does not, I am inclined to choose the one who programs outside of work." The latter is a defensible strategy. The former is a cargo cult hiring practice, should be criticized, and will inevitably offend because it carries with it an implicit assertion of truth for an unproven stereotype. The phrasing of the original post more closely resembles the cargo cult hiring practice statement.
  - raganwald 16y ago
    > you should expect to have your idea criticized Good! One of the reasons I write is to provoke criticism. Often, I have an idea and it seems bulletproof. But when committed to paper--an archaic way to put it--its flaws reveal themselves. Sometimes the act of writing it down forces me to be specific and in doing to I recognize fatal flaws. Sometimes I need to share it and other people point out its flaws. The OP is one of my poorest posts, ever. It was originally a reddit comment as part of an "Ask Reddit." The asker was faced with the question of whether to hire someone who had no interest in programming outside of his job, and in that context my reply was that I would be surprised if I ever even interviewed such a person. Which in itself highlights the discrepancy between strategies (be they cargo cult or not) and metrics. Programming on the side is a terrible metric. It is easy to measure, but so are Lines of Code. The things we want to measure are knowledge, aptitude for learning, skill, and so forth. Programming on the side is a programmer's strategy for achieving something that we want. But it isn't what we want. I was thinking the same thing about a degree. Terrible metric, but I wouldn't object if it was one of the heuristics someone uses when selecting applicants for a position. Here in Ontario job postings almost always say "Degree or equivalent experience" because people have successfully sued for discrimination when companies have ignored obviously qualified applicants who either didn't have a degree or had more than enough experience to compensate. Getting back to your words, although the link-bait title says "I won't hire a programmer who doesn't X," the article said that the real issue was that I was unlikely to interview them because of my strategy for meeting people. Obviously it isn't that hard and fast. If I was looking for people and mentioned it on my blog, someone could easily get in touch with me because they read blogs on company time. Presto, I would be interviewing someone who reads blogs but doesn't do so after hours. Likewise, if someone in my network said, "You really ought to meet Joe P. NoineToFive, he's incredibly organized and does as much work in forty-five hours as you do in ninety," I would interview Joe and if he lived up to the claim, I would have no interest in what Joe does with the remaining 123 hours a week. But the fact that I have to explain these things identifies that the post was hastily written and incoherent! "All else being equal, if I have two job applicants with similar qualifications and demonstrated merit to choose from and one of these applicants programs as a hobby in addition to performing their professional duties and the other does not, I am inclined to choose the one who programs outside of work." Actually, I personally wouldn't say that, because once someone landed in an interview with me, I would have the luxury of examining each person's abilities directly. If all else really was equal, then I'm curious: Is the programmer who doesn't program as a hobby smarter? Is that why he is just as effective as the hobbyist? Or is it that programming as a hobby is a useful strategy for choosing people to interview, but once they're in the do I should ignore it?

raganwald 16y ago

Your argument above intersperses merit with missing the mark. Merit: While it is true that no test can avoid false negatives, it is also true that if the quantity measured by your test is uncorrelated with the phenomenon of interest then your test is neither sensitive nor specific and its PPV and NPV will be poor no matter where you set your decision threshold on the ROC curve. Absolutely, and all strategies can be evaluated as strategies, on this basis. Missing the mark: "If you advocate X you should expect to be criticized." How about, if I advocate X, my argument should be criticized? Criticizing me for my words is the ultimate example of making something personal. I can deal with it, but I suggest you ask yourself whether making it about me is also a way of making it about you, getting emotionally invested in it in a way that is not good for you. JM2C, I am not laden down with emotional intelligence, so I can hardly advise you on how to be happy. Merit: "when you advocate a hiring practice publicly you are making an implicit claim that it has some validity" We agree. Missing the mark: The implied claim that when I advocate that a strategy has some validity as a strategy, I am also advocating it is a metric. Strategies are not metrics. Ultimate straw-man example: One strategy for hiring people on a small scale is to hire people you've worked with before in some capacity and have observed performing well. The ultimate reference check! I suggest that if you don't have a huge requirement for scaling, this can be very effective as a strategy. However, it is trivially easy to demonstrate that this is a terrible metric for evaluating someone's competence. And no, that isn't personal in any way, shape, or form. UPDATE: This is a really fruitful line of discussion, so I'm upmodding you. One thing that occurred to me as an example of a strategy that is not a metric is a University Degree. It's a certain kind of evidence of the ability to follow through on something for a few years, it is allegedly evidence that you have been exposed to basic computer science, and so on. I think it's reasonable to value a degree when hiring as part of a larger strategy. However, I don't think it's reasonable to use a degree as a metric for measuring programmers.

blahblahblah 16y ago
We're in agreement about the "If you advocate X you should expect to be criticized" remark. It was inarticulately phrased. I am only interested in criticizing behaviors and ideas, not people. The intended meaning was "you should expect to have your idea criticized". The key point that I'm making is that, "I won't hire a programmer who doesn't pass my unsubstantiated litmus test based on a stereotype about programmers," is a qualitatively different statement than, "All else being equal, if I have two job applicants with similar qualifications and demonstrated merit to choose from and one of these applicants programs as a hobby in addition to performing their professional duties and the other does not, I am inclined to choose the one who programs outside of work." The latter is a defensible strategy. The former is a cargo cult hiring practice, should be criticized, and will inevitably offend because it carries with it an implicit assertion of truth for an unproven stereotype. The phrasing of the original post more closely resembles the cargo cult hiring practice statement.
- raganwald 16y ago
  > you should expect to have your idea criticized Good! One of the reasons I write is to provoke criticism. Often, I have an idea and it seems bulletproof. But when committed to paper--an archaic way to put it--its flaws reveal themselves. Sometimes the act of writing it down forces me to be specific and in doing to I recognize fatal flaws. Sometimes I need to share it and other people point out its flaws. The OP is one of my poorest posts, ever. It was originally a reddit comment as part of an "Ask Reddit." The asker was faced with the question of whether to hire someone who had no interest in programming outside of his job, and in that context my reply was that I would be surprised if I ever even interviewed such a person. Which in itself highlights the discrepancy between strategies (be they cargo cult or not) and metrics. Programming on the side is a terrible metric. It is easy to measure, but so are Lines of Code. The things we want to measure are knowledge, aptitude for learning, skill, and so forth. Programming on the side is a programmer's strategy for achieving something that we want. But it isn't what we want. I was thinking the same thing about a degree. Terrible metric, but I wouldn't object if it was one of the heuristics someone uses when selecting applicants for a position. Here in Ontario job postings almost always say "Degree or equivalent experience" because people have successfully sued for discrimination when companies have ignored obviously qualified applicants who either didn't have a degree or had more than enough experience to compensate. Getting back to your words, although the link-bait title says "I won't hire a programmer who doesn't X," the article said that the real issue was that I was unlikely to interview them because of my strategy for meeting people. Obviously it isn't that hard and fast. If I was looking for people and mentioned it on my blog, someone could easily get in touch with me because they read blogs on company time. Presto, I would be interviewing someone who reads blogs but doesn't do so after hours. Likewise, if someone in my network said, "You really ought to meet Joe P. NoineToFive, he's incredibly organized and does as much work in forty-five hours as you do in ninety," I would interview Joe and if he lived up to the claim, I would have no interest in what Joe does with the remaining 123 hours a week. But the fact that I have to explain these things identifies that the post was hastily written and incoherent! "All else being equal, if I have two job applicants with similar qualifications and demonstrated merit to choose from and one of these applicants programs as a hobby in addition to performing their professional duties and the other does not, I am inclined to choose the one who programs outside of work." Actually, I personally wouldn't say that, because once someone landed in an interview with me, I would have the luxury of examining each person's abilities directly. If all else really was equal, then I'm curious: Is the programmer who doesn't program as a hobby smarter? Is that why he is just as effective as the hobbyist? Or is it that programming as a hobby is a useful strategy for choosing people to interview, but once they're in the do I should ignore it?