5 ms·

Perhaps the exact wording of the question is less important than the discussion that follows?

• tptacek 13y ago
  No, that's exactly my point: the discussion that follows looks valuable, but isn't, because it isn't rigorous.
  • hippee-lee 13y ago
    So what is valuable? I ding candidates when they can't carry a conversation in an interview setting, technical discussion or otherwise.
    • tptacek 13y ago
      So you ding candidates for lacking an ability (carrying a time-bounded conversation with a stranger who is judging them for a new job) that has nothing whatsoever to do with what they'll actually be doing on the job, and this is something you're happy to tell us? Job interviews are hostile. In fact, they're one of the most hostile experiences most of us have to deal with. I watch candidates in face-to-face interviews very carefully now, because this is a problem I am studying, and when you stop and consider candidates not as gamepieces but instead as human beings, you'll be amazed at what you notice. For instance, a majority of the candidates I've sat down with in interviews have, subtly but noticeably, been physically shaking. Job interviews are like a game of Jeopardy! where one of the players is being filmed and broadcast on live television and the other player is playing from their couch and has been given an answer book, and the poor televised shmoe is judged on the same curve as the person on the couch.
      • bennyg 13y ago
        Assuming two candidates are both qualified to perform a job, and one of them gels and converses with team (aka gets along with), then that person should get the job. The best teams get the best work done, and good teams have amazing communication - also assuming that everyone on said team can actually do the work without bring the others down. A good teammate is a multiplier on total team productivity and efficiency - like in sports and the military. Everyone wishes it wasn't the case, but talent alone doesn't cut it.
        • tptacek 13y ago
          First, you're assuming that the interview process reliably distinguishes between "qualified" and "easy to talk to", but in practice not only do engineering interviews not do that, but in confounding the two they also drastically over-weigh the "easy to talk to" attribute. Second, you're assuming that the way a candidate interacts with your team in an interview is an accurate representation of how they'll gel with the team down the road. Some of the shyest people I've interviewed have gone on to be the ones I actually end up talking to all day. Part of that is because interviews are hostile and make people shy, and part of it is that different people take different amounts of time to open up to strangers. Finally, you're describing the "gut check" interview process used by almost every engineering team since the 1990s. We have evidence for how well that process works. Answer: not well. As an aside: the idea of hiring candidates based on how well team members "get along" with them is repellant to me, as is "culture fit", since they're basically ways of making sure you have a team full of people who listen to roughly the same music, drink roughly the same beer, play roughly the same X360 games, and like roughly the same technology stacks. But that's an emotional argument; the real argument is that it doesn't work.
          • bennyg 13y ago
            The ability for people to gel doesn't guarantee any of the following: listening to the same music, drinking roughly the same beer, playing the same games, and liking the same technology stacks. What I am arguing for is the part after someone's ability to accomplish the work and be creative has been shown. I think you underestimate how many people actually have the smarts and creative ability to do just about all software engineering tasks.
            • tptacek 13y ago
              I think I do the opposite: I think I value candidate potential more than you, because you propose screening based on how cordial or fluent a single interview with their prospective team is, and I don't buy for a second that you get a realistic picture of how well someone will communicate or gel with that team in the course of a single adversarial interview.
      • hippee-lee 13y ago
        So I think you are reading way to much into the comment. It was more meant to probe an expert for his approach to sussing out viable teammates or employees. If I as someone interviewing for a QA role ask them a question such as 'Tell me about the QA work you are most proud of.' And they spend the next 20 minutes not talking about anything QA related it's a red flag. And an important data point to me when doing a retro with my peers. Perhaps that recent scenario is a reflection of the folks my company is able to attract in this area. Conversely, if I am interviewing with you and I ask you, what's the most interesting thing your team has done and how did you help them accomplish it.' And you go into great detail about a deep area in your field that's part of the product your team works on its useful and important information when I need to make a decision. However if you are only interested and talking about me, my technical (or lack of - I know very little about security) understanding and you brush off conversational attempts to gain information that will help me judge the book by its cover it's usefully info when I need to make a decision. I am not sure why your tone was so hostile? I did not mean to imply anything by my question. I am merely looking for other ways that people have found works for them.
  • smsm42 13y ago
    Why would you expect anything rigorous in such a nebulous business as predicting someone's performance in the environment he's never been doing things that he's probably never done before (even if he's done similar things)? I don't think you can have any rigor here. Do you have a proposed rigorous solution?
    • tptacek 13y ago
      Yes, I have written long and specific comments on alternative approaches on HN before and won't bore you with a recap, except to note that I sketched out what I meant by "rigor" in the comment rooting this thread.
      • smsm42 13y ago
        I understand what you mean by rigor but I'm not sure that your quest for rigor in that sense would yield better results. I could be very wrong, of course, my experience is nothing but a relatively small list of anecdotes. But if you indeed have a rigorous way of assessing candidates, I certainly look forward for you to creating the best recruiting agency ever, becoming a billionaire and transforming this field forever (not being sarcastic, btw, I'd like so see some innovation in this field, I'm just not sure how "rigorous" can work here, but if you do - I hope you are right and I am wrong).
        • tptacek 13y ago
          I've written about the results we've seen from more rigorous recruiting, too.
          • smsm42 13y ago
            Care to post a link so I could read it? Thanks in advance.
            • karmelapple 13y ago
              Agreed - let's see a link!