4 ms·

Well mistakes happen - a phone screen can only get you so far, especially in a seller's market where your talent pool is pretty shallow. Though I think a share

by cozuya 12y ago

Well mistakes happen - a phone screen can only get you so far, especially in a seller's market where your talent pool is pretty shallow. Though I think a shared google docs interview is probably the best way to go for the initial screen, I just haven't given or taken one like that yet so it must still be relatively rare.

tptacek 12y ago
We came to the conclusion that phone screens are pretty close to worthless. We'd still have been very upset if we ended up in a face-to-face interview with someone who had no business interviewing for the job. You need to do more than just phone screen (actually: you should eliminate phone screens). (I don't know who's downvoting you but I upvoted.)
- JohnBooty 12y ago
  I'm really fascinated by your replies on this thread; seems you build a really solid process there. I'm curious about the phone screens - how do you screen out the candidates that are grossly unqualified so that they don't progress to the interview stage? In other words, how do you screen out candidates like those Joel describes here: http://www.joelonsoftware.com/articles/ThePhoneScreen.html http://www.joelonsoftware.com/articles/ThePhoneScreen.html
  - tptacek 12y ago
    By my last year at Matasano we were literally discarding the results of phone screens. We did them largely as a pro-forma exercise, and because they made team members more comfortable with the process as a whole. We virtually never selected out candidates based on phone screens. We had a work sample process that kicked in after phone screens, and that cost us almost nothing to run. So we never had an incentive to prevent a candidate from going through that process. Unlike phone screens, the work sample results were strongly predictive. You could bomb phone screens, ace work samples, and end up coming in for in-person interview. (We did "nudge" candidates we felt wouldn't do well on the work sample tests, solely out of concern for not wasting their time, but anyone who wanted to proceed was able to). Phone screens are a waste of time. Spolsky did a really good job of documented the best practices within the framework of unstructured interviews as they were practiced 10 years ago. The problem isn't Spolsky's tactical suggestions; it's that the strategy they're a part of is being discredited.
    - timv 12y ago
      I'm curious how often you had people voluntarily drop out of the process without completing the work sample, and whether that frequency would change if you didn't have a phone screen. (I take it from your questions that you never actually stopped doing phone screens). If my first contact with a company is them asking me to put in hours of my time, when it costs them "almost nothing to run" (your words) then I'd be inclined to pass. The company I work for has a work sample exercise, and we intentionally place it after a phone screen and first interview, because we feel it is (and appears) more fair to the candidates. From our point of view, we'd love to put it up front, as it is the best source of information we get, but if it caused us to lose good candidates before we even started (and we believe it would) that would be a show-stopper.
      - jerematasno 11y ago
        Most of our candidates drop out before the work sample. On the other hand, almost none of our candidates are qualified to work for us when they initially apply. We make it really clear that there will be work samples before people even apply. I think the real question would be "how many qualified applicants don't bother applying because of the work sample," which is a question we can't answer. Given the paucity of unemployed qualified infosec folks, we're comfortable with the tradeoff.
      - Decade 12y ago
        I’m curious exactly how many times a candidate gets to apply to Matasano. Thomas describes the Matasano process as costing “almost nothing,” but that includes running a couple exploit training websites and sending an 852-page book to applicants. Which I’m not inclined to go through immediately, because I need money right now. Do I have to do the microcorruption.com and the cryptopals.com and The Web Application Hackers Handbook before even trying the technical screens and challenges? Or should I try seeing if my existing web application security best practices and rusty MIPS assembly experience are enough to get to where I have enough breathing room to do these exercises?
        jjarmoc 12y ago

by cozuya 12y ago

tptacek 12y ago

We came to the conclusion that phone screens are pretty close to worthless. We'd still have been very upset if we ended up in a face-to-face interview with someone who had no business interviewing for the job. You need to do more than just phone screen (actually: you should eliminate phone screens). (I don't know who's downvoting you but I upvoted.)

JohnBooty 12y ago
I'm really fascinated by your replies on this thread; seems you build a really solid process there. I'm curious about the phone screens - how do you screen out the candidates that are grossly unqualified so that they don't progress to the interview stage? In other words, how do you screen out candidates like those Joel describes here: http://www.joelonsoftware.com/articles/ThePhoneScreen.html http://www.joelonsoftware.com/articles/ThePhoneScreen.html
- tptacek 12y ago
  By my last year at Matasano we were literally discarding the results of phone screens. We did them largely as a pro-forma exercise, and because they made team members more comfortable with the process as a whole. We virtually never selected out candidates based on phone screens. We had a work sample process that kicked in after phone screens, and that cost us almost nothing to run. So we never had an incentive to prevent a candidate from going through that process. Unlike phone screens, the work sample results were strongly predictive. You could bomb phone screens, ace work samples, and end up coming in for in-person interview. (We did "nudge" candidates we felt wouldn't do well on the work sample tests, solely out of concern for not wasting their time, but anyone who wanted to proceed was able to). Phone screens are a waste of time. Spolsky did a really good job of documented the best practices within the framework of unstructured interviews as they were practiced 10 years ago. The problem isn't Spolsky's tactical suggestions; it's that the strategy they're a part of is being discredited.
  - timv 12y ago
    I'm curious how often you had people voluntarily drop out of the process without completing the work sample, and whether that frequency would change if you didn't have a phone screen. (I take it from your questions that you never actually stopped doing phone screens). If my first contact with a company is them asking me to put in hours of my time, when it costs them "almost nothing to run" (your words) then I'd be inclined to pass. The company I work for has a work sample exercise, and we intentionally place it after a phone screen and first interview, because we feel it is (and appears) more fair to the candidates. From our point of view, we'd love to put it up front, as it is the best source of information we get, but if it caused us to lose good candidates before we even started (and we believe it would) that would be a show-stopper.
    - jerematasno 11y ago
      Most of our candidates drop out before the work sample. On the other hand, almost none of our candidates are qualified to work for us when they initially apply. We make it really clear that there will be work samples before people even apply. I think the real question would be "how many qualified applicants don't bother applying because of the work sample," which is a question we can't answer. Given the paucity of unemployed qualified infosec folks, we're comfortable with the tradeoff.
    - Decade 12y ago
      I’m curious exactly how many times a candidate gets to apply to Matasano. Thomas describes the Matasano process as costing “almost nothing,” but that includes running a couple exploit training websites and sending an 852-page book to applicants. Which I’m not inclined to go through immediately, because I need money right now. Do I have to do the microcorruption.com and the cryptopals.com and The Web Application Hackers Handbook before even trying the technical screens and challenges? Or should I try seeing if my existing web application security best practices and rusty MIPS assembly experience are enough to get to where I have enough breathing room to do these exercises?
      - jjarmoc 12y ago