4 ms·

I don't disagree your first point. The problem is that asking questions like FizzBuzz is also a filter in the other way. Higher quality candidates aren't going

by leros 4y ago

I don't disagree your first point. The problem is that asking questions like FizzBuzz is also a filter in the other way. Higher quality candidates aren't going to bother answering your filter question and bail out of the process. I spent several years working on a large tech hiring platform and a thing we noticed again and again is that putting up any filters immediately drove away top tier candidates. They're just not going to jump through hoops when they have so many other options available to them. Now if you goal is to hire a more average quality candidate, then you can get away with filter questions, coding tests, etc.

anyfoo 4y ago
Forgive me for being glib, but if the candidate jumps off because they don't want to answer a very quick, simple, efficient question, that they should fully know is because it is beneficial to filter out complete duds, I'm not sure I want to continue with them anyway? I wouldn't even know how that happened in an in-person or video interview, or even phone screen. "I'm sorry, I do not want to answer your simple upfront question, because it is beneath me"? Well, good luck to your future endeavors then. To be clear, as I explained in my first post, my filter questions are not FizzBuzz, they are much more domain specific. At least if it's past the phone screen stage and for more than an internship. But they are still identifiable as trivial filter question for any serious applicant.
- leros 4y ago
  It's not necessarily that it's beneath them, but more that they have so many options available to them. Imagine you have 10 companies that you are willing to consider: - 5 of them are willing to immediately bring you into a serious interview - 5 of them want you to take a coding test or some other pre-interview step Why would you bother with those last 5 companies? Just go for the companies that respect your resume enough to bring you in for a real interview. Those latter 5 companies will lose out on a lot great candidates due to their filters. I can tell you having worked on tech hiring at scale and having AB tested a lot of these things across many companies, you lose a lot of grade-A candidates with anything beyond bringing them in to interview. The thing about hiring though is that no single company really notices their own inefficiencies. Everyone has their own filters and their own method of incorrectly rejecting great candidates. However, they ultimately hire people so they feel their process works. They don't really know if they could be hiring better people or hiring people faster.
  - anyfoo 4y ago
    > - 5 of them want you to take a coding test or some other pre-interview step I don't see the assumption anywhere that the filter question is a separate step from anyone else here. Even if you mean a phone screen, then the easy filter question is only one of the very first questions of that phone screen, the rest will be more substantive. So let's continue with the filter question not being a dedicated extra step (really, I don't think anybody assumed so): > Why would you bother with those last 5 companies? Just go for the companies that respect your resume enough to bring you in for a real interview. Because presumably I want to work at any of these companies. If the thing that's stopping me from applying to them is answering a trivial question, either my desire was not strong at all to begin with, or there is something seriously wrong with me. Even if this is about the phone screen and not a filter question per se, I'm still not sure I really like the idea of omitting phone screens for more senior positions because you might lose on "better" candidates. For such a senior engineer, a phone screen should basically be a formality, and actually a chance for themselves to dip their toe in. Not just in seeing what kind of technical areas they come up with, but also an opportunity to ask preliminary questions about the company and the position in general. If that is off-putting enough to not consider applying or stop the process, then, again, the desire to work there could either have not been that strong, or the other reason.
    - leros 4y ago
      > So let's continue with the filter question not being a dedicated extra step I can't agree with that. I don't see point of a filter question if its not a separate step before the full interview. If you're going to bring someone in for a proper interview, go ahead and give them a full interview session. It's weird to put a filter a question into a full day's worth of interviewing. And I can say, having seen hiring practices across tons of companies, filter questions are almost always a separate step where candidates have a tendency to fall out of the hiring pipeline. > Even if this is about the phone screen and not a filter question per se, I'm still not sure I really like the idea of omitting phone screens for more senior positions because you might lose on "better" candidates. Phone screens are fine because they're usually more about culture fit and trying to figure out if the candidate and company are right for each other. Experienced candidates that get asked basic technical questions on a phone screen have a tendency to report a negative experience and not have a desire to move forward with the company.
justin_oaks 4y ago
There may be alternative methods to determining a candidate's skill without starting off with extremely low-ball questions. I'm thinking you might get better engagement with a binary-search algorithm. Let's say that a candidate's skill is on a spectrum from 0 to 100. You might start by asking a level 50 question, which if the candidate can't answer then you'll assume they're 0-50. Then you can ask a level 25 question and see how they do there before deciding if their skill level is 0-25 or 25-50. Obviously this is overly simplistic: knowledge/skill can be over several domains and dimensions, one question might not give sufficient signal to determine whether to raise or lower the difficulty of the next question, etc. But I think the principle may be sound. Also, if you're searching for top tier candidates, you might aim for a nerd-sniping [0] approach where you ask about a genuinely interesting problem. The good candidates will be fine musing over the interesting problem and discussing possible solutions. The lower tier candidates will either be completely stumped, give only one solution, or give poor solutions. [0]: https://xkcd.com/356/ https://xkcd.com/356/

by leros 4y ago

anyfoo 4y ago

Forgive me for being glib, but if the candidate jumps off because they don't want to answer a very quick, simple, efficient question, that they should fully know is because it is beneficial to filter out complete duds, I'm not sure I want to continue with them anyway? I wouldn't even know how that happened in an in-person or video interview, or even phone screen. "I'm sorry, I do not want to answer your simple upfront question, because it is beneath me"? Well, good luck to your future endeavors then. To be clear, as I explained in my first post, my filter questions are not FizzBuzz, they are much more domain specific. At least if it's past the phone screen stage and for more than an internship. But they are still identifiable as trivial filter question for any serious applicant.

leros 4y ago
It's not necessarily that it's beneath them, but more that they have so many options available to them. Imagine you have 10 companies that you are willing to consider: - 5 of them are willing to immediately bring you into a serious interview - 5 of them want you to take a coding test or some other pre-interview step Why would you bother with those last 5 companies? Just go for the companies that respect your resume enough to bring you in for a real interview. Those latter 5 companies will lose out on a lot great candidates due to their filters. I can tell you having worked on tech hiring at scale and having AB tested a lot of these things across many companies, you lose a lot of grade-A candidates with anything beyond bringing them in to interview. The thing about hiring though is that no single company really notices their own inefficiencies. Everyone has their own filters and their own method of incorrectly rejecting great candidates. However, they ultimately hire people so they feel their process works. They don't really know if they could be hiring better people or hiring people faster.
- anyfoo 4y ago
  > - 5 of them want you to take a coding test or some other pre-interview step I don't see the assumption anywhere that the filter question is a separate step from anyone else here. Even if you mean a phone screen, then the easy filter question is only one of the very first questions of that phone screen, the rest will be more substantive. So let's continue with the filter question not being a dedicated extra step (really, I don't think anybody assumed so): > Why would you bother with those last 5 companies? Just go for the companies that respect your resume enough to bring you in for a real interview. Because presumably I want to work at any of these companies. If the thing that's stopping me from applying to them is answering a trivial question, either my desire was not strong at all to begin with, or there is something seriously wrong with me. Even if this is about the phone screen and not a filter question per se, I'm still not sure I really like the idea of omitting phone screens for more senior positions because you might lose on "better" candidates. For such a senior engineer, a phone screen should basically be a formality, and actually a chance for themselves to dip their toe in. Not just in seeing what kind of technical areas they come up with, but also an opportunity to ask preliminary questions about the company and the position in general. If that is off-putting enough to not consider applying or stop the process, then, again, the desire to work there could either have not been that strong, or the other reason.
  - leros 4y ago
    > So let's continue with the filter question not being a dedicated extra step I can't agree with that. I don't see point of a filter question if its not a separate step before the full interview. If you're going to bring someone in for a proper interview, go ahead and give them a full interview session. It's weird to put a filter a question into a full day's worth of interviewing. And I can say, having seen hiring practices across tons of companies, filter questions are almost always a separate step where candidates have a tendency to fall out of the hiring pipeline. > Even if this is about the phone screen and not a filter question per se, I'm still not sure I really like the idea of omitting phone screens for more senior positions because you might lose on "better" candidates. Phone screens are fine because they're usually more about culture fit and trying to figure out if the candidate and company are right for each other. Experienced candidates that get asked basic technical questions on a phone screen have a tendency to report a negative experience and not have a desire to move forward with the company.

justin_oaks 4y ago

There may be alternative methods to determining a candidate's skill without starting off with extremely low-ball questions. I'm thinking you might get better engagement with a binary-search algorithm. Let's say that a candidate's skill is on a spectrum from 0 to 100. You might start by asking a level 50 question, which if the candidate can't answer then you'll assume they're 0-50. Then you can ask a level 25 question and see how they do there before deciding if their skill level is 0-25 or 25-50. Obviously this is overly simplistic: knowledge/skill can be over several domains and dimensions, one question might not give sufficient signal to determine whether to raise or lower the difficulty of the next question, etc. But I think the principle may be sound. Also, if you're searching for top tier candidates, you might aim for a nerd-sniping [0] approach where you ask about a genuinely interesting problem. The good candidates will be fine musing over the interesting problem and discussing possible solutions. The lower tier candidates will either be completely stumped, give only one solution, or give poor solutions. [0]: https://xkcd.com/356/ https://xkcd.com/356/