10 ms·

> What’s the best way to test if someone can do a particular job well? Get them to carry out tasks that are part of the job. See if they can do what they say th

by kanox 6y ago

> What’s the best way to test if someone can do a particular job well? Get them to carry out tasks that are part of the job. See if they can do what they say they can do. It’s much harder for someone to lie and mislead an interviewer during actual work than during an interview. Using competency tests for a blinded interview process is also possible—interviewers could look at depersonalized test results to make unbiased judgments. Why would a practical test be more effective than a discussion about prior work experience? Short coding tests under pressure are extremely unrepresentative of real work and I'm not sure homework-style interviews are much better. I personally don't want to spend an entire weekend on your test and I've dropped opportunities for that reason in the past. All these "competency tests" are good for is catching blatant lying. Is this a serious problem? In my view job interviews are mostly about finding a match between skills/interests and project needs. Communications skills are a very big advantage in interviewing and that feels unfair but I'm not sure that true. Communication and persuasion skills are extremely useful and important to all office jobs, even the most technical. I'd bet that people who interview well also write beautiful code comments and commit messages.

maksimilian 6y ago
I imagine a good compromise might be a small piece of "contract work" where a company presents a small representative task as a project for a candidate to complete and the candidate actually gets some form of compensation for their time. There are definitely problems to this approach, but in a best case scenario, the result is a win-win situation even if a candidate doesn't get hired. At the moment every candidate that doesn't get hired represents a net loss for both parties (time for both and opportunity for the candidate)
hellcow 6y ago
> All these "competency tests" are good for is catching blatant lying. Is this a serious problem? Of the people I interview that claim they have years of experience, mastery over multiple languages, and expertise in various frameworks, a solid 80% or more can't pass fizz-buzz in any language they choose.
- smichel17 6y ago
  Claims made on resume or in person? On the seeking side, I've felt pressure to put every technology I've used even in passing on my resume, to satisfy "buzzword bingo" and get through the initial screen, but in an interview I would give a (truthful) answer along the lines of "I don't have a ton of experience, but I know the basics; enough to be confident that I can quickly pick up whatever else I need to learn on the job."
  - hellcow 6y ago
    > Claims made on resume or in person? Both. And it's not "putting every technology I've used even in passing on the resume" that bothers me. It's "I don't know how to write a loop or a function in any programming language."
- kofejnik 6y ago
  Having administered over 600 interviews, I second this. Most people can’t code, despite degrees and years of experience.
leray_J 6y ago
Agree on the test stuff. When talking to someone who can goes deep in details you mostly know that he's not a lier; then you have the trial period to fire him if he is.
kabdib 6y ago
> Is this a serious problem? You would be astonished, absolutely flabbergasted, at the number of "senior engineers" who interview well, talk great about projects, but cannot demonstrate basic programming skills (e.g., convert an integer to a string representation, or write fizzbuzz).
- invalidOrTaken 6y ago
  How does this happen? What (apparently huge) swath of industry am I insulated from where this can take root? Is this just a bunch of wannabes after tech salaries? Or is this is a side-effect of industrial java programming, where their "job" was basically filling in boilerplate? (I am probably unfairly slandering java shops, it seems unlikely, but everything else seems even more unlikely) It boggles my mind.
  - Afton 6y ago
    I think it usually involves 'helper' type folks. They end up on a project, and they 'help' the project get done. But the work they do tends to be just the boilerplate (or cut-and-paste style work), or just the gathering of test resources, or a surprising amount of work that you would assume that the PM or Manager would do, but instead is done by this person (organizing meetings, setting up checklists, emailing people about current issues/priorities). This isn't to say that those people don't add value, but if they do this for a few years, they may find that their ability to do greenfield coding has significantly diminished, so much so that they struggle to write objectively simple code because they've fallen out of practice. Source: I've been one of those people, and I've worked with many. It's an easy rut to fall into, particularly in Test Engineer role (what MS used to call SDET roles).
  - kqr 6y ago
    There's the concept of the "expert beginner" who works in a role when they're never challenged, so they quickly assume they're great at the thing when from a global perspective, the just stopped learning very quickly. There's also the type that accidentally falls into a manager-type role way too soon, before they've actually had a proper chance to learn programming well. Then whatever basic skills they had atrophies quickly.
itgoon 6y ago
Competency tests have helped me get in the door. That said, they seem to work best when they're part of an overall interview process. I think it's a mistake to look at only the highest scores and interview those people, they should only weed out the bottom scores and follow a more typical (do they fit? can they learn it?) process for the rest.
- kanox 6y ago
  > Competency tests have helped me get in the door. How so? The door was already open if the company decided to give you a test. My impression is that coding tests either offer neutral or negative feedback during an interview. It's not like a candidate can manipulate linked lists in a way that shows particular brilliance.
  - kofejnik 6y ago
    Nope, the first hurdle is usually some bs resume filter which removes a lot of competent candidates from consideration
  - itgoon 6y ago
    The test comes before the interview, after speaking to a recruiter.
kube-system 6y ago
> Why would a practical test be more effective than a discussion about prior work experience? Short coding tests under pressure are extremely unrepresentative of real work and I'm not sure homework-style interviews are much better. I personally don't want to spend an entire weekend on your test and I've dropped opportunities for that reason in the past. These are good points, and the tests I have given are specifically designed to avoid these issues. Personally I do this: * Come up with a simple but realistic 1 or 2 hour task, and write up the request in plain English (do not write a tech spec). It should be task you'd expect anyone to be able to easily do on day one without any assistance. * Include a short narrative about the end user that clearly implies a specific requirement or two, but don't spell out it out as an explicit requirement. * Omit a small but important detail that any reasonable person would ask. * Instruct them to provide a solution to the users problem, and ask questions if something isn't clear. These are pretty simple to evaluate too: * Does the code run? (i.e. can the candidate write basic code?) * Does it meet the user's explicit needs? (i.e. can the candidate follow basic instructions?) * Does it meet the user's implicit needs? (i.e. does the candidate write code to spec, or do they think about the bigger picture?) * Did the candidate ask a question about that obvious missing detail? (i.e. does the candidate speak up, or will they make assumptions?) You would be surprised with how many people don't have the ability to apply their ability to write code to solve simple problems.
dean177 6y ago
> All these "competency tests" are good for is catching blatant lying. Is this a serious problem? It absolutely is, there are plenty of companies who have no staff capable of verifying a candidates technical abilities

by kanox 6y ago

maksimilian 6y ago

I imagine a good compromise might be a small piece of "contract work" where a company presents a small representative task as a project for a candidate to complete and the candidate actually gets some form of compensation for their time. There are definitely problems to this approach, but in a best case scenario, the result is a win-win situation even if a candidate doesn't get hired. At the moment every candidate that doesn't get hired represents a net loss for both parties (time for both and opportunity for the candidate)

hellcow 6y ago

> All these "competency tests" are good for is catching blatant lying. Is this a serious problem? Of the people I interview that claim they have years of experience, mastery over multiple languages, and expertise in various frameworks, a solid 80% or more can't pass fizz-buzz in any language they choose.

smichel17 6y ago
Claims made on resume or in person? On the seeking side, I've felt pressure to put every technology I've used even in passing on my resume, to satisfy "buzzword bingo" and get through the initial screen, but in an interview I would give a (truthful) answer along the lines of "I don't have a ton of experience, but I know the basics; enough to be confident that I can quickly pick up whatever else I need to learn on the job."
- hellcow 6y ago
  > Claims made on resume or in person? Both. And it's not "putting every technology I've used even in passing on the resume" that bothers me. It's "I don't know how to write a loop or a function in any programming language."
kofejnik 6y ago
Having administered over 600 interviews, I second this. Most people can’t code, despite degrees and years of experience.

leray_J 6y ago

Agree on the test stuff. When talking to someone who can goes deep in details you mostly know that he's not a lier; then you have the trial period to fire him if he is.

kabdib 6y ago

> Is this a serious problem? You would be astonished, absolutely flabbergasted, at the number of "senior engineers" who interview well, talk great about projects, but cannot demonstrate basic programming skills (e.g., convert an integer to a string representation, or write fizzbuzz).

invalidOrTaken 6y ago
How does this happen? What (apparently huge) swath of industry am I insulated from where this can take root? Is this just a bunch of wannabes after tech salaries? Or is this is a side-effect of industrial java programming, where their "job" was basically filling in boilerplate? (I am probably unfairly slandering java shops, it seems unlikely, but everything else seems even more unlikely) It boggles my mind.
- Afton 6y ago
  I think it usually involves 'helper' type folks. They end up on a project, and they 'help' the project get done. But the work they do tends to be just the boilerplate (or cut-and-paste style work), or just the gathering of test resources, or a surprising amount of work that you would assume that the PM or Manager would do, but instead is done by this person (organizing meetings, setting up checklists, emailing people about current issues/priorities). This isn't to say that those people don't add value, but if they do this for a few years, they may find that their ability to do greenfield coding has significantly diminished, so much so that they struggle to write objectively simple code because they've fallen out of practice. Source: I've been one of those people, and I've worked with many. It's an easy rut to fall into, particularly in Test Engineer role (what MS used to call SDET roles).
- kqr 6y ago
  There's the concept of the "expert beginner" who works in a role when they're never challenged, so they quickly assume they're great at the thing when from a global perspective, the just stopped learning very quickly. There's also the type that accidentally falls into a manager-type role way too soon, before they've actually had a proper chance to learn programming well. Then whatever basic skills they had atrophies quickly.

itgoon 6y ago

Competency tests have helped me get in the door. That said, they seem to work best when they're part of an overall interview process. I think it's a mistake to look at only the highest scores and interview those people, they should only weed out the bottom scores and follow a more typical (do they fit? can they learn it?) process for the rest.

kanox 6y ago
> Competency tests have helped me get in the door. How so? The door was already open if the company decided to give you a test. My impression is that coding tests either offer neutral or negative feedback during an interview. It's not like a candidate can manipulate linked lists in a way that shows particular brilliance.
- kofejnik 6y ago
  Nope, the first hurdle is usually some bs resume filter which removes a lot of competent candidates from consideration
- itgoon 6y ago
  The test comes before the interview, after speaking to a recruiter.

kube-system 6y ago

> Why would a practical test be more effective than a discussion about prior work experience? Short coding tests under pressure are extremely unrepresentative of real work and I'm not sure homework-style interviews are much better. I personally don't want to spend an entire weekend on your test and I've dropped opportunities for that reason in the past. These are good points, and the tests I have given are specifically designed to avoid these issues. Personally I do this: * Come up with a simple but realistic 1 or 2 hour task, and write up the request in plain English (do not write a tech spec). It should be task you'd expect anyone to be able to easily do on day one without any assistance. * Include a short narrative about the end user that clearly implies a specific requirement or two, but don't spell out it out as an explicit requirement. * Omit a small but important detail that any reasonable person would ask. * Instruct them to provide a solution to the users problem, and ask questions if something isn't clear. These are pretty simple to evaluate too: * Does the code run? (i.e. can the candidate write basic code?) * Does it meet the user's explicit needs? (i.e. can the candidate follow basic instructions?) * Does it meet the user's implicit needs? (i.e. does the candidate write code to spec, or do they think about the bigger picture?) * Did the candidate ask a question about that obvious missing detail? (i.e. does the candidate speak up, or will they make assumptions?) You would be surprised with how many people don't have the ability to apply their ability to write code to solve simple problems.

dean177 6y ago

> All these "competency tests" are good for is catching blatant lying. Is this a serious problem? It absolutely is, there are plenty of companies who have no staff capable of verifying a candidates technical abilities