3 ms·

You ever see or suspect fraud in a take home? I can't imagine it doesn't happen giving that cheating is so prevalent in colleges.

tptacek 8y ago
If I was doing this at Google, I would spend a lot of time thinking about test fraud. At a 40-50 person company? Not so much. We do simple follow-up things that raise the amount of effort you'd have to put in to fraudulently submitting work sample tests, and we know pretty exactly how we'd randomize our work sample tests to make it hard to cheat (at least, hard to cheat without doing something we'd be interested in anyways) --- but it's just not worth it right now. Work sample test fraud is one of those things that sounds like a huge deal on message boards, but when you game through what would be involved in doing it in real life, it makes very little sense.
wool_gather 8y ago
To my mind, the simplest and most straightforward way to combat any fraud is also beneficial because it gives you even more information about the candidate: Talk to them about the code they wrote. Have a conversation, as if they were already your co-worker, with the exercise as the subject. Go through it, ask them -- non-adversarially -- why they did X, what they thought about requirement 2, how they could get better test coverage for Z. If you see something that seems to be a mistake, talk about it. If you see something awesome, discuss it. I can't imagine someone incompetent being able to bullshit their way through detailed discussion of code they were supposed to have written. And this provides valuable info about how this person thinks and communicates about the work you want them to do.
- bradleyjg 8y ago
  That seems like a good idea, but does somewhat detract from the notion that once a candidate does the work-sample he is done with the technical part of the process.
  - wool_gather 8y ago
    True, it deviates from tptacek's recommendations; about scoring and identical questions for everyone as well.
    - swish_bob 8y ago
      A nice approach I liked (back when I was interviewed by Thoughtworks years ago) was that one of the interview stages was to take the assignment you'd done and apply some new requirements to the story. It rapidly makes it clear if the candidate actually did the assignment or not.
LaserToy 8y ago
Are yo saying most of candidates commit fraud? Do you have any data on it or just following common sense nonsense?
- bradleyjg 8y ago
  > Are yo saying most of candidates commit fraud? Nope.
  - LaserToy 8y ago
    If majority doesn’t, why create a process that punishes everyone? People are lying on the resumes is a very common argument made by creators of insane interview processes. It feels weird to start a relationship with a company with assumption that I’m a cheater.
    - bradleyjg 8y ago
      Who said anything about punishing anyone? 1) tptacek advocated for a specific kind of interviewing, 2) I asked him if he had ever seen a problem that I thought might be an issue with that method, 3) he explained that he wasn't worried about it because of X, Y, Z but acknowledged that for other companies it could be a problem, 4) I thought his answer was very reasonable. Also wool_gather and swish_bob added some useful ideas. I'm not sure why you felt the need to come out guns blazing.
drb91 8y ago
How can I sign up to take these tests for people? Sounds INCREDIBLY lucrative.

tptacek 8y ago

If I was doing this at Google, I would spend a lot of time thinking about test fraud. At a 40-50 person company? Not so much. We do simple follow-up things that raise the amount of effort you'd have to put in to fraudulently submitting work sample tests, and we know pretty exactly how we'd randomize our work sample tests to make it hard to cheat (at least, hard to cheat without doing something we'd be interested in anyways) --- but it's just not worth it right now. Work sample test fraud is one of those things that sounds like a huge deal on message boards, but when you game through what would be involved in doing it in real life, it makes very little sense.

wool_gather 8y ago

To my mind, the simplest and most straightforward way to combat any fraud is also beneficial because it gives you even more information about the candidate: Talk to them about the code they wrote. Have a conversation, as if they were already your co-worker, with the exercise as the subject. Go through it, ask them -- non-adversarially -- why they did X, what they thought about requirement 2, how they could get better test coverage for Z. If you see something that seems to be a mistake, talk about it. If you see something awesome, discuss it. I can't imagine someone incompetent being able to bullshit their way through detailed discussion of code they were supposed to have written. And this provides valuable info about how this person thinks and communicates about the work you want them to do.

bradleyjg 8y ago
That seems like a good idea, but does somewhat detract from the notion that once a candidate does the work-sample he is done with the technical part of the process.
- wool_gather 8y ago
  True, it deviates from tptacek's recommendations; about scoring and identical questions for everyone as well.
  - swish_bob 8y ago
    A nice approach I liked (back when I was interviewed by Thoughtworks years ago) was that one of the interview stages was to take the assignment you'd done and apply some new requirements to the story. It rapidly makes it clear if the candidate actually did the assignment or not.

LaserToy 8y ago

Are yo saying most of candidates commit fraud? Do you have any data on it or just following common sense nonsense?

bradleyjg 8y ago
> Are yo saying most of candidates commit fraud? Nope.
- LaserToy 8y ago
  If majority doesn’t, why create a process that punishes everyone? People are lying on the resumes is a very common argument made by creators of insane interview processes. It feels weird to start a relationship with a company with assumption that I’m a cheater.
  - bradleyjg 8y ago
    Who said anything about punishing anyone? 1) tptacek advocated for a specific kind of interviewing, 2) I asked him if he had ever seen a problem that I thought might be an issue with that method, 3) he explained that he wasn't worried about it because of X, Y, Z but acknowledged that for other companies it could be a problem, 4) I thought his answer was very reasonable. Also wool_gather and swish_bob added some useful ideas. I'm not sure why you felt the need to come out guns blazing.

drb91 8y ago

How can I sign up to take these tests for people? Sounds INCREDIBLY lucrative.