3 ms·

Unless you have data to show that your "different way" produces better results, your idea sounds exactly the same as every other idea, which is basically garbag

Unless you have data to show that your "different way" produces better results, your idea sounds exactly the same as every other idea, which is basically garbage. Lots of people have lots of ideas on what makes a great interview question, but none of them are backed by data. LeetCode-style algo questions USED to be an indicator of intelligence as per Google's investigation, but now it's so heavily gamed that I doubt there's any signal behind it anymore. Someone needs to spend time and effort and money and actually do random controlled tests to see if people who pass a particular type of test actually make good employees. But so far no one has done this, except Google as far as I've seen. But even now, I think there's no evidence that even algorithm questions are any indicator, given what I've seen.

godelski 2y ago
> Unless you have data to show that your "different way" produces better results, your idea sounds exactly the same as every other idea, which is basically garbage. Do you have evidence that the standard coding interview works? (There's evidence that it doesn't) I'm with you that the claim might be too strong to say "this is the way" but that's because I'm of the (very strong) opinion that interviewing is an extremely fuzzy process and there are no clear cut metrics to measure one's abilities in a significantly meaningful way. a̶l̶l̶ ̶m̶o̶d̶e̶l̶s̶ ̶a̶r̶e̶ ̶w̶r̶o̶n̶g̶ ̶b̶u̶t̶ ̶s̶o̶m̶e̶ ̶a̶r̶e̶ ̶u̶s̶e̶f̶u̶l̶ There are useful interviewing methods but certainly not "the best" method. Trying to just mark checkboxes only leads to mediocre results. The reason we're generally okay with this is because we more often than not don't need rockstars and it doesn't make sense to put a lot of time and energy into this process when we get sufficient results through lazy methods. FWIW, a typical (non-software) engineering job really just involves high level discussions like the OP suggests but even without the implementation. It is generally about seeing how the person thinks/problem solves and looking at problems they solved in the past. It isn't resource intensive and good enough. Because truth is, you don't know what someone is like as an employee until they are an employee (and even then this is fuzzy)
- blindriver 2y ago
  > Do you have evidence that the standard coding interview works? No, I think that they are all garbage. The only way to really hire is by having a vibe check to see if they are someone the team wants to work with, making sure that the person seems competent and has a reasonable chance of being very productive, and then hiring them quick. Give them a month and if they don't seem like a good fit, then fire them, with 2 months severance. This is the only way I've seen that will produce a great team quickly, by hiring quickly and firing quickly. This is similar to what Netflix does but they also pay top of market which not too many companies can afford, but it produces the best results.
  - godelski 2y ago
    Seems we're on the same page: https://news.ycombinator.com/item?id=40291828 https://news.ycombinator.com/item?id=40291828 > Give them a month and if they don't seem like a good fit, then fire them, with 2 months severance. And this! I think this is quite effective and efficient. You aren't wasting anyone's time and making sure the person has adequate time to find their next income source. It immediately makes me respect you and I think would build high employee loyalty.
    - valicord 2y ago
      As long as you want to only hire people who don't have an existing job or competing offers.
  - YZF 2y ago
    But we're talking about "seems competent and has a reasonable chance of being very productive". That's why you're interviewing in the first place. If you don't ask them to demonstrate their competence in any way how do you know they're competent? If they can't "produce" anything during an interview (even something trivial) how do you know they'll be productive? Assuming a complete stranger with no references you can trust. A month might be too short of a time for certain roles. I think people would be hesitant to take a risk with you if they know this is your policy. I'd would say that at any point where someone is clearly not a fit they should be let go. You might know after a month (if they're terrible), you might know after 6 months, they might progress initially but stall. I think e.g. with new grads it's going to take a little longer in general since they have a pretty long growth trajectory.
hintymad 2y ago
All fair points, and I agree with you on rigorous experiments. For now we are talking about only intuitions.
Paul-Craft 2y ago
> LeetCode-style algo questions USED to be an indicator of intelligence as per Google's investigation, but now it's so heavily gamed that I doubt there's any signal behind it anymore. Where can I find the data on this?
YZF 2y ago
Don't discard good anecdotal evidence because there's no proper randomly controlled trial. If you're experienced/senior in the industry you probably worked with dozens of developers closely, with hundreds less closely, you've seen many projects and designs, you've interviewed and was involved in the hiring of dozens of people. This random controlled trial you're talking about is so hard to do because there are infinite confounders. How do you even measure success? After how long? How do you separate politics from technical ability? In reality people above a certain bar are able to make some contribution to most projects. You can't and don't need to staff every project with superstars. The superstars are rare but are not hard to identify. They'll have the track record, reputation and the skills. Another thing to consider is that the best outcome of interviews depends on the quality of the pipeline into those interviews. If you can't get good employees to apply you won't get good employees hired. Pay well and build a company that people want to work for, and then your hiring is going to be easier.
- niccl 2y ago
  I guess we could make some reasonable evidence about hiring approaches based on HN hirers: What percentage of hires using your favourite technique/interview question do you think worked out as a success? My rate is around 50% good, 50% I wish I hadn't hired. Anyone else want to fess up?
  - YZF 2y ago
    I've had a much better rate than that. What does "wish you hadn't hired" mean exactly. Did you completely misjudge them or did you have an idea of what you're getting and still hired them. To be way off on 50% of interviews feels like way too high failure rate.

godelski 2y ago

> Unless you have data to show that your "different way" produces better results, your idea sounds exactly the same as every other idea, which is basically garbage. Do you have evidence that the standard coding interview works? (There's evidence that it doesn't) I'm with you that the claim might be too strong to say "this is the way" but that's because I'm of the (very strong) opinion that interviewing is an extremely fuzzy process and there are no clear cut metrics to measure one's abilities in a significantly meaningful way. a̶l̶l̶ ̶m̶o̶d̶e̶l̶s̶ ̶a̶r̶e̶ ̶w̶r̶o̶n̶g̶ ̶b̶u̶t̶ ̶s̶o̶m̶e̶ ̶a̶r̶e̶ ̶u̶s̶e̶f̶u̶l̶ There are useful interviewing methods but certainly not "the best" method. Trying to just mark checkboxes only leads to mediocre results. The reason we're generally okay with this is because we more often than not don't need rockstars and it doesn't make sense to put a lot of time and energy into this process when we get sufficient results through lazy methods. FWIW, a typical (non-software) engineering job really just involves high level discussions like the OP suggests but even without the implementation. It is generally about seeing how the person thinks/problem solves and looking at problems they solved in the past. It isn't resource intensive and good enough. Because truth is, you don't know what someone is like as an employee until they are an employee (and even then this is fuzzy)

blindriver 2y ago
> Do you have evidence that the standard coding interview works? No, I think that they are all garbage. The only way to really hire is by having a vibe check to see if they are someone the team wants to work with, making sure that the person seems competent and has a reasonable chance of being very productive, and then hiring them quick. Give them a month and if they don't seem like a good fit, then fire them, with 2 months severance. This is the only way I've seen that will produce a great team quickly, by hiring quickly and firing quickly. This is similar to what Netflix does but they also pay top of market which not too many companies can afford, but it produces the best results.
- godelski 2y ago
  Seems we're on the same page: https://news.ycombinator.com/item?id=40291828 https://news.ycombinator.com/item?id=40291828 > Give them a month and if they don't seem like a good fit, then fire them, with 2 months severance. And this! I think this is quite effective and efficient. You aren't wasting anyone's time and making sure the person has adequate time to find their next income source. It immediately makes me respect you and I think would build high employee loyalty.
  - valicord 2y ago
    As long as you want to only hire people who don't have an existing job or competing offers.
- YZF 2y ago
  But we're talking about "seems competent and has a reasonable chance of being very productive". That's why you're interviewing in the first place. If you don't ask them to demonstrate their competence in any way how do you know they're competent? If they can't "produce" anything during an interview (even something trivial) how do you know they'll be productive? Assuming a complete stranger with no references you can trust. A month might be too short of a time for certain roles. I think people would be hesitant to take a risk with you if they know this is your policy. I'd would say that at any point where someone is clearly not a fit they should be let go. You might know after a month (if they're terrible), you might know after 6 months, they might progress initially but stall. I think e.g. with new grads it's going to take a little longer in general since they have a pretty long growth trajectory.

hintymad 2y ago

All fair points, and I agree with you on rigorous experiments. For now we are talking about only intuitions.

Paul-Craft 2y ago

> LeetCode-style algo questions USED to be an indicator of intelligence as per Google's investigation, but now it's so heavily gamed that I doubt there's any signal behind it anymore. Where can I find the data on this?

YZF 2y ago

Don't discard good anecdotal evidence because there's no proper randomly controlled trial. If you're experienced/senior in the industry you probably worked with dozens of developers closely, with hundreds less closely, you've seen many projects and designs, you've interviewed and was involved in the hiring of dozens of people. This random controlled trial you're talking about is so hard to do because there are infinite confounders. How do you even measure success? After how long? How do you separate politics from technical ability? In reality people above a certain bar are able to make some contribution to most projects. You can't and don't need to staff every project with superstars. The superstars are rare but are not hard to identify. They'll have the track record, reputation and the skills. Another thing to consider is that the best outcome of interviews depends on the quality of the pipeline into those interviews. If you can't get good employees to apply you won't get good employees hired. Pay well and build a company that people want to work for, and then your hiring is going to be easier.

niccl 2y ago
I guess we could make some reasonable evidence about hiring approaches based on HN hirers: What percentage of hires using your favourite technique/interview question do you think worked out as a success? My rate is around 50% good, 50% I wish I hadn't hired. Anyone else want to fess up?
- YZF 2y ago
  I've had a much better rate than that. What does "wish you hadn't hired" mean exactly. Did you completely misjudge them or did you have an idea of what you're getting and still hired them. To be way off on 50% of interviews feels like way too high failure rate.