6 ms·

Have we reached the tipping point yet? How much worse does the technical interview process need to get before the industry finally decides enough is enough? W

Have we reached the tipping point yet? How much worse does the technical interview process need to get before the industry finally decides enough is enough? When this all began, it was with the justification that it's not about getting the right answer, it's about seeing how you think and approach a problem. Companies don't even pretend this is the case anymore. It's now just expected rote memorization of algorithm patterns, talk about it with the right terms and key phrases, still pretend like the candidate had some brilliant "aha" moment, when both sides know it's bullshit, and write the solution on the board as fast as possible without errors. A completely pointless process that's now heavily pushed by an entire industry of tech interview prep (websites, books, interview coaches, etc.) When I now read a discussion about tech interviews, I wonder how many are industry shills who want to keep pushing the narrative that the process is great, but you just have to keep practicing and studying more, using the right materials of course. There's also the side effect of ageism. Who other than college grads or seniors in college has the free time to study for this process? Those with families and full time jobs are of course going to have a hard time. Even if they manage to find some free time here and there, how can they compete with someone in college with no responsibilities who has time to hundreds of problems? The simple answer is they can't. The college grad will always look smarter and faster in the interviews.

mkozlows 9y ago
Memorization? It's a factorial function. You shouldn't need to study or memorize anything to implement a factorial function. If you can't come up with this algorithm off the top of your head, that is legit telling something about your problem-solving ability.
- vthallam 9y ago
  Not particularly factorial problem, but the general interview questions require some amount of prior knowledge about the techniques. Topics like math based or bit manipulation, permutations/combinations have questions which are often asked in Interviews and require some prior knowledge to be able to give optimized solutions.
  - chillacy 9y ago
    Also if it's legitimately the first time you're solving something like "all permutations" you'll be slower than someone who has done it before, and that might hurt you too.
- auxym 9y ago
  I'd still ask for the definition of factorial. Sure, for various reasons, most programmers know the factorial, but I'd consider remembering the definition of some mathematical operation only common in certain domains (statistics and finance?), that most people only learn in high school and never use again, to be rote memorization.
  - nine_k 9y ago
    Ability to detect your deficiencies of knowledge and ask questions is hugely important, and is worth testing. Or you can use a search engine.
JustSomeNobody 9y ago
And the college grad will potentially be cheaper. "If you have kids, you're too old to code and should move into management." This is basically what the industry is saying.
tankenmate 9y ago
Coupled with the fact that most recent grads don't have much wisdom or common sense, relatively less so in the coding sense, and much more so in the everyday business environment sense. As a friend of mine used to have in his email sig; "You have a Ph.D.. Good, don't touch anything."
troycarlson 9y ago
Any suggestions for how to improve the process? I've been conducting a lot of technical phone screens lately and I'm always interested in how to make it better for both parties.
- justboxing 9y ago
  I've been attending a lot of phone screens, and in-person interviews for a Senior Software Engineer ( C# / .Net ) Position. Here's some feedback. YMMV. 1. Don't ask questions that are hard to explain over phone. For instance I was asked if I've used Lambdas in C# and I said, "Yes, extensively.", next question was. "Explain to me what a Lambda is.". I gave the textbook definition for it, and he was expecting more, so I simply said "It's hard to explain lambdas, I can show you some source code I've written and that way I'll be able to explain it better.". I thought that was it and that I would be 'screened-out', but I was moved to the next round of on-site interviews! 2. Do ask questions about how the candidate has used the technologies that the job requires, and ask them to explain how they designed and implemented their most recent solution, and where they used what technology, and how. After initial answer, drill deep into areas that are important for the job posted, and from the answers, you can get a fairly good assessment of whether the candidate should be moved to the next stage of the process. 3. After passing the tech phone screen, I've received and completed 3 take-home coding exercises (1 each from 3 different companies), and in 1 of those companies, I was subject to white (water-) boarding exercises on site in addition to the take home test. The white boarding seldom goes well because of the environment and nature of the whole thing. I found the take home exercises to be better than on-site white-boarding because the former is closer to how a future employee would work, than the latter. Also, how a candidate does on algorithm and data structure questions is a false positive (or negative) because these can be practiced, rehearsed, perfected and hence gamed. Coding exercises, or specific question on the candidate's most recent project, on the other hand, cannot be 'gamed' and even if it was, you could tell from listening to the response, or from reading the code... Hope this helps!
  - mkozlows 9y ago
    Take home is great, except people cheat. People even cheat on early screening questions knowing that an interview is coming up. (In a previous role, I gave applicants the same -- very trivial -- question that they had already coded the answer for in a do-at-home screening. I wasn't secretive about this, either, I told them that this is the exact same question they had already answered. A significant percentage of them were unable to figure out the same answer they had provided a few days earlier.)
mrtksn 9y ago
I think the goal is to be able to sell engineers on Amazon. To do this, a spec sheet of each engineer is needed and all these tests are attempt to create a standardised way of generating spec sheets.
dominotw 9y ago
> industry finally decides enough is enough? I think only google needs to decide this. Everyone will just follow what google does.
geebee 9y ago
Great question, when will we decide enough is enough? I actually don't have a problem with the data structures interview exam inherently, my problem is with how it's administered. It doesn't bother actuaries have to take exams to demonstrate an understanding of linear algebra, vector calculus, stats, numerical analysis, and so forth. Nor does it bother me that they have take these exams even if they took the material from a reputable college with a good grade. I think it's great that the actuarial field will allow people from multiple educational paths to take the test, giving them a choice about how they learned the material. But here's the thing - software developers have to take these "interview" exams over and over, at the whiteboard, every time we look for new job. And unlike the actuarial exams, we take these tests under conditions of tremendous secrecy. We don't know if our examiners are qualified, or if their questions are vetted, or if the exam is consistently applied and graded. We don't have a clear and defined study path, aside from books like "cracking the coding interview". And we get no feedback, hell, at times we don't know if our performance was evaluated at all. I've considered this for a while, and I've come to understand that exam-based professions and institutions, such as the medical boards, the bar, universities, have evolved a set of considerations, kind of a student bill of rights. Students must submit to the exams, but they have the right to a fair and consistently administered exam, evaluated by respected people in the profession or field, that the exam questions should not be capricious or utterly unexpected, that there is a preparation path, they get feedback on why they did not pass. What we have in tech is the stress and burden for examinees, in many ways amplified (whiteboard coding, on the spot, is among the more stressful ways to take an exam). But the student bill of rights is largely non-existent, and I think this is the deep and severe problem with tech interviews. Until this changes, to me, nothing can change meaningfully. And I don't just mean in interviews, I think this sort of thing is what is deeply rotten about tech, at the core of many of the problems (in a field where there's a lot of suspicion about bias, are secret tests under capricious conditions a good idea?) And we're expected to come back for more of this, over and over, and to just take it. Oh, by the way, there's a shortage of software engineers, best solved by putting tech companies in charge of the immigration system so they can decide who is and isn't allowed to come to the US based on how well they do on these interview tests/how much they're willing to put up with these interview tests.
- 9y ago
bit_logic 9y ago
Some comments here are asking what would be a better process. I think a big step is just stopping with the requirement to get the optimal solution. If I was doing these interviews, brute force is fine. If they can come up with a brute force and write code for it, they completely pass my coding bar. Some may say that's a low bar, but it's not. Because expecting anything more is going into rote memorization territory of algorithms. It's also not realistic. In the real world, brute force is always the first solution and it's the right first choice for a number of reasons: - Brute force is often simple to understand. That directly translates into maintainable code. - The business constraints on the input size may make the brute force an acceptable solution. - If you need a better optimized algorithm, brute force is a great place to start. It's basically your test case generator for verifying that your more complex algorithm is right. After they get brute force code written, the coding part is over and the rest of the interview is about discussing why it's brute force and what we could do about it. But no more coding is expected. It's a chance to be creative. If they want to talk about improving the algorithm that's fine, let's draw some diagrams. They want to throw more hardware at it? Sure, let's talk about how to scale that. Maybe they've actually seen a business problem before that was similar? Great, let's discuss how you solved it.
- jxramos 9y ago
  +1 for test case generator, some of our libraries even have a dividing path between optimized vs brute force modes just so we can be certain of correctness in unit test and what not. Correctness first with all the testing infrastructure and TODOs to at a later time go in and lay better algorithms down/execute general refactoring.
- dopamean 9y ago
  I really like this comment a lot because it mirrors my experience with brute force type solutions to problems. It's pretty much what I always start with because on my first pass I need to prove that the problem can be solved in the first place. Then that gives me a baseline where I feel comfortable making changes and trying to optimize. Like you said, very often because of business constraints the optimization only needs to go so far and so an "optimal solution" may not necessarily be the one that is fastest in benchmarks. It may be the one I can get out today that solves the business problem now.
NTDF9 9y ago
I LOLed at your comment! I still remember a recent interview I had with a team at Amazon. That team had accomplished NOTHING (and I mean NOTHING OF SIGNIFICANCE) in past 3 years, went through re-orgs and new managers, high attrition, nothing to show for. Here I am with successful products, published code and a track record of delivering. I have the skills to help them out. What happens? I get rejected because I didn't remember how to use a comparator in merging k sorted arrays and write that perfect code on a whiteboard. WTF??!! Did hiring rote memorizers really help that team in Amazon? Why do they not have something productive in 3 years despite having hired experts at tree traversals?
- nine_k 9y ago
  Good thing you did not end up in a lame team like that, with unreasonable decision-making like you've seen. Interviews are a mutual process, where the hiring team must do their best to persuade me to join. (The same way I do my best to persuade them to accept me.) Some of them utterly fail, even if they make an offer.
- geebee 9y ago
  This is a good comment. One thing I'd like to point out is that people often defend technical interview exams on the notion that if someone can't write a factorial function, that's a bad sign. Well, sure, but my experience has been that the technical interview exam is far more elaborate than this. My questions were more along the lines of finding all permutations of a set (usually with some twist, like find all permutations that, when combined, match another member of the set). Or finding all matching sub trees in an binary tree, or something depending on merge sort or quick sort. That sort of thing. Nothing impossible, but considerably more involved than simple recursion. And the teams that interviewed me clearly did expect to see largely working code written on a whiteboard. Overall, it's elaborate enough that people do need to study substantially for their interview exams, and good developers may simply decide they don't want to waste the time on a job that might not even come through anyway. I once learned to do intricate integration by parts in calculus, but I wouldn't be especially interested in training up to do it for a data science interview. If it comes up for reals, I'll deal with it. This is something the industry has inflicted on itself, which is why I feel a lot of irritation when I hear the same companies talk about a severe shortage of talented software developers.
bogomipz 9y ago
>"When this all began, it was with the justification that it's not about getting the right answer, it's about seeing how you think and approach a problem." Indeed. Now the default seems to be companies sending out a Hacker Ranks test consisting of 5 separate problems, a countdown timer and zero human interaction. I can't help but think that recruiters are playing a part in pushing for this online Hacker Rank test bullshit. It feels to me that as the quality of people working as recruiters goes further into the toilet the reliance on the Hacker Rank puzzle test seems to increase.
- nine_k 9y ago
  I still suppose it's a reasonable input filter. If you can solve a few simple coding problems, you're worth talking to face to face onsite. This depends on the problems being reasonable, though.

mkozlows 9y ago

Memorization? It's a factorial function. You shouldn't need to study or memorize anything to implement a factorial function. If you can't come up with this algorithm off the top of your head, that is legit telling something about your problem-solving ability.

vthallam 9y ago
Not particularly factorial problem, but the general interview questions require some amount of prior knowledge about the techniques. Topics like math based or bit manipulation, permutations/combinations have questions which are often asked in Interviews and require some prior knowledge to be able to give optimized solutions.
- chillacy 9y ago
  Also if it's legitimately the first time you're solving something like "all permutations" you'll be slower than someone who has done it before, and that might hurt you too.
auxym 9y ago
I'd still ask for the definition of factorial. Sure, for various reasons, most programmers know the factorial, but I'd consider remembering the definition of some mathematical operation only common in certain domains (statistics and finance?), that most people only learn in high school and never use again, to be rote memorization.
- nine_k 9y ago
  Ability to detect your deficiencies of knowledge and ask questions is hugely important, and is worth testing. Or you can use a search engine.

JustSomeNobody 9y ago

And the college grad will potentially be cheaper. "If you have kids, you're too old to code and should move into management." This is basically what the industry is saying.

tankenmate 9y ago

Coupled with the fact that most recent grads don't have much wisdom or common sense, relatively less so in the coding sense, and much more so in the everyday business environment sense. As a friend of mine used to have in his email sig; "You have a Ph.D.. Good, don't touch anything."

troycarlson 9y ago

Any suggestions for how to improve the process? I've been conducting a lot of technical phone screens lately and I'm always interested in how to make it better for both parties.

justboxing 9y ago
I've been attending a lot of phone screens, and in-person interviews for a Senior Software Engineer ( C# / .Net ) Position. Here's some feedback. YMMV. 1. Don't ask questions that are hard to explain over phone. For instance I was asked if I've used Lambdas in C# and I said, "Yes, extensively.", next question was. "Explain to me what a Lambda is.". I gave the textbook definition for it, and he was expecting more, so I simply said "It's hard to explain lambdas, I can show you some source code I've written and that way I'll be able to explain it better.". I thought that was it and that I would be 'screened-out', but I was moved to the next round of on-site interviews! 2. Do ask questions about how the candidate has used the technologies that the job requires, and ask them to explain how they designed and implemented their most recent solution, and where they used what technology, and how. After initial answer, drill deep into areas that are important for the job posted, and from the answers, you can get a fairly good assessment of whether the candidate should be moved to the next stage of the process. 3. After passing the tech phone screen, I've received and completed 3 take-home coding exercises (1 each from 3 different companies), and in 1 of those companies, I was subject to white (water-) boarding exercises on site in addition to the take home test. The white boarding seldom goes well because of the environment and nature of the whole thing. I found the take home exercises to be better than on-site white-boarding because the former is closer to how a future employee would work, than the latter. Also, how a candidate does on algorithm and data structure questions is a false positive (or negative) because these can be practiced, rehearsed, perfected and hence gamed. Coding exercises, or specific question on the candidate's most recent project, on the other hand, cannot be 'gamed' and even if it was, you could tell from listening to the response, or from reading the code... Hope this helps!
- mkozlows 9y ago
  Take home is great, except people cheat. People even cheat on early screening questions knowing that an interview is coming up. (In a previous role, I gave applicants the same -- very trivial -- question that they had already coded the answer for in a do-at-home screening. I wasn't secretive about this, either, I told them that this is the exact same question they had already answered. A significant percentage of them were unable to figure out the same answer they had provided a few days earlier.)

mrtksn 9y ago

I think the goal is to be able to sell engineers on Amazon. To do this, a spec sheet of each engineer is needed and all these tests are attempt to create a standardised way of generating spec sheets.

dominotw 9y ago

> industry finally decides enough is enough? I think only google needs to decide this. Everyone will just follow what google does.

geebee 9y ago

Great question, when will we decide enough is enough? I actually don't have a problem with the data structures interview exam inherently, my problem is with how it's administered. It doesn't bother actuaries have to take exams to demonstrate an understanding of linear algebra, vector calculus, stats, numerical analysis, and so forth. Nor does it bother me that they have take these exams even if they took the material from a reputable college with a good grade. I think it's great that the actuarial field will allow people from multiple educational paths to take the test, giving them a choice about how they learned the material. But here's the thing - software developers have to take these "interview" exams over and over, at the whiteboard, every time we look for new job. And unlike the actuarial exams, we take these tests under conditions of tremendous secrecy. We don't know if our examiners are qualified, or if their questions are vetted, or if the exam is consistently applied and graded. We don't have a clear and defined study path, aside from books like "cracking the coding interview". And we get no feedback, hell, at times we don't know if our performance was evaluated at all. I've considered this for a while, and I've come to understand that exam-based professions and institutions, such as the medical boards, the bar, universities, have evolved a set of considerations, kind of a student bill of rights. Students must submit to the exams, but they have the right to a fair and consistently administered exam, evaluated by respected people in the profession or field, that the exam questions should not be capricious or utterly unexpected, that there is a preparation path, they get feedback on why they did not pass. What we have in tech is the stress and burden for examinees, in many ways amplified (whiteboard coding, on the spot, is among the more stressful ways to take an exam). But the student bill of rights is largely non-existent, and I think this is the deep and severe problem with tech interviews. Until this changes, to me, nothing can change meaningfully. And I don't just mean in interviews, I think this sort of thing is what is deeply rotten about tech, at the core of many of the problems (in a field where there's a lot of suspicion about bias, are secret tests under capricious conditions a good idea?) And we're expected to come back for more of this, over and over, and to just take it. Oh, by the way, there's a shortage of software engineers, best solved by putting tech companies in charge of the immigration system so they can decide who is and isn't allowed to come to the US based on how well they do on these interview tests/how much they're willing to put up with these interview tests.

9y ago

bit_logic 9y ago

Some comments here are asking what would be a better process. I think a big step is just stopping with the requirement to get the optimal solution. If I was doing these interviews, brute force is fine. If they can come up with a brute force and write code for it, they completely pass my coding bar. Some may say that's a low bar, but it's not. Because expecting anything more is going into rote memorization territory of algorithms. It's also not realistic. In the real world, brute force is always the first solution and it's the right first choice for a number of reasons: - Brute force is often simple to understand. That directly translates into maintainable code. - The business constraints on the input size may make the brute force an acceptable solution. - If you need a better optimized algorithm, brute force is a great place to start. It's basically your test case generator for verifying that your more complex algorithm is right. After they get brute force code written, the coding part is over and the rest of the interview is about discussing why it's brute force and what we could do about it. But no more coding is expected. It's a chance to be creative. If they want to talk about improving the algorithm that's fine, let's draw some diagrams. They want to throw more hardware at it? Sure, let's talk about how to scale that. Maybe they've actually seen a business problem before that was similar? Great, let's discuss how you solved it.

jxramos 9y ago
+1 for test case generator, some of our libraries even have a dividing path between optimized vs brute force modes just so we can be certain of correctness in unit test and what not. Correctness first with all the testing infrastructure and TODOs to at a later time go in and lay better algorithms down/execute general refactoring.
dopamean 9y ago
I really like this comment a lot because it mirrors my experience with brute force type solutions to problems. It's pretty much what I always start with because on my first pass I need to prove that the problem can be solved in the first place. Then that gives me a baseline where I feel comfortable making changes and trying to optimize. Like you said, very often because of business constraints the optimization only needs to go so far and so an "optimal solution" may not necessarily be the one that is fastest in benchmarks. It may be the one I can get out today that solves the business problem now.

NTDF9 9y ago

I LOLed at your comment! I still remember a recent interview I had with a team at Amazon. That team had accomplished NOTHING (and I mean NOTHING OF SIGNIFICANCE) in past 3 years, went through re-orgs and new managers, high attrition, nothing to show for. Here I am with successful products, published code and a track record of delivering. I have the skills to help them out. What happens? I get rejected because I didn't remember how to use a comparator in merging k sorted arrays and write that perfect code on a whiteboard. WTF??!! Did hiring rote memorizers really help that team in Amazon? Why do they not have something productive in 3 years despite having hired experts at tree traversals?

nine_k 9y ago
Good thing you did not end up in a lame team like that, with unreasonable decision-making like you've seen. Interviews are a mutual process, where the hiring team must do their best to persuade me to join. (The same way I do my best to persuade them to accept me.) Some of them utterly fail, even if they make an offer.
geebee 9y ago
This is a good comment. One thing I'd like to point out is that people often defend technical interview exams on the notion that if someone can't write a factorial function, that's a bad sign. Well, sure, but my experience has been that the technical interview exam is far more elaborate than this. My questions were more along the lines of finding all permutations of a set (usually with some twist, like find all permutations that, when combined, match another member of the set). Or finding all matching sub trees in an binary tree, or something depending on merge sort or quick sort. That sort of thing. Nothing impossible, but considerably more involved than simple recursion. And the teams that interviewed me clearly did expect to see largely working code written on a whiteboard. Overall, it's elaborate enough that people do need to study substantially for their interview exams, and good developers may simply decide they don't want to waste the time on a job that might not even come through anyway. I once learned to do intricate integration by parts in calculus, but I wouldn't be especially interested in training up to do it for a data science interview. If it comes up for reals, I'll deal with it. This is something the industry has inflicted on itself, which is why I feel a lot of irritation when I hear the same companies talk about a severe shortage of talented software developers.

bogomipz 9y ago

>"When this all began, it was with the justification that it's not about getting the right answer, it's about seeing how you think and approach a problem." Indeed. Now the default seems to be companies sending out a Hacker Ranks test consisting of 5 separate problems, a countdown timer and zero human interaction. I can't help but think that recruiters are playing a part in pushing for this online Hacker Rank test bullshit. It feels to me that as the quality of people working as recruiters goes further into the toilet the reliance on the Hacker Rank puzzle test seems to increase.

nine_k 9y ago
I still suppose it's a reasonable input filter. If you can solve a few simple coding problems, you're worth talking to face to face onsite. This depends on the problems being reasonable, though.