3 ms·

This reminds me of a time when I didn't do well on an interview that involved data structures. I usually review my old data structures and algorithms book prio

by geebee 13y ago

This reminds me of a time when I didn't do well on an interview that involved data structures. I usually review my old data structures and algorithms book prior to an interview, but I didn't this time because I was very busy. A few months later, I needed to do a lot of tree traversal for an algorithm. I studied up on it and started to feel embarrassed about the questions I had missed. Now that I'd reviewed tree traversal, these questions seemed less esoteric and more fundamental. I mentioned this to a coworker, saying that I could see how an employer that would need someone with knowledge of trees and traversal algorithms wouldn't have wanted to hire me. His response was insightful. "Actually," he said, "what you showed is that as soon as it came up on a real project, you were able to do the research and implement the algorithms." The problem is, how does the employer know you're one of the people who can actually do this? I guess it comes down to confidence. Google accepts that you probably don't know certain algorithms like the back of your hand, but they want to be sure that you can get up to speed with them in a month or so. So they tell you that you should spend a month reading and front loading algorithms and data structures. Then you interview, and they get to see how good you are at this when it is fresh in your mind. Anyone can claim they'd be able to learn, I guess this is how they force you to prove it. BTW, I'm only writing this to explain why they might want to do it, not why it is a good process. I completely understand how it feels from your point of view - I've been there too. Unless I have a particular problem I need to solve, there isn't really any reason for me to reload certain data structures and algorithms into my head. An interview could be a reason, but if I don't get the job, then I just spent the month studying for nothing. It really is nothing, because this is stuff I've already learned. To me, this would be lie having to re-lean difficult integration by parts techniques from calculus . I've done it more than once, I know it's there, but I don't keep it all front-loaded. I'm at the point where I'm not sure I'd do this again. The final straw was an 8 hour take-home assignment where the company didn't get back to me for a month and then had the recruiter call me to say they'd decided not to pursue this anymore. Google should do what it wants, of course, and if people desperately want to work there, they can get away with this sort of interview process. A lot of people (including me) just aren't willing to put in a full month of study just to get an interview. If I was certain to get the job, and the study is part of necessary preparation for the type of work I'd be doing, then sure... (but then why is this completely uncompensated?) My guess is that google is fully aware that they will lose some people who would have been hires, and that they are ok with this tradeoff (ie., they are willing to tolerate a high incidence of false negatives to avoid the harm of a false positive). I respect that, though it does rankle when they also complain to congress about the shortage of good programmers when this shortage is almost built-in to their hiring process.

RogerL 13y ago
Yes, they are vocal about their false negatives, and they are happy with it. As long as the resumes keep flooding in, and they are hiring who they need, why do they care about rejecting qualified people, or wasting their time? OTOH, I care about them wasting my time. "The problem is, how does the employer know you're one of the people who can actually do this" - I addressed this, I guess unclearly. Ask me. Ask me to describe my algorithms. You'll know very quickly if I am misrepresenting myself or not. You'll also see if I get passionate about my work. If I have 37 ideas that I haven't had a chance to develop yet (I hypothesize that any good brain develops more good ideas than it has time to pursue). Did the code ship? That is not ironclad proof, as I spent 4 years at a company and at the end the project was cancelled because of a political shift in Washington (this was defense work), but it can be a useful measure. I know people that spend forever 'tweaking' algorithms, but never producing anything that generates value for the company. Those people are largely useless unless you have a research branch, but would (and do) pass a Google interview with flying colors. I've been told that they do not care about your resume, and that you will not be questioned on it. OTOH, expect red-black tree questions, scaling questions, and so on. But, they selected me. I have not worked in that area of CS. Ask me what I have been doing for the last 25 years, give me some kind of test to make sure I did what I said I did, and you'll quickly know if I can help you out or not. If I can code probabilistic graphical models and make them work in real products (which is what I'm doing at the moment, more or less), I can re-learn graph theory if I need to. On the other hand, if I can learn book answers to Dijkstra's algorithm and code shared_ptr on a whiteboard with no errors, I've proved very little about my productivity and creativity (a question I was asked to do by a Chicago HF trading company, and 'failed' because I didn't do it the 'boost' way, but spent time talking about all the tradeoffs one could make in such a design and made different choices then boost). And that squares with the conclusion of the article - talk to the person about what they did, how they did it, how they dealt with problems. Trouble is, I see no evidence of that in Google's actual interview process. You better be 20 and fresh out of grad school, or willing to grind through a lot of CS theory that you haven't had to do for a long time.
- geebee 13y ago
  I know this is google, and people there are smart, but there is a chance that you're overestimating the interviewer's ability to know quickly if you have these skills. If they're very unfamiliar with the problem space, they may lack confidence in their ability to interview you. On the other hand, if they know the answers in advance (or at least understand the problem very well), they'll be better able to gauge your performance. So they go with red-black trees. Not your skill or ability, as you pointed out, just your performance on some narrowly chosen questions from an area that the interviewer understands well in advance. I've thought about false negatives a lot, but you've raised a good point about the high risk of false positives in this sort of interview culture as well. I will grant that it isn't easy to study for and pass a data structures grilling that can last all day (or longer), even if you have studied for a month. Anyone who can do this is at least capable of writing and understanding fairly complicated code. But you are absolutely right that this may lead to "false positives" as well... they may simply be hiring people who are great at studying for and passing algorithms quizzes. This certainly doesn't mean they'll be successful. In fact... well, you've essentially pointed this out, but what does it say about a motivated and creative person that he or she will drop everything in order to study for something with only passing relevance to the number of projects he or she would like to be pursuing? Well, desperate, I suppose, could be one of the motivating factors. But how many top programmers are desperate? Many would be interested in working for google, but aren't desperate enough to put aside the interesting things they'd like to be working on in order to spend a month preparing for an exam on red-black trees. Maybe this article suggests google may be coming around to realizing that they are doing some harm to their hiring efforts with this approach to technical interviews?

by geebee 13y ago

RogerL 13y ago

Yes, they are vocal about their false negatives, and they are happy with it. As long as the resumes keep flooding in, and they are hiring who they need, why do they care about rejecting qualified people, or wasting their time? OTOH, I care about them wasting my time. "The problem is, how does the employer know you're one of the people who can actually do this" - I addressed this, I guess unclearly. Ask me. Ask me to describe my algorithms. You'll know very quickly if I am misrepresenting myself or not. You'll also see if I get passionate about my work. If I have 37 ideas that I haven't had a chance to develop yet (I hypothesize that any good brain develops more good ideas than it has time to pursue). Did the code ship? That is not ironclad proof, as I spent 4 years at a company and at the end the project was cancelled because of a political shift in Washington (this was defense work), but it can be a useful measure. I know people that spend forever 'tweaking' algorithms, but never producing anything that generates value for the company. Those people are largely useless unless you have a research branch, but would (and do) pass a Google interview with flying colors. I've been told that they do not care about your resume, and that you will not be questioned on it. OTOH, expect red-black tree questions, scaling questions, and so on. But, they selected me. I have not worked in that area of CS. Ask me what I have been doing for the last 25 years, give me some kind of test to make sure I did what I said I did, and you'll quickly know if I can help you out or not. If I can code probabilistic graphical models and make them work in real products (which is what I'm doing at the moment, more or less), I can re-learn graph theory if I need to. On the other hand, if I can learn book answers to Dijkstra's algorithm and code shared_ptr on a whiteboard with no errors, I've proved very little about my productivity and creativity (a question I was asked to do by a Chicago HF trading company, and 'failed' because I didn't do it the 'boost' way, but spent time talking about all the tradeoffs one could make in such a design and made different choices then boost). And that squares with the conclusion of the article - talk to the person about what they did, how they did it, how they dealt with problems. Trouble is, I see no evidence of that in Google's actual interview process. You better be 20 and fresh out of grad school, or willing to grind through a lot of CS theory that you haven't had to do for a long time.

geebee 13y ago
I know this is google, and people there are smart, but there is a chance that you're overestimating the interviewer's ability to know quickly if you have these skills. If they're very unfamiliar with the problem space, they may lack confidence in their ability to interview you. On the other hand, if they know the answers in advance (or at least understand the problem very well), they'll be better able to gauge your performance. So they go with red-black trees. Not your skill or ability, as you pointed out, just your performance on some narrowly chosen questions from an area that the interviewer understands well in advance. I've thought about false negatives a lot, but you've raised a good point about the high risk of false positives in this sort of interview culture as well. I will grant that it isn't easy to study for and pass a data structures grilling that can last all day (or longer), even if you have studied for a month. Anyone who can do this is at least capable of writing and understanding fairly complicated code. But you are absolutely right that this may lead to "false positives" as well... they may simply be hiring people who are great at studying for and passing algorithms quizzes. This certainly doesn't mean they'll be successful. In fact... well, you've essentially pointed this out, but what does it say about a motivated and creative person that he or she will drop everything in order to study for something with only passing relevance to the number of projects he or she would like to be pursuing? Well, desperate, I suppose, could be one of the motivating factors. But how many top programmers are desperate? Many would be interested in working for google, but aren't desperate enough to put aside the interesting things they'd like to be working on in order to spend a month preparing for an exam on red-black trees. Maybe this article suggests google may be coming around to realizing that they are doing some harm to their hiring efforts with this approach to technical interviews?