6 ms·

I interview quite a lot, and one of the things I try to do in an interview is work through a problem that I almost certainly understand more deeply than any of

I interview quite a lot, and one of the things I try to do in an interview is work through a problem that I almost certainly understand more deeply than any of the candidates (both because it's my niche and because I've worked through the exact problem with hundreds of candidates before). There's a certain minimum bar that I expect any educated and intelligent individual to achieve if they're a good match for the job, but then I take it really far beyond what I actually expect them to know, to the point where a lot of the details are unknown, and the details matter a lot in determining what kinds of solutions will be successful. Do they start BS'ing me? Do they confidently walk into really nasty parts of the problem without recognizing the pitfalls? Or do they tell me they're a little stuck? Do they ask which trade-offs to make? Do they ask if they can do some Googling for prior research on the problem? Do they explain what they would try and what pitfalls they would watch our for, or how they would validate their idea? Do they ask if they can make some simplifying assumptions for practical purposes? There's a lot of right and wrong answers but really I want to see their attitude and communication. My first couple of hires before I did this were a brilliant computer scientist and a guy with a business degree and some coding background. I was surprised to see the guy with the business degree excel because we weren't doing many CS-hard problems, and an ability to manage time, talk about the problem, and navigate the project life cycle were far more important than classic algorithm knowledge. He needed to know how to code and debug, for sure, but like most software jobs, his soft skills were a force multiplier. I'm pretty sure this has been done with me - I was given a problem and told to come up with something like an O(n log n) solution, and I honestly couldn't do it. I tried a few different approaches but I could think of really bad pathological cases for all of them. The interviewer gave me a few suggestions which I explored and I kept exploring them and then admitted I didn't see a way forward. I thought I was bombing and then he told me I did really well. I looked up the problem and found out he had done Ph.D. research in that area, and there was no general O(n log n) solution - only solutions that could do that in most practical cases.

moosey 8y ago
I would think that anyone hiring a Ph.D. in Computer Science would be looking for a different kind of output than the knowledge of actually doing software development and the associated project management/time management work. I would imagine that such people, if placed optimally, would be doing largely self-directed work with an expectation of solving a very complicated problem, with work output that looked more like a thesis than a program. I further surmise that very few companies need these kinds of people. If your needs are people that can be managed to do basic programming tasks (my job as well), then that's what you hire. That's something like 99% of all programming work. It's like hiring someone that understands the building of nuclear power plants, and being surprised that they don't understand basic home building codes. The saddest part of it, for me, though, is that someone with a Ph.D. in Computer Science that is searching for a job in the non-research sector is probably having a really bad time. Research is happening, but I don't think it will jump in a very large way until there are a lot more dollars being pumped into research using public money.
- TallGuyShort 8y ago
  Just to be clear, I neither have Ph.D. in CS myself nor am I looking for that in candidates. I agree with everything you say.
jrochkind1 8y ago
I think that would be kinder to the applicants if you told them that there were going to be things you didn't expect them to know required for a good solution and you didn't necessarily expect a good solution but were interested in how they approached it. You'd still be able to tell nearly as much by what questions they asked, and it would be less testing for "How well do they do under unrealistic/unrepresentatives kinds of pressure and in guessing what the boss actually wants when he isn't being honest about it." One of the most unrealistic parts about a code interview is just how high-pressure it is, time sensitive in unrealistic ways, way more impact on your career than usual 5 minutes of coding, no peers to ask questions to, unclear if asking to google will be seen as a negative or not, etc. I think trying to reduce some of these unrealistic pressures, by being clear about what you're expecting (and being in this case up front that you _have_ intentionally given them something out of their domain knowledge) can probably give you a better evaluations. Some people do better under that kind of pressure than others, but if that kind of pressure is not actually representative of the actual job, it may not be what you want to be testing for. Also set expectations for transparency and kindness (instead of bullshitting or preening) by modelling it yourself, right from the start.
- TallGuyShort 8y ago
  I generally do tell them it's a complex problem and I'm looking more for how they approach it than any specific solution. I don't tell them everything I'm looking for because then they'll just pretend to be the way I want them to be. But I disagree that it's unrealistic. I've had far too many employees and coworkers who don't understand what is being asked of them or don't know what they're doing, and try to fake it as long as they can. And if I tell them they need to tell me if there's a problem, they almost always haven't until the pattern continues and becomes a worse problem. If the way they deal with pressure in an interview is to bullshit me, I need to know that long before they're under pressure at work.
  - therealx 8y ago
    I like this idea because it allows me to figure out if they have one of the most important skills: being able to say "I don't know" without wasting a lot of time. IMO anyone else saying you should tell them the solutions might not have answers are dancing around needing to have that skill.
jimbokun 8y ago
" I looked up the problem and found out he had done Ph.D. research in that area, and there was no general O(n log n) solution - only solutions that could do that in most practical cases." The Kobayashi Maru of programming interview questions!
kristopolous 8y ago
If I smell any bs then I immediately change tactics and make sure I put them deep over their head. For example, "what are the strengths and weaknesses of classifying grammars via the Chomsky hierarchy?" There's 3 ways to answer these kinds of questions and it exposes you as either a genius (or in this case a linguist), a non-genius (real geniuses are as common as 7.5 ft tall people) or a bullshitter trying to look like a genius.* I've never met an actual genius in an interview so let me address the other two. That third group is a nonstarter. I've known the person for 10 minutes and they're already trying to deceive me. How the hell am I going to build a product with them? Kim Scott, some SV big wig wrote an entire book on this called Radical Candor. Essentially it's "don't bullshit people (and know where your taking the relationship when you use that rule)" I strongly believe companies work best with the smallest bullshit quotient possible. They may sell more or be more profitable with certain flavors of bullshit, but things are easier and more stuff gets done without it. At the end I would have rather actually made something than duped people with sham products sold on dreams --- * I actually can't answer that question but I could probably identify the neighborhood of a real answer (...something something limitations of the von nuemann model restricting the constructions of parsing systems something something...). The point isn't to answer it, although that would be quite impressive; it's to see how far someone digs into their own BS.
- westoncb 8y ago
  I'm curious about the question here if anyone wants to chime in (this is just an area in CS that I'm interested in but don't have exhaustive knowledge of or anything). > what are the strengths and weaknesses of classifying grammars via the Chomsky hierarchy? Are there alternative classification schemes for grammars? I haven't come across any personally—not for formal languages anyway. Or was something else meant by the question?
- mjcohen 8y ago
  For Chomsky, I would hazard finite state automata, BNF systems, and Turing machines. I might then mumble about number of tapes and time to recognize, and then stop.
  - deleted 8y ago

moosey 8y ago

I would think that anyone hiring a Ph.D. in Computer Science would be looking for a different kind of output than the knowledge of actually doing software development and the associated project management/time management work. I would imagine that such people, if placed optimally, would be doing largely self-directed work with an expectation of solving a very complicated problem, with work output that looked more like a thesis than a program. I further surmise that very few companies need these kinds of people. If your needs are people that can be managed to do basic programming tasks (my job as well), then that's what you hire. That's something like 99% of all programming work. It's like hiring someone that understands the building of nuclear power plants, and being surprised that they don't understand basic home building codes. The saddest part of it, for me, though, is that someone with a Ph.D. in Computer Science that is searching for a job in the non-research sector is probably having a really bad time. Research is happening, but I don't think it will jump in a very large way until there are a lot more dollars being pumped into research using public money.

TallGuyShort 8y ago
Just to be clear, I neither have Ph.D. in CS myself nor am I looking for that in candidates. I agree with everything you say.

jrochkind1 8y ago

I think that would be kinder to the applicants if you told them that there were going to be things you didn't expect them to know required for a good solution and you didn't necessarily expect a good solution but were interested in how they approached it. You'd still be able to tell nearly as much by what questions they asked, and it would be less testing for "How well do they do under unrealistic/unrepresentatives kinds of pressure and in guessing what the boss actually wants when he isn't being honest about it." One of the most unrealistic parts about a code interview is just how high-pressure it is, time sensitive in unrealistic ways, way more impact on your career than usual 5 minutes of coding, no peers to ask questions to, unclear if asking to google will be seen as a negative or not, etc. I think trying to reduce some of these unrealistic pressures, by being clear about what you're expecting (and being in this case up front that you _have_ intentionally given them something out of their domain knowledge) can probably give you a better evaluations. Some people do better under that kind of pressure than others, but if that kind of pressure is not actually representative of the actual job, it may not be what you want to be testing for. Also set expectations for transparency and kindness (instead of bullshitting or preening) by modelling it yourself, right from the start.

TallGuyShort 8y ago
I generally do tell them it's a complex problem and I'm looking more for how they approach it than any specific solution. I don't tell them everything I'm looking for because then they'll just pretend to be the way I want them to be. But I disagree that it's unrealistic. I've had far too many employees and coworkers who don't understand what is being asked of them or don't know what they're doing, and try to fake it as long as they can. And if I tell them they need to tell me if there's a problem, they almost always haven't until the pattern continues and becomes a worse problem. If the way they deal with pressure in an interview is to bullshit me, I need to know that long before they're under pressure at work.
- therealx 8y ago
  I like this idea because it allows me to figure out if they have one of the most important skills: being able to say "I don't know" without wasting a lot of time. IMO anyone else saying you should tell them the solutions might not have answers are dancing around needing to have that skill.

jimbokun 8y ago

" I looked up the problem and found out he had done Ph.D. research in that area, and there was no general O(n log n) solution - only solutions that could do that in most practical cases." The Kobayashi Maru of programming interview questions!

kristopolous 8y ago

If I smell any bs then I immediately change tactics and make sure I put them deep over their head. For example, "what are the strengths and weaknesses of classifying grammars via the Chomsky hierarchy?" There's 3 ways to answer these kinds of questions and it exposes you as either a genius (or in this case a linguist), a non-genius (real geniuses are as common as 7.5 ft tall people) or a bullshitter trying to look like a genius.* I've never met an actual genius in an interview so let me address the other two. That third group is a nonstarter. I've known the person for 10 minutes and they're already trying to deceive me. How the hell am I going to build a product with them? Kim Scott, some SV big wig wrote an entire book on this called Radical Candor. Essentially it's "don't bullshit people (and know where your taking the relationship when you use that rule)" I strongly believe companies work best with the smallest bullshit quotient possible. They may sell more or be more profitable with certain flavors of bullshit, but things are easier and more stuff gets done without it. At the end I would have rather actually made something than duped people with sham products sold on dreams --- * I actually can't answer that question but I could probably identify the neighborhood of a real answer (...something something limitations of the von nuemann model restricting the constructions of parsing systems something something...). The point isn't to answer it, although that would be quite impressive; it's to see how far someone digs into their own BS.

westoncb 8y ago
I'm curious about the question here if anyone wants to chime in (this is just an area in CS that I'm interested in but don't have exhaustive knowledge of or anything). > what are the strengths and weaknesses of classifying grammars via the Chomsky hierarchy? Are there alternative classification schemes for grammars? I haven't come across any personally—not for formal languages anyway. Or was something else meant by the question?
mjcohen 8y ago
For Chomsky, I would hazard finite state automata, BNF systems, and Turing machines. I might then mumble about number of tapes and time to recognize, and then stop.
- deleted 8y ago