7 ms·

At all the companies I’ve run we’ve used a simple whiteboard example for this reason. Don’t worry about typos; how does the person think. “Trick” questions ar

by gumby 2y ago

At all the companies I’ve run we’ve used a simple whiteboard example for this reason. Don’t worry about typos; how does the person think. “Trick” questions are dumb, but you want to get an idea of if the candidate knows their stuff (and let the candidate know what we’re like — interviewing is a sales process in both directions). If someone asks a question like “is it ok to modify the argument” (or says “I’ll assume I can’t modify the argument”) that’s great (or says “I can’t remember if strlen includes the 0 byte so I’ll add 1 — normally I’d look it up”) - again, no trick questions. When I say “simple” it’s something like atoi or “shuffle a deck of cards”: basic, but not 100% trivial like fizzbuzz. On of our best hires was a guy who made a fundamental mistake — when we asked him if it world work as intended he said “I’m a fucking idiot” and fixed it. You’re not supposed to talk like that in an interview I supposed but we read it as a strong positive signal. We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time.

lumb63 2y ago
I have come to like this approach reasonably well. My current company uses a similarly simple problem to evaluate candidates. For me, it was a breeze, and I thought we’d be hiring slouches left and right. Once I started administering the interview, I realized that the majority of candidates absolutely bomb this simple question for one reason or another. It seems that “able to code a simple problem well” is a far less ridiculous proxy for good software engineer than “able to code a hard problem at all”. Much to my surprise.
- Frost1x 2y ago
  “Simple problems” often aren’t as “simple” as the retrospectively sound if you consider everything like stress level, mixed mindset of the interviewee (they’re not just thinking about a programming problem, they’re also thinking about social queues, mixing conversing in, peer judgement, and loads of other anxiety inducing factors), etc. The software industry is notorious for underestimating overall complexity factors, understanding uncertainty, and time to manage them and interviews seem to be no different. So “simple” problems are often good (I agree with you) because they’re actually not so simple for most people when you factor everything in, they’re often reasonably challenging under the set of circumstances. Meanwhile genuinely difficult problems (in any arbitrary setting and longer timelines) require a whole lot of rote training and reusing little clever idiosyncratic techniques that may or may not be generally useful, so you very often end up with a question lottery on whether you already know how to address the problem in question or not correctly vs having some problem where a reasonable solution and expectations around that can be derived in the time (and environment) it’s taking place. Lots of nonsense.
- bena 2y ago
  You test the floor, not the ceiling. Truth is, I'm never looking for "the best", I'm looking for "good enough to do the job competently". Because we can hem and haw over what is best for hours. And we could disagree with what qualities in what proportion make someone "the best". But we can all generally agree with "could do the job". As I've mentioned elsewhere, we're hiring right now and we have the duality of candidates in our pool. There's one guy who is just a hard no from at least two people in the group. And that translates into a no from the third member simply because we're so adamant that he's not the guy. And there's another who we're all feeling really solid on. Not "the best", but solid floor and we feel they'll be able to learn. But being able to weed out people who can't even perform the basics despite having a decent resume is invaluable.
- tnel77 2y ago
  I have a friend who has interviewed about 30-40 software engineering candidates for his company (small, publicly traded company from the midwest). He said that FizzBuzz alone is enough to filter out 25% of applicants. The next slight more challenging question will axe another 30-40% of candidates. I think Leetcode hards exist to stroke the egos of select software engineers.
  - ryandrake 2y ago
    Yea, smaller and medium sized companies just don't have the recruiting infrastructure to weed out the fakers. When I worked in FAANG companies, by the time a candidate got to me, they were pretty good. They went through enough filters and checkpoints and annoying gates that I never really had to do a FizzBuzz equivalent. When I worked for more medium sized companies, I'd get software engineering candidates who obviously did not even remotely know how to code. Imagine the worst possible software candidate, and then lower your expectation even further. One couldn't even tell me what an "int" was. I think a single "for loop on a whiteboard" question would filter out at least 50% of candidates.
    - tnel77 2y ago
      I agree. How are these people graduating? What does that say about our universities?
      - ryandrake 2y ago
        Well, I'm not impartial... I kind of have a chip on my shoulder against these phonies, but if you want me to speculate: they are faking their way through university the same way they fake their way through everything else in their lives: Through things like charm, good looks, Ivy League mannerisms, confidence, a firm handshake, a beautiful smile, that certain way of talking like a salesman. During Character Creation, they put all their skill points into "charisma" and just use that to sail through situations the rest of us have to work through. I think we all encounter these people all the time during our careers, and it's infuriating that what they are doing is reliable and works.
      - 2y ago
dicroce 2y ago
Yeah, that's much better.
giancarlostoro 2y ago
That I wouldnt mind. I didnt do a full on CS degree, that never stopped me from being a good contributor to various teams over 8 years. I have learned the best candidates are the ones who like you say are open about how they messed up, and can get along with your team. It takes one toxic dev to ruin productivity for an entire team.
IshKebab 2y ago
I agree with all of this. Leetcode questions are fine as long as they aren't actually difficult maths puzzles or brain teasers. We also used atoi/itoa loads and it was pretty much the perfect difficulty. Though I still do start with a short fizzbuzz-level question because you would be surprised how many people fail that and it makes it way less awkward if you start with atoi and they can't write a loop. If they fail fizzbuzz I generally just turn the interview into a chat or give them easy questions to practice, rather than say "yeah no thanks", because I feel like the latter is a bit mean, and everyone can benefit from interview practice.
- gumby 2y ago
  > If they fail fizzbuzz I generally just turn the interview into a chat or give them easy questions to practice, rather than say "yeah no thanks", because I feel like the latter is a bit mean, and everyone can benefit from interview practice. It’s kind to politely ramp the interview down in that case but I’ve learned to then cut off any later interviews. It’s a waste of the company’s time and the candidate’s too. If we were going to have lunch the recruiter takes them to lunch anyway, unless the (now former) candidate doesn’t want to.
- throwaway2037 2y ago
  > We also used atoi/itoa Do you mean the C function atoi() to convert a string to number? If yes, I also used the same technical interview question for year. It is a nice "fractally complex" (many levels of depth) question. Everyone can understand it in 10 seconds, but you can really show your experience when you get into the edge cases and unit tests. Plus, if the candidate is a bit junior, you can push their boundaries by asking about different edge case and watch how they react. Similarly: I had a teammate for a few years that asked people to program a very simple linked list class. He was shocked how many mid-level devs couldn't do it. As I recall, he never sent me a bad candidate, so it was good enough for us.
Aurornis 2y ago
> We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time. Trying to debate the interview format or reject interview questions is one of my hard-stop rejection triggers in interviews now. The couple times I’ve been part of hiring pipelines where candidates argued about the interview itself and then got hired anyway, they became extremely difficult employees within the company. Arguing about interview questions turned into arguing about every other ticket, code reviews, and every architecture choice. I’ve had the same experience where we take an actual problem we solved in our codebase and turned it into a small interview problem. Same thing: Some candidates will try to debate the relevance or try to wriggle out of it.
- red-iron-pine 2y ago
  there is a lot of stupid bullshit in most orgs, and things only get fixed by often irritable, unreasonable people. "shut up and do it" was only an acceptable answer when I was in the military. if the interviewee pushes back and find they don't like the answer, then it's a good signal to both parties.
  - ryandrake 2y ago
    This is definitely something that is dependent on the company culture and the personal preferences of the team. I would never want to work with someone who is constantly argumentative over everything, just because he thinks he's pushing back on bullshit. Sometimes you just have to "document your objection, but do it anyway."
  - jasonm23 2y ago
    If that's what you want it's a good signal, many companies also already have their quota of irritable, unreasonable people. Sometimes we want harmony and not drama. Knowing how to push forward without being abrasive is a more valuable trait.
EnergyAmy 2y ago
Offtopic, but there's an interesting parallel between making a mistake like that and AI hallucination. In a human, that behavior is a positive signal, but for many people, that same behavior is proof of how LLMs are just a toy that will never rival human intelligence.
- soerxpso 2y ago
  The difference is that an LLM isn't very good at saying "I'm a fucking idiot" and changing it when asked to double-check (unless you handhold it in the direction of the exact error it's meant to be looking for). Humans recognize their own hallucinations. There's not really any promising work towards getting AI to do the same.
  - gumby 2y ago
    Leela.ai was founded to work on problems like that.
  - EnergyAmy 2y ago
    Have you tried it? They're actually pretty good at it in a lot of scenarios. It's not flawless, but they're only getting better.
krackers 2y ago
>shuffle a deck of cards Fwiw this is surprisingly subtle and non-trivial to get right. A naive "random swap" approach does not produce uniform permutations. Sorting based on a random comparison function can likewise produce biased results depending on the implementation of the sort. Tagging each element with a random number and then sorting based on the tag works (or so I think?) but you need to deal with the case where tags collide. Fisher-yates shuffle is bulletproof but I would not trust myself to write it from memory lest you end up writing Sattolo's algorithm instead.
- gumby 2y ago
  It’s an open ended problem and someone could nerd out on the maths while another could define a card class, a suit class, and use a factory pattern (oh brother…SMH but probably wouldn’t categorically rule someone out for it, just talk some more). At the end of the day the question is: “could this person do the job and could they work with others in the process?”
- chuckadams 2y ago
  The problem isn’t about getting the perfect answer, it’s about observing the process, and conversing about the answer afterwards. I’d be fine hiring the candidate who whiffs the shuffle but knows about Fisher-Yeats and says they’d look for a well-tested implementation and drop it in. I mean that can’t be their answer to everything, but evidence of real experience should grant a Mulligan now and then. Coding interviews cannot be code themselves.

by gumby 2y ago

lumb63 2y ago

I have come to like this approach reasonably well. My current company uses a similarly simple problem to evaluate candidates. For me, it was a breeze, and I thought we’d be hiring slouches left and right. Once I started administering the interview, I realized that the majority of candidates absolutely bomb this simple question for one reason or another. It seems that “able to code a simple problem well” is a far less ridiculous proxy for good software engineer than “able to code a hard problem at all”. Much to my surprise.

Frost1x 2y ago
“Simple problems” often aren’t as “simple” as the retrospectively sound if you consider everything like stress level, mixed mindset of the interviewee (they’re not just thinking about a programming problem, they’re also thinking about social queues, mixing conversing in, peer judgement, and loads of other anxiety inducing factors), etc. The software industry is notorious for underestimating overall complexity factors, understanding uncertainty, and time to manage them and interviews seem to be no different. So “simple” problems are often good (I agree with you) because they’re actually not so simple for most people when you factor everything in, they’re often reasonably challenging under the set of circumstances. Meanwhile genuinely difficult problems (in any arbitrary setting and longer timelines) require a whole lot of rote training and reusing little clever idiosyncratic techniques that may or may not be generally useful, so you very often end up with a question lottery on whether you already know how to address the problem in question or not correctly vs having some problem where a reasonable solution and expectations around that can be derived in the time (and environment) it’s taking place. Lots of nonsense.
bena 2y ago
You test the floor, not the ceiling. Truth is, I'm never looking for "the best", I'm looking for "good enough to do the job competently". Because we can hem and haw over what is best for hours. And we could disagree with what qualities in what proportion make someone "the best". But we can all generally agree with "could do the job". As I've mentioned elsewhere, we're hiring right now and we have the duality of candidates in our pool. There's one guy who is just a hard no from at least two people in the group. And that translates into a no from the third member simply because we're so adamant that he's not the guy. And there's another who we're all feeling really solid on. Not "the best", but solid floor and we feel they'll be able to learn. But being able to weed out people who can't even perform the basics despite having a decent resume is invaluable.
tnel77 2y ago
I have a friend who has interviewed about 30-40 software engineering candidates for his company (small, publicly traded company from the midwest). He said that FizzBuzz alone is enough to filter out 25% of applicants. The next slight more challenging question will axe another 30-40% of candidates. I think Leetcode hards exist to stroke the egos of select software engineers.
- ryandrake 2y ago
  Yea, smaller and medium sized companies just don't have the recruiting infrastructure to weed out the fakers. When I worked in FAANG companies, by the time a candidate got to me, they were pretty good. They went through enough filters and checkpoints and annoying gates that I never really had to do a FizzBuzz equivalent. When I worked for more medium sized companies, I'd get software engineering candidates who obviously did not even remotely know how to code. Imagine the worst possible software candidate, and then lower your expectation even further. One couldn't even tell me what an "int" was. I think a single "for loop on a whiteboard" question would filter out at least 50% of candidates.
  - tnel77 2y ago
    I agree. How are these people graduating? What does that say about our universities?
    - ryandrake 2y ago
      Well, I'm not impartial... I kind of have a chip on my shoulder against these phonies, but if you want me to speculate: they are faking their way through university the same way they fake their way through everything else in their lives: Through things like charm, good looks, Ivy League mannerisms, confidence, a firm handshake, a beautiful smile, that certain way of talking like a salesman. During Character Creation, they put all their skill points into "charisma" and just use that to sail through situations the rest of us have to work through. I think we all encounter these people all the time during our careers, and it's infuriating that what they are doing is reliable and works.
    - 2y ago

dicroce 2y ago

Yeah, that's much better.

giancarlostoro 2y ago

That I wouldnt mind. I didnt do a full on CS degree, that never stopped me from being a good contributor to various teams over 8 years. I have learned the best candidates are the ones who like you say are open about how they messed up, and can get along with your team. It takes one toxic dev to ruin productivity for an entire team.

IshKebab 2y ago

I agree with all of this. Leetcode questions are fine as long as they aren't actually difficult maths puzzles or brain teasers. We also used atoi/itoa loads and it was pretty much the perfect difficulty. Though I still do start with a short fizzbuzz-level question because you would be surprised how many people fail that and it makes it way less awkward if you start with atoi and they can't write a loop. If they fail fizzbuzz I generally just turn the interview into a chat or give them easy questions to practice, rather than say "yeah no thanks", because I feel like the latter is a bit mean, and everyone can benefit from interview practice.

gumby 2y ago
> If they fail fizzbuzz I generally just turn the interview into a chat or give them easy questions to practice, rather than say "yeah no thanks", because I feel like the latter is a bit mean, and everyone can benefit from interview practice. It’s kind to politely ramp the interview down in that case but I’ve learned to then cut off any later interviews. It’s a waste of the company’s time and the candidate’s too. If we were going to have lunch the recruiter takes them to lunch anyway, unless the (now former) candidate doesn’t want to.
throwaway2037 2y ago
> We also used atoi/itoa Do you mean the C function atoi() to convert a string to number? If yes, I also used the same technical interview question for year. It is a nice "fractally complex" (many levels of depth) question. Everyone can understand it in 10 seconds, but you can really show your experience when you get into the edge cases and unit tests. Plus, if the candidate is a bit junior, you can push their boundaries by asking about different edge case and watch how they react. Similarly: I had a teammate for a few years that asked people to program a very simple linked list class. He was shocked how many mid-level devs couldn't do it. As I recall, he never sent me a bad candidate, so it was good enough for us.

Aurornis 2y ago

> We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time. Trying to debate the interview format or reject interview questions is one of my hard-stop rejection triggers in interviews now. The couple times I’ve been part of hiring pipelines where candidates argued about the interview itself and then got hired anyway, they became extremely difficult employees within the company. Arguing about interview questions turned into arguing about every other ticket, code reviews, and every architecture choice. I’ve had the same experience where we take an actual problem we solved in our codebase and turned it into a small interview problem. Same thing: Some candidates will try to debate the relevance or try to wriggle out of it.

red-iron-pine 2y ago
there is a lot of stupid bullshit in most orgs, and things only get fixed by often irritable, unreasonable people. "shut up and do it" was only an acceptable answer when I was in the military. if the interviewee pushes back and find they don't like the answer, then it's a good signal to both parties.
- ryandrake 2y ago
  This is definitely something that is dependent on the company culture and the personal preferences of the team. I would never want to work with someone who is constantly argumentative over everything, just because he thinks he's pushing back on bullshit. Sometimes you just have to "document your objection, but do it anyway."
- jasonm23 2y ago
  If that's what you want it's a good signal, many companies also already have their quota of irritable, unreasonable people. Sometimes we want harmony and not drama. Knowing how to push forward without being abrasive is a more valuable trait.

EnergyAmy 2y ago

Offtopic, but there's an interesting parallel between making a mistake like that and AI hallucination. In a human, that behavior is a positive signal, but for many people, that same behavior is proof of how LLMs are just a toy that will never rival human intelligence.

soerxpso 2y ago
The difference is that an LLM isn't very good at saying "I'm a fucking idiot" and changing it when asked to double-check (unless you handhold it in the direction of the exact error it's meant to be looking for). Humans recognize their own hallucinations. There's not really any promising work towards getting AI to do the same.
- gumby 2y ago
  Leela.ai was founded to work on problems like that.
- EnergyAmy 2y ago
  Have you tried it? They're actually pretty good at it in a lot of scenarios. It's not flawless, but they're only getting better.

krackers 2y ago

>shuffle a deck of cards Fwiw this is surprisingly subtle and non-trivial to get right. A naive "random swap" approach does not produce uniform permutations. Sorting based on a random comparison function can likewise produce biased results depending on the implementation of the sort. Tagging each element with a random number and then sorting based on the tag works (or so I think?) but you need to deal with the case where tags collide. Fisher-yates shuffle is bulletproof but I would not trust myself to write it from memory lest you end up writing Sattolo's algorithm instead.

gumby 2y ago
It’s an open ended problem and someone could nerd out on the maths while another could define a card class, a suit class, and use a factory pattern (oh brother…SMH but probably wouldn’t categorically rule someone out for it, just talk some more). At the end of the day the question is: “could this person do the job and could they work with others in the process?”
chuckadams 2y ago
The problem isn’t about getting the perfect answer, it’s about observing the process, and conversing about the answer afterwards. I’d be fine hiring the candidate who whiffs the shuffle but knows about Fisher-Yeats and says they’d look for a well-tested implementation and drop it in. I mean that can’t be their answer to everything, but evidence of real experience should grant a Mulligan now and then. Coding interviews cannot be code themselves.