7 ms·

In my startup I interviewed a 48 years old senior Java programmer with excellent resume, who took 1hr to write a String.contains(), it only worked for the reque

In my startup I interviewed a 48 years old senior Java programmer with excellent resume, who took 1hr to write a String.contains(), it only worked for the requested 4 letters, didn’t work if a letter was repeated twice, and didn’t work with Chinese characters. At least it had the JUnit. I asked an employee to do it too and he made his code pass the JUnit in 6 minutes. The candidate hated the interview, claiming it was discouraging. Coding is erratic, talent is strange, it really is a craft and we still don’t know how to reliably raise someone to competency.

mattkrause 5y ago
But which is more likely? 1. The candidate was a complete and utter fraud and their previous (and apparently well-regarded) employers were too stupid or negligent to notice this, wasting literally millions of dollars (48-21 * $100,000+). 2. Something about the interview failed to let this person demonstrate the skills that had kept them employed for two decades. Maybe their mind went blank under pressure, or at the end of a long day. Maybe they got hung up on something trivial (that a quick search—-or nudge from the interviewer—-would have resolved), or the question was unclear.
- lumost 5y ago
  To add to this, I’ve found engineers more likely to hang on time series and string manipulation problems. Likely due to a combination of not having to code low level functions in these areas, as well as infrequently encountering the problem.
  - skeeter2020 5y ago
    Yes, strings are hard and times/dates have a ridiculous number of edge cases, and sometimes very poor language support. This works both ways though; if the problem is easy enough (calculate average cycle time) it can give you lots of edge cases to discuss and really show how someone problem solves, which is really the point of a programming interview. If someone even mentioned non-english language support that would be enough for me, forget about implementing it.
    - lumost 5y ago
      I personally dislike giving questions with too many rabbit holes. My observation on a few questions is that it’s a 50/50 shot if the candidate who freezes on a question recognized more nuances than the candidate who didn’t which means I’m not getting any data. Fizz buzz was a great question in that it had pretty much a Boolean success criteria.
      - mattkrause 5y ago
        The interviewer needs to be good/prepared to make it work. If the interviewer only says "Write String.contains that passes these test cases" goes back to playing with their phone, several things may happen. One person will take that absolutely literally ("It's a test; better do as I'm told"), and you'll dismiss that apparent garbage or move onto the "real" assessment where they're hoping to shine. Another will get bogged down in something the interviewer regards as a distraction, and "waste" a bunch of time on something the interviewer regards as a distraction. "He handled unicode, but not substring matching (KMP or Booyer-Moore) or vice versa." Maybe someone will goldilocks it and hit the right (not-explicitly-specified) balance of (also unspecified) features and time, but... If you structure it as "Please, do the dumbest possible thing and we'll iterate"--and don't hold that initial pass against them--I could see it working well.
  - BizarroLand 5y ago
    Sounds like the solution then would be to give interviewers all of the systems and support they would normally have access to on the job and see how well they adapt to a conventional task or an issue that was recently solved by someone in a similar position on your company, and have their result evaluated by someone involved with the implementation or fix. That would tell you if their workflow would fit your company much more than knowing how to run a coding challenge would.
    - lumost 5y ago
      On the other hand, tools tend to be fast to teach and pick up relative to fundamentals. Most companies have rough around the edges tooling that a candidate either wouldn’t know about or need a couple weeks to get productive in.
- nuclearnice1 5y ago
  I would say 1 is possible enough that it’s worth checking for. Remember that insanely too hard programming detail is a reaction against a situation where 99% of candidates couldn’t program at all.[1] [1] https://wiki.c2.com/?FizzBuzzTest https://wiki.c2.com/?FizzBuzzTest
  - padthai 5y ago
    I interview for my company. 80% of the DS applicants (some of them with SWE background) that apply for our senior positions fail with FizzBuzz or some riddle of similar difficulty. This is already pre-filtering for seniors from established companies. We do not pay bad for the market. They also do equally bad with other FizzBuzz-level tests in other areas that they claim to have worked in. It is still a very useful test.
    - alisonatwork 5y ago
      This is exactly my experience too. Sometimes it's incredible just how little applicants understand about how to develop software. I've even interviewed people where they were allowed to have a web browser and IDE while coding a solution, and they still struggled. Personally I am a much bigger fan of using FizzBuzz as a gate than an algorithm question. I think algorithm questions optimize for the kind of developer who doesn't mind memorizing algorithms to get a job, which might be a useful skill, but you can test that same skill of memorization using FizzBuzz, and then you don't end up also filtering out people who can code but don't care about memorizing algorithms. In any case, I always think it's worth using their solution as a jumping-off point to ask other, more language-specific questions. Things like: how would you change this if it was intended for use in a FizzBuzz library, how would you annotate this if you needed it to be injected as a Spring dependency, why did you use a for loop instead of a Java 8 stream (or vice versa), what are the implications of declaring this thing as final or static, can you write a unit test for this, and so on. That's when you can get past the point of memorization into figuring out if they actually understand what they typed, which is helpful to ascertain their level.
      - geebee 5y ago
- BurningFrog 5y ago
  Unfortunately, both are fairly likely.
- stupendoustrace 5y ago
  I had number two happen on an interview recently and I am incredibly happy the interviewer didn't hold it against me. I forgot an otherwise simple word/term, but the pressure of the interview just made my mind go completely blank. I think everyone has a tendency sometimes to forget what it's like to be on the other side, and will hammer on small mistakes, or not consider all the factors.
  - mattkrause 5y ago
    Me too! I'm sure I'm on somone's list of incompetent bozos for an interview that went like this: "Please, describe Python programming language." That's it. I had no idea what I was supposed to be doing, and the two fellows interviewing me would not elaborate. I talked about what I had done with Python. Stony stares. Do I talked about the nature of Python itself (interpreted, multi-paradigm, lexical/LEGB scope, the GIL). Stony stares. I wrote some trivial programs on the board. Stony stares. Had I brought an actual snake, I might have tried to charm it. At the end, the CEO told me they weren't overwhelmingly sold on me, but would think about it. Never heard from them again.
- jnwatson 5y ago
  The second is certainly more likely, but I'd wager the likelihood of the first is greater than 10%. I've encountered my share. There are so many "developers" just faking it, I can certainly understand using a test that would reject 90% of the good candidates if it could reject 99% of the bad ones.
  - deleted 5y ago
    [deleted]
- handrous 5y ago
  I've repeatedly had employers very happy with my abilities & results, and am also entirely sure I've, on a few occasions, convinced interviewers I'm entirely unable to write code and am one of these frauds everyone's sure exist and that they need these coding tests to "catch".
- laurent92 5y ago
  I’m the interviewer, I’m still wondering what happened. This was the introductory question before launching 200 threads and asking him to solve the deadlocks/inefficiencies, which was the real question supposed to let him show off his skills in front of my employees, specifically crafted for him because I wanted to persuade my employees he was an excellent hire. So he had a taylored chance to show off his skills but failed at the introductory question. But on the other hand, how can you be asked “Here’s a substring, return true or false if it contains the substring, this is the introduction of 5 questions so don’t sweat it” and not just write two nested loops and an if? I’d pass on UTF8 problems, but when you’ve been working with Java for CRUD apps, you still should have your UTF8 correct. This is how you end up with passwords that must be ASCII because the programmer is bad.
  - mattkrause 5y ago
    I've seen an actual Nobel Prize winner get stuck describing their research. People's brains just occasionally lock up.
BossingAround 5y ago
What was the goal of the question? Why did you want the person to implement a contains method? Did you really want to verify they understood String implementation in Java? And if the candidate was able to do this in 6 minutes, what would you have thought? "Great, let's hire"? In my humble opinion, the question is a waste of time either way. You'll get much further trying to probe what the candidate does know rather than randomly creating an exercise that you think they "should be able to do if woken up in the middle of the night". People forget how stressful interviews are and how easy it is to assume shared context. The fact that your employee was able to do the test might be indicative of the fact that you share context with the employee that you did not share with the candidate, thus confirming your bias.
- Lio 5y ago
  > The fact that your employee was able to do the test might be indicative of the fact that you share context with the employee that you did not share with the candidate, thus confirming your bias. This! When giving interviews last I really worried if the questions I asked where just indicative of my own Dunning-Kruger effect. i.e. Do I only ask questions I already know the answer to and not questions I don't know the answer to? If I do then am I just filtering for people with the same background and knowledge and missing out on people with other skills I don't know, because they're in my blind spot, I need yet?
  - magnetic 5y ago
    > i.e. Do I only ask questions I already know the answer to and not questions I don't know the answer to? I've been advocating for a "coding interview" where both the interviewer _and_ interviewee draw some random question from leetcode or other problem bank, and try to work at it together. This would show collaboration skills, and you can tell pretty easily how helpful the candidate is with his/her contributions, and whether you find there is an impedance mismatch somewhere. It probably also maps more closely to the kinds of interactions you'd have after the person's been hired. I think it would also help calibrate: if you can't figure it out, is it fair to expect the candidate to figure it out? Maybe it's just a hard problem!
- 5y ago
eloisius 5y ago
Did they have to write it in a google doc with no ability to compile and run their code?
bigbillheck 5y ago
I don't do interviews in my current job (mostly because the pandemic really did a number on hiring) but in my previous job the only coding question I asked was what I thought was a fairly simple string problem. They could assume ascii, use any language they wanted, and make any other assumptions to simplify the problem. My then-co-workers liked to ask harder algorithmic questions but I wanted to give the candidates a little bit of a break with something easier. It didn't always work, but at least I tried.
notJim 5y ago
Curious about the age of the person who supposedly wrote it in 6 minutes. If you're fresh out of school, "String.contains" may be top of mind, but the vast majority of people never write that function in practice, so it's easy to not think about.
wheelinsupial 5y ago
What was the problem statement you gave to the candidate? What were you trying to tease out from the problem statement?
walshemj 5y ago
Why would you not use the language built-ins? I am not a Java programmer but it took me 30 seconds to find the contains() method.

mattkrause 5y ago

But which is more likely? 1. The candidate was a complete and utter fraud and their previous (and apparently well-regarded) employers were too stupid or negligent to notice this, wasting literally millions of dollars (48-21 * $100,000+). 2. Something about the interview failed to let this person demonstrate the skills that had kept them employed for two decades. Maybe their mind went blank under pressure, or at the end of a long day. Maybe they got hung up on something trivial (that a quick search—-or nudge from the interviewer—-would have resolved), or the question was unclear.

lumost 5y ago
To add to this, I’ve found engineers more likely to hang on time series and string manipulation problems. Likely due to a combination of not having to code low level functions in these areas, as well as infrequently encountering the problem.
- skeeter2020 5y ago
  Yes, strings are hard and times/dates have a ridiculous number of edge cases, and sometimes very poor language support. This works both ways though; if the problem is easy enough (calculate average cycle time) it can give you lots of edge cases to discuss and really show how someone problem solves, which is really the point of a programming interview. If someone even mentioned non-english language support that would be enough for me, forget about implementing it.
  - lumost 5y ago
    I personally dislike giving questions with too many rabbit holes. My observation on a few questions is that it’s a 50/50 shot if the candidate who freezes on a question recognized more nuances than the candidate who didn’t which means I’m not getting any data. Fizz buzz was a great question in that it had pretty much a Boolean success criteria.
    - mattkrause 5y ago
      The interviewer needs to be good/prepared to make it work. If the interviewer only says "Write String.contains that passes these test cases" goes back to playing with their phone, several things may happen. One person will take that absolutely literally ("It's a test; better do as I'm told"), and you'll dismiss that apparent garbage or move onto the "real" assessment where they're hoping to shine. Another will get bogged down in something the interviewer regards as a distraction, and "waste" a bunch of time on something the interviewer regards as a distraction. "He handled unicode, but not substring matching (KMP or Booyer-Moore) or vice versa." Maybe someone will goldilocks it and hit the right (not-explicitly-specified) balance of (also unspecified) features and time, but... If you structure it as "Please, do the dumbest possible thing and we'll iterate"--and don't hold that initial pass against them--I could see it working well.
- BizarroLand 5y ago
  Sounds like the solution then would be to give interviewers all of the systems and support they would normally have access to on the job and see how well they adapt to a conventional task or an issue that was recently solved by someone in a similar position on your company, and have their result evaluated by someone involved with the implementation or fix. That would tell you if their workflow would fit your company much more than knowing how to run a coding challenge would.
  - lumost 5y ago
    On the other hand, tools tend to be fast to teach and pick up relative to fundamentals. Most companies have rough around the edges tooling that a candidate either wouldn’t know about or need a couple weeks to get productive in.
nuclearnice1 5y ago
I would say 1 is possible enough that it’s worth checking for. Remember that insanely too hard programming detail is a reaction against a situation where 99% of candidates couldn’t program at all.[1] [1] https://wiki.c2.com/?FizzBuzzTest https://wiki.c2.com/?FizzBuzzTest
- padthai 5y ago
  I interview for my company. 80% of the DS applicants (some of them with SWE background) that apply for our senior positions fail with FizzBuzz or some riddle of similar difficulty. This is already pre-filtering for seniors from established companies. We do not pay bad for the market. They also do equally bad with other FizzBuzz-level tests in other areas that they claim to have worked in. It is still a very useful test.
  - alisonatwork 5y ago
    This is exactly my experience too. Sometimes it's incredible just how little applicants understand about how to develop software. I've even interviewed people where they were allowed to have a web browser and IDE while coding a solution, and they still struggled. Personally I am a much bigger fan of using FizzBuzz as a gate than an algorithm question. I think algorithm questions optimize for the kind of developer who doesn't mind memorizing algorithms to get a job, which might be a useful skill, but you can test that same skill of memorization using FizzBuzz, and then you don't end up also filtering out people who can code but don't care about memorizing algorithms. In any case, I always think it's worth using their solution as a jumping-off point to ask other, more language-specific questions. Things like: how would you change this if it was intended for use in a FizzBuzz library, how would you annotate this if you needed it to be injected as a Spring dependency, why did you use a for loop instead of a Java 8 stream (or vice versa), what are the implications of declaring this thing as final or static, can you write a unit test for this, and so on. That's when you can get past the point of memorization into figuring out if they actually understand what they typed, which is helpful to ascertain their level.
    - geebee 5y ago
BurningFrog 5y ago
Unfortunately, both are fairly likely.
stupendoustrace 5y ago
I had number two happen on an interview recently and I am incredibly happy the interviewer didn't hold it against me. I forgot an otherwise simple word/term, but the pressure of the interview just made my mind go completely blank. I think everyone has a tendency sometimes to forget what it's like to be on the other side, and will hammer on small mistakes, or not consider all the factors.
- mattkrause 5y ago
  Me too! I'm sure I'm on somone's list of incompetent bozos for an interview that went like this: "Please, describe Python programming language." That's it. I had no idea what I was supposed to be doing, and the two fellows interviewing me would not elaborate. I talked about what I had done with Python. Stony stares. Do I talked about the nature of Python itself (interpreted, multi-paradigm, lexical/LEGB scope, the GIL). Stony stares. I wrote some trivial programs on the board. Stony stares. Had I brought an actual snake, I might have tried to charm it. At the end, the CEO told me they weren't overwhelmingly sold on me, but would think about it. Never heard from them again.
jnwatson 5y ago
The second is certainly more likely, but I'd wager the likelihood of the first is greater than 10%. I've encountered my share. There are so many "developers" just faking it, I can certainly understand using a test that would reject 90% of the good candidates if it could reject 99% of the bad ones.
- deleted 5y ago
  [deleted]
handrous 5y ago
I've repeatedly had employers very happy with my abilities & results, and am also entirely sure I've, on a few occasions, convinced interviewers I'm entirely unable to write code and am one of these frauds everyone's sure exist and that they need these coding tests to "catch".
laurent92 5y ago
I’m the interviewer, I’m still wondering what happened. This was the introductory question before launching 200 threads and asking him to solve the deadlocks/inefficiencies, which was the real question supposed to let him show off his skills in front of my employees, specifically crafted for him because I wanted to persuade my employees he was an excellent hire. So he had a taylored chance to show off his skills but failed at the introductory question. But on the other hand, how can you be asked “Here’s a substring, return true or false if it contains the substring, this is the introduction of 5 questions so don’t sweat it” and not just write two nested loops and an if? I’d pass on UTF8 problems, but when you’ve been working with Java for CRUD apps, you still should have your UTF8 correct. This is how you end up with passwords that must be ASCII because the programmer is bad.
- mattkrause 5y ago
  I've seen an actual Nobel Prize winner get stuck describing their research. People's brains just occasionally lock up.

BossingAround 5y ago

What was the goal of the question? Why did you want the person to implement a contains method? Did you really want to verify they understood String implementation in Java? And if the candidate was able to do this in 6 minutes, what would you have thought? "Great, let's hire"? In my humble opinion, the question is a waste of time either way. You'll get much further trying to probe what the candidate does know rather than randomly creating an exercise that you think they "should be able to do if woken up in the middle of the night". People forget how stressful interviews are and how easy it is to assume shared context. The fact that your employee was able to do the test might be indicative of the fact that you share context with the employee that you did not share with the candidate, thus confirming your bias.

Lio 5y ago
> The fact that your employee was able to do the test might be indicative of the fact that you share context with the employee that you did not share with the candidate, thus confirming your bias. This! When giving interviews last I really worried if the questions I asked where just indicative of my own Dunning-Kruger effect. i.e. Do I only ask questions I already know the answer to and not questions I don't know the answer to? If I do then am I just filtering for people with the same background and knowledge and missing out on people with other skills I don't know, because they're in my blind spot, I need yet?
- magnetic 5y ago
  > i.e. Do I only ask questions I already know the answer to and not questions I don't know the answer to? I've been advocating for a "coding interview" where both the interviewer _and_ interviewee draw some random question from leetcode or other problem bank, and try to work at it together. This would show collaboration skills, and you can tell pretty easily how helpful the candidate is with his/her contributions, and whether you find there is an impedance mismatch somewhere. It probably also maps more closely to the kinds of interactions you'd have after the person's been hired. I think it would also help calibrate: if you can't figure it out, is it fair to expect the candidate to figure it out? Maybe it's just a hard problem!
5y ago

eloisius 5y ago

Did they have to write it in a google doc with no ability to compile and run their code?

bigbillheck 5y ago

I don't do interviews in my current job (mostly because the pandemic really did a number on hiring) but in my previous job the only coding question I asked was what I thought was a fairly simple string problem. They could assume ascii, use any language they wanted, and make any other assumptions to simplify the problem. My then-co-workers liked to ask harder algorithmic questions but I wanted to give the candidates a little bit of a break with something easier. It didn't always work, but at least I tried.

notJim 5y ago

Curious about the age of the person who supposedly wrote it in 6 minutes. If you're fresh out of school, "String.contains" may be top of mind, but the vast majority of people never write that function in practice, so it's easy to not think about.

wheelinsupial 5y ago

What was the problem statement you gave to the candidate? What were you trying to tease out from the problem statement?

walshemj 5y ago

Why would you not use the language built-ins? I am not a Java programmer but it took me 30 seconds to find the contains() method.