13 ms·

I realize everybody's going to jump in and rant about algorithms in interviews, but I wish you'd all add something constructive as well. I just had to conduct

I realize everybody's going to jump in and rant about algorithms in interviews, but I wish you'd all add something constructive as well. I just had to conduct a round of interviews in a non-SF large US city, and it was a hellish crapshoot. Resumes are meaningless, and often re-written by recruiters to match the job anyway. Everyone has the same canned answers to the stupid behavioral questions. And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. Is this kind of code problem too complicated in your opinion? For all I join in when complaining about irrelevant algorithmic questions, I have to admit that they at least test something, even if it's just willingness to study for the interview. Instead of reading everybody's complaints about interviewing, I'd love to hear how you think it should be done. Because I have to admit I'm pretty much lost right now.

ale22 7y ago
How much were you paying for these positions?
- DavidWoof 7y ago
  Slightly above market rate for the area, all in the low-to-mid six figures.
  - ale22 7y ago
    Like $110k-$150k?
  - numakerg 7y ago
    Was this a position where they were expected to write code? I'm surprised that developers making six figures couldn't solve a problem that an interviewer considered trivial. Any chance of a sample or maybe an alternate version of the problem?
    - deleted 7y ago
      [deleted]
    - deleted 7y ago
      [deleted]
    - maehwasu 7y ago
      I have had similar experiences as an interviewer. It’s one of those things that you really have to see to believe, but it’s definitely a real thing.
leovander 7y ago
Nest for loop for the technical interview? Where do I apply?
momokoko 7y ago
Is there any chance you could share the problem? Changed enough to protect your identity, of course.
- DavidWoof 7y ago
  It was very similar to the “given an array of stock prices, find the optimal buy and sell indices for the biggest profit" problem that somebody referenced above. But we emphasized repeatedly we weren't looking for the O(n) solution, just the brute force naive solution was 100% OK. And it definitely didn't look like people were freezing up trying to figure out the optimal problem, they were struggling on the basic nested loop.
dmoy 7y ago
> Is this kind of code problem too complicated in your opinion? For all I join in when complaining about irrelevant algorithmic No. Just programming is actually not easy and a lot of people apply for jobs they can't do. Plus, a nonzero number of people freeze up in any interview situation. I once failed an interview loop because I forgot how bucket sort works.
- agentultra 7y ago
  Also 95% of job descriptions list skills candidates will never use and screen candidates with problems they will never encounter. At the final interview to join the SRE team at Google I was asked to implement the kNN algorithm. I barfed at implementing a kD-tree after regurgitating the brute force solution. Has any SRE ever had to implement a kD-tree in < 20 minutes or Google would go down? I asked the interviewer at the end. They had never implemented one on the job. As long as companies insist on these inane rituals I think it’s fair game to optimize for it as an interviewee. It’s stupid but what else can you do?
  - raincom 7y ago
    They could be looking for those who participate in competitive programming contests.
  - CoolGuySteve 7y ago
    The thing that really bugs me is that while Google has a reputation for asking these optimization questions, when it comes to programming artifacts one can inspect client side: - Gmail loads slower than Eudora did on dial-up - Chrome takes so much memory it’s basically a meme now - The Google homepage (a text bar on a white background) is several hundred KB. So what I want to know is, if they hire so many algorithmic whiz kids, where the fuck are they hiding?
    - username90 7y ago
      Very few at Google work at client side stuff, most sought after positions are many layers down the stack like machine learning, distributed infrastructure, programming languages or operative systems.
andrewstuart 7y ago
>>> how you think it should be done. Because I have to admit I'm pretty much lost right now. - Treat recruiting in the same way as you do software development. - Formulate a set of requirements. - Define interview questions that give insight into whether or not the candidate meets those requirements. This is the equivalent of "tests" in the software process. - Specific skills with your technology stack is good, but not necessarily essential. - Ability to discuss sophisticated software concepts, and to explain software that they have built, and how they would build out ideas given to them is good. - Evidence that this person gets stuff done is good (ref Joel Spolsky). Coding tests are, for the most part, garbage. Not because the test is of no value, but because you the employer probably don't evaluate the result properly.
- worble 7y ago
  While I can't say anything from the prospect of an interviewer, as an interviewee I am a big advocate for this style. Give someone a set of requirements, and then talk through how they would solve it. Allow for further clarification, and just talk about tech. It's a little more unstructured and less formal, but surely you'll very quickly pick up what experience they have, whether or not they've made past solutions that'll fit, what new technology they'd like to use, what technology they'd end up actually using, and there's no set right or wrong answer. The interviewer might even end up learning about something they themselves weren't aware of. If someone could blag that while not being able to even write FizzBuzz, all I can say is well played to them.
new299 7y ago
Personally I can't see an issue with very simple FizzBuzz style programming interview questions. I used to ask a simple "count duplicate substrings" question [1]. Maybe some people consider this too hard? I never used to require exact syntax, and would have been happy with pseudo-code. Using libraries is fine etc.. I also found very few people could solve this (similar non-SF large city location). Occasionally, people who could not solve this were hired for other teams. Based on their performance, I don't think I would have been comfortable working with them. I don't think it's unreasonable. But I'm not sure I'd use it as a screen if I was hiring now. I think I'd just have a chat and try and discuss a previous project. After that I'd move to a paid take home project (ideally representing real, useful work). [1] Take a string, for example "ABCCABC" and count the number of times each 3 character substring occurs. In this case the answer would be 2xABC 1xBCC 1xCCA 1xCAB.
- lacker 7y ago
  Of course some people will consider that question to be too hard. But you may not want to hire those people. The point of an interview process is for you to pick the people you want to hire, not to ask questions that everyone agrees aren’t too hard.
  - new299 7y ago
    I'm curious to hear the opinions of people who think that the question is too hard (i.e. it's too complex of a question to ask in an interview situation). Perhaps some people might feel that while they could write code to solve this problem (or similar problems) outside of an interview. However, under the pressure of an interview they would not be able to solve the problem (due to anxiety, stress etc.). If that was relatively common, then such a question would not be a useful interview question. Or would at least not be giving an accurate estimate of a candidates programming ability. This seems unlikely to me, but I would be interested in hearing different view.
    - bpchaps 7y ago
      The fizzbuzz test is not entirely about whether someone can do it. It's also about figuring out the style of the programmer. There's a surprising amount of flexibility to fizzbuzz; how is user input handled? Do they use a linter? How are variables managed/named? There's a lot going on, and it can be gamed just like anything else on both ends. That said, the last time I had a fizzbuzz take-home test, it was infuriating -- my code is online.. what does fizzbuzz prove except to help make a company soc2 compliant?
_hardwaregeek 7y ago
Where are you posting your jobs? Are you paying a recruiter? How much? Maybe you could have interviewers rank candidates (hire; competent, but no hire; slightly incompetent; how did they get here?), then see where the majority of your "how did they get here?" candidates come from.
seanmcdirmid 7y ago
Resumes and prior experience should be taken more seriously, backed by more rigorous verification of said things and real reference checking. In a way, that is what already happens for the more technical positions (someone already knows you can do the job, the interview is just a formality) and sucks for those who don’t have well known enough reputations.
non-entity 7y ago
I realize everybody's going to jump in and rant about algorithms in interviews, but I wish you'd all add something constructive as well. I was like this a couple years ago. I was a self taught, "college is a scam", "practical experience" type guy. I now, however do see immense value in the ability to be able to work through these algorithm questions,especially if you ever want to do something besides web / app development.
- goatinaboat 7y ago
  Algorithms are great but in the real world what matters is being able to recognise a class of problem then go to the literature e.g. Knuth to find the right one. No working programmers know every algorithm off the top of their heads. The classic detect a loop in a linked list question. The original guy took years to devise the algorithm for it. In an interview either you’ve seen it before, in which case you rattle it off, or you have to write a research paper effectively in 5 minutes.
  - alkonaut 7y ago
    This. The problem a programmer needs to solve isn’t remembering or applying the algorithm, it’s identifying what needs to be done. “This problem is (analogous to) loop detection” “This problem requires a sorted list” “This problem can be solved by depth first traversal” Actually sorting a list or doing a traversal is easy after that crucial step. You can look it up. You can’t look up the step that told you what algorithm or data structure to use though.
  - scarejunba 7y ago
    Yeah but there's a difference. We're immersed in a state of better knowledge. It took centuries for us to go from the forms of knowledge that are "truth is self-evident" to "verification:of consequents is the pre-eminent method of knowledge acquisition" to "failure to reject falsifiable ideas is the best method to acquire knowledge". But I took that same path without reading the philosophers in approximately ten years. How? Because the cultural milieu is different. I am a unit of a more capable civilization and I am therefore enhanced. So am I smarter than Karl Popper? Not really. It's just easier to learn something than to find it. So, yes, I know the linked list answer.
dehrmann 7y ago
> nested for-loop problem Ah, dynamic programming.
deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
choppaface 7y ago
I agree a code test is necessary. I’ve seen several panels neglect to do a code test, the candidate was hired, then within a month fired because it was clear they couldn’t do anything (other than, well, argue). I’ve been in a panel where I was the only person who asked a code question, the candidate flunked, and then the VP of Engineering went over my complaints and hired the guy anyways. He had been a Professor of Software Engineering and had a graduate degree from Princeton. Within three weeks, the VP of Engineering had to fire him because he couldn’t make it through a simple code review. BUT the extremely negative sentiment here towards the technical interview process is very well-deserved. Assessment of code (and the selection of problems) is most often no less subjective than any non-technical assessment. Sometimes the interviewer doing the grading is flat out wrong. Several times I’ve been asked the famous “given an array of stock prices, find the optimal buy and sell indices for the biggest profit.” One interviewer was not aware of the linear time solution to this problem, and didn’t believe me when I wrote and tried to explain it to him. But sometimes, the interviewer doesn’t even want you to do well. Once time I was interviewing with an injury that prevented me from typing efficiently. I had a doctor’s note and the injury was quite conspicuous. Nevertheless, three start-ups made me solve problems by typing on a keyboard, which guaranteed an excessively long completion time. Those companies held those results against me. (An it’s not like there is anybody to hold those panels accountable). And then there are those who just don’t care. I had a phone interview with Airbnb that was literally as bad as the stories on Glassdoor: the guy answered the phone in a noisy office (not a conference room), gave no introduction, then simply stated the problem and dumped me into Coderpad. I literally thought it was a prank, since I had met with people at Airbnb face-to-face prior to the call. But the recruiters confirmed the guy was a real employee. The root problem here is there is no feedback loop back to interviewers. The candidates get “feedback,” but people asking code questions, especially new grads, typically get zero assessment on how well they are doing as interviewers. What’s worse is that recruiters and hiring managers both have incentives to deprive ICs of such feedback, since it would invariably make ICs more aware of opportunities outside the company. Until the incentive structure of technical interviewing changes dramatically, we’re stuck with Leetcode and hope for the best. People like Gayle Laakmann are helpful (especially when Facebook gives candidates a live hour-long session with her for free), but these people are ultimately invested in their own income and not the task of fundamentally fixing this broken process.
jjav 7y ago
I can tell you how I do it and would certainly recommend it as the way it should be done. For some context, I've been interviewing software engineers for about 25 years in companies ranging from established multi-nationals to tiny startups in very fast headcount-growth mode. I'm in silicon valley. I can say that I've never regretted a hire I said yes to, so the method works to my satisfaction. It'd be nice to think I have some special skill here but I really don't. This is just how interviewing was done in the 90s. To some of the younger generations, I've been told it sounds crazy. If you send me your resume I'll actually read it, carefully. If the person described in this resume fits the background experience the role needs, you get an interview. During the interview we'll talk about all those projects you worked on that are relevant to this role. Which parts you enjoyed the best and why? Which parts were boring and why? Which parts were the most challenging and why? What you find too easy and why? What would you have done differently? Could you have? If you were to do the same project all over how would you approach it? Other open ended conversations along these lines. I don't ask anyone to whiteboard code, that's not part of the job so it's not part of the interview. No puzzles, no trivia-pursuit style questions. It works great. You can't BS your way through such a conversation with a senior technical peer if you didn't actually do the work described in the resume. You just can't. It is, however, vital that the interviewer must be a expert in the field.
- zerr 7y ago
  I wonder if there is a list of companies doing interviews this way.
  - norcalli 7y ago
    I use the same technique (and I'm 26 now, so relatively young) and it's how I've always interviewed others. Granted, I've only worked at smallish startups, but, either way, the company didn't care about my technique for evaluation, and just my final "yes", "no," or "maybe." So, I think it depends on the engineer. If someone can keep up in a technical conversation about their background with me and answer every question I have about a technical project they did, then basically they pass. It works especially well even if I'm not familiar with their project, because I have an opportunity to learn, so I can ask any question that comes to mind until they teach me what they learned. I did hire someone that I regretted, though, but to be fair, this was among my first interviews. The mistake I made was getting too easily caught talking about programming and technical things without specifically diving deep into his past project. He and I vibed quickly and I liked him, and that felt like enough, but after only a week it seemed obvious that he wasn't going to be producing much code, and we let him go. Otherwise, I've been happy and my ability to discern has only gotten better as I became more experienced. I got a little offtopic, but my main answer to your question was "if the company leaves the decision up to a majority of engineers saying 'yes', then a lot of companies do this." Google does this, the startups I've worked at do this, and some of my friends companies do this.
neetfreek 7y ago
I prefer take-home assignments. I know that they often get a bad rep and have the potential for abuse; for this reason I'd argue for implementing some or other solution in the field. I recently had one requiring me to develop a native mobile application, which I enjoyed. It was interesting, the code is useful down the line, and if I don't land the job, it beefs up my portfolio. Initial screening by recruiters is tough as my background's missing a degree and industry experience. Context: self-taught, started out with game dev, tried going solo - not a runaway success. Looking to move away from the field.
nimrody 7y ago
One company sent me (before the interview) a small technical assignment. After I had submitted the code, an interview was scheduled. The entire interview was an extended code review -- talking about trade-offs, about other potential solutions, etc. I felt this was much better in that it was less stressful, yet allowed me to demonstrate both knowledge and design skills. Another company did something similar but more thorough: They invite candidates for a full day of work where they try to solve a small problem. Then the code is reviewed and evaluated together. They also start the day with a 1-hour overview of their current architecture and you get to ask questions and talk about alternatives. I think this gives both sides a better chance of finding the right fit. I realize this is not always reasonable.
- rhizome 7y ago
  I've been thinking for a while that companies should create their hiring tests from bugs and/or feature requests that came from their actual software in the past. Then they can gauge the quality of the employee for their purposes by comparing the candidate's solution(s) to those the actual employees wrote.
  - nimrody 7y ago
    Bugs I have seen in actual code are divided into two types: those that are due to the way code inter-operates with other code (sort of integration bugs) and tricky bugs (usually edge cases) in some highly dense code. The first type requires the candidate to learn a lot about the existing code base. The second requires more mental work but is atypical of real work. Implementing features sounds like a good idea. Perhaps create a small well defined system which is a simplification of the real world system and let the candidate implement some additional features.
RijilV 7y ago
> Everyone has the same canned answers to the stupid behavioral questions. The art of behavioral questions isn’t “ask and answer” it’s the follow up questions. As you astutely point out, the questions are ‘stupid’. They might as well be “do you want a stick of gum?” Next time you ask those questions do a couple of things. First keep in the forefront of your thoughts what information you’re trying to get out of it and keep the candidate on track answering your data point. Do that by relentlessly asking follow up questions. When you think you have everything ask more. As an anecdote I was being shadowed during an on-site recently. I asked some arbitrary ‘dumb’ behavioral question, went back and forth a bit, wasn’t getting much out of it. I noticed my shadow clearly moving on to the next question in their notes and decided to keep pushing on the original question - why did you do this, what were you trying to solve, what motivated you. Turns out the candidate did all of this to generate new revenue for the company and ended up bringing in $10m a year extra at the small company there currently worked for. Loads of great data, would have never gotten there if I’d settled for the canned answer the candidate had. Behavioral questions aren’t comp-sci trivial questions, you can’t just ask the behavioral equivalent to fizbuz/Fibonacci/floodfill and copy down the answer (and you should never be asking those questions either, but that’s a separate rant). Behavioral questions are stupid and to some degree that’s the point. When you ask your significant other or kids “how was your day?” — guess what, that’s a stupid question too. What matters is what follows from your line of interviewing. If you want to get good at behavioral questions listen to Fresh Air and try to be like Terry Gross.
gaoshan 7y ago
For the code portion of things I stick to a set of increasingly difficult "real world" problems. The first one should be easily answerable by any potential candidate, the last one should be too hard for most people but I'm really looking at how they problem solve and handle themselves. I've recommended people for hiring who eventually gave up on that last one (and who went on to be fantastic in their jobs).
winrid 7y ago
Phone screens should prevent those candidates from ever getting onsite. You'd be surprised (or maybe not now) how many applicants for a senior frontend position can't build a progress bar for the phone screem.
- shagie 7y ago
  I've given phone screens to individuals who turned out to be a different person when they showed up onsite. Not "nice on phone, jerk in real life" but rather "Bob does the phone screen for Charlie (and passes), Charlie shows up at the onsite."
  - farah7 7y ago
    I usually have video off during phone screens. Do you require them on?
watwut 7y ago
I like the for loop code thing. Imo, good hiring process for hiring programmers must involve small piece of code. Not difficult or algorithmic, but something to distinguish those who can't do anything at all.
danmaz74 7y ago
I use basically the same process, but use a very simple programming problem at the end - simple, but amenable to discussing optimization and edge cases. I let the candidate choose the language (or just use pseudo code) and don't care at all about the syntax. I find that this is very useful especially when interviewing juniors who don't have many projects under their belt for the first part. It's also useful when a candidate has good verbalisation skills, but poor programming ones (which happens).
underwater 7y ago
> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. It's possible that you asked the question poorly, or the solution wasn't as obvious as you thought. Designing interview questions is hard[1]. I'll test out new questions on my peers at least two or three times before putting them in front of a candidate. And many don't make the cut. If a good engineer who's relaxed can't solve it easily, then a stressed out candidate will have no hope. [1] This is why I hate seeing candidates share specific questions online. As an interviewer you'll have to scrub a good question, and switch to something you're not as familiar with. This hurts good candidates.
greenyouse 7y ago
I've been through a few different types of interviews and the random algorithm style seems off to me. I recognize that there are companies like FAANGs that want a deep bench and will have expert algorithm people on-call for that one time when it's needed. Most companies should be OK to just present real problems either solved or being worked on which they would expect applicants to be able to solve. This is more true for startups that need less know "how to reverse a binary string" and more "how to properly design a database with 3rd order normalization", etc. If you're presenting an interview question, it should have actual job relevance and your co-workers should be able to solve it in the same time as the candidate. If you're drilling people on non-job qualities (e.g. invert a binary tree for a web dev role...) then you should expect a large difference between audience that can pass your bad interview tests and audience which will perform well at actual job. Not trying to complain. I think job interviews should focus more on the 99% of what you do in your job on a Tuesday. Poor interviews seem to be more gotchas and algorithm tricks to disqualify roles which have little actual use for algos + data structures. The tests might seem too easy but as a front-end engineer I would rather be with a coworker which understands the CSS box model, knows semantic markup for accessibility, and similar web things than a person which is good at creating hash tables and doubly-linked lists in JS. Leet code probably doesn't test vertical centering techniques with CSS but if you're applying for a web dev position you better know them.
dustfinger 7y ago
> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. > Is this kind of code problem too complicated in your opinion? May you please post the problem so that we can provide you with meaningful feedback regarding said problem?
rapht 7y ago
I don't work in tech (though I sometimes think about uuyýjmq
alkonaut 7y ago
If you risk having a bullshitter you need some way of identifying them. The best way is through references, personal projects etc. But if you need to recruit someone without any such credentials then you may need to do a simple coding aptitude test. Could be a code review or a simple excercise but whatever you do, don’t do whiteboard coding and don’t have people recite/implement memorized CS textbook algorithms. Anyone can do that and still no code.
dustfinger 7y ago
> Everyone has the same canned answers to the stupid behavioral questions. If the person conducting the interview thinks the behavioral questions are stupid, then perhaps they are. In that case, don't ask "stupid" behavioral questions. > Resumes are meaningless, and often re-written by recruiters to match the job anyway Was the position entry level? Students coming right out of compsci often have little to no practicable experience. They may have difficulty thinking about what to put in their resume. After one or two years of full time experience that should no longer be an issue. > For all I join in when complaining about irrelevant algorithmic questions, I have to admit that they at least test something, even if it's just willingness to study for the interview. Asking those "stupid" behavioural questions and receiving the same canned answers also demonstrates a willingness to study for an interview. The coding problem should be testing a candidate's problem solving capabilities as practicably required by the role being interviewed for. The chosen problem should reflect the types of problems that they will actually need to solve if hired. For example, you could select a small PR from one of projects being actively developed by the company. The selected PR should involve only one or two classes (assuming a language with classes) and require improvement. You can look through the history of a PR and just pull out a segment that was selected for improvement by the reviewer(s), or have the team select it for you. Then ask the candidate: - to conduct a code review of the PR - to improve the code
detaro 7y ago
> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. How did you run these? While it sounds like something that should be ok even on paper, you can vary comfort a lot through the medium. E.g. for the last interview I had with a substantial coding part, I think being able to do it on my personal machine made a big difference. (I obviously was told before what kind of environment I'd need to have ready)
dccoolgai 7y ago
1) Resume screen 2) 30-minute coderpad/codeshare exercise on a problem/pattern you actually use/encounter during the course of your work over phone. (No inverting binary trees). Expect a 20% pass rate here. 3) Reasonable take-home problem that you've timed 2 of your own staff completing well in 50 minutes. This is where you will get the most complaints from applicants, but that's OK. Let them select themselves out of the process. Expect a 30-40% pass rate here. 4) In-person interview. At this point, you should be mostly committed to hiring the candidate. Do a couple livecoding deals, but be extremely lenient in how you interpret results. Other week, we had a candidate fail the problem, but they kept their composure and showed they knew what they were doing on the way to bombing the problem. Candidate seemed sad at the end of the interview and happily surprised when we extended an offer. IMHO this process works fairly well and does a good job of being economical with people's time.
- mjfisher 7y ago
  I'm curious about the codepad/codeshare approach right at the first touch point. I fully agree that screening actual tech skill early on is important. Do you not find that you commit a lot of engineer time to codeshare interviews that don't work out further down the line?
  - dccoolgai 7y ago
    Codeshare portion is 1 (typically senior) engineer for 30 minutes. Not that bad on a large enough team with everyone taking turns. I should add that we have 3-4 "canned" questions that the HR rep writes down the answer to, so even in the "resume screen" there are a few super-lightweight technical questions.
    - mjfisher 7y ago
      Interesting. I've got a feeling early stages of tech hiring pipelines are very poorly optimized at the minute; it's something I'm working on. Individual experiences help; thank you.
  - dccoolgai 7y ago
toupeira 7y ago
At my current position for a remote job, one of the interview assignments was reviewing a pull request for a very basic example app using their tech stack, and then implementing my own suggestions during a video call (while sharing my screen), updating/running tests, discussing trade-offs etc. I think this was a great way to not only verify coding ability, but also testing team work and communication skills.
bufferoverflow 7y ago
I have a similar experience doing interviews in NYC. We started by asking algo questions, but quickly found out that 95% of the candidates can't answer them. My boss, an ex-programmer, was surprised too, and she asked me to dumb them down, a lot. We ended up with a bunch of really trivial stuff like "write a function to reverse a string" or its slightly harder version - "reverse an integer". Could you give the exact wording of your "nested for-loop" question?
patientplatypus 7y ago
This is surprisingly easy, but you have to understand what the problem is first or the solution won't be apparent. If this sounds patronizing it comes about from my frustration on being on the other side of the table too often. What you want is very good football (soccer) players. Unfortunately you (or upper management) may not know all of the rules to soccer. You may not know the training regime that goes into winning a good soccer game - and it's a big risk spending money training for the big game on the off-season only to lose during the Big Show. So what do you do to test potential candidates? You see if you can get along with them, if they're a team player, and then see how well they play foosball. It's perfect! There are soccer players on the field, there's a goal, it takes skill and coordination. But oh no, it turns out in the population at large really good soccer players really sort of suck at foosball. After all, they'd rather spend their time and energy playing soccer. So now they spend all their time reading up on books about foosball and what the best foosball strategies are. You see where I'm going with this? No one uses algorithms in their day jobs. OK a few of you, but come on man, I make full stack web applications. As do most programmers. So why are you testing theory that has literally nothing to do with the job? Somehow someone thought this was a proxy for smart people, but I mean, if the guy who wrote this "Tech Interview Handbook" was really smart wouldn't he have spent his time writing a cool program? I mean how lame is this? If you want to hire competent engineers tell them what you're building and ask them how they feel is best for them to show they're competent. If you have a big data pipeline in Scala ask if they can construct a data pipeline example that is cool over a few days (take home) or do something similar in the office. Some people like one, some like the other. But just communicate with the people you want to hire! And if not everyone's interview process is the same then maybe that's ok. I mean I just got a guy who sent me some automated code interview program that had a timer that counted down from an hour at the top! WHO THINKS TREATING THEIR POTENTIAL EMPLOYEES LIKE STAR WARS DRONES IS A GOOD THING? All you have to do is treat the people you want to work with like you would want to be treated and demand that they know their shit. This. Isn't. That. Hard.
traverseda 7y ago
> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. Could it be that you're having issues communicating the problem?
wan23 7y ago
The way most fields that require some level of knowledge or ability handle this is by having industry-standard assessments that everyone takes. When doctors interview they get almost entirely behavioral questions because the hiring hospital / office merely needs to check that the doctor is a) licensed and b) certified in whatever specialty they are being hired for. They certainly never get asked whatever the medical equivalent of FizzBuzz might be.
Jach 7y ago
Seems like your problem is basic competency? Move the "can you even code?" question to as early in the filter stage as possible (first 'phone' screening). If you have a lot of applicants, you'll have to do some earlier filtering in the name of time (like on degrees, years experience, "the lucky half"), but don't pretend it's fair or very accurate since both the false positive and false negative rates will be high. I agree that some coding problem needs to be used to try and answer the question, though with the right interviewer they can answer it without seeing code. The problems you use for that don't have to be at octree-collision-detection whatever challenge, a trivial nested for-loop is fine -- fizzbuzz level is fine. Sometimes you can rely on github or strong internal referral to skip this, but watch out, and anyway it's worth giving your questions to people you're sure will do fine (you've timed at least yourself right?) for the benchmark data and because sometimes they don't do fine, perhaps since maybe your question is too much. e.g. Floyd-Warshall can be done simply with a few nested loops, still I would never give it as a problem and I'd expect nearly everyone I've worked with to flunk it given only the standard hour (which really means 45 minutes). Some jobs only need basic competence, so you might want to extend an offer if you've been convinced of its presence. At my last job, which ended up being more technically challenging / interesting than my current job, I was hired after posting my resume to Craigslist which led to exchanging some emails and having lunch with the startup founder to talk about my past work and whether I would be useful for his most pressing work. At my current job, I've been part of on-sites where I've established "can you even code?" is "no". Those were costly failures of not having that answered earlier. But we also like to believe we need more than basic competence, so rejections can still occur because of a lack of "testing mindset" or certain "behavioral answers". Only once you fix your "can you even code?" filter is it even worth considering what else you might want to justify an interview pipeline with more stages than a 'phone' screen or lunch conversation.
- mjfisher 7y ago
  I couldn't agree more with the idea that you should move a 'can you code at all' test to as early as possible in your hiring pipeline. I used to wait to the first in-person interview to try simple fizzbuzz style questions (with the candidates on a machine and a compiler/interpreter). In about a third of cases that meant we'd committed a significant chunk of time to engineers that apparently couldn't solve trivial problems. Now it's one of the first things I check. Done right, it's a relatively small hurdle for capable people to overcome, but really helps as a filter for those who aren't suited to the role. I recently created a service (https://candidatecode.com https://candidatecode.com) to help companies manage issuing and reviewing their coding challenges; I think it's got real potential to help some people out.
ummonk 7y ago
I don’t think the problem lies with your interview if they can’t write a simple nested for loop... Which I think is why a lot of startups locate in the expensive Bay Area - not a lot of cities have a similar concentration of decent talent.
modzu 7y ago
what is everyone smoking? youre still trying to optimize for a test and not a job. dont give them a fizbuz. give them an example of a real problem your engineers need / are trying to solve. how do they respond? thoughts / intuitions / pseudo code. do they show knowledge of the problem space / domain? ot if you just want a kid who can code and they pass the fizbuz but fail at the real job, what does your training/culture look like? who does that really reflect on? it seems to me that interviewing is terribly cargo cult. the problem is real, the practises ostensibly supposed to be solutions are not. /end rant
daphneokeefe 7y ago
> we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. If no one can answer the question, maybe they don't understand it? Maybe there's something unclear in the way it is worded?
wcunning 7y ago
I'm getting up the gumption to look for a different, hopefully better job in the nearer future (in automotive control software, looking to move to AV), and I had a bit of a revelation when talking to a friend of mine who does interviewing. I cannot talk about the things in my current job that would make me a good hire for the things I want to move into. It's all hunting down an obscure bug buried in layers of technical debt and overly complicated standards, but giving any depth beyond that bare platitude requires going into things that my NDA covers. I even work in driver assist technologies, but I can't go into detail about that because I'm working on unreleased, unannounced features. That means that the only thing left is side projects or whiteboard interviews. To be fair, this is at least partly an industry problem due to long product development cycles and a culture of secrecy, but it makes a lot of the better solutions to interviewing unworkable and causes companies to drop back to whiteboard interviews. And as a candidate, my personal maximizing function is to hit the books and be ready for curly braces and logic puzzles. Sometimes there really isn't a better way.
- mixmastamyk 7y ago
  “I’m under NDA” is a valid response. Have some hobby project or floss bug fix on github if there is time to fill.
davidjnelson 7y ago
What works for me is to consider what work the engineer will be doing, and ask them to write code to prove they can do it. For instance, a front end engineer can be expected to be able to write a to do list or similar app in a framework (ideally the one your team uses, but not a hard no hire if not) app with minimal googling (although that’s fine as long as not excessive) in ~45 minutes. Then you have to look at what level of experience they have. Less experience requires more mentoring generally, which may be fine depending on how much time your team budgets for that work. Lastly, measure their body language and tone of voice to check for red flags pointing to difficult communication styles or people who treat others poorly. If all three match, hire!
Aperocky 7y ago
This. I loath writing algorithm on whiteboard, especially the 'catchy' type. But I've interviewed people who can't even write a for loop.. the amount of brain drain in the flyover country is insane.

ale22 7y ago

How much were you paying for these positions?

DavidWoof 7y ago
Slightly above market rate for the area, all in the low-to-mid six figures.
- ale22 7y ago
  Like $110k-$150k?
- numakerg 7y ago
  Was this a position where they were expected to write code? I'm surprised that developers making six figures couldn't solve a problem that an interviewer considered trivial. Any chance of a sample or maybe an alternate version of the problem?
  - deleted 7y ago
    [deleted]
  - deleted 7y ago
    [deleted]
  - maehwasu 7y ago
    I have had similar experiences as an interviewer. It’s one of those things that you really have to see to believe, but it’s definitely a real thing.

leovander 7y ago

Nest for loop for the technical interview? Where do I apply?

momokoko 7y ago

Is there any chance you could share the problem? Changed enough to protect your identity, of course.

DavidWoof 7y ago
It was very similar to the “given an array of stock prices, find the optimal buy and sell indices for the biggest profit" problem that somebody referenced above. But we emphasized repeatedly we weren't looking for the O(n) solution, just the brute force naive solution was 100% OK. And it definitely didn't look like people were freezing up trying to figure out the optimal problem, they were struggling on the basic nested loop.

dmoy 7y ago

> Is this kind of code problem too complicated in your opinion? For all I join in when complaining about irrelevant algorithmic No. Just programming is actually not easy and a lot of people apply for jobs they can't do. Plus, a nonzero number of people freeze up in any interview situation. I once failed an interview loop because I forgot how bucket sort works.

agentultra 7y ago
Also 95% of job descriptions list skills candidates will never use and screen candidates with problems they will never encounter. At the final interview to join the SRE team at Google I was asked to implement the kNN algorithm. I barfed at implementing a kD-tree after regurgitating the brute force solution. Has any SRE ever had to implement a kD-tree in < 20 minutes or Google would go down? I asked the interviewer at the end. They had never implemented one on the job. As long as companies insist on these inane rituals I think it’s fair game to optimize for it as an interviewee. It’s stupid but what else can you do?
- raincom 7y ago
  They could be looking for those who participate in competitive programming contests.
- CoolGuySteve 7y ago
  The thing that really bugs me is that while Google has a reputation for asking these optimization questions, when it comes to programming artifacts one can inspect client side: - Gmail loads slower than Eudora did on dial-up - Chrome takes so much memory it’s basically a meme now - The Google homepage (a text bar on a white background) is several hundred KB. So what I want to know is, if they hire so many algorithmic whiz kids, where the fuck are they hiding?
  - username90 7y ago
    Very few at Google work at client side stuff, most sought after positions are many layers down the stack like machine learning, distributed infrastructure, programming languages or operative systems.

andrewstuart 7y ago

>>> how you think it should be done. Because I have to admit I'm pretty much lost right now. - Treat recruiting in the same way as you do software development. - Formulate a set of requirements. - Define interview questions that give insight into whether or not the candidate meets those requirements. This is the equivalent of "tests" in the software process. - Specific skills with your technology stack is good, but not necessarily essential. - Ability to discuss sophisticated software concepts, and to explain software that they have built, and how they would build out ideas given to them is good. - Evidence that this person gets stuff done is good (ref Joel Spolsky). Coding tests are, for the most part, garbage. Not because the test is of no value, but because you the employer probably don't evaluate the result properly.

worble 7y ago
While I can't say anything from the prospect of an interviewer, as an interviewee I am a big advocate for this style. Give someone a set of requirements, and then talk through how they would solve it. Allow for further clarification, and just talk about tech. It's a little more unstructured and less formal, but surely you'll very quickly pick up what experience they have, whether or not they've made past solutions that'll fit, what new technology they'd like to use, what technology they'd end up actually using, and there's no set right or wrong answer. The interviewer might even end up learning about something they themselves weren't aware of. If someone could blag that while not being able to even write FizzBuzz, all I can say is well played to them.

new299 7y ago

Personally I can't see an issue with very simple FizzBuzz style programming interview questions. I used to ask a simple "count duplicate substrings" question [1]. Maybe some people consider this too hard? I never used to require exact syntax, and would have been happy with pseudo-code. Using libraries is fine etc.. I also found very few people could solve this (similar non-SF large city location). Occasionally, people who could not solve this were hired for other teams. Based on their performance, I don't think I would have been comfortable working with them. I don't think it's unreasonable. But I'm not sure I'd use it as a screen if I was hiring now. I think I'd just have a chat and try and discuss a previous project. After that I'd move to a paid take home project (ideally representing real, useful work). [1] Take a string, for example "ABCCABC" and count the number of times each 3 character substring occurs. In this case the answer would be 2xABC 1xBCC 1xCCA 1xCAB.

lacker 7y ago
Of course some people will consider that question to be too hard. But you may not want to hire those people. The point of an interview process is for you to pick the people you want to hire, not to ask questions that everyone agrees aren’t too hard.
- new299 7y ago
  I'm curious to hear the opinions of people who think that the question is too hard (i.e. it's too complex of a question to ask in an interview situation). Perhaps some people might feel that while they could write code to solve this problem (or similar problems) outside of an interview. However, under the pressure of an interview they would not be able to solve the problem (due to anxiety, stress etc.). If that was relatively common, then such a question would not be a useful interview question. Or would at least not be giving an accurate estimate of a candidates programming ability. This seems unlikely to me, but I would be interested in hearing different view.
  - bpchaps 7y ago
    The fizzbuzz test is not entirely about whether someone can do it. It's also about figuring out the style of the programmer. There's a surprising amount of flexibility to fizzbuzz; how is user input handled? Do they use a linter? How are variables managed/named? There's a lot going on, and it can be gamed just like anything else on both ends. That said, the last time I had a fizzbuzz take-home test, it was infuriating -- my code is online.. what does fizzbuzz prove except to help make a company soc2 compliant?

_hardwaregeek 7y ago

Where are you posting your jobs? Are you paying a recruiter? How much? Maybe you could have interviewers rank candidates (hire; competent, but no hire; slightly incompetent; how did they get here?), then see where the majority of your "how did they get here?" candidates come from.

seanmcdirmid 7y ago

Resumes and prior experience should be taken more seriously, backed by more rigorous verification of said things and real reference checking. In a way, that is what already happens for the more technical positions (someone already knows you can do the job, the interview is just a formality) and sucks for those who don’t have well known enough reputations.

non-entity 7y ago

I realize everybody's going to jump in and rant about algorithms in interviews, but I wish you'd all add something constructive as well. I was like this a couple years ago. I was a self taught, "college is a scam", "practical experience" type guy. I now, however do see immense value in the ability to be able to work through these algorithm questions,especially if you ever want to do something besides web / app development.

goatinaboat 7y ago
Algorithms are great but in the real world what matters is being able to recognise a class of problem then go to the literature e.g. Knuth to find the right one. No working programmers know every algorithm off the top of their heads. The classic detect a loop in a linked list question. The original guy took years to devise the algorithm for it. In an interview either you’ve seen it before, in which case you rattle it off, or you have to write a research paper effectively in 5 minutes.
- alkonaut 7y ago
  This. The problem a programmer needs to solve isn’t remembering or applying the algorithm, it’s identifying what needs to be done. “This problem is (analogous to) loop detection” “This problem requires a sorted list” “This problem can be solved by depth first traversal” Actually sorting a list or doing a traversal is easy after that crucial step. You can look it up. You can’t look up the step that told you what algorithm or data structure to use though.
- scarejunba 7y ago
  Yeah but there's a difference. We're immersed in a state of better knowledge. It took centuries for us to go from the forms of knowledge that are "truth is self-evident" to "verification:of consequents is the pre-eminent method of knowledge acquisition" to "failure to reject falsifiable ideas is the best method to acquire knowledge". But I took that same path without reading the philosophers in approximately ten years. How? Because the cultural milieu is different. I am a unit of a more capable civilization and I am therefore enhanced. So am I smarter than Karl Popper? Not really. It's just easier to learn something than to find it. So, yes, I know the linked list answer.

dehrmann 7y ago

> nested for-loop problem Ah, dynamic programming.

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

choppaface 7y ago

I agree a code test is necessary. I’ve seen several panels neglect to do a code test, the candidate was hired, then within a month fired because it was clear they couldn’t do anything (other than, well, argue). I’ve been in a panel where I was the only person who asked a code question, the candidate flunked, and then the VP of Engineering went over my complaints and hired the guy anyways. He had been a Professor of Software Engineering and had a graduate degree from Princeton. Within three weeks, the VP of Engineering had to fire him because he couldn’t make it through a simple code review. BUT the extremely negative sentiment here towards the technical interview process is very well-deserved. Assessment of code (and the selection of problems) is most often no less subjective than any non-technical assessment. Sometimes the interviewer doing the grading is flat out wrong. Several times I’ve been asked the famous “given an array of stock prices, find the optimal buy and sell indices for the biggest profit.” One interviewer was not aware of the linear time solution to this problem, and didn’t believe me when I wrote and tried to explain it to him. But sometimes, the interviewer doesn’t even want you to do well. Once time I was interviewing with an injury that prevented me from typing efficiently. I had a doctor’s note and the injury was quite conspicuous. Nevertheless, three start-ups made me solve problems by typing on a keyboard, which guaranteed an excessively long completion time. Those companies held those results against me. (An it’s not like there is anybody to hold those panels accountable). And then there are those who just don’t care. I had a phone interview with Airbnb that was literally as bad as the stories on Glassdoor: the guy answered the phone in a noisy office (not a conference room), gave no introduction, then simply stated the problem and dumped me into Coderpad. I literally thought it was a prank, since I had met with people at Airbnb face-to-face prior to the call. But the recruiters confirmed the guy was a real employee. The root problem here is there is no feedback loop back to interviewers. The candidates get “feedback,” but people asking code questions, especially new grads, typically get zero assessment on how well they are doing as interviewers. What’s worse is that recruiters and hiring managers both have incentives to deprive ICs of such feedback, since it would invariably make ICs more aware of opportunities outside the company. Until the incentive structure of technical interviewing changes dramatically, we’re stuck with Leetcode and hope for the best. People like Gayle Laakmann are helpful (especially when Facebook gives candidates a live hour-long session with her for free), but these people are ultimately invested in their own income and not the task of fundamentally fixing this broken process.

jjav 7y ago

I can tell you how I do it and would certainly recommend it as the way it should be done. For some context, I've been interviewing software engineers for about 25 years in companies ranging from established multi-nationals to tiny startups in very fast headcount-growth mode. I'm in silicon valley. I can say that I've never regretted a hire I said yes to, so the method works to my satisfaction. It'd be nice to think I have some special skill here but I really don't. This is just how interviewing was done in the 90s. To some of the younger generations, I've been told it sounds crazy. If you send me your resume I'll actually read it, carefully. If the person described in this resume fits the background experience the role needs, you get an interview. During the interview we'll talk about all those projects you worked on that are relevant to this role. Which parts you enjoyed the best and why? Which parts were boring and why? Which parts were the most challenging and why? What you find too easy and why? What would you have done differently? Could you have? If you were to do the same project all over how would you approach it? Other open ended conversations along these lines. I don't ask anyone to whiteboard code, that's not part of the job so it's not part of the interview. No puzzles, no trivia-pursuit style questions. It works great. You can't BS your way through such a conversation with a senior technical peer if you didn't actually do the work described in the resume. You just can't. It is, however, vital that the interviewer must be a expert in the field.

zerr 7y ago
I wonder if there is a list of companies doing interviews this way.
- norcalli 7y ago
  I use the same technique (and I'm 26 now, so relatively young) and it's how I've always interviewed others. Granted, I've only worked at smallish startups, but, either way, the company didn't care about my technique for evaluation, and just my final "yes", "no," or "maybe." So, I think it depends on the engineer. If someone can keep up in a technical conversation about their background with me and answer every question I have about a technical project they did, then basically they pass. It works especially well even if I'm not familiar with their project, because I have an opportunity to learn, so I can ask any question that comes to mind until they teach me what they learned. I did hire someone that I regretted, though, but to be fair, this was among my first interviews. The mistake I made was getting too easily caught talking about programming and technical things without specifically diving deep into his past project. He and I vibed quickly and I liked him, and that felt like enough, but after only a week it seemed obvious that he wasn't going to be producing much code, and we let him go. Otherwise, I've been happy and my ability to discern has only gotten better as I became more experienced. I got a little offtopic, but my main answer to your question was "if the company leaves the decision up to a majority of engineers saying 'yes', then a lot of companies do this." Google does this, the startups I've worked at do this, and some of my friends companies do this.

neetfreek 7y ago

I prefer take-home assignments. I know that they often get a bad rep and have the potential for abuse; for this reason I'd argue for implementing some or other solution in the field. I recently had one requiring me to develop a native mobile application, which I enjoyed. It was interesting, the code is useful down the line, and if I don't land the job, it beefs up my portfolio. Initial screening by recruiters is tough as my background's missing a degree and industry experience. Context: self-taught, started out with game dev, tried going solo - not a runaway success. Looking to move away from the field.

nimrody 7y ago

One company sent me (before the interview) a small technical assignment. After I had submitted the code, an interview was scheduled. The entire interview was an extended code review -- talking about trade-offs, about other potential solutions, etc. I felt this was much better in that it was less stressful, yet allowed me to demonstrate both knowledge and design skills. Another company did something similar but more thorough: They invite candidates for a full day of work where they try to solve a small problem. Then the code is reviewed and evaluated together. They also start the day with a 1-hour overview of their current architecture and you get to ask questions and talk about alternatives. I think this gives both sides a better chance of finding the right fit. I realize this is not always reasonable.

rhizome 7y ago
I've been thinking for a while that companies should create their hiring tests from bugs and/or feature requests that came from their actual software in the past. Then they can gauge the quality of the employee for their purposes by comparing the candidate's solution(s) to those the actual employees wrote.
- nimrody 7y ago
  Bugs I have seen in actual code are divided into two types: those that are due to the way code inter-operates with other code (sort of integration bugs) and tricky bugs (usually edge cases) in some highly dense code. The first type requires the candidate to learn a lot about the existing code base. The second requires more mental work but is atypical of real work. Implementing features sounds like a good idea. Perhaps create a small well defined system which is a simplification of the real world system and let the candidate implement some additional features.

RijilV 7y ago

> Everyone has the same canned answers to the stupid behavioral questions. The art of behavioral questions isn’t “ask and answer” it’s the follow up questions. As you astutely point out, the questions are ‘stupid’. They might as well be “do you want a stick of gum?” Next time you ask those questions do a couple of things. First keep in the forefront of your thoughts what information you’re trying to get out of it and keep the candidate on track answering your data point. Do that by relentlessly asking follow up questions. When you think you have everything ask more. As an anecdote I was being shadowed during an on-site recently. I asked some arbitrary ‘dumb’ behavioral question, went back and forth a bit, wasn’t getting much out of it. I noticed my shadow clearly moving on to the next question in their notes and decided to keep pushing on the original question - why did you do this, what were you trying to solve, what motivated you. Turns out the candidate did all of this to generate new revenue for the company and ended up bringing in $10m a year extra at the small company there currently worked for. Loads of great data, would have never gotten there if I’d settled for the canned answer the candidate had. Behavioral questions aren’t comp-sci trivial questions, you can’t just ask the behavioral equivalent to fizbuz/Fibonacci/floodfill and copy down the answer (and you should never be asking those questions either, but that’s a separate rant). Behavioral questions are stupid and to some degree that’s the point. When you ask your significant other or kids “how was your day?” — guess what, that’s a stupid question too. What matters is what follows from your line of interviewing. If you want to get good at behavioral questions listen to Fresh Air and try to be like Terry Gross.

gaoshan 7y ago

For the code portion of things I stick to a set of increasingly difficult "real world" problems. The first one should be easily answerable by any potential candidate, the last one should be too hard for most people but I'm really looking at how they problem solve and handle themselves. I've recommended people for hiring who eventually gave up on that last one (and who went on to be fantastic in their jobs).

winrid 7y ago

Phone screens should prevent those candidates from ever getting onsite. You'd be surprised (or maybe not now) how many applicants for a senior frontend position can't build a progress bar for the phone screem.

shagie 7y ago
I've given phone screens to individuals who turned out to be a different person when they showed up onsite. Not "nice on phone, jerk in real life" but rather "Bob does the phone screen for Charlie (and passes), Charlie shows up at the onsite."
- farah7 7y ago
  I usually have video off during phone screens. Do you require them on?

watwut 7y ago

I like the for loop code thing. Imo, good hiring process for hiring programmers must involve small piece of code. Not difficult or algorithmic, but something to distinguish those who can't do anything at all.

danmaz74 7y ago

I use basically the same process, but use a very simple programming problem at the end - simple, but amenable to discussing optimization and edge cases. I let the candidate choose the language (or just use pseudo code) and don't care at all about the syntax. I find that this is very useful especially when interviewing juniors who don't have many projects under their belt for the first part. It's also useful when a candidate has good verbalisation skills, but poor programming ones (which happens).

underwater 7y ago

> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. It's possible that you asked the question poorly, or the solution wasn't as obvious as you thought. Designing interview questions is hard[1]. I'll test out new questions on my peers at least two or three times before putting them in front of a candidate. And many don't make the cut. If a good engineer who's relaxed can't solve it easily, then a stressed out candidate will have no hope. [1] This is why I hate seeing candidates share specific questions online. As an interviewer you'll have to scrub a good question, and switch to something you're not as familiar with. This hurts good candidates.

greenyouse 7y ago

I've been through a few different types of interviews and the random algorithm style seems off to me. I recognize that there are companies like FAANGs that want a deep bench and will have expert algorithm people on-call for that one time when it's needed. Most companies should be OK to just present real problems either solved or being worked on which they would expect applicants to be able to solve. This is more true for startups that need less know "how to reverse a binary string" and more "how to properly design a database with 3rd order normalization", etc. If you're presenting an interview question, it should have actual job relevance and your co-workers should be able to solve it in the same time as the candidate. If you're drilling people on non-job qualities (e.g. invert a binary tree for a web dev role...) then you should expect a large difference between audience that can pass your bad interview tests and audience which will perform well at actual job. Not trying to complain. I think job interviews should focus more on the 99% of what you do in your job on a Tuesday. Poor interviews seem to be more gotchas and algorithm tricks to disqualify roles which have little actual use for algos + data structures. The tests might seem too easy but as a front-end engineer I would rather be with a coworker which understands the CSS box model, knows semantic markup for accessibility, and similar web things than a person which is good at creating hash tables and doubly-linked lists in JS. Leet code probably doesn't test vertical centering techniques with CSS but if you're applying for a web dev position you better know them.

dustfinger 7y ago

> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. > Is this kind of code problem too complicated in your opinion? May you please post the problem so that we can provide you with meaningful feedback regarding said problem?

rapht 7y ago

I don't work in tech (though I sometimes think about uuyýjmq

alkonaut 7y ago

If you risk having a bullshitter you need some way of identifying them. The best way is through references, personal projects etc. But if you need to recruit someone without any such credentials then you may need to do a simple coding aptitude test. Could be a code review or a simple excercise but whatever you do, don’t do whiteboard coding and don’t have people recite/implement memorized CS textbook algorithms. Anyone can do that and still no code.

dustfinger 7y ago

> Everyone has the same canned answers to the stupid behavioral questions. If the person conducting the interview thinks the behavioral questions are stupid, then perhaps they are. In that case, don't ask "stupid" behavioral questions. > Resumes are meaningless, and often re-written by recruiters to match the job anyway Was the position entry level? Students coming right out of compsci often have little to no practicable experience. They may have difficulty thinking about what to put in their resume. After one or two years of full time experience that should no longer be an issue. > For all I join in when complaining about irrelevant algorithmic questions, I have to admit that they at least test something, even if it's just willingness to study for the interview. Asking those "stupid" behavioural questions and receiving the same canned answers also demonstrates a willingness to study for an interview. The coding problem should be testing a candidate's problem solving capabilities as practicably required by the role being interviewed for. The chosen problem should reflect the types of problems that they will actually need to solve if hired. For example, you could select a small PR from one of projects being actively developed by the company. The selected PR should involve only one or two classes (assuming a language with classes) and require improvement. You can look through the history of a PR and just pull out a segment that was selected for improvement by the reviewer(s), or have the team select it for you. Then ask the candidate: - to conduct a code review of the PR - to improve the code

detaro 7y ago

> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. How did you run these? While it sounds like something that should be ok even on paper, you can vary comfort a lot through the medium. E.g. for the last interview I had with a substantial coding part, I think being able to do it on my personal machine made a big difference. (I obviously was told before what kind of environment I'd need to have ready)

dccoolgai 7y ago

1) Resume screen 2) 30-minute coderpad/codeshare exercise on a problem/pattern you actually use/encounter during the course of your work over phone. (No inverting binary trees). Expect a 20% pass rate here. 3) Reasonable take-home problem that you've timed 2 of your own staff completing well in 50 minutes. This is where you will get the most complaints from applicants, but that's OK. Let them select themselves out of the process. Expect a 30-40% pass rate here. 4) In-person interview. At this point, you should be mostly committed to hiring the candidate. Do a couple livecoding deals, but be extremely lenient in how you interpret results. Other week, we had a candidate fail the problem, but they kept their composure and showed they knew what they were doing on the way to bombing the problem. Candidate seemed sad at the end of the interview and happily surprised when we extended an offer. IMHO this process works fairly well and does a good job of being economical with people's time.

mjfisher 7y ago
I'm curious about the codepad/codeshare approach right at the first touch point. I fully agree that screening actual tech skill early on is important. Do you not find that you commit a lot of engineer time to codeshare interviews that don't work out further down the line?
- dccoolgai 7y ago
  Codeshare portion is 1 (typically senior) engineer for 30 minutes. Not that bad on a large enough team with everyone taking turns. I should add that we have 3-4 "canned" questions that the HR rep writes down the answer to, so even in the "resume screen" there are a few super-lightweight technical questions.
  - mjfisher 7y ago
    Interesting. I've got a feeling early stages of tech hiring pipelines are very poorly optimized at the minute; it's something I'm working on. Individual experiences help; thank you.
- dccoolgai 7y ago

toupeira 7y ago

At my current position for a remote job, one of the interview assignments was reviewing a pull request for a very basic example app using their tech stack, and then implementing my own suggestions during a video call (while sharing my screen), updating/running tests, discussing trade-offs etc. I think this was a great way to not only verify coding ability, but also testing team work and communication skills.

bufferoverflow 7y ago

I have a similar experience doing interviews in NYC. We started by asking algo questions, but quickly found out that 95% of the candidates can't answer them. My boss, an ex-programmer, was surprised too, and she asked me to dumb them down, a lot. We ended up with a bunch of really trivial stuff like "write a function to reverse a string" or its slightly harder version - "reverse an integer". Could you give the exact wording of your "nested for-loop" question?

patientplatypus 7y ago

This is surprisingly easy, but you have to understand what the problem is first or the solution won't be apparent. If this sounds patronizing it comes about from my frustration on being on the other side of the table too often. What you want is very good football (soccer) players. Unfortunately you (or upper management) may not know all of the rules to soccer. You may not know the training regime that goes into winning a good soccer game - and it's a big risk spending money training for the big game on the off-season only to lose during the Big Show. So what do you do to test potential candidates? You see if you can get along with them, if they're a team player, and then see how well they play foosball. It's perfect! There are soccer players on the field, there's a goal, it takes skill and coordination. But oh no, it turns out in the population at large really good soccer players really sort of suck at foosball. After all, they'd rather spend their time and energy playing soccer. So now they spend all their time reading up on books about foosball and what the best foosball strategies are. You see where I'm going with this? No one uses algorithms in their day jobs. OK a few of you, but come on man, I make full stack web applications. As do most programmers. So why are you testing theory that has literally nothing to do with the job? Somehow someone thought this was a proxy for smart people, but I mean, if the guy who wrote this "Tech Interview Handbook" was really smart wouldn't he have spent his time writing a cool program? I mean how lame is this? If you want to hire competent engineers tell them what you're building and ask them how they feel is best for them to show they're competent. If you have a big data pipeline in Scala ask if they can construct a data pipeline example that is cool over a few days (take home) or do something similar in the office. Some people like one, some like the other. But just communicate with the people you want to hire! And if not everyone's interview process is the same then maybe that's ok. I mean I just got a guy who sent me some automated code interview program that had a timer that counted down from an hour at the top! WHO THINKS TREATING THEIR POTENTIAL EMPLOYEES LIKE STAR WARS DRONES IS A GOOD THING? All you have to do is treat the people you want to work with like you would want to be treated and demand that they know their shit. This. Isn't. That. Hard.

traverseda 7y ago

> And as for the code, we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. Could it be that you're having issues communicating the problem?

wan23 7y ago

The way most fields that require some level of knowledge or ability handle this is by having industry-standard assessments that everyone takes. When doctors interview they get almost entirely behavioral questions because the hiring hospital / office merely needs to check that the doctor is a) licensed and b) certified in whatever specialty they are being hired for. They certainly never get asked whatever the medical equivalent of FizzBuzz might be.

Jach 7y ago

Seems like your problem is basic competency? Move the "can you even code?" question to as early in the filter stage as possible (first 'phone' screening). If you have a lot of applicants, you'll have to do some earlier filtering in the name of time (like on degrees, years experience, "the lucky half"), but don't pretend it's fair or very accurate since both the false positive and false negative rates will be high. I agree that some coding problem needs to be used to try and answer the question, though with the right interviewer they can answer it without seeing code. The problems you use for that don't have to be at octree-collision-detection whatever challenge, a trivial nested for-loop is fine -- fizzbuzz level is fine. Sometimes you can rely on github or strong internal referral to skip this, but watch out, and anyway it's worth giving your questions to people you're sure will do fine (you've timed at least yourself right?) for the benchmark data and because sometimes they don't do fine, perhaps since maybe your question is too much. e.g. Floyd-Warshall can be done simply with a few nested loops, still I would never give it as a problem and I'd expect nearly everyone I've worked with to flunk it given only the standard hour (which really means 45 minutes). Some jobs only need basic competence, so you might want to extend an offer if you've been convinced of its presence. At my last job, which ended up being more technically challenging / interesting than my current job, I was hired after posting my resume to Craigslist which led to exchanging some emails and having lunch with the startup founder to talk about my past work and whether I would be useful for his most pressing work. At my current job, I've been part of on-sites where I've established "can you even code?" is "no". Those were costly failures of not having that answered earlier. But we also like to believe we need more than basic competence, so rejections can still occur because of a lack of "testing mindset" or certain "behavioral answers". Only once you fix your "can you even code?" filter is it even worth considering what else you might want to justify an interview pipeline with more stages than a 'phone' screen or lunch conversation.

mjfisher 7y ago
I couldn't agree more with the idea that you should move a 'can you code at all' test to as early as possible in your hiring pipeline. I used to wait to the first in-person interview to try simple fizzbuzz style questions (with the candidates on a machine and a compiler/interpreter). In about a third of cases that meant we'd committed a significant chunk of time to engineers that apparently couldn't solve trivial problems. Now it's one of the first things I check. Done right, it's a relatively small hurdle for capable people to overcome, but really helps as a filter for those who aren't suited to the role. I recently created a service (https://candidatecode.com https://candidatecode.com) to help companies manage issuing and reviewing their coding challenges; I think it's got real potential to help some people out.

ummonk 7y ago

I don’t think the problem lies with your interview if they can’t write a simple nested for loop... Which I think is why a lot of startups locate in the expensive Bay Area - not a lot of cities have a similar concentration of decent talent.

modzu 7y ago

what is everyone smoking? youre still trying to optimize for a test and not a job. dont give them a fizbuz. give them an example of a real problem your engineers need / are trying to solve. how do they respond? thoughts / intuitions / pseudo code. do they show knowledge of the problem space / domain? ot if you just want a kid who can code and they pass the fizbuz but fail at the real job, what does your training/culture look like? who does that really reflect on? it seems to me that interviewing is terribly cargo cult. the problem is real, the practises ostensibly supposed to be solutions are not. /end rant

daphneokeefe 7y ago

> we included what we thought was a trivial nested for-loop problem and virtually nobody could even get started on it. If no one can answer the question, maybe they don't understand it? Maybe there's something unclear in the way it is worded?

wcunning 7y ago

I'm getting up the gumption to look for a different, hopefully better job in the nearer future (in automotive control software, looking to move to AV), and I had a bit of a revelation when talking to a friend of mine who does interviewing. I cannot talk about the things in my current job that would make me a good hire for the things I want to move into. It's all hunting down an obscure bug buried in layers of technical debt and overly complicated standards, but giving any depth beyond that bare platitude requires going into things that my NDA covers. I even work in driver assist technologies, but I can't go into detail about that because I'm working on unreleased, unannounced features. That means that the only thing left is side projects or whiteboard interviews. To be fair, this is at least partly an industry problem due to long product development cycles and a culture of secrecy, but it makes a lot of the better solutions to interviewing unworkable and causes companies to drop back to whiteboard interviews. And as a candidate, my personal maximizing function is to hit the books and be ready for curly braces and logic puzzles. Sometimes there really isn't a better way.

mixmastamyk 7y ago
“I’m under NDA” is a valid response. Have some hobby project or floss bug fix on github if there is time to fill.

davidjnelson 7y ago

What works for me is to consider what work the engineer will be doing, and ask them to write code to prove they can do it. For instance, a front end engineer can be expected to be able to write a to do list or similar app in a framework (ideally the one your team uses, but not a hard no hire if not) app with minimal googling (although that’s fine as long as not excessive) in ~45 minutes. Then you have to look at what level of experience they have. Less experience requires more mentoring generally, which may be fine depending on how much time your team budgets for that work. Lastly, measure their body language and tone of voice to check for red flags pointing to difficult communication styles or people who treat others poorly. If all three match, hire!

Aperocky 7y ago

This. I loath writing algorithm on whiteboard, especially the 'catchy' type. But I've interviewed people who can't even write a for loop.. the amount of brain drain in the flyover country is insane.