24 ms·

A framework for grading your performance on programming interview problems

251 points by danielgh7 5y ago

hehetrthrthrjn 5y ago
Performative programming comes full circle.
Aeolun 5y ago
This looks great for leetcode, it’s a bit ruined by my considering that a terrible interview technique in the first place. It can still be good for practice though, it does seem to cover all bases.
- namdnay 5y ago
  > terrible interview technique in the first place. I'm really not a big fan of whiteboard tests, but I have yet to find an initial filtering method that is as quick and cheap as 30mn with coderpad or whatever.
  - odyssey7 5y ago
    Do you find that it’s a good filter as well as being quick and cheap? Couldn’t tell if this was meant to be ironic.
    - namdnay 5y ago
      As a test of bare minimum coding skills, definitely. You wouldn't believe the number of people who apply for junior software engineering positions yet can't do an easy leetcode problem. That's like a medical intern not knowing how to install an IV I don't go further than that (no complexity questions, no silly google-style "let's pretend you haven't learnt this by heart" knapsack problems). And I don't care about syntax. It's just a way to know who is a developer and who is a stackoverflow cut-and-paster
      - latortuga 5y ago
        I am surprised this needs to be said but the medical intern analogy is terrible and emblematic of why this style of interview is terrible. Medical interns will likely be doing things like installing an IV on a daily basis. It is a great work sample task. Your garden variety web developer will not be building algorithms from scratch virtually ever. This is not a valid work sample task.
        atleta 5y ago
        Depends on what you call an algorithm, of course. As GP mentioned bare minimum coding skills, I assume he uses really simple ones. This can, of course be relative, but maybe he was intentional with the analogy. Because you can't really code without writing algorithms. Yes, you are never write a quick sort or implement a red-black tree, but you do have to understand problems and provide some kind of solution to them and then translate that into code. I.e. come up with an algorithm. I used to use two simple problems, both involved math. One of them is a bit harder and the ideal solution assumes that you do remember high school math (not very complicated but something that you may have forgotten or at least something that you won't think should be used there). But it's nice, because you can come up with more and more optimal solutions and the naive one (which is O(n^2), IIRC, really just shows that you managed to understand the problem). I don't use that one anymore though. I almost exclusively just use the one that I thought was embarrassingly simple and which I used to use as a "warm up" exercise. Not more complicated than the fizz-buzz (which I hadn't heard of at the time I picked this one). I'm not telling you here what it is, but it indeed does weed out people who can't really think through a simple problem and then can't think of a solution that consists of several steps following each other. (I.e. an algorithm.) Now I'm not interviewing 'top' developers, just your average web devs most of the time. Another thing I'm doing in the past few years is code review. I have 2-3 pieces of badly written code that I have reviewed and/or rewritten.
        rolfea 5y ago
        I think the problem with your thinking, and something that I've been trying to figure out as I sit on/direct more interviews myself, is that this doesn't actually tell you that the candidate can't code, but just that they couldn't code in that setting with that problem in the way you chose to frame it for the interview. Meaning, I often sit with candidates who come in with 3-5 years experience in industry who can't do a relatively simple programming task. That can either mean that we consistently are drawing applicants who have been pulling the wool over the eyes of their past employers for 1/2 a decade, but actually can't program. Or, it could mean that they get incredibly nervous coding in front of strangers, aren't used to doing more abstract problems, and once they realized this, experiencing a "snow ball effect" of nerves, wherein they train wreck in the interview.
    - IshKebab 5y ago
      Yes it's definitely good for filtering out the people who are very bad.
      - Aeolun 5y ago
        For a certain definition of bad, for sure.
        IshKebab 5y ago
        For any reasonable definition of bad. Unless you're happy unwittingly hiring people who can't even write a for loop.
    - recursive 5y ago
      Not the OP, but absolutely yes to all those things, except being ironic.
- kache_ 5y ago
  git gud
lmilcin 5y ago
I think it is focusing on the wrong parts. My framework for the coding part of the interview is following: * Does the candidate tackle the problem analytically? Being able to think analytically is critical to scaling from solving small problems to solving large problems. * Can the candidate predict what his code is going to do before he executes it? Repetitively executing code to see if it works may seem innocuous, but really is a huge defect in candidate's process which makes it really difficult or impossible for them to write anything non trivial to be reliable. * Can the candidate recognize and make use of an advice? Surprisingly, a lot of people can't recognize when they are given advice even when I directly give them solution to their problem. It is really disheartening to have a candidate dismiss your advice even when repeated multiple times. I frequently give advice to unstuck the candidate because seeing him/her work on the problem is more important to me than seeing if they can solve any particular part of the problem. * Can the candidate recognize choices/tradeoffs that impact performance and/or simplicity of the solution? * Can I have productive discussion with the candidate about designing his solution? Can the candidate converse in a clear and precise language? * Is he/she caring that the resulting code is readable? * Can they diagnose faults in their code effectively? Assuming they executed their solution and it is not doing what it was expected to do, I like to see the candidate follow some kind of analytical process to find out what is causing the problem. I even design the task specifically so that it is very likely they are going to make one particular mistake to give me that opportunity. ** What I specifically ignore in the coding task: * Whether the candidate knows the standard library. I don't care. During normal work they can look it up on google. On the interview I tell them that if they can describe the function I can tell them the name and how to use it. * Whether the candidate can have flashes of insight. I don't get so many candidates to waste them on that kind high false negative signal. I have identified parts of the task that require a flash of insight. I give the candidate a minute or two when they reach it and if I don't see progress I give them a progressively less subtle nudges. * Whether the candidate can do TDD/DDD/etc. It all depends on their experience. If they worked for non-TDD or non-DDD shop for a long time it is going to take some time to adjust. * Whether the candidate can solve a very complex problem. The interview is already stressful enough experience for the candidate. I can learn what I need on a simple problem and I care much more to see the candidate's process rather than results. The problem is just hard enough to be out of grasp of most people at the very beginning and require some actual thinking. Some people get it instantly and then I am disappointed a little bit because I can't see them working on it.
helen___keller 5y ago
I do find it a little ironic that as an industry we started doing these interviews as a "least common denominator" - from senior engineers to fresh college grads, everyone occasionally needs to do a search or a sort or a lookup table, and people should recognize when "loop in a loop" n^2 solutions aren't ideal. And we tuned it from there to into its own sort of mini-SAT with an accompanying preparatory industry and culture. I find it as a reminder of my fungibility as an engineer: I get paid absurd sums because of supply and demand, not because of my skills; anybody who can game the leetcode exam (and the architectural diagram quiz and the personality quiz) can walk in and take the same 300k income I have. If enough people do so, wages could be pushed down by employers. Thankfully for my mortgage, that hasn't happened yet and probably won't happen soon.
- Jensson 5y ago
  > If enough people do so, wages could be pushed down by employers. Thankfully for my mortgage, that hasn't happened yet and probably won't happen soon. You know that already happened? Like 15 years ago was what it looked like before every started practicing, today most people are practicing. If they fail to take your job today then it is because they failed to pass the interviews even with extensive practice.
  - helen___keller 5y ago
    Right now engineering demand is far outstripping supply. If the tides shift and big tech starts tightening their belt, leetcode practitioners will start competing 100 engineers for 10 positions at big tech, and that's when salaries get pushed down. Most books and blogs on professional growth say to have some sort of talent or skill that makes an organization need you. That way, you aren't fungible and have leverage. Right now my "skill" that I demonstrate is leetcode and the only leverage I have is that big tech needs more engineers and have to bid against each other for my fungible work. The moment tides shift, myself and every other leetcode practitioner are basically fucked. When will this happen? Ask a fortune teller.
    - Jensson 5y ago
999900000999 5y ago
I really really liked Interviewing.io for practice. You take a mock interview, white boarding and all. I'm actually very very bad at these interviews. That said, I've also been on the other side of the table. One candidate didn't actually know how to program. I had to argue with almost everyone else in the company to not hire this person, and on top of that I got reprimanded for being condescending. Don't put a programming language on your resume if you've never actually used it. I do wish American employment law was a bit more flexible here, I'd prefer to give someone a small paid assignment and if they're able to complete that great. Right now you need to decide if you're going to hire someone within about 30 minutes of knowing them.
b20000 5y ago
i already went to college, and worked hard to get my CS degree. i don’t do and should not be doing these interviews, and everyone else should do the same. it is the only way to stop this.
- benlivengood 5y ago
  You worked hard but the rich kid next to you in class paid someone to do their homework and steal test keys and maybe even bribed a professor or two. Companies need to filter them out.
  - b20000 5y ago
    you can just ask for projects they worked on and look at their code and discuss that. also, you can’t bribe anyone at universities where i’m from.
    - Jensson 5y ago
      Or you can ask them to code up some algorithms. The companies paying the best choose to do this, and they grew and became much larger than the companies that didn't do it this way, so now most companies do it this way.
      - b20000 5y ago
        leetcode BS interviews are a quite recent fad.
        pedrosorio 5y ago
        Yeah, if you’re old enough to consider Microsoft doing them in the 90s a recent fad, sure.
    - tester34 5y ago
      you'd be shocked how much bullshit occurs on universities across the world, even in EU.
omginternets 5y ago
How does one prove runtime complexity? I would love to know how to do this, just out of interest.
- ipnon 5y ago
  Skiena's algorithm book goes into some detail on proving complexity and proving optimality.
  - omginternets 5y ago
    Thank you. Are you by any chance aware of a more introductory text on the subject?
    - ipnon 5y ago
      Believe it or not Skiena's book is an introduction to algorithms! His lectures for the book are available on YouTube.[0] [0] https://youtube.com/playlist?list=PLOtl7M3yp-DX6ic0HGT0PUX_wiNmkWkXx https://youtube.com/playlist?list=PLOtl7M3yp-DX6ic0HGT0PUX_w...
      - omginternets 5y ago
        Oh, perfect! Video lectures FTW! Thank you once again :)
- lpapez 5y ago
  Usually on interviews it is not actual proving in any mathematical sort of way. It's more like: "I am calling sort() in a loop so obviously this is O(n^2 * log n) complexity" and then the interviewer nods in approval and you move on to the next question.
- danielgh7 5y ago
  I personally like to go line by line in my solution and state the runtime of each line of code. Then I add them together, and write out the limit notation showing as N->inf and how I can cancel certain variables out. Its a little bit of overkill but I've found it helps me stand out a bit during interviews.
qorrect 5y ago
Why did you link the html view now we can't clone it. https://docs.google.com/spreadsheets/d/1gy9cmPwNhZvola7kqnfY3DElk7PYrz2ARpaCODTp8Go/ https://docs.google.com/spreadsheets/d/1gy9cmPwNhZvola7kqnfY... for the lazy
agomez314 5y ago
A pattern I've noticed in the comments is how leetcode "commoditizes" engineers. To which some counter with an argument against this idea. It's clear that companies seek engineers from all around the world to help them in their business. Because they're casting such a wide net, some kind of cheap, quick filter is required to sort out a large segment of the population. What if the initial filter is physical proximity to the company headquarters? What if the companies look for engineers first in their surrounding areas, and provide opportunities for those even without the skills initially set on the job opportunity? My hunch is that, if a person is dedicated, he/she will work hard and acquire the necessary tasks required for the job. The surrounding communities will benefit from the gains by the business, and leetcode-like tests would be unnecessary for the majority of job postings. Curious to hear what others think about this "local-first" approach.
- whimsicalism 5y ago
  Why should people with geographic proximity be disproportionately granted opportunity? Like I fail to see the motivation for doing this outside of base nativism/tribalism.
- curiousllama 5y ago
  I know of a F500 company that takes this approach in IT. They’re constantly understaffed, despite being a legit good place to work, with fair compensation. People tend to leave once they’re trained - it’s pretty natural, given they’re smart, motivated engineers and the company just doesn’t need to solve very interesting tech problems. The company does it for effectively altruistic reasons - they want to help bring tech talking to their declining city.
- chillacy 5y ago
  Software companies tend to be very scalable because they can get their customers from anywhere in the world. Unless the company was itself hyper local (which I’m sure exists but I’m struggling to think of an example) why would they handicap themselves for little benefit?
tristor 5y ago
I think that LC style interviews are an attempt to solve for the fact that programming is a field where a few things happen to be true and broadly true: 1. Significant amounts of people cheated during college, and if asked probably wouldn't even identify their behavior as cheating. 2. Significant amounts of competent people don't even have a degree. 3. Many people with degrees are incapable of actually programming even basic algorithms. 4. There is an incredibly high demand for this job because of its pay, leading many people to pursue this job in the market that have no actual interest or aptitude for the work. 5. Significant amounts of people lie on their resume, or work for body shops that lie on their behalf with or without their knowledge. These truths make it so that something like a degree is not a useful filter and create a situation where every job openings has thousands of applicants, the majority of which are not even remotely qualified. How do you filter out applicants rapidly to narrow down the number of people that your engineers take time away from producing value for the company to interview? Personally, like every other engineer, I hate LC-style interviews, but at the same time, the issues they are intended to solve seem intractable any other way. Fundamentally the LC-interview is intended to be an on-the-spot test of your abilities because nothing on your resume or in your past credentials is trustworthy.
- guessmyname 5y ago
  LC stands for LeetCode → https://leetcode.com/ https://leetcode.com/ in case anyone is wondering. I find it funny how the parent comment is so long but didn’t want to include the full name of the website haha
- deleted 5y ago
  [deleted]
- game_the0ry 5y ago
  I disagree with a lot you have to say, even though you have valid concerns with are shared with a lot of hiring managers I talk to. > 1. Significant amounts of people cheated during college, and if asked probably wouldn't even identify their behavior as cheating. Unless you have some real data backing this up, you can't claim this is true. > 2. Significant amounts of competent people don't even have a degree. Don't disagree here, though the question of whether or not leetcode proves competency for day-to-day software engineering (of which changes given the type of job you are interviewing for) is still debatable. > 3. Many people with degrees are incapable of actually programming even basic algorithms. I have seen this phenomenon and it is a valid concern. However, you can still test for basic algorithms without resorting to leetcode hard questions on phone screens, which happens often (i am speaking from experience). > 4. There is an incredibly high demand for this job because of its pay, leading many people to pursue this job in the market that have no actual interest or aptitude for the work. Agreed, sort of. Its supply and demand - where the supply is artificially kept low at the interview stage be eliminating too many candidates who can't solve leetcode hards. Many of those candidates could be adequate, especially with minimal training. A lot of your concerns are 1st order, surface level, but the solutions to the problem of tech interviews are a couple layers deeper in the stack. Leetcode is a monkey patch, an adequate one but it has its own set of problems that are ignored. That's why we keep seeing discussions about leetcode on HN.
everdrive 5y ago
I'm surprised that no one has suggested the obvious: Yes, tech interviews are flawed in the literal sense: they don't measure genuine or practical programming ability. However, it's illegal for companies to require IQ tests to screen employee hiring, and these sorts of interviews seem like a pretty clear stand-in for an IQ test. It's doesn't really measure your ability to be a programmer, but act as a good-enough replacement for an IQ test: The problems are hard and complex, and require that someone does actually spend the time to learn the problems.
- luis8 5y ago
  Does it really measure IQ? Memorizing the most common algorithms and data structures will eventually land you a good job if you try several times. For example: Binary search Tree traversals Graph searches Sort algorithms Interesting Trees(Trie, RedBlack, etc) Etc many other common data structures or algorithms If you memorize the implementation of those. You only need to make sure you understood the problem and check for edge cases. What I have seen is that a lot of people struggle in the interview because they lose a lot of time trying to implement the algorithms but sometimes know how to get there
- bradlys 5y ago
  It’s not a stand in for an IQ test. If you study the problems, you get a better score. Therefore, not an IQ test.
  - patatino 5y ago
    You can study IQ tests too
xyzelement 5y ago
Whenever a topic like this comes up, people inevitably talk about broken FAANG hiring and the stupidity of whiteboard programing (aka Leetcode.) I often worry that people who say this are somewhat missing the point of what interviewing is about, and thus lower both their own chances of getting more & better offers, and worse, if someone else is influenced by their writing, their odds go down too. I say this as a hiring manager and someone responsible for company-wide engineering recruitment for a long time, not at a FAANG but a company of that caliber who competed w. FAANGs for talent. From that perspective, whiteboard programing provide a meaningful signal (though not the only signal) towards a hiring decision. When I am hiring someone, I care about many things: how do they understand and solve problems, how do they prioritize and trade-off solutions, how do they collaborate and communicate, how they react when they are "stumped", how they react to coaching/hints/feedback/etc. And yes, whether they can code well, and how they deal with pressure. So when I used to go through one of these problems with a candidate, I would form a picture of how they'd interact with their product manager (from the way they asked questions and clarified their understanding of their problem), how they'd interact with their technical team (in the way they explain their thinking and evolve their solution in response to feedback) to how they'd be with me/their manager (from the way they took feedback and reacted to being coached/helped) to how they would be in an incident/outage (based on whether they were able to make progress despite the time pressure of the interview) and ultimately what their code looked like in my repo (from... the way their code looked like.) Yes obviously there are factors that make people perform differently in an interview than at the job and as an experienced interviewer you try to discern that, but the bigger point is if you are ranting against whiteboard coding interviews, but you don't understand what I just wrote, then you are basically arguing from a point of view of ignorance. If you don't understand what the process is meant to do, then of course it seems stupid to you but that's not a problem with the process... I can also look at it from an ever more "real" perspective. Companies hire you to solve problems. To do that, you need the smarts to understand what it takes to solve the problem and the drive and persistence to actually solve it. So if the problem in front of you is "I want to be hired by Google", are you able to recognize that part of the solution is "I need to get good at whiteboard programming" and that a part of that is research and disciplined practice? If not, then you're sort of failing at the meta-interview, because your very approach to problem solving falls short of the level required to get these jobs. By the way, there are plenty of great places to work that hire on other criteria, and what those places expect from programmers day to day is different to some extent than FAANG-caliber. That's totally fine to be working there, I just wish people had the self-awareness to know what FAANGS want and how they screen for it, vs just saying "oh I am not good at that but it's OK cuz it's stupid to begin with."
decebalus1 5y ago
I fail to understand why this content was published as a Google spreadsheet. Or as a spreadsheet in general.
- oneepic 5y ago
  Because it's a rubric? It fits a tabular format.
  - decebalus1 5y ago
    Sure, but plain html supports that. Heck, you can TSV a pastebin.
oneepic 5y ago
I just finished my interview process with a couple FAANGs and a few other companies, and was a little surprised by how "memorizable" it was, depending on the company. I'm not talking about the topics but rather the process -- ie you might follow a framework for coding interviews like 1) ask clarifying questions, 2) run thru an example, 3) state brute force, 4) solve it optimally (hopefully), etc. It felt like some sort of bougie "fashion" (or hazing ritual like everyone likes to call it) that candidates needed to follow in order to land the job. Even for system design I felt the same trend was there.
- curiousllama 5y ago
  Also recently went through a bunch of tech interviews, and had a similar experience. Felt the exact same as doing consulting case interviews in college. I have my script, I recite the relevant variation for the problem, and I pass the interview. It’s great for me that I can almost always pass, but I do feel bad for people who never had a chance to just learn the magic words.
ipnon 5y ago
To all the naysayers, who proudly proclaim their professional pedigree and computer science degrees, but crash and burn through these interviews in vain: if you thought your Macbook Pro was a good investment, if you thought your 4 year degree was worth the time and effort, wait until you discover the returns from spending less than $1,000 USD on preparing for software engineering industry interviews. Swallow your pride and start getting paid!
brailsafe 5y ago
This does seem a little irrelevant unless you're just practicing analog whiteboarding. Almost every company I've interviewed for just buys some inane off-the-shelf Hacker Rank thing and sends it out to the whole world, followed by "I guess now we'll talk".
mlengineerio 5y ago
As someone who wrote about interview prep (mlengineer.io), I encounter people hate this LC-non-sense quite often. There are few things come in mind. 1. When you apply to STEM graduate program, you also need to take GRE. It's clearly that you don't need those GRE absurd words in your study. But I do see some values in people who dedicate their time to learn these GRE materials. Company might see it the same way, they want to hire grinder. 2. Some companies like Intuit, Airbnb try different method, for example, giving candidate 90 minutes to solve one particular engineering problem related to real world. After a while, people share the problem sets and everyone who have prepared can crush them easily. 3. Some of my ex-coworkers who are competent and have good swe skills but they never bother with LC. as a result they stick with underpaid jobs for years. I strongly believe in the near future more companies will adapt different interview structure to evaluate candidates. The current structure heavily reward for grinders.
kenjackz 5y ago
Even though you can perfect this grading rubric, it doesn't guarantee that you will get the job in an actual programming interview.

251 points by danielgh7 5y ago

Performative programming comes full circle.

This looks great for leetcode, it’s a bit ruined by my considering that a terrible interview technique in the first place. It can still be good for practice though, it does seem to cover all bases.

namdnay 5y ago
> terrible interview technique in the first place. I'm really not a big fan of whiteboard tests, but I have yet to find an initial filtering method that is as quick and cheap as 30mn with coderpad or whatever.
- odyssey7 5y ago
  Do you find that it’s a good filter as well as being quick and cheap? Couldn’t tell if this was meant to be ironic.
  - namdnay 5y ago
    As a test of bare minimum coding skills, definitely. You wouldn't believe the number of people who apply for junior software engineering positions yet can't do an easy leetcode problem. That's like a medical intern not knowing how to install an IV I don't go further than that (no complexity questions, no silly google-style "let's pretend you haven't learnt this by heart" knapsack problems). And I don't care about syntax. It's just a way to know who is a developer and who is a stackoverflow cut-and-paster
    - latortuga 5y ago
      I am surprised this needs to be said but the medical intern analogy is terrible and emblematic of why this style of interview is terrible. Medical interns will likely be doing things like installing an IV on a daily basis. It is a great work sample task. Your garden variety web developer will not be building algorithms from scratch virtually ever. This is not a valid work sample task.
      - atleta 5y ago
        Depends on what you call an algorithm, of course. As GP mentioned bare minimum coding skills, I assume he uses really simple ones. This can, of course be relative, but maybe he was intentional with the analogy. Because you can't really code without writing algorithms. Yes, you are never write a quick sort or implement a red-black tree, but you do have to understand problems and provide some kind of solution to them and then translate that into code. I.e. come up with an algorithm. I used to use two simple problems, both involved math. One of them is a bit harder and the ideal solution assumes that you do remember high school math (not very complicated but something that you may have forgotten or at least something that you won't think should be used there). But it's nice, because you can come up with more and more optimal solutions and the naive one (which is O(n^2), IIRC, really just shows that you managed to understand the problem). I don't use that one anymore though. I almost exclusively just use the one that I thought was embarrassingly simple and which I used to use as a "warm up" exercise. Not more complicated than the fizz-buzz (which I hadn't heard of at the time I picked this one). I'm not telling you here what it is, but it indeed does weed out people who can't really think through a simple problem and then can't think of a solution that consists of several steps following each other. (I.e. an algorithm.) Now I'm not interviewing 'top' developers, just your average web devs most of the time. Another thing I'm doing in the past few years is code review. I have 2-3 pieces of badly written code that I have reviewed and/or rewritten.
        rolfea 5y ago
        I think the problem with your thinking, and something that I've been trying to figure out as I sit on/direct more interviews myself, is that this doesn't actually tell you that the candidate can't code, but just that they couldn't code in that setting with that problem in the way you chose to frame it for the interview. Meaning, I often sit with candidates who come in with 3-5 years experience in industry who can't do a relatively simple programming task. That can either mean that we consistently are drawing applicants who have been pulling the wool over the eyes of their past employers for 1/2 a decade, but actually can't program. Or, it could mean that they get incredibly nervous coding in front of strangers, aren't used to doing more abstract problems, and once they realized this, experiencing a "snow ball effect" of nerves, wherein they train wreck in the interview.
  - IshKebab 5y ago
    Yes it's definitely good for filtering out the people who are very bad.
    - Aeolun 5y ago
      For a certain definition of bad, for sure.
      - IshKebab 5y ago
        For any reasonable definition of bad. Unless you're happy unwittingly hiring people who can't even write a for loop.
  - recursive 5y ago
    Not the OP, but absolutely yes to all those things, except being ironic.
kache_ 5y ago
git gud

lmilcin 5y ago

I think it is focusing on the wrong parts. My framework for the coding part of the interview is following: * Does the candidate tackle the problem analytically? Being able to think analytically is critical to scaling from solving small problems to solving large problems. * Can the candidate predict what his code is going to do before he executes it? Repetitively executing code to see if it works may seem innocuous, but really is a huge defect in candidate's process which makes it really difficult or impossible for them to write anything non trivial to be reliable. * Can the candidate recognize and make use of an advice? Surprisingly, a lot of people can't recognize when they are given advice even when I directly give them solution to their problem. It is really disheartening to have a candidate dismiss your advice even when repeated multiple times. I frequently give advice to unstuck the candidate because seeing him/her work on the problem is more important to me than seeing if they can solve any particular part of the problem. * Can the candidate recognize choices/tradeoffs that impact performance and/or simplicity of the solution? * Can I have productive discussion with the candidate about designing his solution? Can the candidate converse in a clear and precise language? * Is he/she caring that the resulting code is readable? * Can they diagnose faults in their code effectively? Assuming they executed their solution and it is not doing what it was expected to do, I like to see the candidate follow some kind of analytical process to find out what is causing the problem. I even design the task specifically so that it is very likely they are going to make one particular mistake to give me that opportunity. ** What I specifically ignore in the coding task: * Whether the candidate knows the standard library. I don't care. During normal work they can look it up on google. On the interview I tell them that if they can describe the function I can tell them the name and how to use it. * Whether the candidate can have flashes of insight. I don't get so many candidates to waste them on that kind high false negative signal. I have identified parts of the task that require a flash of insight. I give the candidate a minute or two when they reach it and if I don't see progress I give them a progressively less subtle nudges. * Whether the candidate can do TDD/DDD/etc. It all depends on their experience. If they worked for non-TDD or non-DDD shop for a long time it is going to take some time to adjust. * Whether the candidate can solve a very complex problem. The interview is already stressful enough experience for the candidate. I can learn what I need on a simple problem and I care much more to see the candidate's process rather than results. The problem is just hard enough to be out of grasp of most people at the very beginning and require some actual thinking. Some people get it instantly and then I am disappointed a little bit because I can't see them working on it.

helen___keller 5y ago

I do find it a little ironic that as an industry we started doing these interviews as a "least common denominator" - from senior engineers to fresh college grads, everyone occasionally needs to do a search or a sort or a lookup table, and people should recognize when "loop in a loop" n^2 solutions aren't ideal. And we tuned it from there to into its own sort of mini-SAT with an accompanying preparatory industry and culture. I find it as a reminder of my fungibility as an engineer: I get paid absurd sums because of supply and demand, not because of my skills; anybody who can game the leetcode exam (and the architectural diagram quiz and the personality quiz) can walk in and take the same 300k income I have. If enough people do so, wages could be pushed down by employers. Thankfully for my mortgage, that hasn't happened yet and probably won't happen soon.

Jensson 5y ago
> If enough people do so, wages could be pushed down by employers. Thankfully for my mortgage, that hasn't happened yet and probably won't happen soon. You know that already happened? Like 15 years ago was what it looked like before every started practicing, today most people are practicing. If they fail to take your job today then it is because they failed to pass the interviews even with extensive practice.
- helen___keller 5y ago
  Right now engineering demand is far outstripping supply. If the tides shift and big tech starts tightening their belt, leetcode practitioners will start competing 100 engineers for 10 positions at big tech, and that's when salaries get pushed down. Most books and blogs on professional growth say to have some sort of talent or skill that makes an organization need you. That way, you aren't fungible and have leverage. Right now my "skill" that I demonstrate is leetcode and the only leverage I have is that big tech needs more engineers and have to bid against each other for my fungible work. The moment tides shift, myself and every other leetcode practitioner are basically fucked. When will this happen? Ask a fortune teller.
  - Jensson 5y ago

999900000999 5y ago

I really really liked Interviewing.io for practice. You take a mock interview, white boarding and all. I'm actually very very bad at these interviews. That said, I've also been on the other side of the table. One candidate didn't actually know how to program. I had to argue with almost everyone else in the company to not hire this person, and on top of that I got reprimanded for being condescending. Don't put a programming language on your resume if you've never actually used it. I do wish American employment law was a bit more flexible here, I'd prefer to give someone a small paid assignment and if they're able to complete that great. Right now you need to decide if you're going to hire someone within about 30 minutes of knowing them.

b20000 5y ago

i already went to college, and worked hard to get my CS degree. i don’t do and should not be doing these interviews, and everyone else should do the same. it is the only way to stop this.

benlivengood 5y ago
You worked hard but the rich kid next to you in class paid someone to do their homework and steal test keys and maybe even bribed a professor or two. Companies need to filter them out.
- b20000 5y ago
  you can just ask for projects they worked on and look at their code and discuss that. also, you can’t bribe anyone at universities where i’m from.
  - Jensson 5y ago
    Or you can ask them to code up some algorithms. The companies paying the best choose to do this, and they grew and became much larger than the companies that didn't do it this way, so now most companies do it this way.
    - b20000 5y ago
      leetcode BS interviews are a quite recent fad.
      - pedrosorio 5y ago
        Yeah, if you’re old enough to consider Microsoft doing them in the 90s a recent fad, sure.
  - tester34 5y ago
    you'd be shocked how much bullshit occurs on universities across the world, even in EU.

omginternets 5y ago

How does one prove runtime complexity? I would love to know how to do this, just out of interest.

ipnon 5y ago
Skiena's algorithm book goes into some detail on proving complexity and proving optimality.
- omginternets 5y ago
  Thank you. Are you by any chance aware of a more introductory text on the subject?
  - ipnon 5y ago
    Believe it or not Skiena's book is an introduction to algorithms! His lectures for the book are available on YouTube.[0] [0] https://youtube.com/playlist?list=PLOtl7M3yp-DX6ic0HGT0PUX_wiNmkWkXx https://youtube.com/playlist?list=PLOtl7M3yp-DX6ic0HGT0PUX_w...
    - omginternets 5y ago
      Oh, perfect! Video lectures FTW! Thank you once again :)
lpapez 5y ago
Usually on interviews it is not actual proving in any mathematical sort of way. It's more like: "I am calling sort() in a loop so obviously this is O(n^2 * log n) complexity" and then the interviewer nods in approval and you move on to the next question.
danielgh7 5y ago
I personally like to go line by line in my solution and state the runtime of each line of code. Then I add them together, and write out the limit notation showing as N->inf and how I can cancel certain variables out. Its a little bit of overkill but I've found it helps me stand out a bit during interviews.

qorrect 5y ago

Why did you link the html view now we can't clone it. https://docs.google.com/spreadsheets/d/1gy9cmPwNhZvola7kqnfY3DElk7PYrz2ARpaCODTp8Go/ https://docs.google.com/spreadsheets/d/1gy9cmPwNhZvola7kqnfY... for the lazy

agomez314 5y ago

A pattern I've noticed in the comments is how leetcode "commoditizes" engineers. To which some counter with an argument against this idea. It's clear that companies seek engineers from all around the world to help them in their business. Because they're casting such a wide net, some kind of cheap, quick filter is required to sort out a large segment of the population. What if the initial filter is physical proximity to the company headquarters? What if the companies look for engineers first in their surrounding areas, and provide opportunities for those even without the skills initially set on the job opportunity? My hunch is that, if a person is dedicated, he/she will work hard and acquire the necessary tasks required for the job. The surrounding communities will benefit from the gains by the business, and leetcode-like tests would be unnecessary for the majority of job postings. Curious to hear what others think about this "local-first" approach.

whimsicalism 5y ago
Why should people with geographic proximity be disproportionately granted opportunity? Like I fail to see the motivation for doing this outside of base nativism/tribalism.
curiousllama 5y ago
I know of a F500 company that takes this approach in IT. They’re constantly understaffed, despite being a legit good place to work, with fair compensation. People tend to leave once they’re trained - it’s pretty natural, given they’re smart, motivated engineers and the company just doesn’t need to solve very interesting tech problems. The company does it for effectively altruistic reasons - they want to help bring tech talking to their declining city.
chillacy 5y ago
Software companies tend to be very scalable because they can get their customers from anywhere in the world. Unless the company was itself hyper local (which I’m sure exists but I’m struggling to think of an example) why would they handicap themselves for little benefit?

tristor 5y ago

I think that LC style interviews are an attempt to solve for the fact that programming is a field where a few things happen to be true and broadly true: 1. Significant amounts of people cheated during college, and if asked probably wouldn't even identify their behavior as cheating. 2. Significant amounts of competent people don't even have a degree. 3. Many people with degrees are incapable of actually programming even basic algorithms. 4. There is an incredibly high demand for this job because of its pay, leading many people to pursue this job in the market that have no actual interest or aptitude for the work. 5. Significant amounts of people lie on their resume, or work for body shops that lie on their behalf with or without their knowledge. These truths make it so that something like a degree is not a useful filter and create a situation where every job openings has thousands of applicants, the majority of which are not even remotely qualified. How do you filter out applicants rapidly to narrow down the number of people that your engineers take time away from producing value for the company to interview? Personally, like every other engineer, I hate LC-style interviews, but at the same time, the issues they are intended to solve seem intractable any other way. Fundamentally the LC-interview is intended to be an on-the-spot test of your abilities because nothing on your resume or in your past credentials is trustworthy.

guessmyname 5y ago
LC stands for LeetCode → https://leetcode.com/ https://leetcode.com/ in case anyone is wondering. I find it funny how the parent comment is so long but didn’t want to include the full name of the website haha
deleted 5y ago
[deleted]
game_the0ry 5y ago
I disagree with a lot you have to say, even though you have valid concerns with are shared with a lot of hiring managers I talk to. > 1. Significant amounts of people cheated during college, and if asked probably wouldn't even identify their behavior as cheating. Unless you have some real data backing this up, you can't claim this is true. > 2. Significant amounts of competent people don't even have a degree. Don't disagree here, though the question of whether or not leetcode proves competency for day-to-day software engineering (of which changes given the type of job you are interviewing for) is still debatable. > 3. Many people with degrees are incapable of actually programming even basic algorithms. I have seen this phenomenon and it is a valid concern. However, you can still test for basic algorithms without resorting to leetcode hard questions on phone screens, which happens often (i am speaking from experience). > 4. There is an incredibly high demand for this job because of its pay, leading many people to pursue this job in the market that have no actual interest or aptitude for the work. Agreed, sort of. Its supply and demand - where the supply is artificially kept low at the interview stage be eliminating too many candidates who can't solve leetcode hards. Many of those candidates could be adequate, especially with minimal training. A lot of your concerns are 1st order, surface level, but the solutions to the problem of tech interviews are a couple layers deeper in the stack. Leetcode is a monkey patch, an adequate one but it has its own set of problems that are ignored. That's why we keep seeing discussions about leetcode on HN.

everdrive 5y ago

I'm surprised that no one has suggested the obvious: Yes, tech interviews are flawed in the literal sense: they don't measure genuine or practical programming ability. However, it's illegal for companies to require IQ tests to screen employee hiring, and these sorts of interviews seem like a pretty clear stand-in for an IQ test. It's doesn't really measure your ability to be a programmer, but act as a good-enough replacement for an IQ test: The problems are hard and complex, and require that someone does actually spend the time to learn the problems.

luis8 5y ago
Does it really measure IQ? Memorizing the most common algorithms and data structures will eventually land you a good job if you try several times. For example: Binary search Tree traversals Graph searches Sort algorithms Interesting Trees(Trie, RedBlack, etc) Etc many other common data structures or algorithms If you memorize the implementation of those. You only need to make sure you understood the problem and check for edge cases. What I have seen is that a lot of people struggle in the interview because they lose a lot of time trying to implement the algorithms but sometimes know how to get there
bradlys 5y ago
It’s not a stand in for an IQ test. If you study the problems, you get a better score. Therefore, not an IQ test.
- patatino 5y ago
  You can study IQ tests too

xyzelement 5y ago

Whenever a topic like this comes up, people inevitably talk about broken FAANG hiring and the stupidity of whiteboard programing (aka Leetcode.) I often worry that people who say this are somewhat missing the point of what interviewing is about, and thus lower both their own chances of getting more & better offers, and worse, if someone else is influenced by their writing, their odds go down too. I say this as a hiring manager and someone responsible for company-wide engineering recruitment for a long time, not at a FAANG but a company of that caliber who competed w. FAANGs for talent. From that perspective, whiteboard programing provide a meaningful signal (though not the only signal) towards a hiring decision. When I am hiring someone, I care about many things: how do they understand and solve problems, how do they prioritize and trade-off solutions, how do they collaborate and communicate, how they react when they are "stumped", how they react to coaching/hints/feedback/etc. And yes, whether they can code well, and how they deal with pressure. So when I used to go through one of these problems with a candidate, I would form a picture of how they'd interact with their product manager (from the way they asked questions and clarified their understanding of their problem), how they'd interact with their technical team (in the way they explain their thinking and evolve their solution in response to feedback) to how they'd be with me/their manager (from the way they took feedback and reacted to being coached/helped) to how they would be in an incident/outage (based on whether they were able to make progress despite the time pressure of the interview) and ultimately what their code looked like in my repo (from... the way their code looked like.) Yes obviously there are factors that make people perform differently in an interview than at the job and as an experienced interviewer you try to discern that, but the bigger point is if you are ranting against whiteboard coding interviews, but you don't understand what I just wrote, then you are basically arguing from a point of view of ignorance. If you don't understand what the process is meant to do, then of course it seems stupid to you but that's not a problem with the process... I can also look at it from an ever more "real" perspective. Companies hire you to solve problems. To do that, you need the smarts to understand what it takes to solve the problem and the drive and persistence to actually solve it. So if the problem in front of you is "I want to be hired by Google", are you able to recognize that part of the solution is "I need to get good at whiteboard programming" and that a part of that is research and disciplined practice? If not, then you're sort of failing at the meta-interview, because your very approach to problem solving falls short of the level required to get these jobs. By the way, there are plenty of great places to work that hire on other criteria, and what those places expect from programmers day to day is different to some extent than FAANG-caliber. That's totally fine to be working there, I just wish people had the self-awareness to know what FAANGS want and how they screen for it, vs just saying "oh I am not good at that but it's OK cuz it's stupid to begin with."

decebalus1 5y ago

I fail to understand why this content was published as a Google spreadsheet. Or as a spreadsheet in general.

oneepic 5y ago
Because it's a rubric? It fits a tabular format.
- decebalus1 5y ago
  Sure, but plain html supports that. Heck, you can TSV a pastebin.

oneepic 5y ago

I just finished my interview process with a couple FAANGs and a few other companies, and was a little surprised by how "memorizable" it was, depending on the company. I'm not talking about the topics but rather the process -- ie you might follow a framework for coding interviews like 1) ask clarifying questions, 2) run thru an example, 3) state brute force, 4) solve it optimally (hopefully), etc. It felt like some sort of bougie "fashion" (or hazing ritual like everyone likes to call it) that candidates needed to follow in order to land the job. Even for system design I felt the same trend was there.

curiousllama 5y ago
Also recently went through a bunch of tech interviews, and had a similar experience. Felt the exact same as doing consulting case interviews in college. I have my script, I recite the relevant variation for the problem, and I pass the interview. It’s great for me that I can almost always pass, but I do feel bad for people who never had a chance to just learn the magic words.

ipnon 5y ago

To all the naysayers, who proudly proclaim their professional pedigree and computer science degrees, but crash and burn through these interviews in vain: if you thought your Macbook Pro was a good investment, if you thought your 4 year degree was worth the time and effort, wait until you discover the returns from spending less than $1,000 USD on preparing for software engineering industry interviews. Swallow your pride and start getting paid!

brailsafe 5y ago

This does seem a little irrelevant unless you're just practicing analog whiteboarding. Almost every company I've interviewed for just buys some inane off-the-shelf Hacker Rank thing and sends it out to the whole world, followed by "I guess now we'll talk".

mlengineerio 5y ago

As someone who wrote about interview prep (mlengineer.io), I encounter people hate this LC-non-sense quite often. There are few things come in mind. 1. When you apply to STEM graduate program, you also need to take GRE. It's clearly that you don't need those GRE absurd words in your study. But I do see some values in people who dedicate their time to learn these GRE materials. Company might see it the same way, they want to hire grinder. 2. Some companies like Intuit, Airbnb try different method, for example, giving candidate 90 minutes to solve one particular engineering problem related to real world. After a while, people share the problem sets and everyone who have prepared can crush them easily. 3. Some of my ex-coworkers who are competent and have good swe skills but they never bother with LC. as a result they stick with underpaid jobs for years. I strongly believe in the near future more companies will adapt different interview structure to evaluate candidates. The current structure heavily reward for grinders.

kenjackz 5y ago

Even though you can perfect this grading rubric, it doesn't guarantee that you will get the job in an actual programming interview.