21 ms·

I understand the frustration, but at the end of the day I think they do serve a purpose. They're not great at that purpose, but they're good enough and they gen

• hiAndrewQuinn 2y ago
  Bingo. A good hire is a windfall, a bad hire can be an existential threat to the team or the entire company.
  • KaiserPro 2y ago
    > a bad hire can be an existential threat to the team or the entire company. Totally, but leetcode wont filter them out. Incompetent is fairly easy to navigate around, pathological bastardness is impossible to work past. I know that some companies use psychometric tests, but they are not based in anything remotely scientific, but are also super easy to game.
    • Aurornis 2y ago
      > Totally, but leetcode wont filter them out. Coding challenges won’t filter every pathological bad hire, but coding challenges do catch a lot of bad hires. LeetCode shouldn’t be the entirety of the interview. It’s just part of it.
      • KaiserPro 2y ago
        > but coding challenges do catch a lot of bad hires. leetcode doesn't catch bad hires, as someone whos done a lot of interviews, we could easily catch "bad" coders with a very simple whiteboard test or a "look at this code and tell me what you think" but people being bad at code aren't the company killers, its the toxic people who are "good" at coding that do. They can be very good at these kind of tests, but are absolute shites to work with. I think we actually agree more that I let on, and you are right that coding tests should only be a part of the interview process.
        • stephenblum 2y ago
          You are right, there's not enough time to filter a bad hire. Not enough time to fully vet anyone during an interview. We can work a challenge together to see how you work collaboratively
  • beej71 2y ago
    But you hired them at-will, right? The bad hires I've seen that are existential threats were only there because management refused to fire them.
    • FabHK 2y ago
      Depends on the jurisdiction, needless to say. I've interviewed in London, say.
    • hiAndrewQuinn 2y ago
      "At-will", he says to the expat to Finland. I wish. It's nearly impossible for me, or any FTE I know here, to get fired. (To be fair, I have never to my knowledge been considered 'a bad hire' by anyone except Home Depot.) But in fact even in at-will situations, firing aversion seems to be a real and expensive issue that all kinds of managers in organizations face [1] [2]. So, in one sense I agree, bad hires are existential threats "only" because we don't fire them fast enough. On the other hand, have you ever had to fire someone yourself? Try it. It's heart-wrenching, even to a heartless utility monster like yours truly. [1]: https://www.econlib.org/archives/2012/06/firing_aversion_2.html https://www.econlib.org/archives/2012/06/firing_aversion_2.h... [2]: https://www.econlib.org/firing-and-the-left/ https://www.econlib.org/firing-and-the-left/
      • beej71 2y ago
        I've never had to fire a bad employee. So every time has been awful. But I would have gleefully fired the bad ones had I the authority to do so. :)
• fzeroracer 2y ago
  How do you know they produce false negatives over false positives? How do you know if your top candidates simply didn't cheat over the honest candidates that produced worse but honest code?
  • deleted 2y ago
    [deleted]
  • mvdtnz 2y ago
    How do you cheat at live coding?
    • ljm 2y ago
      I’ve interviewed people who wore an earpiece and were fed instructions on the fly, and also people who kept their cameras turned off to try and hide the fact they were getting assistance. You started picking up on patterns like them repeating your questions, or spending a lot of time in silence (asking their friend a question) without any other communication or activity on the screen. Like they weren’t there.
      • shitlord 2y ago
        I once interviewed someone who was on a call with someone but messed up their audio feed. I could hear the third person but the person I interviewed could not.
        • mrbombastic 2y ago
          That is hilarious
    • lokhura 2y ago
      Nowadays it's easy by using undetectable interviewing AI tools. See: - https://ultracode.ai https://ultracode.ai - https://interviewsolver.com https://interviewsolver.com Well worth it. Highly recommend these tools to play the dumb interviewing game.
• kristopolous 2y ago
  There's an entire industry of bootcamps built on the premise of teaching you how to pass this exact test in like 6 weeks. And because they ask pretty irrelevant questions, the 6 weeker will do better then the recent Stanford PhD who has been studying a specific problem for 5 years or the multimillionaire who spent the past 10 years building and selling successful software companies. Those people will fail and your 6 weeker who's never heard of cmake, diff or gdb will pass. When I see there's leetcode as the hiring process I presume the place is full of bozos because the process actually strongly preferences them.
  • alexey-salmin 2y ago
    I believe this is a common myth. I've spend around seven years teaching programming and people who can go from zero to decent problem solving in just six weeks are virtually non-existent. Either it wasn't a true "zero" but rather they were really good e.g. at advanced math or physics and managed to see some parallels, or it has to be a genius. I'd be very interested in hiring such a person.
    • kristopolous 2y ago
      Leet code is not "decent problem solving skills" but it's almost always about three types of programs: sliding window, linked list rearrangement, and BFS. The code to solve basically all of them look nearly identical after you classify it and the questions asked (what's the big O of this) are also all identical. It's an extremely narrow and teachable class of problems. There's outliers for sure but the 80th percentile are basically just the same damn things slightly rephrased. It's extremely gameable and doesn't test the kind of breadth say, fixing a crashing program would show. Here's a list of 16 https://github.com/Chanda-Abdul/Several-Coding-Patterns-for-Solving-Data-Structures-and-Algorithms-Problems-during-Interviews https://github.com/Chanda-Abdul/Several-Coding-Patterns-for-... ... That's the 99th percentile. If you think someone passing that means they can fix the conda bug in your k8s cluster or whatever real problem you have, dream on. It'd be like hiring a cook by quizzing them on the names of kitchen utensils.
      • 2y ago
• watwut 2y ago
  They do produce plenty of false positives - just not "you hire an idiot" kind. You will hire a person completely unsuitable for the actual job, but is good at leet. Issue is not the lack of intelligence. Issue is not being software engineer despite being good at algorithms. Or not being good at whatever your position requires. I do work in a team where majority was hired by puzzles of sorts. All of them are smart. They are not software engineers and it shows massively.
  • resource_waste 2y ago
    >They are not software engineers and it shows massively. Coming from Real Engineering and making 2x more programming, I lol at people who say there is any Engineering in software. We are programmers dealing with layers upon layers of abstraction. Knowing to optimize for time by using vectors is more of an art, than a science. I did do safety critical C code and Assembly which are probably my hole in the whole 'its not engineering'. But javascript/python/backend work: programming.
    • lupusreal 2y ago
      The fact that in 99% of cases "software engineer" is just an aggrandizing title given by management to any and every programmer at their company should put a bullet in the head of the pretentions of this being an engineering field.
      • ljm 2y ago
        The term is used ubiquitously in the field of technology and has been for a long time. We also have Tech or Software Architects, and quite obviously they don't do Real Architecture™ or civil engineering, but the concept makes sense. It's just a job title, arguably one without the protections or standards afforded to chartered engineers, and with an incredibly low barrier to entry. Pretty much the reason we have to do this ridiculous dance with tech tests, LeetCode, and 12 stage interview processes.
        • lupusreal 2y ago
• logicchains 2y ago
  Hard leetcode questions hire grinders, not thinkers. Many of those questions were thesis/publication-worthy decades ago when first solved; it's unlikely for even a very intelligent person to produce a similar result in just tens of minutes. So if someone solves it, it's much more likely to be because they saw it or a similar question while grinding practice questions than that they genuinely derived the optimal solution from scratch.
  • concordDance 2y ago
    I disagree with this. I can solve the majority of leetcode hard problems in under an hour despite not having seen them before and I'm far from the smartest person I've met.
    • batshit_beaver 2y ago
      Congratulations, you've memorized leetcode patterns. Proves nothing beyond that (except that you've also had the time recently to practice leetcode).
      • concordDance 2y ago
        I have never practiced leetcode puzzles or memorised solutions to them. I don't understand why you jumped to this conclusion.
        • kirth_gersen 2y ago
          Probably because you said you "can solve the majority of leetcode hard problems in under an hour". That could easily give the impression that you have practiced them. If you haven't practiced them, then how do you know you can solve them?
          • concordDance 2y ago
            Sample of 5-6 I tried for fun. Along with a handful of similar puzzles from other places.
• drewcoo 2y ago
  If, after all of the normal interviews, you randomly selected candidates who passed and do not make them offers . . . you generally produce false negatives (assuming the interviews mostly worked). But it's entirely pointless.
• NhanH 2y ago
  > They're not great at that purpose, but they're good enough and they generally produce false negatives (smart people fail) rather than false positives (you hire an idiot). I've seen this reasoning, but I'm wondering if this is actually true, especially if you also believe the other myth in hiring: most people applying are not qualified for the job. Since most hiring actually requires you to fill the position in the end (in other words, you can't leave it open indefinitely if no one qualifies as eventually the pressure to hire mount up), every false negatives does increase the odd of you hiring a false positives as well. Just take the example at the extreme, if there is only ever one person on Earth qualified for your job and you got a false negatives on him, there is no other way but getting an idiot in the job.
  • breckenedge 2y ago
    In my years of hiring experience, it doesn't hold up. I've hired folks who could leet code all day long, but couldn't develop a feature from scratch, or figure things out without enormous hand holding. I want to hire problem solvers who also know how to code. I stopped doing leet-code interviews about 7 years ago. By all means have people code in interviews, but don’t do leet code.
    • FabHK 2y ago
      How do you test whether they're problem solvers that also know how to code, without leet-code interviews?
      • breckenedge 2y ago
        Giving people real world problems and pairing with them on finding a solution.
        • rented_mule 2y ago
          I'm retired now, but I used a variant of this that worked well for me. In my experience, much of what we do as engineers is learn new tools and application domains and apply what we learn to solve problems. So I wanted to test how good a candidate is at leaning and whether they appear to enjoy it (good and enjoy often correlate). We would work together at the whiteboard, with me teaching them the basics of our application domain and our internal distributed computation framework. Then there were a series of 5 increasingly difficult iterations we'd work through. First, do a simple calculation. Now there's a need for a more difficult computation. And an even harder one. Then explain that it needs to run over billions of log lines, and it's I/O bound, so let's make it faster. Finally, a problem that requires them to probe me deeply about the nature of the data and only needs a hand-wavy answer (calculate a median in a single pass with limited memory - not too bad once you extract from me that the samples are all integers between 0 and 500). To the candidate, this looked like iteration driven by evolving requirements, not 5 interview questions. I mostly let them drive, but I jump in now and then to keep things on pace. And I would try to answer any question the same way I would as a teammate / mentor. They don't need to get every problem right. The increasing difficulty is designed so that almost everyone gets something right and something wrong (< 1% gets all the way through the last one). That's an opportunity for me to give supportive feedback to see how they respond. The best responses to feedback yielded something like "oh cool, I can also use that over here to simplify the code". The worst was someone insisting that you can flip the order of addition and division without changing the answer (i.e., order of operations doesn't matter), me saying "I think it does matter, try doing 1/2 + 1/4 in different orders", and them saying "no I have a math degree, I know what I'm talking about." What they do need to do is be able to solve problems collaboratively, extract learning from an eager mentor, and extrapolate from those learnings to solve problems. The best candidates start finding trade-offs across the application domain / tech tooling boundary that I haven't even brought up. One of the downsides is that it takes a while to calibrate as an interviewer, and that calibration is not entirely transferrable across interviewers, as candidates will respond differently to different interviewers. I did this interview thousands of times (my record was 14 times in one day, and frequently 20+ times a week - we were scaling an org from ~50 engineers to over 1,000). I never stopped learning about signals I was seeing, but I was mostly calibrated after a few dozen candidates. Leet code calibration is much more transferrable, but what use is that if it's not measuring what matters? One thing that I think leet code interviews get very wrong is that interviews involve two sides trying to evaluate each other. Wearing people down with leet code is not a good pitch for your work environment. Many candidates told me at the end of my interview that they were shocked at how much fun/educational an interview could be, and many hires said that this interview style was the reason they accepted our offer over more lucrative offers, because they felt they would learn more and have more fun with us.
• grishka 2y ago
  Okay, do do quizzes, but let people google goddamn stuff because that's just how software engineers operate. Don't make it an exam. Don't mimic the worst thing about the education systems. Both in the school and in the university, these goddamn exams were the worst because they tested memory first and everyone else second, and I'm such a kind of person that I could never remember things on command. It was a real struggle. I can't be the only one. I've never been through the "regular" IT hiring process myself, but I've interviewed several candidates for an Android developer position. I liked asking questions and looking at how the person thinks. I didn't expect exact correct answers for every question, I just wanted to see whether they have a fundamental understanding of Android. After that, they had to build a small app at home (not my initiative) to show their skills.
  • throwaway2037 2y ago
    I like this idea as the more advanced version of whiteboard programming from the year 2000. Even if you still do whiteboard programming, give them a PC / tablet, and allow them to do some Google searches. Then, you could even make the problem a bit harder to solve. Again: You are looking at the candidate's problem solving process holistically. I also very much agree with your second paragraph. Some of the most pleasant surprises I had during interviews were candidates who had very limited domain knowledge, but tried as hard to possible to make educated guesses, or ask intelligent questions. It shows a lot about a candidate.
  • diehunde 2y ago
    What are you going to google though? For most of these LC questions you only need to know how to use arrays, maps and loops. If I'm interviewing you and you tell me you need to google how to append something to an array or check if the element is in a map then I'll think you just haven't written enough code yet. There are better questions where I do think google should be allowed but they aren't LC-style. Some examples are building APIs or operating on files
• gumby 2y ago
  At all the companies I’ve run we’ve used a simple whiteboard example for this reason. Don’t worry about typos; how does the person think. “Trick” questions are dumb, but you want to get an idea of if the candidate knows their stuff (and let the candidate know what we’re like — interviewing is a sales process in both directions). If someone asks a question like “is it ok to modify the argument” (or says “I’ll assume I can’t modify the argument”) that’s great (or says “I can’t remember if strlen includes the 0 byte so I’ll add 1 — normally I’d look it up”) - again, no trick questions. When I say “simple” it’s something like atoi or “shuffle a deck of cards”: basic, but not 100% trivial like fizzbuzz. On of our best hires was a guy who made a fundamental mistake — when we asked him if it world work as intended he said “I’m a fucking idiot” and fixed it. You’re not supposed to talk like that in an interview I supposed but we read it as a strong positive signal. We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time.
  • lumb63 2y ago
    I have come to like this approach reasonably well. My current company uses a similarly simple problem to evaluate candidates. For me, it was a breeze, and I thought we’d be hiring slouches left and right. Once I started administering the interview, I realized that the majority of candidates absolutely bomb this simple question for one reason or another. It seems that “able to code a simple problem well” is a far less ridiculous proxy for good software engineer than “able to code a hard problem at all”. Much to my surprise.
    • Frost1x 2y ago
      “Simple problems” often aren’t as “simple” as the retrospectively sound if you consider everything like stress level, mixed mindset of the interviewee (they’re not just thinking about a programming problem, they’re also thinking about social queues, mixing conversing in, peer judgement, and loads of other anxiety inducing factors), etc. The software industry is notorious for underestimating overall complexity factors, understanding uncertainty, and time to manage them and interviews seem to be no different. So “simple” problems are often good (I agree with you) because they’re actually not so simple for most people when you factor everything in, they’re often reasonably challenging under the set of circumstances. Meanwhile genuinely difficult problems (in any arbitrary setting and longer timelines) require a whole lot of rote training and reusing little clever idiosyncratic techniques that may or may not be generally useful, so you very often end up with a question lottery on whether you already know how to address the problem in question or not correctly vs having some problem where a reasonable solution and expectations around that can be derived in the time (and environment) it’s taking place. Lots of nonsense.
• johnnyanmac 2y ago
  >because a false negative is low cost (maybe you spend 50% longer interviewing candidates) this would make sense if they filled these positions in 2,3 weeks tops. But that's the insane part; I hear of interviews going on for 6-7 stages now, lasting almost 6-10 weeks. At that point I feel you are no longer avoiding false positives, but simply hiring the most desperate programmers willing to jump the hoops. The best candidates will get an offer mid interview at that rate. >But just don't! It's fine! just move on, I've been "just moving on" for some 9 months now. There's only so much morale in a tank, no matter how BS you know the whole process is. I just wanna move on in life, not be bogged down on how many permutations you can color an N x M grid with.
  • throwaway2037 2y ago
    First, I am sympathetic to your experience. It must be difficult. > I hear of interviews going on for 6-7 stages now, lasting almost 6-10 weeks. From this story, it is clear that your market is clearly in favour of the employer. One year ago, it was a different story: half the time. Another thing that will hurt: If you are amazing, that 6-10 weeks will suddenly become 1-2 weeks. In my experience, nothing hurries the interview process more than telling an employer about late round interviews (or offers) at another competitor.
• Suzuran 2y ago
  That false negative can be pretty damn expensive if the good engineer you didn't hire could have saved your ass from the situation the less-good engineer you did hire couldn't handle.
  • FabHK 2y ago
    The latter would be a false-positive, which by assumption is both expensive and rare.
• hintymad 2y ago
  The problem with Leetcode-style test nowadays is that there's too much cramming. Otherwise, leetcode-style interview has certain correlation with one's geekiness and raw talent. Just this week my team solved a long-tail performance issue by first building a simulation of a queuing system, and also built a customized Coffman–Graham algorithm to resolve the order of execution of a complex task graph. I would certainly appreciate that my team members just get it when someone mentions basic CS concepts like topological order in a graph or queuing theory.
• jandrewrogers 2y ago
  FWIW, I have come across examples in the wild of engineers who were extraordinarily polished and facile at everything leetcode, and poor at computer science. Not many, but some. This is a somewhat recent phenomenon. Best tests are simple and common computer science problems in algorithms and data structures with no tractable solutions. There is no facile answer, you truly have to reason from first principles for any particular scenario because there is no alternative.