17 ms·

AI killed the tech interview. Now what?

• joshstrange 2y ago
  AI did not kill tech interviews and anyone saying so is probably not interviewing correctly or trying to outsource the work to one of these other companies. I’ve run interviews in the last year where I told people they were free to use LLMs in the initial coding (which we don’t watch in realtime) but when they are explaining the code to us we ask them to make a change (a tiny additional feature, 2-6 lines of code depending on how they originally solved it). The people who can’t are immediately disqualified (in our heads, obviously we were cordial in the meeting). Just to note, we let them use LLMs for the change too, it doesn’t help the people who are completely relying on the LLM. You can use LLMs all you want IF, and only if, you understand what it’s outputting and can build on top of that and/or modify the code it wrote. As for the deepfake video stuff, ehh, I don’t know as I’ve never seen it “in person” (only videos of how it could look) so I’m not sure if I would notice it or not. We have multiple interviews and I’m confident that speed of answers would give away someone using AI or they would say something stupid the AI hallucinated. That confidence comes from talking to LLMs extensively. They can BS for a little bit but then it becomes clear they full of it. Also LLMs aren’t going to answer questions like “tell me about a project you worked on at a previous company” or “describe a time where you had to fix production issue and how you went about it”. I had many people trip up here and be unable to describe a past project with any level of detail. Bottom line, I ask a lot of questions a LLM cannot answer well (not on purpose, these questions were written prior to thinking about LLMs/AI and I ask for people to make a minor edit to code “they” just wrote. Lastly, the interview process is only the first step. What are these people doing after they get hired? Are they continuing to pay for AI tools to help them lie about who they are? it sure seems like their skill, or rather lack thereof, will become readily apparent even if somehow they made it past the interview process (which again I don’t understand unless you have a bad interview process).
• jarsin 2y ago
  I never understood why Big Tech never setup contracts with all the SAT and ACT test centers across the country. Even before Zoom with Codepads it would have made sense for the recruiters to send potential candidates to a test center to do a pre assessment rather than waste time with engineers sitting on prescreen calls all day.
  • neofrommatrix 2y ago
    What exactly are you suggesting here? A standardized test that applies to all your job applications? Or, a candidate having to drive to a test center for every company they apply to? Or something else?
    • jarsin 2y ago
      Standardized plus the ability for companies to do their own test after they pass the standard one. So go get prescreened at test center then use that test to apply for jobs. Company either flys you in for in-person or sends you back to test center to do live remote interview in controlled environment.
      • recursive 2y ago
        That was essentially triplebyte.
        • thaumasiotes 2y ago
          No, that was Triplebyte's marketing. Their actual product was completely different.
        • benatkin 2y ago
          That was part of it. Another part was it being socially awkward in a way that I think outsourcing testing needn't be.
      • AlexGizis 2y ago
        It's been tried and failed: Sun Microsystems pushed certifications in the 90s. Pass the test on some technology, get the certification. Then they studied performance. The result? More certifications implied a worse employee. The reason was the top performing employees had no time to study for the exams, but the managers of the bottom performing employees were happy to send them off to training and testing. And then the certification fad came mostly to an end.
• JCM9 2y ago
  So there’s AI that’s really good at doing the skills we’re hiring for. We want you to not use AI so we can hire you for a job that we’re saying we’re going to replace with AI. Sounds like a great plan.
  • trescenzi 2y ago
    The article actually takes a stance that if you incorporate AI you can learn how the person is at doing things the AI cannot.
  • kagakuninja 2y ago
    No. Using AI requires a depth of knowledge to spot the mistakes in the generated. code, and to know how to fit all the snippets of code in to something that works. We need to know that the developer actually has skills and isn't just secretly copying the answer off of a hidden screen. We are interviewing now, and some cantidates are obviously cheating. Our interview process is not leet code based, and reasonably chill, but we will probably have to completely rethink the process. Since we are hiring contractors, in theory we can let them go after a couple months if they suck, but we haven't tested out how this will work in practice.
• stevage 2y ago
  Interestingly I find AI is actually better at that kind of CS whiteboard question (implementing a binary search tree) than that "connecting middlewares to API" type task. Or at least, it's more straightforward to apply the AI, when you want a set of functions with clearly defined interfaces written - rather than making a bunch of changes across existing files.
  • ttyprintk 2y ago
    I’ve wondered about the kind of person who starts white boarding with the pros and cons of several AI offerings. As if, confronted with a problem domain, they are choosing or hiring AI before architecture. As an interviewer, how should I adapt my questions? Something like, “How would you prompt it to add fuzzing?” “How would you prompt it to describe how each change might affect our stack?”
• sulami 2y ago
  I've been arguing that "AI" has very little impact on meaningful technical interviews, that is ones that don't test for memorization of programming trivia: https://blog.sulami.xyz/posts/llm-interviews/ https://blog.sulami.xyz/posts/llm-interviews/
  • pavel_lishin 2y ago
    That link doesn't work for me.
  • OptionOfT 2y ago
    A couple of weeks ago I interviewed at a place where I had to do a take-home exercise. It's fine, I don't mind. No Leetcode. Just my own IDE, my own shortcuts, and write a piece code that solves a problem. I was asked whether I used AI/LLM for the solution. I didn't. I felt like using an LLM to solve the problem for me wasn't the right way of showcasing knowledge. The role was for some form of 'come in with knowledge and help us'. The response to that was basically: everybody here uses AI. I declined the follow-up interview, as I felt that if all you have is the speed of AI to be ahead of your competitors, you're not really building the kind of things that I want to be a part of. It basically implies that the role is up in the air as soon as the AI gets better.
    • jochem9 2y ago
      When I started coding I did it in notepad. I thought it was hardcore and cool. I was young and stupid. Then I adopted an IDE and I became much better at writing code. To me AI is just another tool that helps me solve problems with code. An auto complete on steroids. A context aware stack overflow search. Not wanting to adopt or not even work somewhere where colleagues use it, sounds to me like coding in notepad AND in the process scoffing those who use an IDE. Besides, if AI gets to the point it can replace you, it will replace you. Better to start learning how to work with it so you can fill whatever gap AI can't.
      • userbinator 2y ago
        I still mainly use a text editor after several decades, and do a lot of thinking and initial design with pencil and paper. IDEs just get in the way. I've seen the type of code AI generates. It might work, but if you think that's good or that massaging it so it works will make you any better, I have some bad news for you...
• userbinator 2y ago
  Stop remote tech interviews Unless the job you're interviewing for is remote-only, this makes perfect sense. If you expect your candidates to be able to work in your office, they should be interviewed there.
• rvz 2y ago
  Or you can just given them a way to bypass all of that, and ask them about any significant project that the candidate did build (which is relevant to the job description, open or closed source that is released) or even open source contributions towards widely used and significant projects. (Not hello world, or demo projects, or README changes.) Both scenarios are easily verifiable (can check that you released the project or if you made that commit or not) and in the case of open-source, the interviewer can lookup at how you code-review with others, and how you respond and reason about the code review comments of others all in public to see if you actually understand the patches you or another person submitted. A conversation can be started around it and eliminates 95% of frauds. If the candidate cannot answer this, then no choice but give a leetcode / hackerrank hard challenge and interview them again to explain their solution and why. A net positive to everyone and all it takes to qualify is to build something that you can point to or contribute to a significant open source project. Unlike Hackerrank which has now become a negative sum race to the bottom quest with rampant cheating thanks to LLMs. After that, a simple whiteboard challenge and that is it.
  • janalsncm 2y ago
    This would be a nice interview for candidates who have open source contributions, but many who have day jobs do not. Or their open source code is 5 years old and not representative of their current skill set.
    • OutOfHere 2y ago
      There is no shame in taking time off after leaving a job to develop or contribute to an open source project or two. The world would be a better place for it.
      • janalsncm 2y ago
        Well the reason I don’t has nothing to do with shame and everything to do with time. I’m allocating my extra time to work, which (on a good day) makes my company money. For a candidate who does have OS contributions, that’s great but most will not. And the more senior they are the less likely I would imagine.
• torben-friis 2y ago
  My last interview, for the job I'm currently employed in, asked for a take home assignment where I was allowed to use any tool I'd use regularly including AI. Similar process for a live coding interview iterating on the take home that followed. I personally used it to speed up wirting initial boilerplate and test suites. I fail to see why this wouldn't be the obvious choice. Do we disallow linters or static analysis on interviews? This is a tool and checking for skill and good practices using it makes all sense.
  • knowaveragejoe 2y ago
    As someone on the other side of the table, I don't care if you used AI to complete a take-home project. I care if you can explain the strengths and weaknesses of the approach it took, or if you chose to steer it in one direction or another. It usually becomes quite clear those who actually understand what the AI actually did for them.
• neofrommatrix 2y ago
  I've been considering using a second webcam stream focused on my screen just to assure hiring managers that I don't have ChatGPT on my screen, or anywhere else. Kind of like chess players do it sometimes on online tournaments. I've been hearing people complain about cheating a lot.
  • userbinator 2y ago
    Ask for an in-person interview instead of voluntarily encouraging more mass surveillance.
    • MathMonkeyMan 2y ago
      But what if I bring my AI butt plug?
    • neofrommatrix 2y ago
      Company wide policy, apparently.
• mixedCase 2y ago
  I've been interviewing a bunch of developers the past year or so, and this: > Architectural interviews are likely safe for a few years yet. From talking to people who have run these, it’s evident that someone is using AI. They often stop with long pauses, do not quite explain things succinctly, and do not understand the questions well enough to prompt the correct answer. As AI gets better (and faster), this will likely follow the same fate as the rest but I would give it some years yet. Completely matches my experience. I don't do leet code BS, just "let's have a talk". I ask you questions about things you tell me you know about, and things I expect of someone at the level you're selling yourself at. The longest it's taken me to detect one of these scumbags was 15 minutes, and an extra 5 minutes to make sure. Some of them make mistakes that are beyond stupid, like identity theft of someone who was born, raised and graduated in a country whose main language they cannot speak. The smartest ones either do not know when to stop answering your questions with perfect answers (they just do not know what they're supposed to not know), or fumble their delivery and end up looking like unauthentic puppets. You just keep grinding them until you catch em. I'm sure it's not infallible, but that's inherent to hiring. The only problem with this is cost, you're going to need a senior+ dev running the interview, and IME most are not happy to do so. But this might just be what the price of admission for running a hiring pipeline for software devs is nowadays. Heck, now feels like a good time to start a recruitment process outsourcing biz focused on the software industry.
  • azthecx 2y ago
    I think this approach is not very favoured by hacker news but it's also what I prefer. It's so much easier to quickly gauge a minimum level of the basic programming knowledge and other sw knowledge by just asking some simple directed questions. I once got a guy that claimed to have implemented multiple default HTTP JSON REST APIs and somehow had never: - tested his API with JSON payloads, serialise serialise - never queried his APIs manually or semi automatically (no knowledge of curl, postman or anything similar)
• deleted 2y ago
  [deleted]
• BiraIgnacio 2y ago
  I don't think it did, if anyone cares. The way I've been advocating to my colleagues who are concerned about "cheating" is that there's probably a problem with the interview process. I prefer to focus on the think, rather than the solve. Collaborate, as opposed to just do. Things that really tell me if I can work with that person and if together, we can make good things.
• ivanjermakov 2y ago
  In my uni days, I respected professors who designed exam in a way where students can utilize whatever they could to complete the assignment, including internet, their notes, calculators, etc. I think the same applies to good tech interview. Company should adapt hiring process to friend with AI, not fight.
  • esafak 2y ago
    What would you suggest?
    • ivanjermakov 2y ago
      Ask questions that are tricky to cheat, evaluate candidates by their thought process in solving a problem and ability to clarify, discuss and justify their decisions.
• ytch 2y ago
  If you are using Internet, Google, Stackoverflow at work, why insist that interviewee need to solve problems on their own?
  • userbinator 2y ago
    To test their inherent thinking skills and base knowledge. There's a huge difference between occasionally looking up something, and practically leaning on it. Ironically, the mass degradation of search engine result quality within the past ~decade has made it much harder for people to do the latter, and when they do, it shows much more clearly.
• OutOfHere 2y ago
  Show the remote candidate an AI's deficient answer to a well-asked question, and ask the candidate if they understand what exactly is wrong with the AI's assessment, or what the follow-up/rewritten prompt to the AI should be. Compile a library of such deficient chats with the AI.
  • MathMonkeyMan 2y ago
    AIs are probably good at answering such questions.
• moanadavidson 2y ago
  [dead]
• hcfman 2y ago
  Very funny :) I too failed an interview at google, also related to binary search on a white board. I never write with pens. I'm on keyboards the whole time, my handwriting is terrible. I've built a search engine for two countries and then I was failed by a guy that wears cowboy hats to work at google in Ireland. Not a lot of cows there I'm guessing. (No offence to any real cowboys that work at google of course). I did like the free flight to Ireland though and the nice lunch. Though I was disappointed I lost "Do no evil" company booklet.
  • thefz 2y ago
    TBH, not the #1 source of animal meat, though plenty of cows in Ireland.
    • hcfman 2y ago
      Good for google. Plenty of passable interview candidates then. Dang! I knew it was a mistake leaving my hat at home. Little things like that people tend to forget.
• hcfman 2y ago
  Don't forget to wear your cowboy hat when interviewing at google. Very important.
• twelve40 2y ago
  Prediction: faangs will come up with something clever or random or just fly everyone onsite, they are so rich and popular, they can filter by any arbitrary criteria. Second-rate companies will keep some superficial coding, but will start to emphasize more of the verbal parts like system design and retrospective. Which sucks, because those are totally subjective and mostly filters for whoever can BS better on the spot and/or cater to the interviewer's mood and biases better. My favorite still: in-person pair programming for a realistic problem (could be made-up or shortened, but similar to the real ones on the job). Use whatever tools you want, but get the correct requirements and then explain what you just did, and why. A shorter/easier task is to code review/critique a chunk of code, could even just print it out if in person.
  • nprateem 2y ago
    It's not that hard. Just ask them to explain the code. Then ask them how they'd change it for several different scenarios.
    • Gormo 2y ago
      I've taken this approach, and found that it's trivially easy to distinguish people relying on LLMs from people who have thought the problem through and can explain their own decision-making process. I had a couple of people who, when asked to explain specific approaches reflected in their code, very obviously typed my question right back into ChatGPT and then recited its output verbatim. Those interviews came to an end rather quickly. One of my favorite ones was when I asked a candidate to estimate the complexity of their solution, and ChatGPT got it wrong, giving O(log(n)) for an O(n) algorithm. When I asked leading questions to see if the candidate could see where the error came in, they starting verbatim reciting a dictionary definition of computational complexity, and could not address the specifics of the problem at all.
• micheles 2y ago
  Nowadays I am on the other part of the fence, I am the interviewer. We are not a FAANG, so we just use a SANE interview process. Single interview, we ask the candidate about his CV and what his expectations are, what are his competences and we ask him to show us some code he has written. That's all. The process is fast and extremely effective. You can discriminate week candidates in minutes.
  • FirmwareBurner 2y ago
    >we ask him to show us some code he has written How do you expect them to get access to the property internal Git repo codebase and approval from their employer's lawyers to show it to third parties during the interview? Sounds like you're only selecting Foss devs and nothing more.
    • ramon156 2y ago
      Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show. If that's the case however, just let them make a small project over the weekend and then do another interview where you ask stuff about what they've made. It's not that deep
      • solumos 2y ago
        I don't code much outside of work. I have hobby projects from 10+ years ago, but they're not much more than landing pages copied from templates and wordpress installs. I mostly work in backend/data/platform engineering professionally. If I were asked to make a small project over a weekend, I'd be likely to decline rather than doing a more standard interview, or I'd use AI to do it in a reasonable timeframe (which seems to defeat the purpose as it relates to this discussion)
        • noirbot 2y ago
          This points to another issue - when I do code outside of work, it's often specifically to try out things I don't do at work. After a day of doing backend work, I'll maybe put together a basic web UI for something. That code is likely awful because I just need it to be functional more than good, and also probably not related to the work I'd be hired to do. My most recent real "side projects" are a terrible OSS monte carlo simulator tool that I contributed to, but cannot explain most of the code for, and a half-working React application that has performance issues I never fixed. Both are years old at this point. I'm not sure what an interviewer would gain from those.
• jakubmazanec 2y ago
  The problem isn't AI, the problem is companies don't know how to properly select between candidates, and they don't apply even the basics of Psychometrics. Do they do item analysis of their custom coding tests? Do they analyse the new hires' performances and relate them to their interview scores? I seriously doubt it. Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks. [1] https://en.wikipedia.org/wiki/Psychometrics https://en.wikipedia.org/wiki/Psychometrics
  • michpoch 2y ago
    > Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks. What kind of desperate candidate would agree to that? Also, what do you expect to see from the person in a few weeks? Usual onboarding (company + project) will take like 2-3 months before a person is efficient.
    • jakubmazanec 2y ago
      Candidate would be compensated, obviously. That's why it's expensive. You don't need him to become efficient. Also I don't think it is always necessary to have such long onboarding. I'll never understand why a new hire (at least in senior position) can't start contributing after a week.
      • michpoch 2y ago
        > can't start contributing after a week. Because you have zero context of what the org is working on.
      • michpoch 2y ago
        > Candidate would be compensated, obviously. That's why it's expensive Ok... take me through it. I apply to your company and after a short call you offer me to spend 4 weeks working at your place instead of an interview. I go back to my employer, give them resignation letter, work the rest of my notice period (2 months - 3 months), working on all handovers, saying goodbyes. Unless the idea is to compensate me for the risk (I guess at least 6 months salary, probably more), then I do not see how you'd get anyone who is just a poor candidate to sign up for this. > You don't need him to become efficient So what will you see? Efficiency, being independent and being a good team player are the main things that are difficult to test during a regular interview.
• brainwipe 2y ago
  I've accidentally been using an AI-proof hiring technique for about 20 years: ask a junior developer to bring code with them and ask them to explain it verbally. You can then talk about what they would change, how they would change it, what they would do differently, if they've used patterns (on purpose or by accident) what the benefits/drawbacks are etc. If they're a senior dev, we give them - on the day - a small but humorously-nasty chunk of code and ask them to reason through it live. Works really well and it mimics the what we find is the most important bit about coding. I don't mind if they use AI to shortcut the boring stuff in the day-to-day, as long as they can think critically about the result.
  • ttyprintk 2y ago
    This is a fine way. I’ll say that the difference between a senior and a principal is that the senior might snicker but the principal knows that there’s a chance the code was written by a founder.
    • ilc 2y ago
      And if the Principal is good, they should stand up and say exactly why the code is bad. If there's a reason to laugh because it is cliche bad, they should say so. If someone gave me code with if (x = 7) { ... } as part of a C eval. Yeah, you'll get a sarcastic response back because I know it is testing code. What I think people ignore is that personality matters. Especially at the higher levels. If you are a Principal SWE you have to be able to stand up to a CEO and say "No, sir. I think you are wrong. This is why." In a diplomatic way. Or sometimes. Less than diplomatic, depending on the CEO. One manager that hired me was trying to figure me out. So he said (and I think he was honest at the time). "You got the job as long an you aren't an axe murderer." To which I replied deadpan: "I hope I hid the axe well." (To be clear to all reading, I have never killed someone, nevermind with an axe! Hi FBI, NSA, CIA and pals!) Got the job, and we got along great, I operated as his right hand.
  • coffeefirst 2y ago
    Yep. I've also been using an AI-proof interview for years. We have a normal conversation, they talk about their work, and I do a short round of well-tested technical questions (there's no trivia, let's just talk about some concepts you probably encounter fairly regularly given your areas of expertise). You can tell who's trying to use AI live. They're clearly reading, and they don't understand the content of their answers, and they never say "I don't know." So if you ask a followup or even "are you sure" they start to panic. It's really obvious. Maybe this is only a real problem for the teams that offloading their interviewing skills onto some leetcode nonsense...
• brainwipe 2y ago
  I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice. That discussion went on for a good few minutes and by the end of it I was teetering on leaving because the fit wasn't good. I agreed to do it as long as they understood that I felt it was a terrible way of assessing someone's ability to code. I was allowed to use any programming language because they knew them all (allegedly). The solution was a pretty obvious bit-shift. So I wrote memory registers up on the board and did it in Motorola 68000 Assembler (because I had been doing a lot of it around that time), halfway through they stopped me and I said I'd be happy to do it again if they gave me a computer. The offered me the job. I went elsewhere.
  • thecleaner 2y ago
    Take a bow.
    • user432678 2y ago
      And my axe…
  • Clubber 2y ago
    >I was allowed to use any programming language because they knew them all (allegedly). After 30 years of doing this, I find that typically the people who claim to know a lot often know very little. They're insecure in their ability so much that they've tricked themselves into not learning anything.
  • Rhapso 2y ago
    Yeah, very bad fit. Surprised they made an offer. Folks getting mad about whiteboard interviews is a meme at this point. It misses the point. We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem. It isn't that your interviewer knew all the languages, but that the language didn't matter. I didn't get this until I was giving interviews. The instructions on how to give them are pretty clear. The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving. I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
• CharlieDigital 2y ago
  Code reviews. Teams are really sleeping on code reviews as an assessment tool. As in having the candidate review code. A junior, mid, senior, staff are going to see very different things in the same codebase. Not only that, as AI generated code becomes more common, teams might want to actively select for devs that can efficiently review code for quality and correctness. I went through one interview with a YC company that had a first round code review. I enjoyed it so much that I ended up making a small open source app for teams that want to use code reviews: https://coderev.app https://coderev.app (repo: https://github.com/CharlieDigital/coderev https://github.com/CharlieDigital/coderev)
  • wruza 2y ago
    Is there a site where one could review some code and see what many others say about it and their experience level? I guess it would degrade to stackoverflow-like poems eventually, but still interesting.
    • CharlieDigital 2y ago
      Not that I know of! It would be interesting, but I agree it would need to be content moderated to some extent.
    • gcrout 2y ago
      This could be like rap genius for code!
    • mclau156 2y ago
      github and issues page
    • supriyo-biswas 2y ago
      > Is there a site where one could review some code and see what many others say about it and their experience level? https://codereview.stackexchange.com https://codereview.stackexchange.com
  • jghn 2y ago
    This is harder than it sounds, although I agree in a vacuum the idea is a good one. So much value of the code review comes from having actual knowledge of the larger context. Mundane stuff like formatting quirks and obvious bad practices should be getting hoovered up by the linters anyways. But what someone new may *not* know is that this cruft is actually important for some arcane reason. Or that it's important that this specific line be super performant and that's why stylistically it's odd. The real failure mode I worry about here is how much of this stuff becomes second nature to people on a team. They see it as "obvious" and forgot that it's actually nuance of their specific circumstances. So then a candidate comes in and misses something "obvious", well, here's the door.
• thecleaner 2y ago
  I know nobody likes doing tech interviews but how has AI killed it ? Anyways you do want to know basics of computer science, it is a helpful thing to know if you ever want to progress beyond CRUD shitshovelling. Also wtf is inverting a binary tree ? Like doing a "bottom-view". That shit is easy.
• yungporko 2y ago
  nah ai killed stupid tech interviews. you can easily get an idea of someones competence by literally just talking to them instead of making them do silly homework exercises and testing their rote memorisation abilities.
  • zingababba 2y ago
    This is the real answer. However to gauge competence you must first have it. The fact that most people don't is why we are in this position in the first place.
• Mekoloto 2y ago
  I just ask to share a text editor and write down my questions. Its critical anyway because often then not its not always clear for tech questions what exactly i asked (linux command for example). This blocks their screen too. and yes we do know very soon if you look somewere else, take time or rephrase the question to get more time. If you able to fake it, at that point you should just get th ejob anyway :P
• samartin 2y ago
  Funny enough, the songs from the website Coding For Nothing about grinding LeetCode and endless take-home projects seem very relevant, and everything nowadays feels like a meme. Tech interviewing has become a weird survival game, and now AI is flipping the rules again. If you need a laugh: https://codingfornothing.com https://codingfornothing.com
• teeray 2y ago
  Licensing. We do the Leetcode interview in a controlled testing center. When you apply for a position, I look up your license number, then I know you can leetcode without wasting any of my developer resources on determining that.
  • aleph_minus_one 2y ago
    > Licensing. We do the Leetcode interview in a controlled testing center. Congratulations, you are now a "Certified Leetcoder (tm)". :-( Seriously: what a lot of people write down here is that a lot of programming jobs don't involve code puzzle skills, but are often rather about putting stuff/APIs together in the currently fashionable framework. This makes becoming a Certified Leetcoder (tm) just another useless hoop to jump through. (Just to be clear: for those few programming jobs that demand the employee to solve algorithmic puzzles regularly, doing them in a job interview makes sense. But these jobs are rare.)
    • teeray 2y ago
      > This makes becoming a Certified Leetcoder (tm) just another useless hoop to jump through. And this differs from the status quo how? Employers obviously find value in this signal for better or worse. We’re just making it so it only needs to be done once, by trained proctors, instead of for every position you apply for.
  • plasticchris 2y ago
    Isn’t this basically triplebyte? I did their process and every company still wanted to do their leetcode interview after.
    • rachofsunshine 2y ago
      Former head of product there here: no, we didn't do this kind of identity verification. That would have prohibitively damaged people's willingness to actually do the interview. We did use various means to try to identify people signing up multiple times, and we caught plenty of people trying to duplicate themselves that way, but you didn't have to physically go to a center to do our interview. I've considered something like that for my current company, which is doing basically the same thing, but: (a) this has not, in practice, been a problem for us in identifying good candidates and, far less importantly: (b) you need very high scale to have interviewers everywhere that candidates are OR you're paying extra for a third-party controlled environment (c) scheduling and cancellations become more difficult and costly respectively
• alsobrsp 2y ago
  I mostly skipped the technical questions in the last few interviews I have conducted. I have a conversation, ask them about their career, about job changes, about hobbies, what they do after work. If you know the subject, skilled people talk a certain way, whether it is IT, construction, sailing. I do rely on HR having, hopefully, done their job and validated the work history. I do have one technical question that started out as fun and quirky but has actually shown more value than expected. I call it the desert island cli. What are your 5 linux cli desert island commands? Having a hardware background, today, mine are: vi, lsof, netcat, glances, and I am blanking on a fifth. We have been doing a lot of terraform lately I have had several interesting responses Manager level candidate with 15+ years hands on experience. He thought it was a dumb question because it would never happen. He became the teams manager a few months after hiring. He was a great manager and we are friends. Manager level to replace the dumb question manager. His were all Mac terminal eye candy. He did not get the job. Senior level SRE hire with a software background. He only needed two emacs and a compiler, he could write anything else he needed.
  • aleph_minus_one 2y ago
    > I have a conversation, ask them about their career, about job changes, about hobbies, what they do after work. If you know the subject, skilled people talk a certain way, whether it is IT, construction, sailing. My experience differs a lot. Many insanely skilled people are somewhat "weird" (including possibly - being a little on the spectrum, - "living a little bit in their own world", - having opinions on topics that are politically "inappropriate" (not in the sense of "being on the 'wrong' side of a political fence", but rather in the sense of "an opinion that is quite different than what you have ever heard in your own bubble", and is thus not "socially accepted") - being a little bit "obnoxious" (not in bad sense, but in a sense that might annoy a particular kind of people)) What you consider to be "skilled people" is what I would rather call "skilled self-promoters" (or possibly "smooth talker"). "Skilled people" and "skilled self-promoter" are quite different breeds of people.
    • alsobrsp 2y ago
      > My experience differs a lot. Many insanely skilled people are somewhat "weird" (including possibly I am actually a bit weird myself, so I can relate. > What you consider to be "skilled people" is what I would rather call "skilled self-promoters". "Skilled people" and "skilled self-promoter" are quite different breeds of people. I don't mean that they have told me that they are skilled, or that their resume has implied it. I mean that they actually have the skills. Self-promoters that don't know the information always look good on paper, but after a few minutes of talking to them you can tell that they don't quite match. Before IT, I was a live sound engineer TV, theater, music. There was also a entertainment university starting up around the same time. They were pumping out tons of "trained" engineers that looked good on paper but couldn't mix for shit. I think we can blame them for the shitification of pop music.
• eek2121 2y ago
  This whole conversation is depressing me. when I left work a couple years ago due to health reasons, AI was just beginning to become a problem. Now, thanks to a clinical study, I may possibly be able to return to work, and it sounds like the industry has changed drastically. Not looking forward to it.
  • richardwhiuk 2y ago
    I think that the effects at the moment are highly exaggerated in the tech media than the reality on the ground. How long that will remain true is a very open question, where different folks have widely differing timelines on when they expect AI to have highly meaningful impacts.
  • grandempire 2y ago
    It hasn’t. Most businesses have continued operating the way they have for years. SV Startup hiring is the most trendy and not representative.
• user99999999 2y ago
  Never really liked leetbro interviews. Always reeked of “SO YOU THINK YOU CAN CODE BRO? SHOW ME WHAT YOU GOT!” The majority of my work over 10+ years of experience always relied on general problem solving and soft skills like collaborating with others. Not rote memorization of in order traversal.
• perlgeek 2y ago
  Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead? There's very likely a real answer to that question, and that answer should shape the way that engineer should be assessed and hired. For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way. Then you could, in an interview, give a bit of context and then ask the candidate to think out loud about an example feature request. It seems to me the heart of the problem is that companies aren't very clear about what value the engineers add, and so they have trouble deciding whether a candidate could provide that value.
  • bitwizeshift 2y ago
    Tech interviews in general need to be overhauled, and if they were it’d be less likely that AI would be as helpful in the process to begin with (at least for LLMs in their current state). Current LLMs can do some basic coding and stitch it together to form cool programs, but it struggles at good design work that scales. Design-focused interviews paired with soft-skill-focus is a better measure of how a dev will be in the workplace in general. Yet, most interviews are just “if you can solve this esoteric problem we don’t use at all at work, you are hired”. I’d take a bad solution with a good design over a good solution with a bad design any day, because the former is always easier to refactor and iterate on. AI is not really good at that yet; it’s trained on a lot of public data that skews towards worse designs. It’s also not all that great at behaving like a human during code reviews; it agrees too much, is overly verbose, it hallucinates, etc.
  • nottorp 2y ago
    Problem is, company A doesn't need an engineer to solve those interview questions but real problems.
    • okdood64 2y ago
      This is the answer. Let's not pretend 95% of companies are asking asinine interview questions (though I understand the reasons why) that LLMs can easily solve.
• deleted 2y ago
  [deleted]
• LunicLynx 2y ago
  It’s simple don’t have a tech interview that does not relate to the job. Show code, ask questions about it that requires opinion.
  • dahart 2y ago
    This oft repeated talking point lacks perspective on what companies want and how interviews work. It also doesn’t address the AI problem. Interviews are screening for multiple things, not just the ability to do one specific technical job. More often than not, technical coding ability is not even at the top of the priority list. Interviews are looking for well-rounded candidates who can do more than 1 job. Companies want to know if you can change jobs easily, they want to know if you’re average, better than the average programmer, or exceptional. They want to know if you’ll make a good manager after a few years, how good you are with people, how well you prioritize and communicate. I had a professor in college that graded tests with the median skewed low, centered on a D. He complained that the usual practice of putting it on C or B made it so he could clearly see the difference between F-, F, and D- students, while the A students were all clumped together. He wanted to identify the hard workers and superstars in the class, see who was A vs A+ vs A++. It freaked everyone out when grades came out much lower than expected, but he renormalized at the end and people with test scores in the Cs and Ds got As and Bs in the class. Be careful what you wish for. It’s competitive right now and interviews that limit screening to ability to do basic job-level coding and don’t screen for knowledge and soft skills and exceptionalism will make it harder for people who are good to demonstrate they’re better than people who are mediocre or use AI. Is that what you want?
• rachofsunshine 2y ago
  The death of shitty interviews has been greatly exaggerated. AI might make e.g. your leetcode interview less predictive than it previously would have been. But was it predictive in the first place? I don't think most interviews are written by people thinking in those terms at all. If your method of interviewing never depended on data suggesting it actually, you know, worked in the first place, why would it matter if it starts working even worse? Insofar as it makes the shittiness of those interviews more visible, the effect of AI is a good thing. An interview focused on recall of some specific algorithm was never predictive, it's just now predictive in a way that Generic Business Idiots can understand. We frequently interview people who both (a) claim to have been in senior IC roles (not architect positions, roles where they are theoretically coding a lot) for many, many years and (b) cannot code their way out of a paper bag when presented with a problem that requires even a modicum of original reasoning. Some of that might be interview nerves, of course, but a lot of these people are not at all unconfident. They just...suck. And I wonder if what we're seeing is the downstream effects of Generic Business Idiots hiring primarily people who memorize stuff than people who build stuff.
  • mattkrause 2y ago
    > A lot of these people … just suck. Another possibility is that their job subtly drifted. I wrote a lot of code as a grad student but my first interviews afterward were disasters. Why? Because I’d spent the last few months writing my thesis and the few months before that writing a very specific kinds of code (signal processing, visualization) that were miles away from generic interview questions like “Make the longest palindrome.”
    • rachofsunshine 2y ago
      We don't ask "make the longest palindrome". We ask "convert this English into code that does what it says". If you want to make the discussion more concrete, we have a public practice problem [1] that we send out with our interview bookings so that people know what to expect. The real problems we ask are very similar to it. Do you feel like there's anything there that any reasonably skilled programmer shouldn't be able to figure out on the fly? [1] https://www.otherbranch.com/shared/practice-coding-problem https://www.otherbranch.com/shared/practice-coding-problem
• voidr 2y ago
  No it didn't, you just need to stop asking questions an LLM can easily solve, most of those were probably terrible questions to begin with. I can create a simple project with 20 files, where you would need to check almost all of them to understand the problem you need to solve, good luck feeding that into an LLM. Maybe you have some sneaky script or IDE integration that does this for you, fine, I'll just generate a class with 200 useless fields to exhaust your LLM's context length. Or I can just share my screen and ask you to help me debug an issue.
• TransAtlToonz 2y ago
  I had no idea people took hackerrank as a serious signal rather than as a tool for recent graduates to track interview prep progress. Surely it has all the same issues AI does: you have no way of verifying that the person who takes the interview actually is responsible for that signal. I don't see AI as a serious threat to the interview process unless your interview process looks a lot like hackerrank.
  • shermantanktop 2y ago
    Your “unless” covers a huge swath of this industry, at the low end and at the high end. Excluding places that do that leaves you with what exactly? Boutique shops filled with 20 year veterans?
    • TransAtlToonz 2y ago
      What do you mean by the "high" end? I would consider this sort of interview style necessarily precluding such a place from being considered a high-quality work-place. Not only is it a miserable way to interview, it's not an effective signal for engineer quality beyond rapid code snippet production. > Excluding places that do that leaves you with what exactly? Boutique shops filled with 20 year veterans? We are on a VC forum—I imagine small shops focused on quality are quite common here.
      • shermantanktop 2y ago
        “High end” was meant as shorthand for FAANG …high comp, not necessarily high tech complexity. You are correct about the deficiencies of the whiteboard interview. It is not a sane way to hire an individual. It makes sense as a way to hire someone in the top 20% from a large unfiltered pool. So wrt high/low, that’s what FAANG companies have to do, and for many nontechnical companies they outsource this work or emulate FAANG practices for no good reason. My point was that there are very few places that don’t do this.
        • TransAtlToonz 2y ago
• karaterobot 2y ago
  The best interview process I've ever been a part of involved pair programming with the person for a couple hours, after doing the tech screening having a phone call with a member of the team. You never failed to know within a few minutes whether the person could do the job, and be a good coworker. This process worked so well, it created the best team, most productive team I've worked on in 20+ years in the industry, despite that company's other dysfunctions. The problem with it is the same curse that has rotted so much of software culture—the need for a scalable process with high throughput. "We need to run through hundreds of candidates per position, not a half dozen, are you crazy? It doesn't matter if the net result is better, it's the metrics along the way that matter!"
  • chiph 2y ago
    Was this an in-person pair programming session or remote via shared desktop? Because the latter is garbage. You can't see the other person or read their body language. It also gives off "proctored test" vibes. Or worse than vibes - where you're expected to clear your room, swing your camera around, and never look away.
  • ukd1 2y ago
    Pairing on something close to whatever real work they'd be doing, but familiar to the applicant is my favorite way to evaluate someone (e.g. choose a side project, pre agree adding a feature). I don't care if someone uses modern tools (like AI assists), google, etc - e.g. "open book" - as that's the how they want to work. Evaluating their style of thinking / solving problems, comms, and output is the win.
  • agustechbro 2y ago
    Beautifully articulated truth
  • m11a 2y ago
    I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)? Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source. Indeed, the other problem is what you mention: it doesn't scale to high throughput.
• siva7 2y ago
  Why don't we simply ask the AI how to conduct a tech interview nowadays?
• advael 2y ago
  I think that a mythology about where the difficulty in working with computers lies has made the relationship between businesses and the people they hire to do this stuff miserable for quite some time "Coding", as in writing stuff in programming languages with correct syntax that does the thing asked for in isolation, has always been a very dumb skill to test for. Even before we had stackoverflow syntactic issues were something you could get through by consulting a reference book or doing some trial and error with a repl or a compiler. That this is faster now with internet search and LLMs is good for everyone involved, but the fact that it's not what matters remains The important part of every job that gets a computer to do a thing is a combination of two capabilities: Problem-solving, that is, understanding the intended outcome and having intuition about how to get there through whatever tools are available, and frustration tolerance: The ability to keep trying new* stuff until you get there Businesses can then optimize for things like efficiency or working well with others once those constraints are met, but without those capabilities you simply can't do the job, so they're paramount. The problem with most dinky little coding interviews wasn't that you could "cheat", it's thst they basically never tested for those constraints by design, though some clever hiring people manage to tweak them to do so on an ad hoc basis sometimes * important because a common frustration failure mode is repetitive behavior. Try something. Don't understand why it doesn't work. Get more frustrated. Try the same thing again. Repeat
• orsenthil 2y ago
  No, it has not. We still have the same situation.
• deleted 2y ago
  [deleted]
• adregan 2y ago
  What I've been thinking about leetcode medium/hard as a 30-45 minute tech interview (as there are a few minutes of pleasantry and 10 minutes reserved for questions), is that you are only really likely to reveal 2 camps of people—taking in good faith that they are not "cheating". One who is approaching the problem from first principles and the other who knows the solution already. Take maximum subarray problem, which can be optimally solved with Kadane's algorithm. If you don't know that, you are looking at the problem as Professor Kadane once did. I can't say for sure, but I suspect it took him longer than 30-45 minutes to come up with his solution, and I also imagine he didn't spend the whole time blabbering about his thought process. I often see comments like: this person had this huge storied resume but couldn't code their way out of a paper bag. Now having been that engineer stuck in a paper bag a few times, I think this is a very narrow way to view others. I don't know the optimal way to interview engineers. I do know the style of interview that I prefer and excel at[0], but I wouldn't be so naive to think that the style that works for me would work for all. Often I chuckle about an anecdote from the fabled I.P. Sharp: Ian Sharp would set a light meter on his desk and measure how wide an interviewees eyes would get when he explained to them about APL. A strange way to interview, but is it any less strange than interviewing people via leetcode problems? 0: I think my ideal tech screen interview question is one that 1) has test cases 2) the test cases gradually ramp up in complexity 3) the complexity isn't revealed all at once; the interviewer "hides their cards," so to speak 4) is focused on a data structure rather than an algorithm such that the algorithm falls out naturally rather than serves as the focus. 5) Gives the opportunity for the candidate to weigh tradeoffs, make compromises, and cut corners given the time frame. 6) Doesn't combine big ideas (i.e. you shouldn't have to parse complex input and do something complicated with it); pick a single focus. Interviews I have participated and enjoyed like this: construct a Set class (union, difference, etc); implement an rpn calculator (ramp up the complexity by introducing multiple arities); create a range function that works like the python range function (for junior engineers, this one involves a function with different behavior based on arity).
  • dbrumbaugh 2y ago
    >Take maximum subarray problem, which can be optimally solved with Kadane's algorithm. If you don't know that, you are looking at the problem as Professor Kadane once did. I can't say for sure, but I suspect it took him longer than 30-45 minutes to come up with his solution, and I also imagine he didn't spend the whole time blabbering about his thought process. This is something that drives me nuts in academia when it comes to exam questions. I once took an exam that asked us to invent vector clocks from whole cloth, basically, having only knowledge of a basic Lamport clock for context. I think one person got it--and that person had just learned about vector clocks in a different class. Given some time, it's possible I could have figured it out. But on an exam, you've got like 10-15 minutes per question. The funny thing about it is that I do the same damn thing from the other side all the time when working with students. It's incredibly tempting once you know the solution to a problem (especially if you didn't "solve" it yourself, but had the solution presented to you already) to present the question as though it has an obvious solution and expect somebody else to immediately solve it. I'm aware of the effect, I've experienced it many times, and I still catch myself doing it. I've never interviewed a candidate for a job, but I can only imagine how tempting it would be to fall into that trap.
• hamuraijack 2y ago
  I'm someone who hated leetcode style interviews for the longest, but I'm starting to come around on them. I get that these style of questions are easy to game, but I still think they have _some_ value. The point of these style of questions was supposed to test your ability to problem solve and come up with a good solution given the tools you knew. That being said, I don't think every company should be using this type of question for their interviews. I think leetcode style questions should be reserved for companies that are pushing the boundary of the industry since they're exploring charted territory and need people who can come up with unique solutions to problems no one really knows. I think most companies would be fine with some kind of pairing problem since most people are probably solving engineering problems instead of computer science problems. But none of this matters, since, we all know that even if we went that direction as an industry, the business people would fuck it up some how anyways.
  • abhijeetpbodas 2y ago
    > reserved for companies that are pushing the boundary of the industry In a world where every company beleives (or wants to beleive) that they are doing some ground-breaking, bleeding edge work (see any tech company blog and you can only find hyped technologies in there), I do not think one can expect companies to do a fair assessment of if they really are doing such work.
• janoc 2y ago
  I miss one option from the list of non-solutions the author presents there - ditch the idiotic whiteboard/"coding exercise" interview style. Voila, the AI (non)problem solved! This sort of comp-sci style exam with quizzes and what not maybe somewhat helps when hiring junior with zero experience fresh out of school. But why are people with 20+ years of easily verifiable experience (picking up a phone and asking for references is still a thing!) being asked to invert trees and implement stuff like quicksort or some contrived BS assignment the interviewer uses to boost their own ego but with zero relevance to the day to day job they will be doing? Why are we still wasting time with this? Why is always the default the assumption there that the applicants are all crooked hochstaplers that are lying on their resumes? 99% of jobs come with probationary period anyway where the person can be fired on the spot without justification or any strings attached. That should be more than enough time to see whether the person knows their stuff or not after having passed one or two rounds of oral interviews. It is good enough for literally every other job - except for software engineering. What makes us the special snowflakes that people are being asked to put up with this crap?
• boogieknite 2y ago
  one hiring manager told me they dont do code challenges. they said, "why would someone take a job they couldnt do?" isnt it that simple?
• wkat4242 2y ago
  What a BS article. As they say, just do the interview in person. Problem solved. Not sure about the US but 99% of jobs here in Spain are hybrid or onsite ("presencial"), not fully remote. They're acting like all jobs are remote and it's impossible to do an interview in person. Also, does it really matter? If a person is good at using AI and manages to be good at creating code with that, is it really so much worse than a person that does it from the top of their head? I think we have to drop the idea that AI is going to go away. I know it's all overhyped right now but there is definitely something to it. I think it will be another tool in our toolboxes. Just like stackoverflow has been for ages (and that didn't kill interviews either).
  • janoc 2y ago
    Costs money (travel) and time (company's) plus a lot of this is outsourced to agencies to do the screening. Yes, it is disgusting. Sadly also very common.
    • wkat4242 2y ago
      Ah weird, here in Europe if you apply for a job in another city or country they won't fly you out there. You can come on your own dime and usually you wouldn't even tell them you don't live there yet (after all if you don't even live there, why would they bother with you, it's only extra hassle for them when you start). Probably C-suite roles are an exception to this. But they're an exception to pretty much everything. Roles with particular foreign languages (e.g. support) too but this is also an edge case. Moving personnel between countries when they are already working for the company does happen. They did it for me. But at that point they already know what they have.
  • morgannewman 2y ago
    [dead]
  • kagakuninja 2y ago
    Our US mega corp is having us hire all remote contractors, in person is completely out of the question.
• tuan 2y ago
  I feel like we, SWEs, have been over-engineering our interview process. Maybe it's time to simplify it, for example, just ask questions based on the candidate's resume instead of coming up with random challenges. I feel like all the new proposals seem overly complicated, and nobody, interviewer or interviewee, is happy with any of them.
  • beezlebroxxxxxx 2y ago
    Definitely over-engineering. But I also think the industry is just extremely bad at hiring anything above junior or entry-level. Job postings are so generic and interchangeable between companies that they don't actually tell you what the role is or what the company is looking for. Everyone wants to cast the widest possible net so that they catch some wunderkind genius out of thousands. Then, they wonder why they can't find the exact person they're looking for to solve the problem they're filling the role for. In reality, job postings should be incredibly specific, with specificity rising as the role requires more experience and problem solving. You'll get less applicants (or will be able to clearly screen out the people who don't meet the specific requirements) but you'll get ones that actually match what you are looking for and can actually solve the problem your company is trying to solve with filling the role. Then the conversation/interview is much more important and both sides feel like they have some "stakes in the game".
  • ktusznio 2y ago
    This risks hiring candidates who can present themselves and their past projects very well but fail to actually write code and ship anything on the job. I've seen it happen.
  • smw 2y ago
    You would be very amazed at how many people with reasonably strong resumes can't write _any_ code. Google for fizzbuz, it's a dumb problem, but candidates often can't solve similar problems with a _take_home_ interview.
• codr7 2y ago
  The best interview processes are chill, laid back, open ended. That's the only way you're going to get relevant information. I've been verified to the moon and back by Apple and others for roles that could never have worked. The problem is that when it comes to the hiring process, everyone is suddenly an expert; no matter how dysfunctional, inhumane and destructive their ideas are.
• paulcole 2y ago
  > I think the image below pretty much sums it up The image below does sum it up but not in the way the author thinks. Google wants to hire people who complete their hiring process. They're OK with missing out on some people who would be excellent but who can't/won't make it through their hiring process. The mistake may lie in copying Google's hiring process.
• codr7 2y ago
  The current job market is so messed up that I honestly can't see myself getting a job until we hit a wall and people start using their brains again. I have 26 years of solid experience, been writing code since I was 8. There should be a ton of companies out there just dying to hire someone with that kind of experience. But I'm not perfect, no one is; and faking doesn't work very well for me.
  • ipunchghosts 2y ago
    I am with you! Been programming since I was 10 and have 20YoE. Many of my prototypes have grown into full fledged products, I have 40+ published papers, and I am regularly sought out for advice and help by those who know me. Everyone i have been, I am always told I am a good catch. However, I won't do leet coding. I want to hear about why I should come work for u. What about my works makes u think I could help ubm with your problem. Then let's have a talk about your problems and where I can create value for you. My experience in hiring is that leet coders are good one trick ponies. But long term don't become technical peers.
    • codr7 2y ago
      Part of the problem is there just aren't a lot of people out there who can correctly judge that level of experience, and looking up the spectrum tends to simply look weird.
      • ipunchghosts 2y ago
        I agree. Do you have any thoughts on how to mitigate this? After all, its in your best advantage and the companies to hire someone with talents because of the value they can bring.
        • codr7 2y ago
          Not really, except having a lot of experience on board from the start. Which will also naturally attract more of the same.
        • 2y ago
• deleted 2y ago
  [deleted]
• parliament32 2y ago
  The inconvenient truth is that everything circles back to in-person interviews. The article addresses this: >A lot of companies are doing RTO, but even companies that are 100% in-office still interview candidates from other cities. Spending money to fly every candidate out without an aggressive pre-screen is too wasteful. No, accidently hiring someone who AI'd their way through the interview costs orders of magnitude more to undo. It's absolutely worth paying for a round trip flight and a couple days of accommodations.
• __MatrixMan__ 2y ago
  Maybe we don't need employers. Maybe we need a bunch of 1-person companies. I don't think AI is yet the force multiplier that makes that feasible for the masses, but who knows what things will look like in a few years.
  • jimbokun 2y ago
    That is the end point of AI in the economy, until we are removed from the economy entirely. Everyone on the planet is a one person startup, using AI and robotics to do all the actual work.
    • abenga 2y ago
      What's the point of humans (other than the ai and robotics ip owners) in this scenario?
      • deleted 2y ago
        [deleted]
      • __MatrixMan__ 2y ago
        Probably the same as it ever was. I couldn't tell you what that is, but it never had anything to do with economics anyway. Having a job has always been a necessary evil. If the time is coming when that changes, all the better. If we can't figure out how to behave in the presence of plenty, we don't deserve utopia anyway.
      • LPisGood 2y ago
        You’re asking what is the point of being alive if you don’t have to work at a company?
  • pizzafeelsright 2y ago
    Anyone with skill and agency needs capital, exposure, resources. Companies currently provide this.
• cryptozeus 2y ago
  One option is to make the interviews harder and let candidates use ai to see how they can work with ai and actually build working product. They will be using ai in the job anyway so let them use it instead of asking stupid algorithm questions to sort an array
• dowager_dan99 2y ago
  The best interview process I've ever had was going to work with former coworkers, aka no real process. A couple of quick calls with new people who deferred strongly to the person who knew me, my work, and my values. Nothing else has the signal value. Of course the problem is this can't scale or be outsourced to HR, but is this a bug or a feature?
• dowager_dan99 2y ago
  Anyone who suggests a paired programming solution is right, and answering the wrong question. Unless/until we return to a covid-like market the process will never be optimized for the candidate, and this is just too expensive an approach for employers. In this market I think the answer is hire less.
• mempko 2y ago
  I cannot emphasize this enough. Coding is the EASY part of writing software. You can teach someone to code in a couple of months. Interviews that focus on someone's ability to code are just dumb. What you need to do is see how well they can design before writing software. What is their process for designing the software they make? Can they architect it correctly? How do they capture user's mental models? How do they deal with the many "tops" that software has?
• madmaniak 2y ago
  Come on - it was already dead for a long time.
• toofso 2y ago
  If using AI is cheating then one solution as the author mentions is have the interview take place at an office but I'm surprised another approach isn't more readily available: having the candidate the the test remotely at a trusted 3rd party location.
• jrochkind1 2y ago
  > Tech interviews are one of the worst parts of the process and are pretty much universally hated by the people taking them. True. > One of the things we can do, however, is change the nature of the interviews themselves. Coding interviews today are quite basic, anywhere from FizzBuzz, to building a calculator. With AI assistants, we could expand this 10x and have people build complete applications. I think a single, longer interview (2 hours) that mixes architecture and coding will probably be the way to go. Oh.... yeah, that sounds just... great.
• sarchertech 2y ago
  There’s no other industry* that interviews experienced people the way we do. So maybe just do what everyone else does. Everyone is so terrified of hiring someone that can’t code, but the most likely bad hires and the most damaging bad hires are bad because of things that have nothing to do with raw coding ability. *Except the performing arts. The way we interview is pretty close to the way musicians are interviewed, but that’s also really similar to their actual job.
• jimmyjazz14 2y ago
  When it comes to interviews I generally stick to asking fairly easy questions but leaving out key details, I care a lot more about candidates asking following ups and talking through the code they are writing over what code they actually produce. If a candidate can ask questions when they don't understand something and talk through their thought process then they are probably going to be a good person to work with. High level design questions are often pretty valuable I find as well, which I usually don't require code for I just ask them to talk through their ideas of how they would design an application.
• shipscode 2y ago
  1point3acres is massacring tech interviews right now. Having to pay $80/month to some China based website where NDA-protected interview questions are posted regularly, then being asked the same questions in the interview, seems insane. It also feels like interviewers know this and assume you studied the questions, they seem incapable of giving hints, etc if you don't have the questions memorized. AI is the least of it.
• neilv 2y ago
  > What are our options? * Take a candidate's track record into account. Talk with them about it. * Show that you're experienced yourself, by being able to tell something about what someone would be like to work with, by talking with them. * Get a reputation for your company not tolerating dishonesty. If someone cheats in an interview and gets caught, they're banned there, all the interviewers will know, and the cheater might also start to get a reputation beyond that company. (Bonus: Company reputation for valuing honesty is attractive to people who don't want dishonest coworkers.) * Treat people like a colleague, trying to assess whether it's a good match. You're not going to be perfectly aligned (e.g., the candidate or the company/role might be a bit out of the other's league right now), but to some degree you both want it to be a good match for both parties. Work as far as you can with that. (Don't do this: Leetcode hazing, to establish the dynamic of them being there to dance for your approval, so hopefully they'll be negged, and will seek your approval, won't think critically about how competent and viable your self/team/company are, and will also be less likely to get uppity when you make a lowball offer. Which incidentally places the burden of rehearsing for Leetcode ritual performances upon the entire field, at huge cost.)
• jwmoz 2y ago
  None of this makes any sense. Why should I complete a tech test interview if I have 15 years of experience at X top firm? I would have done it already anyway. I had a ‘principal engineer’ at last place who grinded leetcode for 100 days and still failed a leetcode interview. It’s utter nonsense. A conversation with technical questions and topics should suffice. Hire fast and fire people.
• slackfan 2y ago
  It hasn't killed the interview, it's killed the career field. Most people just haven't realized this yet.
• fragmede 2y ago
  here's the question, here's the code that ChatGPT produced. what's wrong with it?
• LaGrange 2y ago
  "One of the things we can do, however, is change the nature of the interviews themselves. Coding interviews today are quite basic, anywhere from FizzBuzz, to building a calculator. With AI assistants, we could expand this 10x and have people build complete applications. I think a single, longer interview (2 hours) that mixes architecture and coding will probably be the way to go." If that's where the things are going, I'm retraining to become a line cook at McDonalds.
• darkhorse222 2y ago
  I think code design can often cover just as much as actual code anyway. Just describe to me how your solve it, the interfaces you'd use, and how you'd show me you solved it. As an interviewee it's insane to me how many jobs I have not gotten because of some arbitrary coding problem. I can confidently say after having worked in this field for over a decade and at a FAANG that I am a very capable programmer. I am considered one of the best on every team I've been on. So they are definitely selecting the wrong people IMO.
• gaoshan 2y ago
  Our tech screen is having the candidate walk me through a small project that I created to highlight our stack. They watch my screen and solve a problem to get the app running then they walk me through a registration flow from the client to the server and then returning to the client. There are no gotchas but there are opportunities to improve on the code (left unstated... some candidates will see something that is suboptimal and ask why or suggest some changes). We get to touch on client and browser issues, graphQL, Postgres, Node, Typescript (and even the various libraries used). It includes basic CRUD functionality, a third party API integration, basic security concerns and more. It's just meant to gauge a minimal level of fluency for people that will be in hands on keyboard roles (juniors up to leads, basically). I don't think anyone has found a way to use AI to help them (yet) but if this is too much for them they will quickly fall flat in the day to day job. Where we HAVE encountered AI is in the question/answer portion of the process. So far many of those have been painfully obvious but I'm sure others were cagier about it. The one incident that we have had that kind of shook us was when someone used a stand-in to do the screen (he was fantastic, lol) and then when we hired him it took us about a week to realize that this was a team of people using an AI avatar that looked very much like the person we interviewed. They claimed to be in California but were actually in India and were streaming video from a Windows machine to the Mac we had supplied for Teams meetings. In one meeting (as red flags were accumulating) their Windows machine crashed and the outline of the person in Teams was replaced by the old school blue screen of death.
• slowhadoken 2y ago
  LLMs killed busy work. Now people have to actually talk to each other and they're finding out that we've been imitating functionality instead being functional.
• varelse 2y ago
  [dead]
• leeny 2y ago
  We did an experiment at interviewing.io a few months ago where we asked interviewees to try to cheat with AI, unbeknownst to their interviewers. In parallel, we asked interviewers to use one of 3 question types: verbatim LeetCode questions, slightly modified LeetCode questions, and completely custom questions. The full writeup is here: https://interviewing.io/blog/how-hard-is-it-to-cheat-with-chatgpt-in-technical-interviews https://interviewing.io/blog/how-hard-is-it-to-cheat-with-ch... TL;DR: - Interviewers couldn't tell when candidates were cheating at all - Both verbatim and slightly modified LeetCode questions were really easy to game with AI - Custom questions were not gamable, on the other hand[1] So, at least for now, my advice is that companies put more effort into coming up with questions that are unique to them. It's better for candidates because they get better signal about the work, it reduces the value asymmetry (companies have to put effort into their process instead of just grabbing questions from LeetCode etc), and it's better for employers (higher signal from the interview). [1] This may change with the advent of better models
  • physicles 2y ago
    A couple years ago, we were using a take home coding assignment as a hiring signal. It was a small API based off something I'd built for an internal tool. It was self-contained and relatively easy to explain. The .md file was about two pages. I recently fed it into ChatGPT and asked it to do the assignment. It did it perfectly -- I read the code in detail and couldn't find any issues. So custom questions are off the table now, too. We'll be using a code review instead for the next round.
• EGreg 2y ago
  It's a tricky subject, because what if people who use AI are just better together? And what if in a year from now, AI by itself is better? What's the point of hiring anyone? Perhaps this is the issue behind the problems being described, which might be mere symptoms. There are tons of very smart teams working on software that will basically replace the people you're hiring.
• iancmceachern 2y ago
  "Video killed the radio star"
• enricojr 2y ago
  How about paid internships as a way to filter candidates? As in, hire a candidate for a small period of time, like 2 weeks or something, and have them work on a real task with a full-time employee and use their performance on that to decide whether or not to hire. I realize it's not easy for smaller companies to do, but I think it's the single best way to see if someone's fit for a job
• K0balt 2y ago
  So, when AI can pass the tech interview seamlessly, I guess we can just hire it? Maybe the future will be human shills pretending to be job candidates for shady AI “employment agencies” that are actually just (literally) skinning gpt6 apis that sockpuppet minimum wage developing nation”hosts”?
• anacrolix 2y ago
  thank fuck. they are terrible. being interviewed by CTOs just out of university, with no experience for a senior in everything role. they ask you to do some lame assignment, a pet problem not once looking at 20 years of GitHub repos and open source contributions.
• jelambs 2y ago
  > Using apps like GitHub Co-pilot and Cursor to auto-complete code requires very little skill in hands-on coding. this is a crazy take in the context of coding interviews. first, because it's quite obvious if someone is blindly copy and pasting from cursor, for example, and figuring out what to do is a significant portion of the battle, if you can get cursor to solve a complex problem, elegantly, and in one try, the likelihood that you're actually a good engineer is quite high. if you're solving a tightly scoped and precise problem, like most coding interviews, the challenge largely lies in identifying the right solution and debugging when it's not right. if you're conducting an interview, you're also likely asking someone to walk through their solution, so it's obvious if they don't understand what they're doing. cursor and copilot don't solve for that, they make it much easier to write code quickly, once you know what you're doing.
• aforwardslash 2y ago
  I've conducted hundreds of tech interviews in the past decade, and I can tell you, it takes roughly 5 minutes to understand if a candidate is a good fit or not. I have no problem in having candidates using google, ai or whatever to answer technical stuff - if a candidate can write a query, read the results and be able to explain what he just read, he would be an instant hire. It seems that people keep focusing on the wrong skillset; sure, many companies think knowing sort algorithms and binary trees is super important for a CRUD maintenance job, but they forget the most important tech trait - the ability to adapt and evolve; the ability to learn.
• xnyan 2y ago
  I was on a technical interview panel last week where some one of the candidates was using a tool that transcribed our call and fed answers to the candidate in real time. The first problem was the answers were extremely AI. You could instantly tell that everything they said was kind of off strange. I would not be surprised if this is something that can be improved or addressed, however what will be harder to solve is the fact that the candidate was clearly reading answers. You could see their eyes flick up and parse the text the model was feeding them very clearly. It could not be more obvious.
• dumbledoren 2y ago
  > Change our interviews to allow AI: We start testing prompting and refactoring rather than testing coding ability. Can someone program LLMs to get the outcomes they want during an interview? This is a huge shift change and would be uncharted territory. That is the likely outcome. Its not too difficult to incorporate AI into the process of assessing skill: There is always a risk that the AI will hallucinate or make noticeable mistakes, and being able to detect them would be a confirmation of the engineer's knowledge. Even more so in cases of seniors and architects.