13 ms·
An invisible desktop application that will help you pass technical interviews
- deleted 2y ago[deleted]
- thesaintlives 2y agoSo cheat to get a job then get fired when they see you are useless? Genius..
- tags2k 2y agoI do agree, however from my experience some folk smart enough to build this kind of system will struggle in an interview situation. Maybe rather than outright cheating, this could be more of a leg-up for those who find interviews difficult. Still wouldn't use it myself.
- mog_dev 2y agoTechnical interviews are useless and its normal that their victims fight back
- vanhakoira 2y agoI heavily disagree with this. Technical interviews are great if done correctly. Or do you think that technical interviews hold such trait that is always present, that you can label all of them useless?
- 4dregress 2y agoI think they’re useless unless they touch some part of the domain you’re howling to work in.
- therealdrag0 2y agoWhy do they need to touch the domain? The point of generic interviews is to find smart people who can learn any domain.
- 4dregress 2y agoI’m using the term domain very broadly here, I apologise. What I mean is, if the job your applying for is mostly about working with implementing 3rd party APIs IMO it makes sense that your tech test represents that to some degree. Can you understand basic documentation. Can you use data structures. Can you write tests and understand what actually needs to be tested. Can you handle exceptions. Do you try and get the entire resource? Believe it or not I’ve joined a team in the past who owned a service where no one knew what pagination was. Bonus points if you demonstrate that you understand what a transaction is. And I forgot to add, can you design a sensible database schema. Can a candidate talk about those simple concepts? Now, would I be able to do a tech test based around domains in machine learning? I would have to do a lot of research first!
- tamimio 2y agoI think a better option is to have a system in place where companies hire them for a month or so then after that period you choose whoever made a better impact. And before you say it will be costly, most companies can afford it, especially if it will lead to a better outcome. That way they get tested on the actual systems and not just some useless questions from the internet.
- AndrewDucker 2y agoAs someone with a job I'm not giving up my job security to take a job that might go away after 1 month. Doesn't matter how much more it pays - I need income every month to pay rent!
- tamimio 2y agoThat’s a fair point, maybe the proposed system can be adjusted for a 2hours a day then. The point is to test that person on the systems rather than just QA style.
- toast0 2y agoSure, that's costly for employers, but it's also costly for the candidates. If you're planning to have N candidates do this when you only have 1 position, you're disrupting the others' lifes so much. It's like one of those chef competition shows. You better pay these people at least 6 months salary to show up for this. Plus, do you really have the supervisory system in place to have N new hires and evaluate their impact after 1 month. Do you have the work for them to do? Or are they all going to independently work on the same project? I ask a useless question in interviews, but asking every candidate the same question lets me calibrate the candidates as well as improve how I present the question so we can get to the problem solving and programming skills I want to see. Some questions are worse than others, and I hope mine is less worse; I've had several people tell me they enjoyed working on my question, although I also had a candidate refuse to work on it and ask for another question (which I didn't have, and I was the last interview of the day, so I returned the balance of his time and walked him out)
- rvz 2y ago? "Victims" This just makes it much worse for the industry that is already weeding out frauds or those that claim / lie about their achievements without backing them up when asked very technical tests. I would say the online technical interview solutions like leetcode, hackerrank, etc are losing their worth given they can be easily cheated but you are just cheating yourself in the on-site interview.
- wang_li 2y agoIf you don't like technical interviews then what the industry needs is proper and formal certification process akin to the medical, legal, and engineering fields. In the absence of some outside resource to validate that a person has the basic fundamental skills, then you have to ask questions to determine if the applicant knows any of those skills.
- cynicalsecurity 2y agoThe recruitment process is broken.
- tjpnz 2y agoIn tech? Unless you really are useless you'll be more likely to coast.
- andai 2y agoObligatory "I don't agree with this and think it seems like a pretty depressing way to live", but: A colleague yesterday was telling me about how much he and all his friends lied to get their jobs, and he suggested I do the same, because if I don't, I'm competing against people who do, and said that's unfair (for me). It wasn't even an office job, but I've seen the same thing there too. (Also at university...)
- qwer1234321 2y agoI'm not under pressure so it's easy to say this, but I'd rather not get a job than lie during interview.
- ttyprintk 2y agoI encourage anyone to stand their ground on this. Competing with liars is an enormous soft-skill challenge. But it doesn’t exercise critical thinking. Structuring a personal plan that builds those soft skills is really annoying. Especially if you just want to solve complex problems all day. That said, - the interviewer likely embellished or lied to get his job. Be careful. - a lot of this thread is about avoiding a bad hire. In engineering, someone technical who can detect deception is too valuable to put in the interviewer role. Send them to conferences to “poach”. Just like the soft skills of competing with liars, you can also improve detecting deception.
- snozolli 2y agoIn engineering, someone technical who can detect deception is too valuable to put in the interviewer role. Send them to conferences to “poach”. Those are wildly different skill sets and/or personality traits.
- bartread 2y agoI’d say chances of getting busted are high as well. The invisibility of the window to common interview software won’t do anything mask candidates’ own suspicious behaviour. I was doing a 45 minute screener interview with a guy the other day who I could see kept glancing over at another window or screen as he was typing in code. Every couple of characters or so he was pausing looking over, then he’d type a handful more characters, then pause and glance again. He was typing incredibly slowly as well, like a character or two per second, even when he was typing. It was honestly a bit of a weird experience to watch unfolding. His solutions were spot on though. I didn’t make an issue of it, partly because I already had other concerns as well. I can’t think of a situation where there’s anything to gain by getting into an argument with a candidate: I simply didn’t invite him to the next interview stage.
- herghost 2y agoGong back about 20 years we had to introduce a "pre-screening" interview for infosec roles because of the calbire of candidates we were getting through. They'd made it through the HR "sift" by having the right words on their CVs but we found that a huge majority of these candidates didn't know even entry-level basics. There was initially a "quick fire" round where we did some real basic definitions - Confidentiality, Integrity, Availability, risk, etc, etc. Even back then candidates would hit us with the: "Hmm, great question" /tap tap tap tap [Wikipedia definition of confidentiality]
- OutOfHere 2y agoThat is completely ridiculous because the candidate could be looking at a relevant documentation page, which is fair game. Next time, try a voice interview.
- mda 2y agoSince interview process is also flawed in many ways luck based, if you cheat and moderately capable, you can totally stay there and even go up in ranks.
- hiAndrewQuinn 2y agoPeople rarely get fired solely due to technical incompetence, in my experience. Technical competence happens to be much easier to test for than most of the other factors, and it can compensate to some degree for e.g. bad interpersonal skills. This is indeed part of why tech is such an autism-friendly field. But I've only seen a few people get let go for being great team players, excellent requirements analysts, etc., but just totally unable to code - for better or worse. (More common is they get shifted to a role which better aligns with their "true" skillset. Why didn't they get this role in the first place? Again: Hard to test for.)
- kstenerud 2y agoIt's starting to feel more like "cheat the job interview because the cat-and-mouse game has pushed everything to the point where you can't legitimately get an offer for a job you're actually fully qualified for anymore" That's not quite where things are now, but the initial phases are beginning to show.
- pixelsort 2y agoPerception on the degree of severity seems to depend largely upon the size and structure of your social networks. In the post-GPT4 tech economy, your work friends are also the only worthwhile recruiters.
- fasthandle 2y agoThere's a positive association between 1337code interview hurdles and lack of management competence. Someone without the confidence to hire someone without going through a bunch of hoops to justify themselves is even less likely to admit humility and say they were wrong. Which makes places that employ these interview techniques terribly passive aggressive places to be.
- davedx 2y agoFeels like in some sections of the US like big tech, the talent vetting process has evolved into some adversarial battle of attrition where the idea is to grind down the pool of candidates until only a few are left rather than find some potential new colleagues. Tools like this have emerged as a direct response to that process. What a bizarre way to begin a working relationship.
- stego-tech 2y agoSadly, these sorts of “practical interviews” aren’t just limited to Big Tech. There’s been an arms race for thirty years now between applicants and employers, the latter creating new hurdles to overcome and the former finding new ways to bypass unnecessary hurdles. Employers demanded applications instead of resumes, so candidates used copy machines. Then they wanted block lettering, so candidates used type writers. ATS systems could search for keywords, so candidates flooded their resumes with them. Then employers allowed online applications, and the floodgates opened proper - applications from all over the world, desperate for good employment. So they made you create an account, and applicants wrote or used bots to automate it - which became site features like “Quick Apply” on LinkedIn and Indeed. Practical interviews used to be a single round for only the most senior roles, but then employers wanted everyone to be as qualified as those roles but not pay them that much, which started the cheating arms race. Ultimately, the problem remains a glut of qualified talent demanding salaries to pay the bills, and employers who want to pay as little as possible. The two sides are adversaries by default, and few companies do the work of mending the relationship into something more cooperative.
- dingnuts 2y agothis does NOT match my experience as an interviewer. Our company went through piles of candidates who were not qualified and it took us months to find a good fit after going through hundreds of resumes and scores of interviews. I hope the folks we hired are paid enough to retain because there were was absolutely no glut of talent. It was hard to find the folks we wound up hiring. There was a glut of applications, yes, but the vast majority were totally unqualified.
- colinb 2y agoI’m reasonably certain that I interviewed someone using this or something like it in the last few days. Lots of eye scanning while looking above the window they’d been typing into. Pauses. Big pastes. One of my excellent colleagues noticed that the candidate made use of exciting C++ casts before ever defining the variable’s types. Complete inability to explain or debug the code just written. So. Frustrating in two dimensions. First a waste of time for everyone. Second, occasional signs of real ability make me think the candidate might’ve made it work honestly. The fool. This, and this alone, makes me pine for in-person interviews. But I suspect those won’t be back for some time. (For good reasons that are out of scope here.)
- deleted 2y ago[deleted]
- xandrius 2y agoIt sounds like you want sympathy but it just shows that some people just don't understand the power dynamics of an interview. I believe live coding is a scourge and we should get rid of it altogether. Get a tiny open ended project, which requires more design/architecture skills than actually finishing the solution. You do it at your own time and shouldn't take more than 1-2h. You submit it and provide your mental process in writing. We then hop on a call and we can dicuss it further, talking about possible issues and what to do to get it released if a requirement changed last minute. If you can do that, come across amicable, knowledgeable and experienced enough given the seniority of the role, and feel like you are not reading from an hidden ai window then you pass. "Exciting casts" are absolutely worthless in real life, everyone has to balance shipping vs code artistry.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- pavlov 2y agoWhy bother with this style of interview where the candidate is under extreme time pressure and isn’t allowed to use any of the research tools they would normally use in their job? What is this controlling for? It barely made sense when these were conducted in person, and it’s completely inane over video meeting. Instead of grilling people, talk to them. Give them meaningful take-home exercises that you can discuss together during the interview. If they can explain the decisions they made and discuss alternatives and trade-offs, that’s a much better indicator of job performance than pretending to invent a leetcode party trick algorithm in 10 minutes.
- deleted 2y ago[deleted]
- CastFX 2y agoI agree but a big problem remains. How do you make it scale for 100k applications?
- disgruntledphd2 2y agoFilter people out by requiring them to count the number of s's in sensational.
- pavlov 2y agoEmploying someone is renting their labor. How do you find an apartment rental in a large city with thousands of options? You can’t expect to visit them all. And you certainly can’t come up with increasingly bizarre torture tests for landlords to pass (“I might rent from you if you can answer enough trick questions about tenant law within 15 minutes”). To find the apartment, you probably sample enough of the options to form a picture of what’s available, and then make a decision knowing that you didn’t have perfect information but you had to move ahead. The same applies to hiring.
- bix6 2y agoExcept landlords have almost all the power, just like the job gatekeepers. A landlord gets to choose who rents their property so they aren’t under any obligation to answer your questions, they’ll just find a renter who doesn’t ask questions. When you have all the power it inevitably gets abused which is exactly what we’ve been seeing in rental with things like real page.
- briandear 2y agoWhen LeetCode and HackerRacnk shut down, the industry will be a much better place.
- deleted 2y ago[deleted]
- tjpnz 2y agoWhat options are available for the on-site?
- jasode 2y agoJust in case this repo isn't a joke and some job seekers are seriously thinking of this tactic... Companies already know about cheat methods like this and candidates still have to demonstrate skills in-person on a whiteboard in a 2nd round of tech interviews. A high-paying FAANG job isn't going to hire candidates based on just one remote Zoom tech interview. That still may not deter some folks and they'll try to continue to find some cheating technique to use in front of the whiteboard. In that case, some creativity is going to be required. E.g. : https://www.google.com/search?q=chess+cheating+anal+beads https://www.google.com/search?q=chess+cheating+anal+beads The issue is that "chess board moves" is very low bandwidth and can be efficiently encoded into pseudo-quasi- Morse Code vibration schemes. However, if you're asked to "invert a binary tree" or any other open-ended random puzzles, it's more difficult to compress a secret 2-way high-bandwidth communication with ChatGPT into a hidden butt plug. (But then again, if a candidate is actually able to fool people with hidden ChatGPT hacks in a face-to-face interview, maybe FAANG should hire them based on that alone.) Keep on the lookout for a github repo with ChatGPT butt-plug communication.
- throwaway71271 2y agolooking at the despair at https://www.reddit.com/r/cscareerquestions https://www.reddit.com/r/cscareerquestions i am completely certain people are not above cheating even if its not an app, everything is so cheap and small now, 1mm camera and microphone, and very tiny ear piece (invisible with longer hair), or even earless headphones, realtime whisper turbo and groq llama3.3 can absolutely pass most interviews and answer most questions even without the tech, i am sure people are now cheating on face to face interviews as well i worked at fairly big company, and did maybe 500 interviews, i never knew how the candidate would look in advance, so i dont see why cant someone else shows up to the interview, pass it and then the real person shows up for the job
- xandrius 2y agoThe problem today is not that to get into FAANG you've got to jump through hoops and loops like a well-trained cage animal (aka fresh graduate) but that this is getting applied more and more to literally any startup under the sun, even some with 0 funding. As a first engineer for a tiny startup, show me some past projects and let's grab a (virtual) coffee talking tech, that's all. Now it's like McDonald's requesting to perform Michelin level dishes under 45 minutes with Gordon Ramsey in the room while still ending up flipping burgers minimum wage at the end of it all. If this keeps going and getting worse, I wouldn't mind a butt-plug communication, as long as I can pick my favourite AI (I personally kind of prefer Claude right now over ChatGPT.)
- deleted 2y ago[deleted]
- neilv 2y agoOne of the few things that I hate more than popular 'technical interview' methods is... lying.
- sirwhinesalot 2y agoI don't get why companies waste time with this sort of interview that just turns into this stupid cat and mouse game. Even ignoring this sort of software, someone cramming leet code exercises is the same thing in my view (they won't remember or use in their work 99% of that stuff). Just talk to the person, talk about the sorts of problems you expect them to solve. Can they understand the problem? Do they have ideas on how to solve it? Maybe give them a small take home exercise related to the work they'll be doing. Discuss it with them to see if they were the ones that actually wrote it. Sometimes you don't even need the take home exercise. If they have niche knowledge on the subject you care about they're probably good enough by default (since they are clearly invested and studious in that case). I only hired a few people in my career but every single one was a huge boon to the team, still around even after I quit as a manager. I didn't even give them a take home exercise. Didn't need to.
- xandrius 2y agoSame! This topic gets me soo irritated. If someone is supposed an expert in a field (hence them being an interviewer) how can an average joe fool them into thinking they know more than they actually do during a conversation? Even now with LLMs generating almost real time, there is always a slight delay and we all know that LLMs are far from perfect. It seems the tech community on one hand complains at how bad LLMs are for doing deep tech while at the same time fears them as they can make any candidate pass interviews. So which one is it? Also, the simplest trick even old-timers professor already know: 1) Prepare an open ended question about X 2) Ask it to ChatGPT, Claude, etc. 3) Take note of the titles, keywords used, etc. 4) Ask it to the candidate 5) Compare responses
- jochem9 2y agoNot all interviewers are experts. If you are building out your business you will be hiring roles that don't exist yet. Of course that brings different requirements for that first hire, but judging their skill level might prove very hard.
- IshKebab 2y agoI think it's worthwhile doing simple coding tests to check that the interviewee can actually code. It seems crazy but I've interviewed plenty of people that couldn't write a simple for loop. I agree asking difficult leetcode questions is less useful because it depends so much on whether people have practiced (or now whether they cheat with AI). Often these questions are so difficult there's no way you could come up with a solution in an interview situation if you hadn't already seen it. Getting rid of all coding interviews is stupid though. Other industries would kill to have such an easy to test skill! We shouldn't throw away such a nice thing just because some people take it too far.
- qwer1234321 2y agoI might be calling obvious but if candidate uses something like this, is this not lying to potential employer? I'd rather be rejected than got job I'm not qualified to do, that's why I never prepare for interview above studying the actual company- and that is more to give me proper signal if I want to work for them, I want to be motivated to make move on more than just money. I also have never done leetcode, though I like competitive programming problems, especially bot programming contests on codingame.com
- tamimio 2y ago> that's why I never prepare for interview above studying the actual company You and me both. I just read a bit about the company before applying for the job. In the interview, I ask the interviewer only two questions: what’s the management style? Is it a servant leadership style with autonomous work, or is it a hierarchical task-based one? The second question is, what does an average day-to-day job look like? Sure, there are few stressful days and others are relaxed, but on average you can tell. Is it chaotic, always running against deadlines and having new tasks thrown at you, or is everything sorted out properly? Sometimes, I also ask about the meetings. I personally don’t like useless meetings. A lot of middle management uses meetings as part of the power dynamics and exerting power rather than actually discussing and solving issues.
- pmg101 2y agoOf course I would also want the information from your second question, but it never occurred to me that asking directly would get me that information. I guess if it's a leaf node they might just tell you the truth, but in general you'd expect the interviewer to say what they think you want to hear wouldn't you?
- shlant 2y ago> I'd rather be rejected than got job I'm not qualified to do As many people have pointed out, the interview process has diverged from real-world, day-to-day tasks you would be expected to accomplish once you get the job. Not actually being able to do a leetcode test (either due to lacking knowledge or interview stress) might not have any reflection on someones ability to succeed in the role they are interviewing for. How many people are dealing with sorting algorithms or traversing binary trees without access to the internet as part of their job?
- rvz 2y ago> Never fail an online technical interview again All these online assessment solutions such as leetcode, hackerrank, etc are essentially close to useless given all these AI based cheaters and LLM products being used by candidates. On-site interviews it is then, and this time you cannot use those LLMs to save you.
- wojciechpolak 2y agoHow often do doctors have to prove they can still recall every obscure medical fact when changing hospitals? When a lawyer moves to a new firm, do they have to retake the bar or answer weird hypothetical cases all over again? Aren't their credentials, past cases, and reputation usually enough? Why do we in tech keep forcing experienced professionals to jump through these puzzle-like hoops instead of trusting their track record?
- bilvar 2y agoBecause it's much easier to get-by faking it as a software engineer compared to a lawyer or doctor. We've all have had these colleagues - and I've seen them jumping to other companies easily.
- karaterobot 2y agoI don't know if that's true. I'm not saying software engineers don't fake it, but I think many doctors and lawyers are faking it, too, if we're defining faking it as looking up information, or even assisting with your core job responsibilities. I don't consider that faking it, but I think that's what we're talking about, isn't it? One time I asked my optometrist what he was doing when he left me in the examination room for 15 minutes. He said he was looking up the details of the medicine he was going to prescribe me. I don't consider that incompetence, and it sounds a lot like what I do when I need to know about some technical issue. Everybody in my industry knows that Googling stuff is a huge part of the job, but you're not allowed to acknowledge that in an interview. I think it is actually an interesting and relevant question: why don't past performance or job references play more of a role in hiring, compared to technical screens many people hate and think are pointless? Checking references and former calling employers—if it's done at all—is often a pro forma final step, rather than the first step, which might plausibly be a more effective way to screen candidates.
- NegativeK 2y agoI don't think that's faking it at all. Being able to research is part of any knowledge based profession. Faking it is shirking your duties. Most bad medicine doesn't result in obvious harm/cost to a patient or a board complaint. A poorly managed medical practice that runs off support staff and makes patients wait three hours isn't going to threaten a license, and that's faking practice management. A primary care doctor who punts anything beyond the very, very basics to third party vendors or specialists is faking it. In my major metropolitan area, general practice veterinarians are often absolute crap at doing their (required) medical notes. If you ask them for copies, they'll send you invoices. If you push for real notes, they'll use their 48 hour response window to make something up. They almost always get away with it because during future visits, the vet (whether it's the same person or not) will ask a bunch of questions to (necessarily) fake knowing what was done previously. It's obviously going to affect patient care and cost, and if you look at board complaints that aren't thrown out, the "punishment" is usually because the board saw that the vet has shit for notes. And don't even get me started on vets using AI, which results in hallucinations in the notes -- which I think are notably worse for patient care than just empty notes. I think that's a good example of even good doctors faking it.
- sangeeth96 2y agoSomeone beat my lazy ass to it XD Had something in the works when I first got to know about multimodal GPT and electron made it super simple to prototype something. Refined it a bit to even do the invisible window thing after feedback from my peers. Didn’t have the ahem courage to do it. Also recently noticed there are actual companies around this space who are marketing this off as “interview assistance”. Two of them I’m aware of: 1. https://www.senseicopilot.com/ https://www.senseicopilot.com/ 2. https://www.finalroundai.com/ https://www.finalroundai.com/ —— I’m sure the knee-jerk reaction from the tech industry to these tools would be: 1. Return to in-person interviews 2. Force install spyware/vanguard like crap to scan your whole personal computer before doing the interview If you are a tech leader/senior engineer with influence on these matters, I *HOPE*, that this is not the path you take. It’s high time FAANG and others stop gatekeeping with these useless rounds that are equivalent to forcing people to hand-solve complex math equations instead of using calculators. I also partly blame the YT influencers who keep peddling this BS but they will stop if the companies themselves take a stance. You can rather use the same 1 hour slots to have a conversation, pair program on a realistic problem required for the role you’re hiring for where you and the interviewee can both benefit from the knowledge transfer. I’ve tried this in many of the interviews I’ve conducted and always went away learning something from an excellent candidate or a candidate who didn’t meet the expectations, going out knowing where they fall short.
- rvz 2y ago> If you are a tech leader/senior engineer with influence on these matters, I HOPE, that this is not the path you take. On-site interviews should be the standard path to take to detect the frauds and cheaters before they become potential bad hires. Online technical assessments now have little added value given that AI tools like finalround.ai, etc just make it easy for frauds to cheat the technical interview. This is actually a strong argument to a return to on-site interviews and should just draw the line on this and detect the frauds altogether.
- ttyprintk 2y agoThe fair compensation to fly someone internationally for a face-to-face might make sense if they stay on for 15 years.
- dweekly 2y agoThe long term "solution" here for candidates will devolve to a second device that uses a camera and microphone to monitor the device from which they are being tested and presents feedback on a second monitor visible to the applicant but not interviewer. For interviewers, the solution will be proctored testing centers with metal detectors and all devices required to be in a locker outside the test area. (This is how the FAA conducts its written exams.)
- PartiallyTyped 2y agoThis whole thing is getting ridiculous and is a symptom of wealth inequality and our cutthroat society that demands we pay for the privilege of being able to work.
- dweekly 2y agoMaybe? It's also true that pretty much the most powerful AIs in existence are available to anyone willing to fork over twenty bucks a month and you'd be hard pressed to get better algorithmic outputs as a nation state with $100m to blow. So there's an angle that suggests that this is not primarily a story about wealth inequality but rather savvy in wielding tools.
- tail_exchange 2y agoThis kind of tool is why companies will push back towards on-site interviews, and consequently RTO, given that the people who interviewed are the ones who could easily go to the office to be interviewed. Honestly, it's hard to blame the company. If the employee is willing to lie and cheat even before they were hired, it's expected that companies will take away their tool to do it: remote calls and WFH. As always, the honest worker gets shafted.
- tmpz22 2y agoYeah tech has always been a meritocracy built on ethics and we need to remember that.
- tail_exchange 2y agoTrust. Some people think that trust just means a someone blindly trusting another, but of course that's not how it works. People will be trusted to an extent that they can be trusted, and this goes for company-employee relations too. If people prove they cannot be trusted with remote interviews, companies will take remote interviews away. This was a luxury given to us during the pandemic, and it's now being rolled back because of tools like these. I remember people here on HN complaining about CEOs mandating RTO without data to support it. Well, this certainly doesn't help.
- archagon 2y agoThis is still not data.
- rqtwteye 2y agoFor a while my company did aptitude tests that basically are measuring certain types of intelligence. I personally found a very good correlation between somebody doing well on those and them fitting into the team. For some reason (legal?) we stopped and now we are doing interviews that have a way worse correlation.
- codedokode 2y agoI wonder: if a LLM can answer typical interview question better than a human, what kind of questions should we ask so that using a LLM becomes worthless? Something that LLMs are forbidden to talk about? Something that LLM doesn't know well? In my experience, LLMs are pretty bad with low-level languages like Rust/C when you need to deal with pointers and ownership, avoid extra copies. Another option is to ask for something impossible, as LLMs usually do not admit inability to solve the problem and prefer to hallucinate a fictional solution.
- therealdrag0 2y ago“Forbidden to talk about”. That’s thought provoking and hilarious. I’m just imagining politically incorrect leetcode descriptions.
- jarsin 2y agoI expect version 2 to be an AI clone of myself so I don't even have to take the interview.
- rahimnathwani 2y agoHere is the bit that makes it invisible: https://github.com/ibttf/interview-coder/blob/main/electron%2FWindowHelper.ts#L105 https://github.com/ibttf/interview-coder/blob/main/electron%...
- vunderba 2y agoGreat job on further eroding the trust from a prospective employer. Require a formal degree in CS? That's gatekeeping. Need to pass a whiteboard exam? Not representative of the actual work. Live coding session? Biased against people who don't perform well under pressure. Take home project? It's too much work to do for free. Showcase a personal portfolio? Not fair to people with families or other obligations. Either you enforce a minimal level of competency upfront in the form of an academic degree, industry-standard exams such as a PE Exam, etc. OR you push the entire responsibility of vetting prospective applicants downstream to the employer—which is exactly why interviews are multiple week long gauntlets. The tech world likes to complain about all of this but other occupations 100% DO have high standards - it's just that it's paid up-front. Want to become a lawyer? - You've got to pass the LSAT, get into law school, and pass the bar. Want to become a doctor? - You've got to pass the MCAT, get into medical school, and do residency. Want to become a pilot? - You've got to get your PPL, pass your check ride, do your IFR, multi-engine, commercial rating, ATP God there are some days that I ABSOLUTELY HATE THIS INDUSTRY.
- thot_experiment 2y agoIf your hiring process is broken by people using AI then the process is either not evaluating the right things, or maybe it's fine because realistically speaking you're hiring a dev who's going to be using AI anyway.
- sgarland 2y agoI work in infra. I don’t want an incident to be the time that I find out my coworkers don’t actually know how anything works, and are frantically posting questions to ChatGPT. That said, I also don’t know how to adequately test, other than to carefully watch people’s eyes during questions; even that isn’t foolproof.
- viraptor 2y ago"tell me about potential issues while doing X", then dig into answers? We may get there one day with LLMs, but currently it would be extremely hard to keep the conversation context and process the longer answers while talking to someone. The voice recognition quality and generating speed would be an issue too if you wanted to monitor the whole conversation. Currently the cheating mostly aims for coding tasks because they're pretty simple to do and self-contained. Much less for helping with a good dialog.
- trunnell 2y agoI empathize with new grads. It can be hard to stand out especially at the beginning of your career. But I predict this won’t do what the authors claim: it won’t fix anything, and it won’t change technical interviews that much. At most it might increase the number of companies that require remote candidates to fly in and interview in person to do the same practical coding problems they used to do online. Which would be the standard from just a few years ago, anyway. Practical coding problems (also known as work simulation interviews) are the most effective and least biased tools available for evaluating candidates. Not every hiring manager does a good job using these tools, but if you see misuse, take it as a sign that the manager might not be good to work for. I have trouble understanding the authors thinking here that anything would be worth the cost of this: it’s a trust-busting tool where people are deceiving their potential employer. Trust is foundational to everything, in work and in life. We need more trust in our society, not less.
- bastawhiz 2y agoThe thing I see people getting wrong with live coding interviews (speaking as an interviewer) is that their code rarely matters. If you write some code, I might ask "So what kinds of failure modes will this code need to account for?" Whether you named your variables well or wrote comments didn't matter. If you can't talk about your work or explain your thinking or discuss tradeoffs or question constraints or show your thought process, that's what's going to fail the interview for you. Getting the exact right solution is bonus points, but the job isn't to consistently get things 100% right writing code by hand.
- hnfong 2y agoWhat you said is true, but it's much easier to do if you (as the interviewer) have full control over the questions and evaluation of the candidate in your interview. For larger orgs at scale, often the interviewer needs to stick to a script, ostensibly to avoid personal bias (which probably does happen to a degree), and to make scores between interviewers comparable. In such cases it might be difficult for the interviewer to argue that the candidate has done well despite "not getting the answer correct". A lot of the grievances people have for the interview process are artifacts of scaling the interview process for the largest tech companies. As for the smaller ones, it's basically good ol' cargo-culting. Small startups that follow FAANG processes probably don't realize they can customize their hiring process to their advantage.
- dambi0 2y agoIf you wanted a completely invisible system you could do it via audio. Speech to text and back again and use a two pairs of same model wireless headphone with say the left ear connected to a hidden device and the right ear to the interview machine. Other than a bit of practice it would be completely undetectable visually. It’s extra nice that we encourage thinking out loud. We should assume people are doing this. The solution is interviews that don’t require information that can be looked up or in-person interviews or permitting the use of AI.
- xutopia 2y agoI simply give option of candidates to use AI helpers they want. Still a lot come up short. All you need is a baseline that is the same for every candidate and let them differentiate themselves with the same tools they'd use in real live setting anyway.
- hackable_sand 2y agoTech companies are so tall, they've become scared of their own shadows.
- maxnevermind 2y agoI'd rather just refuse to have a LeetCode interview style interviews all together.
- rolandthomas 2y ago[dead]
- gtirloni 2y agoIn my recent experience, CS courses should incorporate acting classes to help future job seekers. Right now more interviewees than I expected are doing a really bad job at trying to fool interviewers with LLMs and tools like this one. It's blatant obvious and too easy to catch cheaters (and then they are banned from any future positions on ethical grounds).
- kazinator 2y agoI'm thinking: could you just not use second (or third ...) monitor, while only the primary desktop is being recorded?
- biggedyb 2y ago:) this took a bit of thought to get the right words.. I am ALL for this, for a number of reasons: - will help people 'find the words' - it's pretty easy to spot someone who doesn't actually know what they are talking about, even if what they are saying is correct and valid - actual technical capability is only a small part of the job. :D so, 'find the words', interviews can be stressful as all hell for both sides, as someone who assumes the power of the interviewer that's a heavy stick to wield over someone who is trying to explain why they very much want to be paid by you. Communication is _hard_, even if you gel with someone, actually explaining something to them in a way that both people completely grok can be really hard. If someone joined and front and centre said 'I'm using AI to help me with my answers because I find interviews hard' I'd sit up straight and be all 'cool! lets do this!' 'Are you sure about that? what did it feel like?', ok so this one is VERY subjective but it's a pretty solid baseline to go on. Interviews should be a verification that they actually are interested in working with you and that you are interested in working with them. So to be able to make that judgement you and they have to get to a point where the interview can be concluded, and whilst there's a fair chunk of other stuff to go through, technical understanding is important. To be clear, lack of it is fine, that's why different levels exist, you want to be a software engineer but you have no experience or skill? Well ok but you will be starting at the bottom. Now to that, there will sometimes be people who want to 'show a higher level of capability than they have current attained' and that's what my job is, to try and see that when it happens. It's pretty easy to do to be fair, humans experience deeply, computers do not. So if I want to talk about Entity Framework Linq helpers then I'll be looking for the spark of excitement when it suddenly pinged to them that this is WAY easier! And pretty! It makes the queries dance! ..and yes, that's my experience, but it doesn't take much to guide people on sharing their excitement in this stuff, and frankly if you get to the end of the interview and _haven't_ found that excitement or deep interest then skill level aside, you've probably already got your answer. Finally is around how relevant is a specific level of technical capability? As I said, that's what we have graded levels for (junior, senior etc), and besides this whole bag (for me at least) is much more about how you solve problems than what you specifically know. I've seen people wildly switch careers into software and sure, there's a lot of hand-holding and stuff early on cos it's a proper throw into the deep end, but it's the mind, not the current capability.
- edude03 2y agoHonestly I worry more about hiring an oppositional / defiant / insubordinate “asshole” more than the AI use itself because what does it say about an applicant if their first instinct when challenged is to say “your process is invalid, I’m going to cheat instead”?
- RecycledEle 2y agoAs much as I hate cheating, I wonder if these apps will lead to invisible and ubiquitous AI helping us navigate life. As a teacher, it is my job to make sure my students control that AI instead if being controlled by the AI.
- markgoho 2y agowhat interviews get wrong is they typically assess a candidate's lower level thinking skills rather than their higher level thinking skills cf. https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ Typical interview question (lower-level thinking, remember): List the primitives in JavaScript. (remember) Better interview question (lower-level thinking, apply): When would you use an object instead of an array in JavaScript? Really good interview question (higher-level thinking, evaluate): Here's some raw data that we'd like to put in a database, what data structures would you recommend and why?