7 ms·
> Such test filter out the obvious frauds. if you need garbage like leetcode to filter out "obvious frauds" you got a whole lot of problems in your company/tea
by bdangubic 2mo ago
> Such test filter out the obvious frauds.
if you need garbage like leetcode to filter out "obvious frauds" you got a whole lot of problems in your company/team/...
- WalterBright 2mo agoA colleague of mine came to me for advice. He wanted one of those high dollar jobs at a well-known MegaCorp. He knew a leetcode test was used by them. I suggested that his top priority was to study the leetcode books for the next 3 weeks. I told him those three weeks would be the best ROI he'd ever make. He did so. He aced the test. He got the job with a huge offer. He agreed that the ROI was out of the park! P.S. I'm curious what your test is for obvious frauds?
- timr 2mo ago> He did so. He aced the test. He got the job with a huge offer. He agreed that the ROI was out of the park! ...and everybody clapped? I have news for you: passing an online leetcode is now so easy that an AI can do it. If you think you're catching the cheaters, you're just wrong. Leetcode was always a stupid test of competence, but it was cheap and it had a high true-negative rate, which filtered the riff-raff. That isn't true anymore.
- WalterBright 2mo agoHe went in for a monitored taking of the test.
- timr 2mo agoSure, sure. So now we have the worst of all possible worlds: an expensive interview process where the candidate has to be onsite, and a stupid evaluation mechanism that doesn't at all reflect real-world job requirements. And in exchange for the inconvenience of an in-person interview, the candidate gets treated like a number and is met with a stupid memorization gauntlet that tests a skill that quite literally will never be used again now that we have AI. Sounds great. Man, there are times when I really hate this industry.
- bdangubic 2mo ago> Man, there are times when I really hate this industry. Or as I'd like to call it - Wednesday :)
- nocman 2mo ago> an expensive interview process where the candidate has to be onsite I'm sorry, but if I'm doing the hiring, the person who isn't willing to come on site to do an interview will be one of the first people off the list to hire (I mean, unless it is a remote job, and the person is nowhere near the employer, obvious exceptions would apply). In-person interviewing can be extremely valuable. Will it filter out every bad candidate? No, some people excel at faking it. However it can be a big help in finding a good candidate.
- timr 2mo ago> I'm sorry, but if I'm doing the hiring, the person who isn't willing to come on site to do an interview will be one of the first people off the list to hire Nobody said anything about not coming on site. That's how we used to do it, in the ancient pre-history of 2019. What I said was that if you bring someone on site only to leetcode them, you deserve not to hire anyone. The downside, of course, is that you don't have the cheap, dumb pre-filter of a memorization test anymore, so you'll have to figure out a better way to winnow down the applicant pool whom you're willing to pay to bring on site. Maybe finally software engineering will achieve a level of interview maturity seen in other professional fields...but I doubt it.
- WalterBright 2mo agoIf you're willing to travel to be on site, you should be willing to study for the leetcode test.
- eloisius 2mo agoSo if it’s just a shit-test to see if the applicant has enough grit to grind through an arbitrary filter, and not to measure their aptitude, why not use Minecraft or something? Come into our office and show us how long it takes you to kill the End Dragon.
- zzrrt 2mo agoDoesn't this imply that, had your friend not taken your advice, he would have failed and thus been an "obvious fraud", but a mere 3 weeks transformed him from fraud to competent programmer?
- inigyou 2mo agoNo it means he fooled the interviewers
- wyclif 2mo agoI don't think he "fooled the interviewers." The point of the story is that he focussed down on what the company's culture decided was important.
- SoftTalker 2mo agoIt's like cramming for an exam. Yes you can get enough information in your short term memory to pass, even do well. But a year later (or a month) it's all gone. You never really knew it.
- WalterBright 2mo agoTrue, but it shows a willingness and ability to do the work.
- rightbyte 2mo agoSuch recruiting selects for KPI hunters. It is probably bad for an org to select for people that game it rather than doing "what is right". A dice throw and some chitchat would be better recruiting.
- WalterBright 2mo agoThese arguments all sound familiar to the "high stakes testing" debate about the SATs. The tests were unfair, did not measure college readiness, could be gamed, etc. Then universities dropped the SAT requirements. Even MIT was suckered into dropping them. It was a disaster. It turns out the SATs were a very strong predictor of college success. MIT and others reinstated the SATs.
- selcuka 2mo agoI'm confused. Your example contradicts your point. How is it a good filter for fraud if it can be aced by only studying for 3 weeks?
- WalterBright 2mo agoThe frauds and incompetents don't want to do the work.
- esseph 2mo agoNo but they want the money, so they'll do the bare minimum to get in that door.
- tombert 2mo agoAll this proves is that BigCo puts a lot of value in these idiotic leetcode tests. It doesn’t imply at all that it gets rid of “obvious frauds”, and in fact seems to suggest the opposite given that someone was able to “ace” the test just by buying a book and studying it for three weeks.
- watwut 2mo agoYour friend does not sound like an obvious fraud. He sounds like someone talented and capable of learning.
- marcus_holmes 2mo agoLet's assume that instead of leetcode, the company had decided that the qualifying test for programmers would be sketching a vase of flowers (about as relevant as leetcode for commercial programming). Your friend would have spent three weeks practicing their sketching, right? And they would have made a decent drawing at the interview. And that would have the exact same ROI as spending those weeks on practicing leetcode. The point is not that studying leetcode is good ROI. The point is that large orgs are testing for something that isn't relevant to commercial coding.
- WalterBright 2mo agoWhat do you use to narrow down 100 candidates to 2?
- marcus_holmes 2mo agoWell, in this case, sketching a vase of flowers. This is classic Streetlight Effect [0]. Just because it's easy to test leetcode, and it appears to have something vaguely to do with coding, it gets used. Like I said in other posts, I would use the "pick a bug and fix it" method. But I'd also screen down from 100 candidates to ~20 or so before getting into technical interviews. And, again, I don't think leetcode will scale well to 100 interviews either. You need technical staff in that interview room, and if you subject your team to 100 technical interviews using leetcode they're not going to be happy about it. [0] https://en.wikipedia.org/wiki/Streetlight_effect https://en.wikipedia.org/wiki/Streetlight_effect
- esseph 2mo ago> P.S. I'm curious what your test is for obvious frauds? I interview them and ask questions! Seems good so far.
- bdangubic 2mo agoWe pull in everyone from the Team who is available at interview time into a room with a big a$$ whiteboard and we talked to the candidate. We work through the problem, either current one we are working on or recent one that we completed or we have few "golden ones" we like to have a chat about. Afterwards the Team decides whether or not they want to continue solving problems with this candidate or not. Exactly what we do on day-to-day basis is what we do when we are trying to grow the team. I would guess that 85% of my team would fail leetcode-style screening right now (which is good cause it is garbage)
- leptons 2mo agoI took an online technical interview recently where there was a check-box they made you click before every section, promising you would not use AI to solve the test for you. After they rejected me I had to point out that their test is selecting for the cheaters.
- WalterBright 2mo agoI would view skeptically any testee who turned in a perfect score. When you check in at the airport, the machine asks you to certify you are not carrying banned items. Of course, one wanting to break the law and carry banned items would certify it. The purpose is that if you get caught with the banned items, and you certified you did not, it's an extra charge they can paste you with.
- inigyou 2mo agoOnce we carried dangerous (not banned) items and selected yes and the first action by the attendant was to come over and say we selected yes by mistake and she selected no for us
- sscaryterry 2mo agoHonestly this is purely because time is money. Its a blunt instrument that is surprisingly effective. Unfortunately, it also removes a fair amount of competent people too, that simply don't think/operate in a leetcode mindset.
- WalterBright 2mo agoI'm a little hesitant to accept the notion that people are really great engineers but cannot handle basic algorithms and data structures.