8 ms·
I moved into running development at my company two years ago - and one thing I quickly learned is that I had to do some level of skill verification with job app
by indymike 5y ago
I moved into running development at my company two years ago - and one thing I quickly learned is that I had to do some level of skill verification with job applicants. Why? Because many of the applicants I was interviewing could breeze through anything non-technical, and would do well if we stayed on the happy path (i.e. show me things you've built, tell me how this function you built works). When I asked them simple questions like, what type will this python expression return (and yes there was -> int: in the signature, they would be incorrect.
So, I gave a few interviewees a code test, and they failed miserably. Mind you, nothing hard - it was a test can I give you a simple function to write, and can you 1) declare a function or class that accepts the right inputs, 3) does what the spec asks, and 3) has the correct output format. The test took 10 minutes to an hour, and was totally open book. The result was pretty staggering. 2/3 of developers would fail the test, and not for esoteric reasons. They would fail because they used the wrong boolean operator, make simple type-casting mistakes, or fail to return data in the correct type (i.e. return a string and instead return a bool or an int). When they failed, it wouldn't be just one thing, it would be two or three mistakes.
So, now I code test everyone, and it has nothing to do with your salary. If we expect you to program in your job, we will verify that you can actually string together 10-15 lines of code. I've only had a handfull of applicants call the test leetcode (I think it's a lot simpler than that), but the reality is there are just a lot of people who can't really code out there.
- gverrilla 5y agoOut of curiosity: this test you gave your interviewees, was it done on a computer or pen & paper?
- indymike 5y agoComputer. Web based, copy/paste enabled, no cheat tracking. So in short we let them use an IDE, cheat and copy/paste and they still flunk it.
- lfowles 5y agoHeh. I'll never forget the first time I started an interview process with an automated online test with a "cheat tracker". What it ended up really doing was flunking me for alt tabbing to documentation...
- throwaway2037 5y agoSorry, I am laughing out loud at this response. I love how you wrote "cheat". Like! I know... not my best HN post. :)
- cercatrova 5y agoI don't understand your comment, what was funny, I don't see them writing cheat unusually?
- indymike 5y agoHopefully, cheat isn't a normal state :-)
- jrochkind1 5y agoI'm surprised that someone who couldn't say what type a python expression returns could do well on "show me things you've built", "tell me how this function you built works". Do you have an expalanation/guess? Were they giving you rehearsed answers to specific code that, well, they didn't necessarily actually build themselves?
- indymike 5y agoBest guess is someone else did the work and explained it to them.
- Aeolun 5y agoYou’d pop that bubble in a few directed questions wouldn’t you?
- hallway_monitor 5y agoYou definitely can but it's not easy. These people are really good at bullshitting, that's why you have to back them into a corner. The simplest way to do that is to have them prove, live in the interview, that they can write code.
- marcosdumay 5y agoYou mean stuff like asking what type a function returns?
- seanbarry 5y agoI have had the same experience. Some things I've witnessed: - multiple "senior" data scientists who cannot write a function definition in python - a "senior full stack engineer" with 5+ years of web dev experience who took 20 mins to change the width of a div in a web app - many "mid/senior level" full stack engineers who don't know the difference between sync/async, or have any clue about how the JS event loop works There are so many candidates out there who have what looks like impressive CVs, but can hardly string a program together.
- vsareto 5y agoYour first two are pretty bad (possibly panic induced), but the last one might not be realistic. You can rack up multiple years of experience working on low user count apps without knowing those things. You can still write Javascript and solve business problems without knowing those things. But also, seniority doesn't determine if someone knows those things. If I asked you for a list of things I should know by the time I get to mid/senior, someone else will have different expectations. Everyone is kind of inventing their own requirements on what someone ought to to know at junior/mid/senior levels.
- seanbarry 5y ago> You can still write Javascript and solve business problems without knowing those things. It depends what the requirements of the role are. For some frontend React work maybe, but the engineers we were trying to hire would be working on APIs and processes that handle high volumes of streaming data. In that context using sync methods over async would significantly impact throughput. When asked "why are you using a sync method here" the answer would often be "I want the code to pause here before the next line executes". It's a fundamental misunderstanding of what's happening under the bonnet that IMO is required for this sort of task. Also - I agree that labels like "mid" or "senior" shouldn't carry a specific list of knowledge requirements, but they are useful to set benchmarks for what a candidate should be able to roughly offer in exchange for the level of comp they are requesting.
- e4e78a06 5y agoIf you don't know the difference between sync and async you're not qualified to solve problems with any realistic level of scale (>10 concurrent users which is a very low bar) or complexity (have you really never had to code logic that uses 2 or more concurrent network calls?). Might be excusable for a junior engineer but definitely not someone who describes themselves as senior.
- chrisweekly 5y ago> "reality is there are just a lot of people who can't really code out there" -> reality is there are just a lot of people who can't really code -- IN THE ARTIFICIAL CONTEXT OF AN INTERVIEW - out there. FIFY.
- cloverich 5y agoMy pet analogy for this scenario is: Go to all of your employees. One by one ask them to solve a programming challenge for you. Let them know if they don't produce a suitable result (to your liking) in the next 40 minutes, you will fire them immediately. See how they do. I don't think its wrong to do these types of interviews. But interview stress is real, and it affects people in strange ways. I bombed one of these interviews once and the interviewer was convinced I was "not a front-end programmer". I had just left a job where I had written literally hundreds of thousands of lines of (modern) front-end code, daily, for the last several years. More complex than what I'd be writing at that company (I was eventually hired in spite of this particular interview). And oddest of all, that interviewer was (very soon after) let go. Such is the world of programming interviews. So yeah. I don't think its necessarily the worst. It might even be the best way to interview candidates. But we all need to take our asessemtns with a huge grain of salt and stop being so judgemental.
- cercatrova 5y agoYour analogy doesn't really seem too similar to interviews. There will be more stress about getting fired and losing your income than potentially failing an interview which has no effect on your current income, since you still have your current job while interviewing, presumably.
- cloverich 5y agoThat would be the case for all new grads (substantial portion of applicants), first time applicants (who may be working some other, terrible job), poeple who were laid off / fired, and people who have taken time off. Its not meant to be a perfect analogy, its mean to nudge people into the realization of how stressful interviewing can be. Especially if it is for a company you really want to work for.
- haspok 5y agoAre you US based? Here in Europe this sometimes happens (candidates can't code at all), but _very_ rarely, maybe 1 out of 100? Especially as you move eastwards, the base coding abilities start to improve massively for the average candidate - in the UK, if you are stupid you can easily get by on the social welfare system, in Eastern Europe, if you are stupid, you starve. As simple as. There was a time I used to ask Fizzbuzz-style warmup questions on interviews, but then I stopped because the signal it gave was pretty much useless. The only positive case I can remember was when a tester applied for a dev position.
- Zvez 5y agoI worked in Russia and used to think this way too. But once we managed to hire person how couldn't code. At all. He was intelligent during the interview, talked his way through system design section. But during the probation period he basically couldn't do anything, even simple stuff. After that I started to block any candidate, who couldn't provide code solution for at least simple problem.
- TameAntelope 5y agoEr, that explanation doesn’t really hold, considering if you’re stupid you starve in the US too, and we get tons of idiots applying for jobs they’re completely unqualified for.
- nopenopenopeno 5y agoWell, a python return type doesn’t really mean anything except to a type checker.
- willcipriano 5y agoRun mypy on it and see if it complains. If it does I think it will even tell you what type is being emitted instead.
- nopenopenopeno 5y agoThe question wasn’t what mypy will do. The question was what type will the function return.
- willcipriano 5y agomypy error: function 'x' returns type 'y' expected type 'z' I don't recall the exact message but I think it will emit something along those lines.
- comonoid 5y agoThe question wasn’t what mypy will do. The question was what type will the function return.
- deleted 5y ago[deleted]
- p_j_w 5y agoThat was my first thought as well. If someone asked me that during an interview, I might assume the type annotation for the return type was something meant to throw me off.
- indymike 5y agoThe value being returned in the correct type is especially important in languages without static type checking. The next line may try to add the dictionary you returned to an integer or something crazy.
- Draiken 5y agoI've had similar experiences to yours, but maybe this doesn't fit the leetcode people are complaining about. We hire with a similar setup, but we try to keep the test extremely simple and close to reality. Something they would likely encounter on their day-to-day. The majority of the developers fail at it. Senior, mid-level, juniors, etc. It doesn't matter. Which always surprises me. People who claim to have 5-6 years of experience can't even refactor some code. Or they can refactor and can't explain why they'd do it. It's bizarre. But I would never classify this as leetcode because these are common tasks that any developer should be able to solve and not purely algorithmic questions that will rarely (if ever) be used in real life.
- avgDev 5y agoIf you have 0 passion for programming and land a gig at a corp that doesn't care about code quality at all, you could do that job for few years and just coast/do bare minimum. I'm an IC and I could deliver software with all methods in a single class, nobody would care I think, as long as it worked as intended. I happen to like programming and as I am improving my skills every time I touch an existing project I look for things I could have done better. If major changes are being made, I often upgrade frameworks or do a partial rewrite to improve things. I know people who just do stuff without trying to understand why. If I cannot explain why I am using dependency injection and what problem it solves, I am not doing it correctly.
- rr808 5y agoAgreed you need some tests but what you're asking is a different order of magnitude than leetcode problems.
- phphphphp 5y agoI don't mean this to be incendiary, but, do you think perhaps this reflects on your recruitment pipeline, the standards you hold interviewees during the discussions about their experience, and the way you review resumes? There is of course a possibility that someone has rehearsed their interview happy path down to the word and they can easily describe complex technical concepts without understanding them because they're quoting a script, but an interview that can be gamed in such a way may highlight a problem with the interview. If most of your interviewees are failing to write basic code, the issue probably isn't the applicants.
- visch 5y agoI've had a very similar experience to this, these are literally with candidates that are extremely "over qualified" based on their resumes. We're talking masters degrees, doctorate degrees, etc. They can talk like they have years and years of experiance at this place, that place, you ask about projects they'll dive into high level details about them, everything sounds good (can still filter 80% out with these questions to be fair). Get to the code question, boom fail. I dislike leet code, but you do need to be able to code something (Even if it's syntactically incorrect) on the fly and reason with me about what and why you're doing something.
- cloverich 5y agoFor starters having a masters or PhD implies _less_ programming experience, generally speaking. Its a huge plus for a variety of positions but IME PhD or masters candidates were always a little more raw engineering wise. The way you get good at coding solutions quiclky and on the spot is by writing a lot of code.
- throwawaygh 5y agoParticularly PhDs in Math, EE, or areas of CS that aren't engineering heavy. You hire them to solve problems that your engineers can't solve, not (just) to write code. Masters degrees are a total crap shoot. I honestly ignore them entirely unless there's a thesis.
- deleted 5y ago[deleted]
- throwaway2037 5y agoWoah. This is a great post! Zero trolling: Can you share your interview question? We would love to have something similar! Deeper question: Did you think about why some people are able to "show me things you've built, tell me how this function you built works", but fail miserably on a straight-forward coding test? Sometimes, during an interview, it seems obvious the person is very intelligent, but having a bad day. Some part of interviews is pure performance to me.
- neilsimp1 5y agoNot OP, but we often include these two in our interviews: 1. Write code that will find the second largest item in an array on ints. 2. Write SQL that selects the second highest number from a DB table. I'm floored how few candidates can do these. Many end up giving up or fiddling for 10 minutes before giving something like `SELECT table WHERE 2`.
- deckard1 5y agowouldn't be surprised people miss the SQL question. Full stack devs are rather rare nowadays, and it's a bit obscure SQL trivia considering the answer is dependent on which SQL db you're using.
- throwawaygh 5y agoModulo syntax/keywords, I think the following query would work in any modern dialect: SELECT num FROM (SELECT num FROM nums ORDER BY num DESC LIMIT 2) ORDER BY num ASC LIMIT 1; which is actually nearly identical to the simplest Python solution in terms of the computation that's specified: lambda xs: None if len(xs) == 0 else sorted(xs)[-2:][0] I think the reason the question is tricky for some is because they either: 1) haven't used SQL in a long time and forgot all the syntax, or else 2) don't really "grok" SQL as a model of computation (as opposed to e.g. a data structuring language or something like that). But both of those seem like good reasons to disqualify someone from a full stack role?
- SpicyLemonZest 5y ago
- wnolens 5y agoNot really sure, but it sounds like some of things you are testing might be syntax - which makes sense if it's critical to your hiring need - ex. hiring senior C++ folks to start a new project - otherwise every new job I've started in this industry thrusts me into a new language and I have no problem sending CRs almost immediately (given Google and an IDE). I'm generally more a fan of keeping skill-based examination too, but making it very easy and language agnostic. Harder than FizzBuzz of course, but nowhere close to dynamic programming. Ideally, if a candidate has never seen it before, a basic tree traversal is enough. But since 100% of people have practiced that before interviewing, I invent a problem by ripping out a little piece from the codebase I'm working on. It usually amounts to.. Show me some clear control flow, factor that code into a subroutine, iterate over a collection and call that subroutine, contemplate the performance characteristics in a discussion (not bigO complexity, more so consider the I/O device..caching..)
- AlwaysRock 5y agoI know a company that use fizzbuzz and only fizzbuzz as their technical exercise. The people who are going to fail a technical fail at fizzbuzz. I cant say its perfect... But they have a really great Clojure team and everyone else seems to have a difficult time hiring for it. They IPO'ed last year too...
- vmception 5y agonot complaining about fizzbuzz at all, but this is just a PSA to tell the interviewer to end the interview if the candidate gets a good solution in 5 minutes. Adding this "but can you tweak it moooooore" mess is just awkward and the candidate hasn't seen two dozen iterations from all the other candidates.
- hiq 5y agoOne common question of this type (I've had it, and heard a few others as well) is: implement a function returning the factorial of its input (so n!). Basically FizzBuzz. I recently got that question after a homework with about 200 lines of python written. If I had failed at this question, the company would have concluded that I had somebody else do the homework for me, so I understood where they came from with this.