17 ms·
I used to hate it, but have now performed enough interviews of supposedly senior devs with decades of experience. I've literally seen someone who did firmware
by monocasa 9y ago
I used to hate it, but have now performed enough interviews of supposedly senior devs with decades of experience.
I've literally seen someone who did firmware for the space shuttle grind for 45 minutes on fizz buzz without making progress. Like, I'd they struggle for ten minutes or so, I take a step back and say "Ok, screw syntax, let's just vaguely talk about what needs to happen and mock up some pseudo code.". Even then, grinding for another half hour with no progress.
- speby 9y agoI agree w/ you here. And I think a lot of people have some personal experiences encountering this kind of situation. It seems baffling, right? And yet, you have to really wonder ... what did that person do writing firmware for the space shuttle? Did they really do that? Or did they somehow lose their mind after they left and now couldn't do a simple set of modulus operations on a FizzBuzz test? How can that happen? Even reading this kind of thing makes me totally scratch my head.
- collyw 9y agoBrain freeze under interview conditions? One thing that I have noticed is that taking in an interview seems to engage a different part of your mind from coding. I get quite into the flow of chatting, then have to jump into code. Its quite a context switch. Plus the pressure of someone staring at you. Then you feel stupid for not answering straight away. The you get self conscious and start doubting yourself. What was the question again? I don't think there is anything especially unusual about it at all.
- stevenwoo 9y agoOff the top of my head the space shuttle firmware was written by a team that adherred to a very systemic process to ensure it met the spec/reduce the number of bugs, it was often cited in old software engineering papers/books in the time period as the best organization in the world, IIRC in terms of software engineering measurements like defects per lines of code. It would be important to know if someone came from that at the beginning of the space shuttle program when they were creating the standards for work, starting out or if they came on later after the process had been perfected. I worked in a different contractor in NASA/Clear Lake at the time but even before I read those papers the organization responsible for the shuttle firmware was held in the highest regard.
- newnewpdro 9y agoWhich may have proven nothing more than this person was too nervous to function.
- aje403 9y agoIf you've been coding for 30 years and you get too nervous to write fizzbuzz when talking to people, that doesn't mean they should hire you, that means you either have personal life problems going on or you need to see a psychologist
- newnewpdro 9y agoHistorically, and if we're talking about someone with 30 years programming experience, this history is very relevant - programming has been a popular field for smart but antisocial/unsocialized nerds who had very few if any friends growing up and have spent much of their life alone at a computer. For a person with that background, simply sharing a relatively small private room with a total stranger can be overwhelming. It doesn't matter what trivial test you're giving them, if they're not able to think. The test is failing to actually determine if the person can do the technical part of the job, instead it's testing how the person performs in close quarters shared with unfamiliar people. The whole giving homework for technical vetting is a great way to get around this class of problems.
- aje403 9y agoBut what if you're looking to hire people who go outside sometimes and can talk to people and can write code too? Or should that be illegal?
- talmand 9y agoNow we're tacking on an additional aspect of the job that we didn't know about before. If the job requires a social interaction with people outside of the company then, of course, it becomes a factor. Let's not be silly in an effort to just be right and prove someone else wrong.
- pwthornton 9y agoWe have to separate whether or not someone doesn't know how to do a job from whether someone doesn't know how to interview. This is one of the biggest things I try to guard against. Some people just interview better than a lot of other people, but that doesn't mean they do anything other than interview well.
- monocasa 9y agoOh totally. Fizz Buzz shouldn't take you the whole 45 minutes, so the remainder is just a conversation where I dig down in their experience on a hunt for bullshit.
- Clubber 9y agoI think FizzBuzz is about as hard as I would make a technical interview. Maybe some basic questions about database design; what's an index; what's an array, etc. The dig down and hunt for bullshit is the most effective way for a tech person to interview another tech person in my opinion.
- mywittyname 9y agoThis is always my approach too. Pick a few elementary questions about a technology and ask a developer how to do them. Simple stuff, like how to build an array with integers 0-5, or iterate over a map.
- throwaway84742 9y agoThat sounds to me like fight or flight response. Happens to me sometimes, and when it does I can’t code worth a damn, in spite of having a decade and a half of experience on the top projects at some of the top companies in the industry. This, in fact, happened to me the first time I interviewed with Google: I bombed that pretty spectacularly. The second time I applied I already had a couple of very generous offers elsewhere, so I didn’t care as much and was less nervous during my interviews, so I passed.
- monksy 9y agoI did that with the google phone screen. It didn't help that the interview really took a "well that's interesting" approach and basically killed the interview.
- TheGRS 9y agoHence the take home interview. At least at the last company I worked at we began doing these because we could tell that the stress was filtering out some otherwise decent candidates. At least that was one of the reasons, it also gave us a chance to talk about technical solutions the candidate produced in a more stress-free way.
- Clubber 9y agoSo you decided to filter out good candidates in a new way; the take home test.
- TheGRS 9y agoI don't think this assertion that the technical interview is inherently filtering out great candidates is being very honest (you responded similarly to another comment of mine the same way). One quality being tested is "can you try to do the thing that we're asking for?" That's a measurement in competence and the willingness to get something done. If said candidate is saying "this test is ridiculous and a waste of my time!" then my common sense instinct will say that they're probably not a great fit even if they're the greatest programmer in the world. At least on the take homes I've issued they shouldn't be taking more than a few hours. These stories about creating applications that take 3 days are certainly problems and recruiters need to lighten up a little, but I don't think the take home is inherently flawed just by its existence.
- adreamingsoul 9y agoCheck your premise. It sounds to me that an assumption is being made because they didnt meet the expectations within a fixed time period. The danger with this is that this is not a real world scenerio of how two people would approach a task or problem. It’s also important to recognize that everyone is different with regards to how they process information. Stress has a real biological effect on how the brain processes information, and who doesnt feel stressed during an interview. I’m somewhat autistic and struggle to process verbal information. But I try not to share that with people because I choose not to be labeled by it. It’s unfortunate that in our society that we have to conceal things like that. All of us could be better at trying to understand people around us and accepting them for who they are instead of scrutinizing them for what they are not.
- deleted 9y ago[deleted]
- tytytytytytytyt 9y ago> It sounds to me that an assumption is being made because they didnt meet the expectations within a fixed time period. Of course, you have a limited amount of time to interview people. Please let me know when you figure out how to give someone an infinite amount of time to answer questions during a 1 hour interview. > The danger with this is that this is not a real world scenerio of how two people would approach a task or problem. That doesn't mean it's not a reasonable way to test basic skills. It's even too easy to test that by itself. What good alternative do you have? > All of us could be better at trying to understand people around us and accepting them for who they are instead of scrutinizing them for what they are not. How exactly would you structure a 1 hour interview so that it's not, "scrutinizing them for what they are not"? Or are you just having fun being preachy here?
- chrisbennet 9y agoI realize that you’re doing the best that you know how given the time pressures you have as an interviewer. There is no perfect answer to this problem. Here’s the thing though, you are “leaving money on the table” when your interview process deselects great programmers because of a mistaken belief that Test A is a good proxy for developer talent. When HN developers are telling that Test A is a bad proxy for developer competence, perhaps, just perhaps, you might consider trying to improve it with the advice given. The upside of being able to accomodate developers who “lock up” in stressful interviews, is that you now have access to great developer that are (according to your competitors) “impossible to find”.
- psyc 9y agoPerhaps it's because you insist on testing their ability to live code on the spot as a performance piece, and misinterpreting that as testing their ability to code. If you're turning down someone who wrote space shuttle firmware, because they "can't do fizzbuzz", then you are administering a terrible test and getting a false negative. I have 20 years of industry experience. Everything on my resume is the truth. I've worked on serious code, for serious projects, at serious companies. At work, my performance reviews have been stellar. When I do live coding interviews, I'm being tested with tasks that are utterly remedial relative to what I've done professionally over long periods of time. Yet I flub these remedial tasks. It could be because I'm nervous. Could be because the interviewer keeps interrupting me mid-line, so I can't think. Could be they are too terse, and I can't drag enough information out of them. Could be the problem is slightly more complex than I can accomplish under any circumstances in a live performance, but that I could complete correctly in a closed room. These interviews are degrading, disrespectful, and they tell lies to the misguided interviewer. What do they think they're testing the candidate for? Why does it matter whether you can code something remedial, live? Is live performance part of the job function? Do other professions do this? We should end this practice.
- Arzh 9y agoThen the other option is to do a small take home test. Are you ok with that?
- psyc 9y agoYes. I strongly prefer an offline test, if it realistically wouldn't take more than a few hours.
- Arzh 9y agoYeah we try to have a limit of 4 hours for this kinda thing. We understand that it is the interviewee's free time but when you need to screen 12 people for a job it is not worth a Senior Engineers time for a small startup.
- resolaibohp 9y agoIn my experience, candidates who have experience in dev work but could not pass our interview tests always end up in a dev role somewhere else shortly afterwards. Even some of them go on to find much more success. Either there are a lot of companies which employ unskilled developers who cannot even code fizzbuzz, or interviewing is not a great way to understand specific types of people.
- poulsbohemian 9y agoI’m going to show some vulnerability… this happened to me this week. I’m probably like a lot of you here: professional software developer for 20 years, have shipped countless products in multiple languages / technologies. Work has appeared on tv and print media multiple times. Have generated millions and millions in cost savings and revenue for employers and clients. But I totally failed a technical screen. It’s been a week of reflection as a result. I nearly canceled the interview to begin with because the entire process felt like a cattle call - totally cold, the company had shared no information about themselves or the role. There were ten minutes of rapid fire (“you have one minute to answer this question.”) followed by the proclamation that the next 45 minutes would be spent on “as many coding problems as we can get through.” Within the first few minutes I was able to describe a general solution to the problem, but over the next 40 I totally struggled to produce a working solution. Now, sitting here I could likely write it in 5 minutes. The problem was: --I was completely frustrated by the experience from the beginning --I was in a code editor that I wasn’t familiar with --I had limited access to documentation --An interviewer looking over my shoulder harassing me --I realized that while the problem was simple, there were things I’d just never needed in that particular language before, so didn’t have a complete understanding of what was available to me. --I focus on design before implementation, where the interviewer just wanted me to jump in and bang out code. --I realized that the last time I wrote this particular bit of code was 20+ years ago, as a college freshman, and certainly not in the language I was using today. So I agree with the other commenters: we should all check our premises before we disregard another experienced programmer as as hopeless hack. I wrote a long blog article this week about what I think the interview process should be, will be posting it soon. There are ways to fix this process.
- psyc 9y agoThere's no shame in it. Same thing happened to me last week, and I was furious about it. I've resolved not to take any more live coding interviews, on the grounds it is uninformative, degrading hazing. We're composers, but we're being tested as if we're live concert pianists. Then companies complain that there's a talent shortage! This hurts everybody, and it needs to stop. For my part, I will refuse to participate any further.
- geebee 9y agoTo me the problem usnt fizzbuzz, its “find all matching subtrees in a binary tree” in 45 min at a whiteboard. I dont need to hit the book to write fizzbuzz, and i could probably code up dfs and bfs without prep but i dont walk around ready to retake my undergrad algoriths midterm. Especially when i really dont even know what the topic will be. Im aware that the question i posed isnt terribly difficult and perhaps it does say somethingabout me that i need a bit more time to refresh my data structure to do this at a whiteboard. But i do want to be clear that fizzbuzz is not representative of the 5+ hour whiteboard grilling ive gone through at nany of these interviews.
- monksy 9y agoThe problem with the BFS/DFS questions is that you're playing a forced game of knifey spoony: They expect you to write it in the recursive fashion and then try to stump you with "oh but how does that do on large structures?!" .. Stackoverflow. Then they'll try to force you to do tail recursion.
- ajeet_dhaliwal 9y agoI agree, may be it’s just the places I’ve applied in the past but ive never been asked anything near as straightforward fizzbuz.
- ubernostrum 9y agoI always end up linking this piece in response to people like you, and here I am again. These folks have real data on lots of interviews, and have done some analysis on them: http://blog.interviewing.io/you-cant-fix-diversity-in-tech-without-fixing-the-technical-interview/ http://blog.interviewing.io/you-cant-fix-diversity-in-tech-w... The takeaway that's relevant to you is the section titled "Interview outcomes are kind of arbitrary". Specifically, things like: As you can see, roughly 25% of interviewees are consistent in their performance, but the rest are all over the place. And over a third of people with a high mean (>=3) technical performance bombed at least one interview. If you set up a typical tech-company interview process, and if you equate "didn't pass this" with "complete impostor and utterly incapable of coding in any capacity" (as most people with your approach do), well, congratulations. You're almost certainly rejecting a bunch of qualified people, simply because you're terrible at interviewing them and can't actually tell from your process whether or not they're qualified. The most generous thing I can say is that it's not entirely your fault. You adopted ideas and approaches from people who seemed authoritative but turned out to be the actual impostors in this situation. But now you have a choice: you can reject that, now you know the problems, or you can double down. I suggest not doubling down.
- deskamess 9y agoAgreeing with and expanding on the interview studies... In most cases, the job that one ends up doing has very little to do with what you were tested for in an interview. Languages usually have some sort of approved/well know implementation of common data structures and algorithms. When you are asked to do some BFS/DFS/binary tree work in 45min or less you are unlikely to do that for the job. Someone has already provided an implementation. I venture, you are most likely to see it bought up in your next interview (whenever that is). The question is how can you test for experience in writing code that is maintainable, readable, 'instrumentable' and contributes to a better system. How often does a candidate refer and defer to a decent third party library instead of feeling they need to (re)implement a well known data structure or algorithm? What have they done to help automate things so they can free themselves and their peers from minutae (aka how willing are they to put more work on machines?). What more do they think they can do. How well do they mentor or are ready to be mentored? Given a sample project, how would they go about running the technical side of things. If they focus on code, you should try to motivate them to expand to other areas (without telling them what they are). How cognizant are they of the ecosystem; or does their vision end at coding + unit test. Lots of intangibles that cannot be measured with a github profile or a white board interview but probably contribute more to picking out better candidates.
- itronitron 9y agoI think you should consider that most interviewees expect to be asked any question from their entire work history which doesn't leave much room for anything else. It's probably worth separating the problem-solving interview and the techno-archaeological trivia game into separate days.
- keithnz 9y agoThe amount of times I've heard interviewers go "Oh I had this super experienced guy who but in the interview he couldn't even do some simple programming problem". To me that is a failure in the interview process. I've seen this when I've interviewed people. If people don't do well when their resume implies that they should, I'll discuss it with them and say we want to get some kind of confidence in their abilities. Then work out a way to do that. This has worked out really well in my experience. I've ended up with devs who are quite different from me, one person of note (who we employed), was much slower at solving problems but far far more methodical and exhaustive in approaching problems and often notices very subtle details (which is a good thing in firmware development!).
- Clubber 9y agoThe only way I could see that scenario could ever occur is if the candidate outright lied on his resume. A quick solution is to check references (call the school said person graduated from, call references, call references via their company line, etc). Sounds like the interviewer didn't bother checking references.
- monocasa 9y agoThe majority of shops that firmware devs come from (and particularly defense) have a policy of only verifying the dates of employment and nothing else.
- Clubber 9y agoReferences would be personal references - someone with a cell number and call the college he attended; they should also verify enrollment and completion. You don't want to be calling the HR department. Also, a secret is if the reference (not HR) only verifies employment, that's a negative review. They are concerned about liability. If it was a good employee, the references have no qualms about saying so; it's the negative aspects that carry legal liability. You need to ask for those, not just prior companies. My resume has a "References available upon request" at the bottom and I'd happily give them. Is that no longer practiced? It used to be a requirement.
- profalseidol 9y agoHow did he do the firmware of the space shuttle without knowing such basic coding?
- monocasa 9y agoI think he probably didn't really, he was just dead weight on his team.
- ereyes01 9y agoSaid space shuttle firmware engineer probably never had to solve a pop-quiz problem on a whiteboard. It's much more likely that he/she deliberated for weeks about a simple change with the rest of their team, and perhaps even with the person/astronaut they might kill if the change had a bad bug. Who cares if this person can't solve fizz-buzz on the whiteboard? That's probably not even a relevant skill to the job opening you're filling, or a skill they have ever needed to hone or practice. It's like determining a marathon runner's worth by timing their 100 yard dash. If I ever need to do another coding interview again, I'm gonna troll the person asking the question by pulling out my phone, googling their question, and consulting the stack overflow post in the search results to solve the problem. And why not, this is how I (and the vast majority of people) will actually get the job done in a real situation.
- cpolis 9y agoDeveloping software requires high level/architecture type decisions that you mention in your first paragraph. It also requires solving many small fizz-buzz size subtasks, which are necessary and demonstrate competency. A marathon runner doesn't need a stellar 100 yard dash time, but they should at least be able to jog 100 yards.
- jonhendry18 9y ago"It also requires solving many small fizz-buzz size subtasks" If you're working on the shuttle, those small tasks generally won't be surprises. They will have been planned out well in advance.
- eadmund 9y ago> Said space shuttle firmware engineer probably never had to solve a pop-quiz problem on a whiteboard. It's much more likely that he/she deliberated for weeks about a simple change with the rest of their team, and perhaps even with the person/astronaut they might kill if the change had a bad bug. > Who cares if this person can't solve fizz-buzz on the whiteboard? That's probably not even a relevant skill to the job opening you're filling, or a skill they have ever needed to hone or practice. Thing is, shuttle-style development is also probably not even a relevant skill to the job opening he's filling. Odds are, that job requires something more akin to FizzBuzz ('I want to do something. How do I do it?') than to shuttle-firmware.
- cs02rm0 9y agoI avoid doing interviews, but I've suggested fizz buzz a few times to people I've worked with because it's so simple but once outside the environment of your usual IDE seems surprisingly simple to screw up. Whenever I've suggested it a usually more junior guy has scoffed at how trivial it is... I'm yet to then have one supply a complete and flawless solution. These are from people I work with, respect and happily rely on to be able to code. Their recognition that the problem isn't so trivial for themselves is the only thing I've got out of it - I'm no closer to understanding if it's just a poor test or not. There first time I came across the problem was as a candidate and I screwed up the upper bound of my for loop just because I was slightly thrown by starting from 1 instead of the almost obligatory zero. Naming things, caches and off by one errors. My worst candidate experience was only a couple of years ago when they wanted me to reason about booking cinema tickets over the phone (like it's the 90s?!) I've never booked cinema tickets and I think they thought I was joking, but they went at me for at least half an hour on what users would need to supply other than location, film title, date and maybe time, to begin the process of booking tickets. Why they didn't just tell me what attribute they were after I'll never know - eventually they just wrapped it up.
- jonhendry18 9y ago" Why they didn't just tell me what attribute they were after I'll never know" Number of seats?
- cs02rm0 9y agoPossibly. Ha.