16 ms·

Being a good programmer has a surprisingly small role in passing programming interviews. And that just says it all, doesn't it? I agree that interviews sh

Being a good programmer has a surprisingly small role in passing programming interviews. And that just says it all, doesn't it? I agree that interviews should test candidates on certain basic skills, including (time/space) complexity analysis. But do you really learn anything by asking the candidate if they can recite the time complexity of a moving window average algorithm (as I was asked to do by an interviewer yesterday)? What does the candidate's ability to code a palindrome checker that can handle punctuation and spaces tell you about their ability to deliver robust, maintainable code? I don't have the answer, but I just don't see how the questions typically asked in programming interviews give you a good picture of the candidate's actual programming ability. I much prefer "homework" projects, even if they involve me working "for free", because I feel like they ask for actual programming skills rather than the "guess the algorithm" lottery of phone screens and whiteboard coding.

digler999 11y ago
> including (time/space) complexity analysis. I think this is one of the most inane things to be asked during an interview. personally, I've never found myself in a situation where I truly needed to choose between a vector/map/list/hashmap. Or had to find the O(x^n) and replace it with O(x^2) Obviously it depends on the application, but many jobs are simply maintenance coding: find bug, fix bug, test fix. Often times it makes absolutely no difference whether you use a list or a vector, or else you'll get the paradoxical "vector-is-always-faster" because of locality of reference. In my (admittedly limited ) experience, most of the effort is spent simply making it work, not being bogged down because you used a map instead of a hashmap, or didn't know about some esoteric, bleeding-edge probabilistic data structure.
- pessimizer 11y ago
  Honestly, I consider an instinct for complexity analysis the most important thing I learned in school, and the thing that I've gotten the most use out of. I don't know what case you're making here: are you saying that high-level architecture is so hard that choosing a map or a hashmap should be a coinflip, or the one you see first? Having had some criteria for making the choice makes my life a lot better when everybody is panicking because something is running like shit and no one understands why.
  - digler999 11y ago
    To me it was a bunch of rote memorization, just like a biology course. I never - never - have needed to know how bubblesort/heapsort/mergesort actually work, except to appease interviewers. I'm not saying I'm pro writing-inefficient-code, but if you want to talk big-O during an interview, I"m going to roll my eyes about as much as you asking me who the 19th president was.
    - democracy 11y ago
      I ask such questions at the end of an interview, but mostly to see the sanity/reaction or thinking process - wrong answer would do, rolling eyes - would not :)
  - b1449835 11y ago
    Oh shit, my code is O(1), looking at it in the profiler tells me that it's not a bottleneck (it says < 5ms). Yet this single function call somehow takes 78ms in wall time when it should at worst be 1ms (and even that is too much). The reason? JVM classloading/JITing. This quite annoying when you have shortlived applications.
- jerf 11y ago
  "Or had to find the O(x^n) and replace it with O(x^2)" The other thing that really seals the deal for me as an inferior interview question is that you don't need to have a clue what O(x^n) is to wrap some code in a simple time call, see that the code you think ought to run in microseconds is running in seconds, by visual inspection notice stupid nested loops, and fix it. Self-taught programmers may not be able to say "O of exx to the enn" but that doesn't stop them from fixing it. So... seriously, what good is the question anyhow?
  - tomjen3 11y ago
    What if it isn't stupid code? What if it is straight forward and simple and looks correct but is just slow because it has to traverse a list instead of using a hash-table or some other data-structure. You can't get there by intermediate steps, you have to rip out the code that uses it and rewrite it with a hashmap. You can only do that if you know the space complexity and when a hash and a list is appropriate.
    - jerf 11y ago
      "You can only do that if you know the space complexity and when a hash and a list is appropriate." You seem to be confusing "knows how to say 'oh of enn squared'" with "the loop gets slow when I iterate over a lot of things". One is generally a product of education, the other, merely experience. The idea that a thing can only be learned in a classroom is perhaps in the top ten most pernicious ideas in the modern world, and probably one of the more surprising ones to show up in that list. You do not need special courses to discover that your code runs slowly, and as I've seen often enough, having had those special courses does not confer immunity against writing slow code. Now, I am also a believer that formal education has its place, and if you are going to get a formal education in computer science, big-O analysis absolutely must be part of it or you are literally missing out on an entire rather important sub-discipline. But the idea that it's some sort of touchstone between Good and Bad programmers is just ludicrous nonsense. Slow code is slow. There are abundant tools that can be used to figure out why. If you can't work out why your O(n^3) loop is running slowly after a couple of years of practicing the art, you don't need formal education, you need a different job.
- ktRolster 11y ago
  Most likely you think about it, though. When you're coding, and you have a triple loop, do you think, "Oh, this is O(n^3). Is n going to be too big here?" It may be something that's so intuitively obvious to you, that you don't even think about it. So you naturally use the hashmap, where someone else might try a list and then start doing a lookup in a loop. Then while that particular instance might not break things, it'll slow things down, so overall the application feels sluggish instead of snappy.
  - eecks 11y ago
    I've never ever thought that to myself. Then again my code never has triple loops.
    - Retra 11y ago
      So do you never call anybody else's functions, or do you always verify that they don't do any loops in those calls?
- mech4bg 11y ago
  I've seen this plenty of times. I've worked both on a trading platform and a large website, and both times encountered many performance issues that were solved with a more appropriate algorithm or data structure. I've even seen this with a list as small as 10 items - a O(n^3) algorithm was making multiple network calls each time; changing it to O(N) alone made a huge improvement in speed.
  - jkyle 11y ago
    This does come up a lot. But I think it's more telling if a programmer knows how to profile his program and find the performance bottlenecks, recognize them for what they are, then fix them appropriately than if they can recall a specific optimization for a specific use case on demand.
  - kevan 11y ago
    Most of my work on an ecommerce platform doesn't need much attention to algorithmic complexity, but everyone on my team still curses the guy who wrote an O(n^4) algorithm in our checkout pipeline (discounts, promos, shipping, tax, etc). More than a couple items in your cart and you couldn't checkout because the thread would spin forever. I want to work with a team of people who can recognize these things immediately, even if it's not an absolute requirement for the job.
    - im_down_w_otp 11y ago
      That has a lot more to do with mechanical sympathy, and awareness of when you've exchanged cleverness for complexity disguised as cleverness, than it does with knowing the big-O of operations on a datastructure. What you want the person to identify is that they've made your simple iterative checkout process into multiple unbounded tree traversals with no circuit breaker. Knowing that searching the tree is O(log(n)) isn't very helpful when your problem is an inability to identify that you've made (n) an unnecessarily huge problem space.
- tomjen3 11y ago
  I am sure the RubyGems authors thought the same way until somebody sped the process up dramatically because they could do space-time analysis[0]. [0]: https://news.ycombinator.com/item?id=9195847 https://news.ycombinator.com/item?id=9195847
- ken47 11y ago
  I've never found myself in a situation where I truly needed to choose between a vector/map/list/hashmap. Or had to find the O(x^n) and replace it with O(x^2) This entirely depends on the kind of product you are working on. When you get to a large scale with any programming project, optimizing computational resources will cut costs, and can often add value to the customer as well.
  - rhizome 11y ago
    This is special pleading, though, right? It being relevant to "any" large scale project really just means that projects can get big enough for it to matter, but how many jobs involve this family of software? How many interviews for positions directly related? Very few, I would guess.
    - NotSammyHagar 11y ago
      I find myself having to think about efficiency at least a couple times a week. I'm working on database implementation, and in the query processor we have to consider all the time how to evaluate various things efficiently.
      - rhizome 11y ago
        Taking this at face value, were any of the techniques you use in these tasks directly addressed during your interview there?
  - Cakez0r 11y ago
    I would say it starts becoming important for any application that has multiple concurrent users. If it's a single user running it on some device and there's no shared resource (I.E. back end), then chances are it's not going to be an issue.
- EpicEng 11y ago
  That's fine, as long as you _never_ need to trust said dev to do anything complex. It's fine to have mediocre developers perform mediocre tasks, but if you want more from them someday you may be in trouble.
  - digler999 11y ago
    to me, complex, and complex-ITY are entirely different matters. I want a smart programmer who can figure out really complex bugs (something you cant figure out from google/wikipedia). Not someone who memorized the big-O performance tables of 8 different data structures (something you CAN look up on wikipedia ).
    - was_boring 11y ago
      Exactly. I once identified memory leaks in a managed runtime implementation, worked around them, and showed that third party networking libraries from the hardware vendor were causing irrecoverable crashes but I can't spout off runtime complexity of algos. The former saved a multi million dollar contract and my company's reputation, the latter is something I look up when I need it.
    - EpicEng 11y ago
      Optimally, I would like both. I agree that wrote memorization is not a very useful skill, but I'd like to know that they can grasp concepts, and I find it hard to believe that there are a bunch of skilled devs out there with great potential who don't or can't understand algorithmic complexity. Interviewing is hard though, I'm not trying to say that any one question or metric should necessarily rule someone out.
  - FLUX-YOU 11y ago
    That's not really fair, people don't grow unless they're challenged. Some people really do never advance past a certain point, but a lot of people write someone off as mediocre when they're just still in the process of gaining skill. Also, they only assess ONCE, which is pretty bad if we're trying to establish how good you are forever and always.
- munchbunny 11y ago
  I think it's easy enough to ask for this skill if the job would require the interviewee to apply this skill. If your job is mostly frontend, yeah, you probably won't need to worry about this problem. But if you're hiring somebody to work on graphics? You better be doing complexity estimates in your sleep.
- sklogic 11y ago
  That guy in Google who screwed up Android thought just like you. Now, as the number of text messages stored on your phone grows, the entire system slows down. Such a pity that cretin was not screened out on an interview!
- plasticchris 11y ago
  This can happen even on small systems. I once replaced a O(n^4) with O(n^2*ln(n)) on an embedded target which made a minutes long user facing process take seconds. The catch is that the original used a good algorithm, but made an implementation error which I caught in profiling. So complexity analysis is good, but the only way to get better at something is to measure it.
- nathanaldensr 11y ago
  Here's how you should reply: "Sorry, I don't have those complexities memorized. When I really need to look them up (which is nearly never), I refer to bigocheatsheet.com."
- robryan 11y ago
  You should have an intuitive idea around complexity and when it makes sense to optimise. I don't think though you need to know that for example fastsortx is n log n in the best case off the top of your head in an interview situation. You should be able to reason your way through why it is faster than some other sort though.
  - nsxwolf 11y ago
    What the heck is fastsortx? This is another problem I have. I study all these algorithm books then when I get to the interview they ask something that either isn't in the books or they've come up with some nickname for it and expect me to know it.
    - democracy 11y ago
      Even Google doesn't know...
    - rhaps0dy 11y ago
      It's probably a placeholder. Same as writing <sort algorithm>.
- democracy 11y ago
  That's how I failed my Google interview - they said they expect programmers for any position to have good knowledge of algorithms, yeah, well... :) Frankly I don't like trivia-style interviews.
- b1449835 11y ago
  Because in most cases n is a very low number. If you only have 10 elements the algorithm doesn't matter.
- collyw 11y ago
  Exactly. Performance issues happen. Most of them can be solved by re-indexing a database or adding caching. If I really need speed at the algorithm level I can look it up.
quarktasche 11y ago
Wouldn't the question about the moving average be a good one though? It should be quite obvious that you can calculate a moving average without remembering more than the current average and the window size. Given that, time and space complexity should be obvious. I think it would be a fine question to see if the interviewee has any idea what complexity means. Or was it a more complicated moving average case (exponential etc) where the algorithm was given and you were asked to determine the complexity?
- collyw 11y ago
  How many jobs will it be relevant to?
  - quarktasche 11y ago
    Which part? Figuring out how to efficiently calculate a moving average seems like a good question for basic maths skills to me. About complexity theory, there is already a lot of discussion about it's relevancy in the comments. My point was meant more along the line that, if you value a basic understanding of complexity theory, the question asked of the GP seems reasonable.
riazrizvi 11y ago
I have come to believe it is part of an industry-wide negging style to keep people in their place.
lewisl9029 11y ago
I personally also prefer take-home projects over being grilled on my ability to solve obscure algorithms problems under pressure. However, this second route also comes with a number of issues. The most annoying of which in my experience is the amount of time investment each interview requires from the candidate. At least in the traditional technical interview, interviewers and candidates tend to be roughly equally invested in the interview process in terms of time spent, so interviewers have to value candidates' time because failure to do so will end up wasting their own time as well. In take-home projects, this balance in time investment completely breaks down, and thus there's very little left to discourage interviewers from issuing ridiculously time-consuming projects. I've done a few of these in the past, mostly to get some practical experience with a new framework & ecosystem I'm not too familiar with yet, but I think in the future I'll likely just politely decline any projects that looks unreasonably time-consuming, or at least try asking for certain adjustments to the spec first.
- throwaway12324 11y ago
  Ironically enough, Triplebyte's own take-home projects were some of the worst I've ever had, and did a horrible job of respecting the candidate's time. When I went through the their take-home interview process, there were 4 projects to choose from, with only one having anything remotely to do with my area of expertise (it was a multiplayer game, and I was looking to work as a web front-end/full-stack developer). For all their talk on how you should select a practical project to talk about because it correlates better to ability to get real work done, it's really rather ironic just how utterly academic and unpractical the projects they offered were (that game was literally the most practical one on the list). Now they tell you that you're expected to spend at most 3 hours on the project. This might be true for some of the other more academic projects on the list if you had any expertise in the respective areas, but it definitely wasn't true for the multiplayer game project I chose, which had a non-trivial front-end and back-end component, for which testing alone could easily take 3 hours (Or maybe I'm just bad, which is definitely possible, but I've asked many of my more experienced peers how long they think a project like this might take for them, and the lowest estimate I got was a whole work day of 8 hours). Now they also tell you that it's OK if you don't finish the project. There was also a second part to the interview where they may ask for extensions to the original project, where they'd give us more time to work on it, so I assumed if we don't finish, they'd ask for us to finish the original spec along with some extensions for the second interview. Since I had already finished the front-end component for the game, and had worked on the project for well over the expected 3 hours, I decided to call it a day and work on preparing for some of the other interviews I had that week. However, that it was OK to not finish the project for the first interview might have been an outright lie. I won't ever know for sure because I was rejected after the first interview for not finishing the back-end as well as being unable to resolve some performance issues the interviewer pointed out (which at no point in the interview did he even ask for me to fix, so I assumed fixing that performance issue would also be a part of the extension). Incidentally, I went back to the project a while after the interview and resolved the performance issue in 5 minutes flat. The whole process just left a rather bitter taste in my mouth, which was all the more disappointing because I went in with great hopes after reading all their great blog posts on HN. For anyone else considering Triplebyte, I'd highly recommend going with their traditional interview route until someone at Triplebyte can confirm the process has changed for the better. At least with that route, if you get rejected, you'd have wasted less time in the process.
jguegant 11y ago
I already saw such a colleague training hard at competitive programming, everyday. Yet, he wrote that kind of C++ code: ressource AClass::CopyACriticalRessource() { { std::lock l; } // Optimization for the lock. return ressource_; } Having a good understanding of every aspect of your architecture, taking your time instead of rushing for blazing fast (O(1)) but hacky solution, always having in mind that someone must have already solved your problem... these are some valuable skills not coverable during a 1h interview in front of a white board. I would like someone googling a solution in front of me during an interview, trying to UNDERSTAND it, check its complexity, compare it with other search hits, adapt it to his problem, ask for my review. The best technical interview being a homework with enough time for a normal human being, requiring tricky algorithms to solve it in a fancy way, a good architecture and great computer-science knowledge in general. Afterwards, a code-review/interview of 1 hour with debriefing on all the choices.
thearn4 11y ago
> I much prefer "homework" projects I helped a group within my organization with their hiring process recently, and we had pretty good success with assigning a short "take-home" exercise, vs. trying to haze them with a programming problem over a google hangout interview. A problem focusing on a small part of what that group does, but scoped to be doable with 1-2 hours of work.
- jonesb6 11y ago
  Why don't companies offer it as an option? "Hey here's a fizz buzz question, if you feel confident answering it now go for it, otherwise we'd be just as happy for you to do it in a take-home fashion?"
- megablast 11y ago
  I guess it works well for people unemployed with nothing else to do, no family, lots of time. Everyone else, this sucks!
  - buzzdenver 11y ago
    Interviewing does take time. Most people will be able to spare an hour to find a new job.
  - kylequest 11y ago
    So you are saying you don't spend any time preparing for your interviews :-)
- hemc 11y ago
  Homework projects don't work some cases, when developers come with a refined copy/pasted code or code by another programmer. Better way is to go through the previous code that developer has written or go through the code written by other programmers to know the capability of the programmer to understand different code patterns
  - whyever 11y ago
    > Better way is to go through the previous code that developer has written or go through the code written by other programmers to know the capability of the programmer to understand different code patterns Doesn't this have exactly the same problems as homework projects? You can still copy and paste or plagiarize.
mettamage 11y ago
Here's an idea (perhaps naive): why not add all the methods and give them optionally to the programmers. E.g. - normal interview is mandatory (you need something to base your ideas on) - portfolio is optional but when chosen it weighs heavily - take home project is optional but when chosen it weighs heavily - etc. With 'weighs heavily' I mean that it influences the criteria for which those methods are good for. The downside of this is that interviews are more unstandardised, which is a trade-off worth considering.
draw_down 11y ago
If you're a programmer, you like to solve puzzles, so think of passing the interview as just another puzzle! I mean, yeah, it shouldn't be, but here we are.
- kafkaesq 11y ago
  If you're a programmer, you like to solve puzzles, No -- we like solving problems. Especially those that have a legitimate context (i.e. are explicitly linked to some actual, real, business or social problem). And for which our skills are truly relevant needed (specifically, for which no one has a readily available answer, at the moment). But made-up "puzzles"? For which the asker already knows the answer, so they sit back and watch us dance? Not so much.
Bahamut 11y ago
Whiteboard coding doesn't necessarily have to involve guessing the algorithm - any whiteboard questions I've asked avoided doing so. On the flip side, my experience with take home projects is that they are much more of a hazing/pressure cooker ritual - at the least, the time pressure of a Google/FB/etc. interview lasts only a half hour. Most will expect candidates to spend a huge amount of time, which many won't have if they're actively job searching, or are busy people in general - I prefer to pour that extra time into open source work, since at the least it benefits others. One company assigned a project to me once that suspiciously seemed like implementing the company's whole business model. It doesn't take long to figure out if someone has the right skills oftentimes if you ask the right questions - candidates shouldn't be penalized for companies' ineptitude in assessing interviewees.
- rqebmm 11y ago
  > I prefer to pour that extra time into open source work, since at the least it benefits others As someone who pushed his company to adopt a "take home" assignment for our interview process, it should be perfectly reasonable to reply with "here's my commit history on a project relevant to what you're hiring for." I much prefer giving (and taking) take home assignments because it lets the interviewer see _what someone will actually produce on the job_. If you can do that without jumping through our specific hoops, great. If not, that's why we have the assignment. It also at least somewhat ameliorates the pressure cooker of an interview, which many people cope with poorly. It can be hard enough to communicate clearly when all eyes are on you, let alone get your thoughts together and solve a logic problem. If that's actually analogous to your work environment, well...
tn13 11y ago
I don't think that is really a problem. We hire people to do a good job and most of the times kick-ass programmer isn't the best person for the job. At least for a smaller company we need someone who can understand business needs, can communicate better and Add Value. This is a far complex job than implementing red-black trees. Consider this: We need to find the most shared URLs on Facebook in last 24 hours. I can perfectly see why a lousy coder might achieve this objective better than a kick-ass one. A coder with average skills quickly figured out that Buzz-sumo has a public webpage with that content which can very easily be scraped using phantomjs. Job got done. Another great coder suggested to me I should buy $10K per month Facebook firehose. He is not wrong and that is a good solution too but he failed to see that we are building a POC and not a full featured product. Needs of companies are complex, many times if you can pay great salary just having a filter for IQ is good enough. But in most cases I think it is far better to look at an individual and judge.
- paulryanrogers 11y ago
  Would the Buzz-sumo programmer be expected to investigate their terms of service too? (Often scraping is prohibited.)
  - tn13 11y ago
    I am just giving an example. For a POC it does not matter much.
soham 11y ago
Homework projects could work well if the hiring requirements are small, but won't work well when a company has to hire say 25 engineers a quarter, which we had to. At that point, the process becomes too long, and it's easy to lose good candidates to a long process. This method of interviewing has been around ever since, and is going to be around for the foreseeable future. Nobody loves it, including the interviewers, but there just isn't a better way to do it at any sort of scale. Especially when there are much bigger problems to solve when you're running a business. It's best to take the bull by the horns. I run http://InterviewKickstart.com http://InterviewKickstart.com, which is a bootcamp for preparing for such technical interviews. We do almost exactly what is in the blog post. It works. Spectacularly.
- thesimpsons1022 11y ago
  i was about to sign up but then it asked for my phone number. why do you need this information?
  - soham 11y ago
    Simply because it is quite difficult to explain the concept in writing. What does it mean to have an intense bootcamp just to prepare for interviews? What's the method? etc. I want to take the time to talk to everyone who is interested in the course. Because the concept is new, people have all sorts of questions. Can't possibly address all of them in writing. I don't have a team of salespeople and haven't spent a penny on advertising. If you still prefer email, please feel free to send one. It's on the site. It may just take longer to do back and forth. Thanks for considering!
    - kafkaesq 11y ago
      Simply because it is quite difficult to explain the concept in writing. Then maybe it's not such a good concept.
      - soham 11y ago
        Or possibly because it's a new concept, which takes more text to describe. More text than what can be included above the fold, and more than what most people read attentively these days. After all, it's an 8-week intensive course, mostly for CS grads, that grills you hard, and is not cheap. As a consumer hence, I'd highly prefer to talk with someone. Not to mention, all educational institutes have an enrollment process that needs you to talk to a human. At some point, when it becomes more common and well-accepted, we will condense it, but it feels a little too early to do so.
shasta 11y ago
> But do you really learn anything by asking the candidate if they can recite the time complexity of a moving window average algorithm (as I was asked to do by an interviewer yesterday)? This is actually a trivially easy question that gets to the heart of whether you understand the point of moving averages or not (that you can update a sum by subtracting out the value leaving the window and adding in the value entering the window). "Programming ability" isn't one dimensional. Whether or not the above question is useful depend entirely on what skills are important in a candidate.
- Cakez0r 11y ago
  It's not so much specifically about the moving average problem. I would want to see that a candidate can reason about the performance of some code / algorithm. I would not expect them to be able to recite the performance for specific algorithms from memory, however.
  - shasta 11y ago
    I agree, but I suspect "recite" was editorializing by the parent.
    - quanticle 11y ago
      It wasn't. I was literally asked, "What is the time complexity of the moving window average algorithm over an array?" and when I asked for clarification, I could hear an edge of... I guess frustration in my interviewer's voice. Granted, by this time, we'd been through a couple of other problems, and time was running short, but I still think it was pretty unprofessional of the interviewer to let frustration or any other sort of negative emotion show during the interview. That, more than anything else, contributed to my own frustration and perception of unfairness in the entire interview process.
      - shasta 11y ago
        Right, but "recite" was editorializing. That implies that the interviewer expected you to produce the answer from memory, as opposed to thinking about it. It's an easy question if you're familiar with moving window averages and know what the interviewer intends. If it was asked apropos of nothing, a request for context seems reasonable, though. It sounds like you probably had a bad interviewer. There seems to be no shortage of software interviewers lacking in "people skills."
eranation 11y ago
When I was interviewing in the past I was usually refusing any take home projects, if you don't want to spend 30 minutes talking to while watching me code it tells me something about your priorities. However whiteboarding is also terrible. I prefer getting a real world / work related scenario problem, and solving it TOGETHER with the interviewer: if I'm stuck, as opposed to a hackerank codility challenge, there is a good chance they will let me know and hint in the right direction, and they will also see how I communicate, how I think, how I respond to hints. An online challenge can be easily done by the candidate sitting next to 2 more senior coder friends who help him / her along the way. Unless it is proctored, you can never know. I had people whispering someone answers during a phone screen, people typing my question in Google and reading me the first result (I google it at the same time). I think that giving you a hackerank or codility take home problem unless this is an entry level job will simply drive away experienced people. If someone can pass your take home test easily, you will still need to phone screen them before you fly them over in most cases, and if they are that good, they will probably prefer companies that don't waste their time and jump straight to the real person phone screen. However, some people are more nervous when someone is watching over their shoulder, so here is what I would do: I would ask the candidate what they prefer: 1. work related a hands on assignment with plenty of time (e.g. should take 30 mins but you give 1 hour) and ability to search online and ability to compile use an IDE just like in a real world scenario 2. Skype screen with a lot of small questions on a topic they really feel they know about (no Googling allowed), and a relatively smaller coding challenge (something you can code in 15 minutes) 3. Work related scenario but with a real person, where there is no one best answer to the question, but more of a balance of tradeoffs and more open ended but 100% involves coding (just like most phone screens, but more work related and not just puzzles) 4. The standard puzzle challenge but alone - you have 30 minutes to solve the problem once you see it (Googling is allowed or the test is proctored to make sure you don't, then you get more time) 5. the classic - a cracking the code interview kind of question, but with a real person, code sharing (for crying out loud not google docs, at least have your candidates use something that offers color coding and easier indentation) - some might choose still this If you let your candidate chose what is best for them, you already made an amazing impression, and you might have much less false negatives. Just my opinion
cheez 11y ago
The problem is that most companies want the take home in addition to algorithm interviews, keeping the worst characteristics of both approaches.
WalterBright 11y ago
Every week HN has a topic on the front page about how inaccurate and unfair job interviews are. They're always going to be inaccurate and unfair. Interviews must use proxies for candidate quality, and there will always be false positives and false negatives. The most practical thing is to work with it as it is.
Someone 11y ago
"But do you really learn anything by asking the candidate if they can recite the time complexity of a moving window average algorithm" No, but it may be a good question, anyways. A good interviewer probes you about things you might encounter in the job to find the point where you cannot recite things, and looks how you handle that. Having said that, I don't understand the focus on coding interviews I read on HN. I don't know whether it is cultural difference between the USA and Europe, whether I just haven't noticed how you are supposed to prepare for interviews, or whether I am too smart to need to bother, but when I apply for a job, I look up what the company does, try to figure out what its culture is, but do not prepare for the technical side of things. An interview isn't a one-sided affair of you being quizzed, it is two parties figuring out whether they fit together,
blakecaldwell 11y ago
Coding interviews are very stressful, and churn through a lot of really awesome potential hires. I imagine there are tons of false negatives, but it's nearly impossible for a terrible programmer to get through a gauntlet of programming interviews. However, I agree. They don't give a full picture of a developer's abilities. As a developer, I prefer take-home projects. As an interviewer, I prefer a few coding interviews, followed by a take-home project.
- platform 11y ago
  I have not tried it. But in my next business venture, I plan to actually do peer programming with the candidate. This is will allow members of my team (or myself) to get to know the candidate, evaluate the 'wave lengths', and how effective he/she is at finding patterns on the internet/books -- rather than thinking things up. It also demonstrates to the candidate commitment on our side, and it naturally forces us to find problems of a proper size/effort (as we are spending the effort too). We would not do it for all applicants, though -- only for once that pass basic screening / competency process (there are no trick questions or exercises there
  - blakecaldwell 11y ago
    Yeah. You can save a lot of your time, and the candidates' time with a simple coding interview - even FizzBuzz will knock out half of the candidates. Then pair up and/or give a take-home, then discuss the solution in a one-on-one session.
  - quanticle 11y ago
    I have not tried it. But in my next business venture, I plan to actually do peer programming with the candidate. As a candidate, I can highly recommend peer programming. One of the greatest interview experiences I had was interviewing with a local shop where the employee and I reimplemented a Set class using TDD and pair programming. The employee sat at the keyboard, so I didn't have to deal with not knowing keyboard shortcuts or the unfamiliar operating system (OSX), but he was very careful to only write the code that I asked him to write, and to let me make my own mistakes. It was one of the most enjoyable, and, dare I say it, relaxing interview experiences I've had. Though I didn't get an offer, I wouldn't hesitate to recommend that company to any of my peers who're looking to make a change. Also, like you said, it was a second tier screen; after an initial screen consisting of a more traditional phone interview where I had to write some Javascript.
im_down_w_otp 11y ago
Ah. I just provide all my answers in an obscure language they won't know, but has lots of "nerd-cred", so they feel uncomfortable questioning it. Like OCaml, Erlang, or Haskell. Then in the rare case they question it, which usually goes something like, "Can you do it in Java? We don't use whatever language it is you're using there." I respond with, "Oh, will I be spending a lot of time at this job coming up on-the-fly with somebody's PhD thesis from the 60's?" To which the answer is always, "No." Which finally ends the conversation with me saying, "Well then you asked me to solve an irrelevant problem, so I'm happily providing you an irrelevant answer." I have a fair bit of contempt for the pitiful state of technical hiring and assessment process out there. I think that comes mostly from my opinion that hiring is probably the most important thing any company or manager does, and that if any one of these companies/interviewers put even 10% the amount of effort into learning about educational/occupational assessment, psychology, and neurology that they do into obsessing over the dubious idiosyncrasies of the latest framework-rehash-of-the-month, then these terrible interview practices wouldn't endure very long, and we'd all be spared the indignity of the farce on both sides.
- xerophyte12932 11y ago
  > Which finally ends the conversation with me saying, "Well then you asked me to solve an irrelevant problem, so I'm happily providing you an irrelevant answer." So how many of those companies made an offer?
  - im_down_w_otp 11y ago
    3 out of 5 where it's come down to that kind of bizarro pissing contest.
RichieAHB 11y ago
Unfortunately, this is much the same as virtually any broad examination process. I'm primarily thinking of schooling up to 16. There's a lot of work to be done in ironing out one-size-fits-all testing in a lot of areas, primarily those that "require" a human to do the processing of the data.
braythwayt 11y ago
This is a general problem: > Being a good president has a surprisingly small role in > being elected president.
highCs 11y ago
I have a theory: I think the coding skills of a programmer are function of the number of programs he's written - from scratch.
- collyw 11y ago
  Maybe, but at the moment I have inherited a bit of a mess of an application in my new job. I think it takes more skill to work out what it is doing and replace the crap code with something simpler and more efficient, more maintainable and easier to read.
lazerwalker 11y ago
My favorite sorts of interviews are ones where you're expected, either by pairing with an employee on your own, to fix a bug in an open-source project you use. Has all of the benefits of a take-home project over whiteboarding, but the end result is you have something tangible to show for your efforts even if you don't get the job.
- nathan_long 11y ago
  This. Also, any interview where the candidate pairs with an employee lets both of them see how they like working together.
restalis 11y ago
"Being a good programmer has a surprisingly small role in passing programming interviews." "And that just says it all, doesn't it?" An interview is for both prospective employer and potential employee to meet and find out about each other. If you as a candidate notice something you don't like during the interview (including the interview itself), that already is a valuable information for you, as you already may assume things based on it (like, what kind of people you may find past the in-process interview as they all more or less were filtered in by it). So you can always stop and say "thank you" and walk away without wasting more time. You'll allow yourself to stay longer only when the interview's quality warrants it.

digler999 11y ago

> including (time/space) complexity analysis. I think this is one of the most inane things to be asked during an interview. personally, I've never found myself in a situation where I truly needed to choose between a vector/map/list/hashmap. Or had to find the O(x^n) and replace it with O(x^2) Obviously it depends on the application, but many jobs are simply maintenance coding: find bug, fix bug, test fix. Often times it makes absolutely no difference whether you use a list or a vector, or else you'll get the paradoxical "vector-is-always-faster" because of locality of reference. In my (admittedly limited ) experience, most of the effort is spent simply making it work, not being bogged down because you used a map instead of a hashmap, or didn't know about some esoteric, bleeding-edge probabilistic data structure.

pessimizer 11y ago
Honestly, I consider an instinct for complexity analysis the most important thing I learned in school, and the thing that I've gotten the most use out of. I don't know what case you're making here: are you saying that high-level architecture is so hard that choosing a map or a hashmap should be a coinflip, or the one you see first? Having had some criteria for making the choice makes my life a lot better when everybody is panicking because something is running like shit and no one understands why.
- digler999 11y ago
  To me it was a bunch of rote memorization, just like a biology course. I never - never - have needed to know how bubblesort/heapsort/mergesort actually work, except to appease interviewers. I'm not saying I'm pro writing-inefficient-code, but if you want to talk big-O during an interview, I"m going to roll my eyes about as much as you asking me who the 19th president was.
  - democracy 11y ago
    I ask such questions at the end of an interview, but mostly to see the sanity/reaction or thinking process - wrong answer would do, rolling eyes - would not :)
- b1449835 11y ago
  Oh shit, my code is O(1), looking at it in the profiler tells me that it's not a bottleneck (it says < 5ms). Yet this single function call somehow takes 78ms in wall time when it should at worst be 1ms (and even that is too much). The reason? JVM classloading/JITing. This quite annoying when you have shortlived applications.
jerf 11y ago
"Or had to find the O(x^n) and replace it with O(x^2)" The other thing that really seals the deal for me as an inferior interview question is that you don't need to have a clue what O(x^n) is to wrap some code in a simple time call, see that the code you think ought to run in microseconds is running in seconds, by visual inspection notice stupid nested loops, and fix it. Self-taught programmers may not be able to say "O of exx to the enn" but that doesn't stop them from fixing it. So... seriously, what good is the question anyhow?
- tomjen3 11y ago
  What if it isn't stupid code? What if it is straight forward and simple and looks correct but is just slow because it has to traverse a list instead of using a hash-table or some other data-structure. You can't get there by intermediate steps, you have to rip out the code that uses it and rewrite it with a hashmap. You can only do that if you know the space complexity and when a hash and a list is appropriate.
  - jerf 11y ago
    "You can only do that if you know the space complexity and when a hash and a list is appropriate." You seem to be confusing "knows how to say 'oh of enn squared'" with "the loop gets slow when I iterate over a lot of things". One is generally a product of education, the other, merely experience. The idea that a thing can only be learned in a classroom is perhaps in the top ten most pernicious ideas in the modern world, and probably one of the more surprising ones to show up in that list. You do not need special courses to discover that your code runs slowly, and as I've seen often enough, having had those special courses does not confer immunity against writing slow code. Now, I am also a believer that formal education has its place, and if you are going to get a formal education in computer science, big-O analysis absolutely must be part of it or you are literally missing out on an entire rather important sub-discipline. But the idea that it's some sort of touchstone between Good and Bad programmers is just ludicrous nonsense. Slow code is slow. There are abundant tools that can be used to figure out why. If you can't work out why your O(n^3) loop is running slowly after a couple of years of practicing the art, you don't need formal education, you need a different job.
ktRolster 11y ago
Most likely you think about it, though. When you're coding, and you have a triple loop, do you think, "Oh, this is O(n^3). Is n going to be too big here?" It may be something that's so intuitively obvious to you, that you don't even think about it. So you naturally use the hashmap, where someone else might try a list and then start doing a lookup in a loop. Then while that particular instance might not break things, it'll slow things down, so overall the application feels sluggish instead of snappy.
- eecks 11y ago
  I've never ever thought that to myself. Then again my code never has triple loops.
  - Retra 11y ago
    So do you never call anybody else's functions, or do you always verify that they don't do any loops in those calls?
mech4bg 11y ago
I've seen this plenty of times. I've worked both on a trading platform and a large website, and both times encountered many performance issues that were solved with a more appropriate algorithm or data structure. I've even seen this with a list as small as 10 items - a O(n^3) algorithm was making multiple network calls each time; changing it to O(N) alone made a huge improvement in speed.
- jkyle 11y ago
  This does come up a lot. But I think it's more telling if a programmer knows how to profile his program and find the performance bottlenecks, recognize them for what they are, then fix them appropriately than if they can recall a specific optimization for a specific use case on demand.
- kevan 11y ago
  Most of my work on an ecommerce platform doesn't need much attention to algorithmic complexity, but everyone on my team still curses the guy who wrote an O(n^4) algorithm in our checkout pipeline (discounts, promos, shipping, tax, etc). More than a couple items in your cart and you couldn't checkout because the thread would spin forever. I want to work with a team of people who can recognize these things immediately, even if it's not an absolute requirement for the job.
  - im_down_w_otp 11y ago
    That has a lot more to do with mechanical sympathy, and awareness of when you've exchanged cleverness for complexity disguised as cleverness, than it does with knowing the big-O of operations on a datastructure. What you want the person to identify is that they've made your simple iterative checkout process into multiple unbounded tree traversals with no circuit breaker. Knowing that searching the tree is O(log(n)) isn't very helpful when your problem is an inability to identify that you've made (n) an unnecessarily huge problem space.
tomjen3 11y ago
I am sure the RubyGems authors thought the same way until somebody sped the process up dramatically because they could do space-time analysis[0]. [0]: https://news.ycombinator.com/item?id=9195847 https://news.ycombinator.com/item?id=9195847
ken47 11y ago
I've never found myself in a situation where I truly needed to choose between a vector/map/list/hashmap. Or had to find the O(x^n) and replace it with O(x^2) This entirely depends on the kind of product you are working on. When you get to a large scale with any programming project, optimizing computational resources will cut costs, and can often add value to the customer as well.
- rhizome 11y ago
  This is special pleading, though, right? It being relevant to "any" large scale project really just means that projects can get big enough for it to matter, but how many jobs involve this family of software? How many interviews for positions directly related? Very few, I would guess.
  - NotSammyHagar 11y ago
    I find myself having to think about efficiency at least a couple times a week. I'm working on database implementation, and in the query processor we have to consider all the time how to evaluate various things efficiently.
    - rhizome 11y ago
      Taking this at face value, were any of the techniques you use in these tasks directly addressed during your interview there?
- Cakez0r 11y ago
  I would say it starts becoming important for any application that has multiple concurrent users. If it's a single user running it on some device and there's no shared resource (I.E. back end), then chances are it's not going to be an issue.
EpicEng 11y ago
That's fine, as long as you _never_ need to trust said dev to do anything complex. It's fine to have mediocre developers perform mediocre tasks, but if you want more from them someday you may be in trouble.
- digler999 11y ago
  to me, complex, and complex-ITY are entirely different matters. I want a smart programmer who can figure out really complex bugs (something you cant figure out from google/wikipedia). Not someone who memorized the big-O performance tables of 8 different data structures (something you CAN look up on wikipedia ).
  - was_boring 11y ago
    Exactly. I once identified memory leaks in a managed runtime implementation, worked around them, and showed that third party networking libraries from the hardware vendor were causing irrecoverable crashes but I can't spout off runtime complexity of algos. The former saved a multi million dollar contract and my company's reputation, the latter is something I look up when I need it.
  - EpicEng 11y ago
    Optimally, I would like both. I agree that wrote memorization is not a very useful skill, but I'd like to know that they can grasp concepts, and I find it hard to believe that there are a bunch of skilled devs out there with great potential who don't or can't understand algorithmic complexity. Interviewing is hard though, I'm not trying to say that any one question or metric should necessarily rule someone out.
- FLUX-YOU 11y ago
  That's not really fair, people don't grow unless they're challenged. Some people really do never advance past a certain point, but a lot of people write someone off as mediocre when they're just still in the process of gaining skill. Also, they only assess ONCE, which is pretty bad if we're trying to establish how good you are forever and always.
munchbunny 11y ago
I think it's easy enough to ask for this skill if the job would require the interviewee to apply this skill. If your job is mostly frontend, yeah, you probably won't need to worry about this problem. But if you're hiring somebody to work on graphics? You better be doing complexity estimates in your sleep.
sklogic 11y ago
That guy in Google who screwed up Android thought just like you. Now, as the number of text messages stored on your phone grows, the entire system slows down. Such a pity that cretin was not screened out on an interview!
plasticchris 11y ago
This can happen even on small systems. I once replaced a O(n^4) with O(n^2*ln(n)) on an embedded target which made a minutes long user facing process take seconds. The catch is that the original used a good algorithm, but made an implementation error which I caught in profiling. So complexity analysis is good, but the only way to get better at something is to measure it.
nathanaldensr 11y ago
Here's how you should reply: "Sorry, I don't have those complexities memorized. When I really need to look them up (which is nearly never), I refer to bigocheatsheet.com."
robryan 11y ago
You should have an intuitive idea around complexity and when it makes sense to optimise. I don't think though you need to know that for example fastsortx is n log n in the best case off the top of your head in an interview situation. You should be able to reason your way through why it is faster than some other sort though.
- nsxwolf 11y ago
  What the heck is fastsortx? This is another problem I have. I study all these algorithm books then when I get to the interview they ask something that either isn't in the books or they've come up with some nickname for it and expect me to know it.
  - democracy 11y ago
    Even Google doesn't know...
  - rhaps0dy 11y ago
    It's probably a placeholder. Same as writing <sort algorithm>.
democracy 11y ago
That's how I failed my Google interview - they said they expect programmers for any position to have good knowledge of algorithms, yeah, well... :) Frankly I don't like trivia-style interviews.
b1449835 11y ago
Because in most cases n is a very low number. If you only have 10 elements the algorithm doesn't matter.
collyw 11y ago
Exactly. Performance issues happen. Most of them can be solved by re-indexing a database or adding caching. If I really need speed at the algorithm level I can look it up.

quarktasche 11y ago

Wouldn't the question about the moving average be a good one though? It should be quite obvious that you can calculate a moving average without remembering more than the current average and the window size. Given that, time and space complexity should be obvious. I think it would be a fine question to see if the interviewee has any idea what complexity means. Or was it a more complicated moving average case (exponential etc) where the algorithm was given and you were asked to determine the complexity?

collyw 11y ago
How many jobs will it be relevant to?
- quarktasche 11y ago
  Which part? Figuring out how to efficiently calculate a moving average seems like a good question for basic maths skills to me. About complexity theory, there is already a lot of discussion about it's relevancy in the comments. My point was meant more along the line that, if you value a basic understanding of complexity theory, the question asked of the GP seems reasonable.

riazrizvi 11y ago

I have come to believe it is part of an industry-wide negging style to keep people in their place.

lewisl9029 11y ago

I personally also prefer take-home projects over being grilled on my ability to solve obscure algorithms problems under pressure. However, this second route also comes with a number of issues. The most annoying of which in my experience is the amount of time investment each interview requires from the candidate. At least in the traditional technical interview, interviewers and candidates tend to be roughly equally invested in the interview process in terms of time spent, so interviewers have to value candidates' time because failure to do so will end up wasting their own time as well. In take-home projects, this balance in time investment completely breaks down, and thus there's very little left to discourage interviewers from issuing ridiculously time-consuming projects. I've done a few of these in the past, mostly to get some practical experience with a new framework & ecosystem I'm not too familiar with yet, but I think in the future I'll likely just politely decline any projects that looks unreasonably time-consuming, or at least try asking for certain adjustments to the spec first.

throwaway12324 11y ago
Ironically enough, Triplebyte's own take-home projects were some of the worst I've ever had, and did a horrible job of respecting the candidate's time. When I went through the their take-home interview process, there were 4 projects to choose from, with only one having anything remotely to do with my area of expertise (it was a multiplayer game, and I was looking to work as a web front-end/full-stack developer). For all their talk on how you should select a practical project to talk about because it correlates better to ability to get real work done, it's really rather ironic just how utterly academic and unpractical the projects they offered were (that game was literally the most practical one on the list). Now they tell you that you're expected to spend at most 3 hours on the project. This might be true for some of the other more academic projects on the list if you had any expertise in the respective areas, but it definitely wasn't true for the multiplayer game project I chose, which had a non-trivial front-end and back-end component, for which testing alone could easily take 3 hours (Or maybe I'm just bad, which is definitely possible, but I've asked many of my more experienced peers how long they think a project like this might take for them, and the lowest estimate I got was a whole work day of 8 hours). Now they also tell you that it's OK if you don't finish the project. There was also a second part to the interview where they may ask for extensions to the original project, where they'd give us more time to work on it, so I assumed if we don't finish, they'd ask for us to finish the original spec along with some extensions for the second interview. Since I had already finished the front-end component for the game, and had worked on the project for well over the expected 3 hours, I decided to call it a day and work on preparing for some of the other interviews I had that week. However, that it was OK to not finish the project for the first interview might have been an outright lie. I won't ever know for sure because I was rejected after the first interview for not finishing the back-end as well as being unable to resolve some performance issues the interviewer pointed out (which at no point in the interview did he even ask for me to fix, so I assumed fixing that performance issue would also be a part of the extension). Incidentally, I went back to the project a while after the interview and resolved the performance issue in 5 minutes flat. The whole process just left a rather bitter taste in my mouth, which was all the more disappointing because I went in with great hopes after reading all their great blog posts on HN. For anyone else considering Triplebyte, I'd highly recommend going with their traditional interview route until someone at Triplebyte can confirm the process has changed for the better. At least with that route, if you get rejected, you'd have wasted less time in the process.

jguegant 11y ago

I already saw such a colleague training hard at competitive programming, everyday. Yet, he wrote that kind of C++ code: ressource AClass::CopyACriticalRessource() { { std::lock l; } // Optimization for the lock. return ressource_; } Having a good understanding of every aspect of your architecture, taking your time instead of rushing for blazing fast (O(1)) but hacky solution, always having in mind that someone must have already solved your problem... these are some valuable skills not coverable during a 1h interview in front of a white board. I would like someone googling a solution in front of me during an interview, trying to UNDERSTAND it, check its complexity, compare it with other search hits, adapt it to his problem, ask for my review. The best technical interview being a homework with enough time for a normal human being, requiring tricky algorithms to solve it in a fancy way, a good architecture and great computer-science knowledge in general. Afterwards, a code-review/interview of 1 hour with debriefing on all the choices.

thearn4 11y ago

> I much prefer "homework" projects I helped a group within my organization with their hiring process recently, and we had pretty good success with assigning a short "take-home" exercise, vs. trying to haze them with a programming problem over a google hangout interview. A problem focusing on a small part of what that group does, but scoped to be doable with 1-2 hours of work.

jonesb6 11y ago
Why don't companies offer it as an option? "Hey here's a fizz buzz question, if you feel confident answering it now go for it, otherwise we'd be just as happy for you to do it in a take-home fashion?"
megablast 11y ago
I guess it works well for people unemployed with nothing else to do, no family, lots of time. Everyone else, this sucks!
- buzzdenver 11y ago
  Interviewing does take time. Most people will be able to spare an hour to find a new job.
- kylequest 11y ago
  So you are saying you don't spend any time preparing for your interviews :-)
hemc 11y ago
Homework projects don't work some cases, when developers come with a refined copy/pasted code or code by another programmer. Better way is to go through the previous code that developer has written or go through the code written by other programmers to know the capability of the programmer to understand different code patterns
- whyever 11y ago
  > Better way is to go through the previous code that developer has written or go through the code written by other programmers to know the capability of the programmer to understand different code patterns Doesn't this have exactly the same problems as homework projects? You can still copy and paste or plagiarize.

mettamage 11y ago

Here's an idea (perhaps naive): why not add all the methods and give them optionally to the programmers. E.g. - normal interview is mandatory (you need something to base your ideas on) - portfolio is optional but when chosen it weighs heavily - take home project is optional but when chosen it weighs heavily - etc. With 'weighs heavily' I mean that it influences the criteria for which those methods are good for. The downside of this is that interviews are more unstandardised, which is a trade-off worth considering.

draw_down 11y ago

If you're a programmer, you like to solve puzzles, so think of passing the interview as just another puzzle! I mean, yeah, it shouldn't be, but here we are.

kafkaesq 11y ago
If you're a programmer, you like to solve puzzles, No -- we like solving problems. Especially those that have a legitimate context (i.e. are explicitly linked to some actual, real, business or social problem). And for which our skills are truly relevant needed (specifically, for which no one has a readily available answer, at the moment). But made-up "puzzles"? For which the asker already knows the answer, so they sit back and watch us dance? Not so much.

Bahamut 11y ago

Whiteboard coding doesn't necessarily have to involve guessing the algorithm - any whiteboard questions I've asked avoided doing so. On the flip side, my experience with take home projects is that they are much more of a hazing/pressure cooker ritual - at the least, the time pressure of a Google/FB/etc. interview lasts only a half hour. Most will expect candidates to spend a huge amount of time, which many won't have if they're actively job searching, or are busy people in general - I prefer to pour that extra time into open source work, since at the least it benefits others. One company assigned a project to me once that suspiciously seemed like implementing the company's whole business model. It doesn't take long to figure out if someone has the right skills oftentimes if you ask the right questions - candidates shouldn't be penalized for companies' ineptitude in assessing interviewees.

rqebmm 11y ago
> I prefer to pour that extra time into open source work, since at the least it benefits others As someone who pushed his company to adopt a "take home" assignment for our interview process, it should be perfectly reasonable to reply with "here's my commit history on a project relevant to what you're hiring for." I much prefer giving (and taking) take home assignments because it lets the interviewer see _what someone will actually produce on the job_. If you can do that without jumping through our specific hoops, great. If not, that's why we have the assignment. It also at least somewhat ameliorates the pressure cooker of an interview, which many people cope with poorly. It can be hard enough to communicate clearly when all eyes are on you, let alone get your thoughts together and solve a logic problem. If that's actually analogous to your work environment, well...

tn13 11y ago

I don't think that is really a problem. We hire people to do a good job and most of the times kick-ass programmer isn't the best person for the job. At least for a smaller company we need someone who can understand business needs, can communicate better and Add Value. This is a far complex job than implementing red-black trees. Consider this: We need to find the most shared URLs on Facebook in last 24 hours. I can perfectly see why a lousy coder might achieve this objective better than a kick-ass one. A coder with average skills quickly figured out that Buzz-sumo has a public webpage with that content which can very easily be scraped using phantomjs. Job got done. Another great coder suggested to me I should buy $10K per month Facebook firehose. He is not wrong and that is a good solution too but he failed to see that we are building a POC and not a full featured product. Needs of companies are complex, many times if you can pay great salary just having a filter for IQ is good enough. But in most cases I think it is far better to look at an individual and judge.

paulryanrogers 11y ago
Would the Buzz-sumo programmer be expected to investigate their terms of service too? (Often scraping is prohibited.)
- tn13 11y ago
  I am just giving an example. For a POC it does not matter much.

soham 11y ago

Homework projects could work well if the hiring requirements are small, but won't work well when a company has to hire say 25 engineers a quarter, which we had to. At that point, the process becomes too long, and it's easy to lose good candidates to a long process. This method of interviewing has been around ever since, and is going to be around for the foreseeable future. Nobody loves it, including the interviewers, but there just isn't a better way to do it at any sort of scale. Especially when there are much bigger problems to solve when you're running a business. It's best to take the bull by the horns. I run http://InterviewKickstart.com http://InterviewKickstart.com, which is a bootcamp for preparing for such technical interviews. We do almost exactly what is in the blog post. It works. Spectacularly.

thesimpsons1022 11y ago
i was about to sign up but then it asked for my phone number. why do you need this information?
- soham 11y ago
  Simply because it is quite difficult to explain the concept in writing. What does it mean to have an intense bootcamp just to prepare for interviews? What's the method? etc. I want to take the time to talk to everyone who is interested in the course. Because the concept is new, people have all sorts of questions. Can't possibly address all of them in writing. I don't have a team of salespeople and haven't spent a penny on advertising. If you still prefer email, please feel free to send one. It's on the site. It may just take longer to do back and forth. Thanks for considering!
  - kafkaesq 11y ago
    Simply because it is quite difficult to explain the concept in writing. Then maybe it's not such a good concept.
    - soham 11y ago
      Or possibly because it's a new concept, which takes more text to describe. More text than what can be included above the fold, and more than what most people read attentively these days. After all, it's an 8-week intensive course, mostly for CS grads, that grills you hard, and is not cheap. As a consumer hence, I'd highly prefer to talk with someone. Not to mention, all educational institutes have an enrollment process that needs you to talk to a human. At some point, when it becomes more common and well-accepted, we will condense it, but it feels a little too early to do so.

shasta 11y ago

> But do you really learn anything by asking the candidate if they can recite the time complexity of a moving window average algorithm (as I was asked to do by an interviewer yesterday)? This is actually a trivially easy question that gets to the heart of whether you understand the point of moving averages or not (that you can update a sum by subtracting out the value leaving the window and adding in the value entering the window). "Programming ability" isn't one dimensional. Whether or not the above question is useful depend entirely on what skills are important in a candidate.

Cakez0r 11y ago
It's not so much specifically about the moving average problem. I would want to see that a candidate can reason about the performance of some code / algorithm. I would not expect them to be able to recite the performance for specific algorithms from memory, however.
- shasta 11y ago
  I agree, but I suspect "recite" was editorializing by the parent.
  - quanticle 11y ago
    It wasn't. I was literally asked, "What is the time complexity of the moving window average algorithm over an array?" and when I asked for clarification, I could hear an edge of... I guess frustration in my interviewer's voice. Granted, by this time, we'd been through a couple of other problems, and time was running short, but I still think it was pretty unprofessional of the interviewer to let frustration or any other sort of negative emotion show during the interview. That, more than anything else, contributed to my own frustration and perception of unfairness in the entire interview process.
    - shasta 11y ago
      Right, but "recite" was editorializing. That implies that the interviewer expected you to produce the answer from memory, as opposed to thinking about it. It's an easy question if you're familiar with moving window averages and know what the interviewer intends. If it was asked apropos of nothing, a request for context seems reasonable, though. It sounds like you probably had a bad interviewer. There seems to be no shortage of software interviewers lacking in "people skills."

eranation 11y ago

When I was interviewing in the past I was usually refusing any take home projects, if you don't want to spend 30 minutes talking to while watching me code it tells me something about your priorities. However whiteboarding is also terrible. I prefer getting a real world / work related scenario problem, and solving it TOGETHER with the interviewer: if I'm stuck, as opposed to a hackerank codility challenge, there is a good chance they will let me know and hint in the right direction, and they will also see how I communicate, how I think, how I respond to hints. An online challenge can be easily done by the candidate sitting next to 2 more senior coder friends who help him / her along the way. Unless it is proctored, you can never know. I had people whispering someone answers during a phone screen, people typing my question in Google and reading me the first result (I google it at the same time). I think that giving you a hackerank or codility take home problem unless this is an entry level job will simply drive away experienced people. If someone can pass your take home test easily, you will still need to phone screen them before you fly them over in most cases, and if they are that good, they will probably prefer companies that don't waste their time and jump straight to the real person phone screen. However, some people are more nervous when someone is watching over their shoulder, so here is what I would do: I would ask the candidate what they prefer: 1. work related a hands on assignment with plenty of time (e.g. should take 30 mins but you give 1 hour) and ability to search online and ability to compile use an IDE just like in a real world scenario 2. Skype screen with a lot of small questions on a topic they really feel they know about (no Googling allowed), and a relatively smaller coding challenge (something you can code in 15 minutes) 3. Work related scenario but with a real person, where there is no one best answer to the question, but more of a balance of tradeoffs and more open ended but 100% involves coding (just like most phone screens, but more work related and not just puzzles) 4. The standard puzzle challenge but alone - you have 30 minutes to solve the problem once you see it (Googling is allowed or the test is proctored to make sure you don't, then you get more time) 5. the classic - a cracking the code interview kind of question, but with a real person, code sharing (for crying out loud not google docs, at least have your candidates use something that offers color coding and easier indentation) - some might choose still this If you let your candidate chose what is best for them, you already made an amazing impression, and you might have much less false negatives. Just my opinion

cheez 11y ago

The problem is that most companies want the take home in addition to algorithm interviews, keeping the worst characteristics of both approaches.

WalterBright 11y ago

Every week HN has a topic on the front page about how inaccurate and unfair job interviews are. They're always going to be inaccurate and unfair. Interviews must use proxies for candidate quality, and there will always be false positives and false negatives. The most practical thing is to work with it as it is.

Someone 11y ago

"But do you really learn anything by asking the candidate if they can recite the time complexity of a moving window average algorithm" No, but it may be a good question, anyways. A good interviewer probes you about things you might encounter in the job to find the point where you cannot recite things, and looks how you handle that. Having said that, I don't understand the focus on coding interviews I read on HN. I don't know whether it is cultural difference between the USA and Europe, whether I just haven't noticed how you are supposed to prepare for interviews, or whether I am too smart to need to bother, but when I apply for a job, I look up what the company does, try to figure out what its culture is, but do not prepare for the technical side of things. An interview isn't a one-sided affair of you being quizzed, it is two parties figuring out whether they fit together,

blakecaldwell 11y ago

Coding interviews are very stressful, and churn through a lot of really awesome potential hires. I imagine there are tons of false negatives, but it's nearly impossible for a terrible programmer to get through a gauntlet of programming interviews. However, I agree. They don't give a full picture of a developer's abilities. As a developer, I prefer take-home projects. As an interviewer, I prefer a few coding interviews, followed by a take-home project.

platform 11y ago
I have not tried it. But in my next business venture, I plan to actually do peer programming with the candidate. This is will allow members of my team (or myself) to get to know the candidate, evaluate the 'wave lengths', and how effective he/she is at finding patterns on the internet/books -- rather than thinking things up. It also demonstrates to the candidate commitment on our side, and it naturally forces us to find problems of a proper size/effort (as we are spending the effort too). We would not do it for all applicants, though -- only for once that pass basic screening / competency process (there are no trick questions or exercises there
- blakecaldwell 11y ago
  Yeah. You can save a lot of your time, and the candidates' time with a simple coding interview - even FizzBuzz will knock out half of the candidates. Then pair up and/or give a take-home, then discuss the solution in a one-on-one session.
- quanticle 11y ago
  I have not tried it. But in my next business venture, I plan to actually do peer programming with the candidate. As a candidate, I can highly recommend peer programming. One of the greatest interview experiences I had was interviewing with a local shop where the employee and I reimplemented a Set class using TDD and pair programming. The employee sat at the keyboard, so I didn't have to deal with not knowing keyboard shortcuts or the unfamiliar operating system (OSX), but he was very careful to only write the code that I asked him to write, and to let me make my own mistakes. It was one of the most enjoyable, and, dare I say it, relaxing interview experiences I've had. Though I didn't get an offer, I wouldn't hesitate to recommend that company to any of my peers who're looking to make a change. Also, like you said, it was a second tier screen; after an initial screen consisting of a more traditional phone interview where I had to write some Javascript.

im_down_w_otp 11y ago

Ah. I just provide all my answers in an obscure language they won't know, but has lots of "nerd-cred", so they feel uncomfortable questioning it. Like OCaml, Erlang, or Haskell. Then in the rare case they question it, which usually goes something like, "Can you do it in Java? We don't use whatever language it is you're using there." I respond with, "Oh, will I be spending a lot of time at this job coming up on-the-fly with somebody's PhD thesis from the 60's?" To which the answer is always, "No." Which finally ends the conversation with me saying, "Well then you asked me to solve an irrelevant problem, so I'm happily providing you an irrelevant answer." I have a fair bit of contempt for the pitiful state of technical hiring and assessment process out there. I think that comes mostly from my opinion that hiring is probably the most important thing any company or manager does, and that if any one of these companies/interviewers put even 10% the amount of effort into learning about educational/occupational assessment, psychology, and neurology that they do into obsessing over the dubious idiosyncrasies of the latest framework-rehash-of-the-month, then these terrible interview practices wouldn't endure very long, and we'd all be spared the indignity of the farce on both sides.

xerophyte12932 11y ago
> Which finally ends the conversation with me saying, "Well then you asked me to solve an irrelevant problem, so I'm happily providing you an irrelevant answer." So how many of those companies made an offer?
- im_down_w_otp 11y ago
  3 out of 5 where it's come down to that kind of bizarro pissing contest.

RichieAHB 11y ago

Unfortunately, this is much the same as virtually any broad examination process. I'm primarily thinking of schooling up to 16. There's a lot of work to be done in ironing out one-size-fits-all testing in a lot of areas, primarily those that "require" a human to do the processing of the data.

braythwayt 11y ago

This is a general problem: > Being a good president has a surprisingly small role in > being elected president.

highCs 11y ago

I have a theory: I think the coding skills of a programmer are function of the number of programs he's written - from scratch.

collyw 11y ago
Maybe, but at the moment I have inherited a bit of a mess of an application in my new job. I think it takes more skill to work out what it is doing and replace the crap code with something simpler and more efficient, more maintainable and easier to read.

lazerwalker 11y ago

My favorite sorts of interviews are ones where you're expected, either by pairing with an employee on your own, to fix a bug in an open-source project you use. Has all of the benefits of a take-home project over whiteboarding, but the end result is you have something tangible to show for your efforts even if you don't get the job.

nathan_long 11y ago
This. Also, any interview where the candidate pairs with an employee lets both of them see how they like working together.

restalis 11y ago

"Being a good programmer has a surprisingly small role in passing programming interviews." "And that just says it all, doesn't it?" An interview is for both prospective employer and potential employee to meet and find out about each other. If you as a candidate notice something you don't like during the interview (including the interview itself), that already is a valuable information for you, as you already may assume things based on it (like, what kind of people you may find past the in-process interview as they all more or less were filtered in by it). So you can always stop and say "thank you" and walk away without wasting more time. You'll allow yourself to stay longer only when the interview's quality warrants it.