24 ms·

The best engineering interview question I've ever gotten

893 points by db48x 5y ago

deleted 5y ago
[deleted]
db48x 5y ago
I have to agree; I think that question is better than any I have ever gotten in an interview.
camgunz 5y ago
Yeah I do love this. I think it still has some performance element to it, but I would relish it. Definitely good as an option I think.
blatherard 5y ago
Is there a filter that automatically removed "The best" from the submission title?
- dang 5y ago
  Yes. We glued it back.
  - postingposts 5y ago
  - notreallyserio 5y ago
    Thanks dang, you're
  - db48x 5y ago
    That‘s actually pretty hilarious. I submitted this, made my comment, and then immediately went to make a sandwich. I never noticed the truncated title; I didn’t even look. Thank you!
- kaashif 5y ago
  This truly is one of the engineering questions of all time, in my opinion.
dataflow 5y ago
For a junior-ish software engineer level (say, 1-3 years out of college) who claims to have experience in C, this seems like a fine question, but for a senior software engineer (which I think is what he was interviewing for) it doesn't seem all that great, except maybe as a first-round filter [1]? Not that it's terrible, and I guess if the candidate fails it then it certainly tells you a lot—but if they complete it successfully then it doesn't really seem to tell you much about their software engineering skills or depth of understanding. It's better than some silly challenges companies give, but there's so much room to ask more probing questions or provide more challenging tasks to help you gauge where the upper limits of a candidate's abilities are, instead of just the lower limits. [1] Edit: Looks like it indeed was a filter question, as I suspected: https://news.ycombinator.com/item?id=31065819 https://news.ycombinator.com/item?id=31065819
- elromulous 5y ago
  This is an issue with many coding problems. I'm not sure if there is a term for this is detection / estimation theory, but it provides what I'm going to call asymmetric information. In this particular case, negatives are true, but all positives must be presumed false. If you make a problem too hard, you're faced with asymmetry in the other direction.
  - dataflow 5y ago
    I think the usual solution to that is to follow it up with "what if you wanted to {introduce new/harder requirement}?" and ask them to explain & make as much of the changes as time allows. To give an example, if you were to initially ask someone to implement depth-first traversal recursively, you might follow that up with asking them why you might end up needing to do that iteratively, then you might have them do that iteratively for pre-order depth-first traversal (easier), then follow that up with post-order iterative traversal (harder), then you might ask whether/why it might perform poorly (e.g., re-visited nodes blowing up exponentially), then ask them how they might mitigate it (keeping track of visited nodes), then ask them how they might mitigate the space requirement (lots of possibilities), then ask them how they might handle parallelizing it for some particular problem... I can keep going, but the point is you can keep something like this going until they start to struggle, and that at least tells you the upper limits of their abilities on at least that axis. And you can pace it by e.g. skipping stages as you get a better sense of their skills.
kashav 5y ago
Got the same question for a new grad position around 2019. Only had an hour that time. Great question.
notreallyserio 5y ago
I'd find this question frustrating because it feels like a trap. I was looking for the trick -- was I missing something in the question that made the obvious approach untenable? No, the answer is simple, just add a way to multiply numbers with the multiply operator.
- melony 5y ago
  And correctly dealing with arithmetic overflow.
- Syonyk 5y ago
  It's "Advanced FizzBuzz." "Can you understand a complex code base, figure out how to zoom in on the section of interest, and add a function?" Far, far too many people interviewing for coding positions can't do that. Though I question how many could whip out FizzBuzz in 5 minutes and would then fail the more complex version. At some level, either you can code, or you can't. And, importantly, at some point in your career (if you go down the standard management track), you will typically lose the ability to code. If you've not done it in 5 years, you probably can't do it, in a practical sense. Not that you can't re-learn, but you can't just jump into a coding interview and ace it, either. If you find yourself at a point in your career where coding interviews are a thing, "having a side hobby project that requires coding" is a very useful way to be really quite good at coding interviews. Plus, if it's your hobby project, there's no "Hrm... can I talk about that?" sort of problems when you're asked about a time you X'd with Y. You can talk about your personal hobby project absolutely as much, and in depth, as you want.
  - mikeryan 5y ago
    This isn’t an “Advanced FizzBuzz” if it takes three hours or even one hour. The whole point of a FizzBuzz question is that it’s a fast, like 2-3 minutes fast, heuristic for separating the wheat from the chaff and this isn’t that at all. At some point it’s no longer a FizzBuzz and just a coding challenge. Even the author’s answer reflects that (though he calls it a FizzBuzz) I don’t know a thing about memcached internals but, as presented here, with the potential encoding and concurrency issues means that there’s likely levels to the complexity here that will likely take someone quite a bit of time to resolve if they’re not familiar with memcached internals. But again memcached and even C++ isn’t my thing. ¯\_(ツ)_/¯
detinho 5y ago
What kind of skill does this type of test measures? I think the main ones are: * How much about the language the candidate knows. Not the control flow and data types, but how to navigate and where to look for information. And that leads to bellow * how quickly one can understand a codebase (at least part of it) and start delivering What else? If that's one of those "think out loud" tests it's possible to get a clue of how the candidate thinks, but it's not specific of this test.
- teraflop 5y ago
  I'd say your second bullet point encompasses a number of sub-skills, such as: * Understanding common coding patterns, and inferring design intent from them * Quickly prioritizing "critical" vs. "nice-to-have" components of a desired feature (and reprioritizing as you learn more about the implementation constraints) * Being able to take a partially-complete or slightly buggy implementation, and quickly identify what's wrong with it (e.g. by interpreting compiler errors or using a debugger) * Gaining confidence in the correctness of code by spending a reasonable amount of time on testing, without going overboard
- avip 5y ago
  The skill of "getting something done without breaking anything" which is basically our profession.
- eyelidlessness 5y ago
  If I encountered this test, here’s what I’d think (or hope) is being evaluated: - Do you understand the problem, or seek better understanding if not? (Obviously the problem here isn’t extrapolating arithmetic, but identifying the importance of an atomic operation.) - Do you recognize the value of the change, or question it if it feels non-obvious? - How do you think about approaching an unfamiliar codebase/code path? - How do you think about the challenges you encountered while working on the problem? - What else felt important to you that I’m not looking for? All of these questions provide a lot more information than “can you write code?” Especially meaningful for evaluating actual fit is reactions to walking into pre-existing code. In fact I think this has been, at least subconsciously, my best heuristic for assessing colleagues’ skill maturity. “Juniors” will stare at a problem or ask a lot of easily answered questions; “mid level” devs will bang their heads trying to answer without asking; “senior” devs will have a good intuition for which challenges are discoverable and which are best solved by asking a human or a google or whatever, or at least for recognizing the distinction after some time exploring the problem.
ankrgyl 5y ago
Author of the interview question here. I came up with the idea after working at the company for a couple months and realizing that the skill of "diving into an unfamiliar area of the code and quickly figuring it out" was very important for us. Database codebases are large and complex -- so much so that almost every new feature you work on for the first year or so feels a lot like doing the question. Over the years of evaluating with it, we learned a lot about its quirks (both false positives and false negatives). At one point, someone got so frustrated that they threw their laptop on the floor. Happy to answer any questions about it!
- dataflow 5y ago
  I definitely appreciate the intent of it, and think measuring along that axis is incredibly important (and rare). But as someone who continually struggles more to dive into some codebases than my peers, I can say that this task is far too easy and wouldn't help you gauge my ability to do that at all. Something that requires implementing a new feature that isn't a near-mirror of an existing one (ideally one that requires a bit of thought at the design level) would probably be a much better measure, but I don't know what that might be for something like memcached.
  - deleted 5y ago
    [deleted]
  - ankrgyl 5y ago
    That's a great point! We didn't actually think of this question as much of a "measure" (and I think the blog post alludes to this as well with a "Fizz Buzz" analogy). We thought of it more like a filter — if you couldn't pass, then you _may_ not be adept at navigating through a database-y codebase at the pace that was necessary (at that time) for the work we did. We had other questions (which I won't spoil) that tested more complex system-design skills, without having to implement them within an existing codebase. We felt this provided clear independent signals (ability to "hack" within a codebase and ability to reason through complex system design) vs. conflating the two. As the team grew, and we needed folks who were much better at one or the other, this helped us round out the team with folks who excelled at either skill.
avip 5y ago
I salute this dude's commitment to Chesterton's fence. If only my teammates had this mindset...
- db48x 5y ago
  Yea, any candidate who works Chesterton’s fence into their answer somehow gets bonus points from me.
victorronin 5y ago
Got mixed feeling - As people pointed out, it's advanced FizzBuzz (really advanced) - I can easily see a lot of people being unable to solve it, not because they can't do it, but because of a "performance anxiety". The bar here is quite high, and the result is binary (does it work or not). - I like that it is quite close to real life (that people have to read code, figure it out, write code). On another hand, again, what is not real, that you are parachuted into a unknown (quite big) codebase and expect to add a small feature in 3 hours.
- theamk 5y ago
  The result is not binary - you can have it fully working, or forget binary protocol, or syntax accepted but multiplication not happening; or correct code changes done, but code does not build... That is one of advantages of longer questions (compared to 5 minute one): even if task is not fully done, there are plenty of other signals.
- andrewflnr 5y ago
  Given that they're specifically hiring for high-performance database work in C++, I think it's a appropriate to have a pretty dang high bar. For less-intense work, I would definitely want to formulate a less intense version of this style of problem.
kcarter80 5y ago
Why couldn't a potential hire be given 5 minutes to think about a solution and then be asked what their plan is? What benefit is added by 36x-ing the time allowed that justifies putting the candidate through such pain?
- ed25519FUUU 5y ago
  This is the right way to handle this problem. Everything after 5-10 minutes waste both your time and the interviewer’s time. There’s nothing you can discover that’s important after walking through their plan other than to figure out if they are familiar with code bases like the one you’re using.
  - db48x 5y ago
    I disagree with that. If you’re hiring a programmer, surely you want to seem them write some code. This question has the added benefit that they don’t have to write a whole program from scratch, they have to deal with a real–world program instead of a toy program created specifically for interview purposes, and they have to demonstrate that they can read and understand other people’s code. The latter seems really important to me, as apparently it was to the author of the question, because we spend so much of our time improving code that has already been written instead of writing completely new programs. Of course, I am especially good at these software–archaeology skills, so I suppose I could be biased.
  - hn_throwaway_99 5y ago
    Couldn't disagree stronger with this. In fact, I love this problem, in that it gets rid of most of the complaints about engineering tech interviews: (1) it's "real world" (for the job in question), (2) can be reasonably done in the time alotted, (3) involves a task that is most common for developers (modifying existing code), (4) shouldn't require extra preparation. I have seen many times where a user can actually explain the solution to a problem, but the translation from algorithm-to-code is extremely slow, or not done correctly at all. There is real skill in being able to output code quickly, even if you already know the English-language description.
  - 5y ago
a-dub 5y ago
i could do this one in my sleep. wish more were like it...
nine_k 5y ago
Just for the record. Last time I was given to code something up within 3 hours, my task was to write a Minesweeper clone (pretty graphics optional). I ended up with a workable console-mode implementation, using Python.
karlakush 5y ago
yokoprime 5y ago
There is a reason memcached does not have mult. One could argue even incr and decr should be handled by clients. Starting to add more arithmetic support is scope creep. Why stop there, why not add sqr or bitwise ops, maybe a text based adventure game? Just blindly implementing this feature without any questions should be a huge red flag
- pubby 5y ago
  You could say the same thing about web apps not needing fizzbuzz, but that wouldn't get you hired. As an aside, I think atomic incr/decr are useful. I don't see how you would implement them client side without some more complicated mechanism (like CAS).
  - macintux 5y ago
    Absolutely. At Basho, the company learned over time that developers needed help using a distributed key/value store correctly, and started building CRDTs into the server.
  - yokoprime 5y ago
    > You could say the same thing about web apps not needing fizzbuzz, but that wouldn't get you hired. fizzbuzz is not great, but its also implemented as a stand alone webapp (or console app), not added to an existing codebase. im not against incr/decr, my point is moving beyond these simple operators is pushing the product into feature bloat territory.
- db48x 5y ago
  I don’t completely disagree with you, but every interview question has some artificiality. This interview question cleverly moves that artificiality out of the way, so that it isn’t part of the programming task. Since the point of the question is to test the candidate’s programming ability, that is a great boon. And it’s not like you can’t chime in after you’ve done the task to say that you don’t think that the multiplication operation is a good fit for memcached’s role. You might even get extra points for that; nobody wants to hire someone who can’t think intelligently about the tasks they are given. Just don’t blow it by refusing to do the programming task on those grounds; sometimes we don’t get the opportunity to choose every task we work on. Sometimes they turn out to be a bad idea, and yet it is still to our benefit to complete those tasks to the best of our ability.
k2xl 5y ago
Somewhat got confused by the question given part 1. I didnt think we were asked to modify the source but just write a middle layer. Considering you want to stay in sync with the main codebase for memcache with updates and what not, my (potentially incorrect) solution was somewhat hacky 1. Client saves value in local memory. Then issues a command to update the number to a string version of the number (maybe with a flag telling any client to parse it as integer for get). Assuming setting a value is “safe”, this would cause other concurrent clients to error if they are trying to incr (according to example from part 1) 2. Client that did this update now multiplies that local variable. 3. Client updates value to final answer. If a failure/crash happened you could still sort of retrieve the original value since it is a string version of itself. Anyway it is hacky and sort of messes up if someone attempts to get the variable while it is a string. So I probably would fail this question.
- IggleSniggle 5y ago
  I went down a very similar train of thought, making the same assumption. I think this makes us “type 1” engineers: wrongly assume that we are not allowed to break the existing code contract, and attempt to use it to implement a new solution “over” the old. Extending without modifying. I took that as a kind of lesson in and of itself. I’ve certainly had to face code monstrosities and cut through multiple layers to discover the simple rewrite before. If somebody had modified instead of extended sooner, if someone assumed the existing solutions were not so sacred, maybe a monstrosity could have been avoided in the first place.
  - k2xl 5y ago
    On the flip side I've also experienced the opposite. Someone modifies existing code from an open source library and then we aren't able to safely upgrade it because of tons of merge conflicts with the modifications that were added.
abhishekjha 5y ago
Why is the type1 programmer being looked down upon? I mean I am sure that I want to make sure how the lock implementation is. I also want to know if it is modular enough to be called somewhere or is it baked in the add step. I sure do want to multiple times read the add implementation and make sure there isn't a magic somewhere. And finally this problem looks a lot like can you see a pattern!! I mean sure but a lot of people who might go to interview with standards of competitive programming might take the whole 3 hours because there's a lot to rule out. I guess this is a fair question with a proper IDE and grep tools. The timing does look outrageous but if you are interviewing at a company which builds databases as a product, you better know how basic operations are provided. Redis source code is fun to read as well.
- ankrgyl 5y ago
  Fun fact: we tried it out with Redis, but the question worked out to be too easy because of how nicely the codebase is laid out.
  - CamperBob2 5y ago
    Now that's a compliment. "We couldn't use your code as a basis for interviews because it was too obvious how it worked and how to modify it." Definitely a condition worth striving for.
  - deleted 5y ago
    [deleted]
- ww520 5y ago
  Yes, type1 is actually better than type2. Type2 is hyper-focus and is just blindly copying and pasting the incr portion as multi. Type1 actually is curious about the code base and wants to explore it a bit, to see how the locking is working and validate its correctness, to explore different approaches to see if incr can be used to implement multi. Type1 will end up learning a lot more about the codebase and do well in the long term.
  - californical 5y ago
icedchai 5y ago
So I hacked it out in under a half hour. Basically, I wound up cloning some code paths instead of modifying "add_delta". Not sure if I'd pass or fail... but it is a good question to see if someone is comfortable navigating a new code base.
- jldugger 5y ago
  > Basically, I wound up cloning some code paths instead of modifying "add_delta" This is more or less covered in part 2, and is indeed considered the path to passing.
soneca 5y ago
In the end of part two there is this: > ”Basically, this question is to software engineers as FizzBuzz is to programmers.” What’s the implied difference between software engineer and programmer here? I don’t get it.
- macintux 5y ago
  In this context I'd say: writing a piece of software (programming) vs modifying an existing, large project in a clean and correct way (engineering). I can write a pretty good standalone tool to solve a problem, but throw me into an unfamiliar code base and ask me to make a significant change that meets all the necessary style/correctness criteria is a whole 'nother level, at least in an hour interview.
dgs_sgd 5y ago
i received this question when i interviewed with them back in 2019. It’s definitely up there for my most enjoyed interview experienced
davidst 5y ago
This is a great interview question I don't want to be 'that guy' by picking at what may be a detail but there is another response to look for from a candidate. We should think very carefully about adding a multiplication command because it introduces a failure mode that may be unanticipated by the client. Code that previously worked could begin to fail after this command goes into use. Specifically, if the client needs to revert a series of operations on integers, and if the operations are transitive, there is no need for a mechanism to ensure they occur in any particular order (the usual caveat about working within the limits of precision applies.) This holds true for addition and for multiplication, in isolation, but is not true if they are combined. Change the order and the end result will change. Adding multiplication puts a burden on the client to understand this risk and be explicit in the ordering. Some people may argue that the client should already understand it and they have a point. We can't defend against every possible misunderstanding. I think there is a good discussion to be had on this question and if the candidate were to go there, it would be a favorable sign of experience on their part.
- jameshart 5y ago
  This is a REALLY SOLID response to the question, and I'd hope it would get some strong 'hire' points. But, to be fair, there's already technically this issue in the memcached API as described in the post, in that it supports both append and add, and "append 0" (on a value that add could also act upon) is effectively the same thing as "mult 10". If a field contains "1" and successive "add 1" and "append 0" commands come in, depending on what order they arrive in the result could be 20 or 11. So I think the interviewer would be justified in saying 'yeah, let's assume we've evaluated that risk and we plan on adding some really thorough documentation warning people about that risk, so can you just go ahead and try and implement it?' But absolutely, this was the thought that came into my mind when reading the spec too. Not enough developers think about APIs in terms of compositional, algebraic terms, and being able to see that adding 'multiply' and 'add' together at the same precedence in an API might cause trouble is a really valuable skill.
  - akhmatova 5y ago
    This is a REALLY SOLID response to the question, and I'd hope it would get some strong 'hire' points. I agree -- but unfortunately the interview question doesn't filter for this level of thinking. If anything, it filters for the exact opposite: your ability and willingness to shove a random new feature into the codebase in 3 hours (or GTFO) -- stability and other consequences be damned.
mihaitodor 5y ago
To everyone praising how good this challenge is, calm down. There's nothing great about making somebody sweat for 3 hours during an interview. It's even worse if "someone got so frustrated that they threw their laptop on the floor" like the top comment claims. Yes, it would be great to avoid hiring people who have a bad temper and tend to throw tantrums when nervous, but, then again, it's beyond me why anyone would need to prove they can cope with such pressure without breaking down. We're talking about a desk job here, which requires writing good quality code according to some requirements during normal working hours, not disarming a bomb with a timer attached to it.
- theamk 5y ago
  If "3 hours" is "such pressure" for you, then maybe some teams are not a good fit for you? For example, I have broken the build in the past - and this prevents everyone else on your team from merging PRs. In this case the expectation was that I fix it "soon" - once I finish my current meeting/lunch/conversation. If someone is prone to breaking down from pressure given 3 hour deadline, they would have a breakdown right there.
  - fzeroracer 5y ago
    Generally speaking if your build process is so fragile that one person creating a breaking build results in a chain of failures and a pressure crunch then that's a work environment problem. I would consider that sort of thing completely unreasonable and consider finding a new job immediately. If you find yourself in that sort of thing frequently then perhaps you should reconsider normalizing it and instead push for actual fixes. You look at the issue and say it's a personal issue. I look at it and see a dysfunctional development environment issue. Now this doesn't always hold true and sometimes devs have to be held to higher pressure environments (such as having on-call duties): but those kind of things are usually better compensated for and called out as part of the job duties.
    - vimwizard 5y ago
      In an ideal world, no one would ever have to work under pressure, and while some people in this industry don't, many (I would bet a majority) of people do or have in the past. This seems especially true for startups trying to break into a market with competition. Either you have been lucky with the places you've worked at, have sifted through tens of shops to find a good one, or have very little experience in the industry. I really do hope that none of my assumptions are true and that one day I can find a functional, low-pressure team to work with.
loxias 5y ago
I miss questions like this. Things that are closer to actually measuring "so, can you code?" and not if you can do CS puzzles, or how well you can shoehorn corporate "values" into answers. I "passed the shit out of this question" and have failed my last several interviews... :( I miss 2010.
CatAtHeart 5y ago
For anyone else who was initially confused like I was: On first look, I thought the solution had to involve exploiting commands over telnet to inject a mult command. It took me until looking at part 2 to realize it involved actually modifying the source code (which makes sense in retrospect). If you are looking to actually try and solve it, keep that in mind and don't look at part 2 :)
sanderjd 5y ago
Yep, I love this kind of thing and it's waaay more similar to my actual job than the little self-contained problem on a "whiteboard" style. The similar but different style question that I got (at Stripe, fwiw) in my recent job search that I also really loved was: Here's a failing test against a complex codebase; figure out what's wrong and then fix it. This flexes similar muscles I think: figure out enough of the flow of a complex codebase to understand where the problem manifests, and how to modify things to fix it without breaking other stuff. The really nice thing is: if all the tests pass at the end, you've done it. Then, time allowing, you can go work on improving the fix; making it more consistent with the architecture, etc.
cafxx 5y ago
> Via its incr and decr commands, memcached provides a built-in way to atomically add k to a number. But it doesn’t provide other arithmetic operations; in particular, there is no “atomic multiply by” operation. Your programming challenge: Add a mult command to memcached. Protip for people in junior roles: if you're interviewing for a more senior position, the correct answer would *not* be to jump in straight and waste 3 hours implementing something that then you have to maintain forever. The best answer, from an *engineering*[1] perspective, would instead be noticing (or knowing) that memcached supports a CAS command (https://github.com/memcached/memcached/wiki/Commands#cas https://github.com/memcached/memcached/wiki/Commands#cas) that would allow to implement equivalent functionality without changes in memcached, and try to confirm why that would not be a viable solution. If, and only if, CAS is confirmed not to be a viable solution (e.g. excessive contention, unacceptable pX latency, ...) then you should spend the 3 hours (+ all the maintenance effort required until the end of life for that solution). The risk, in case you jump acritically on the solution, is to show you did not spend any time trying to actually understand the problem and you did not consider the pros and cons of viable alternatives (and this is a problem both during an interview, as well and especially when you're doing actual work), something that is normally frowned upon in senior roles. Obviously, being an interview, it's all play pretend so you'll eventually have to complete the coding exercise, but if I was interviewing you for a senior position and you skipped the part above it would be a pretty big red flag (same in the unlikely case I were to ask to implement some functionality that is commonly found in the standard library of most programming languages: huge red flag if you don't ask why the functionality provided by the standard library is not a viable solution). [1]: w.r.t. software engineering being "programming integrated over time" (https://adamj.eu/tech/2021/11/03/software-engineering-is-programming-integrated-over-time/ https://adamj.eu/tech/2021/11/03/software-engineering-is-pro...)
- theamk 5y ago
  During interview, we are not modelling the whole process -- no one would tell a senior engineer just a single sentence "please implement mult command" -- there will be a longer preceding story: maybe the latency has to be very low, or it is a request from a stubborn customer, or you are profiling optimization... So for the reasons of time, we assume that the previous steps have been done, and it was decided that the mult command is the way to go. If I were interviewer and a candidate would mention CAS command, I'd compliment them for their memcached knowledge, and tell that the it is not a viable solution. And this would not affect my evaluation of this stage one way or another.
JohnTHaller 5y ago
I really like this sort of question since it allows a candidate to work their own way while showing their process and lets the company see if the applicant can handle the sort of thing that will be a good part of their job, at least for the first year or so. The popularity of white boards and writing binary tree traversal pseudo code is frustrating to me. I haven't done an interview where that's a part of it because the thought of it freaks my brain out a bit. Which seems odd since I regularly perform improv comedy and am considering teaching it in the near future.
hoosieree 5y ago
"Chesterton's Fence" is pragmatic for getting things done quickly, but I hope people actually come back later to decide if the fence should stay or go. Leaving fences without questioning why they exist builds tech debt.
muh_gradle 5y ago
I haven't checked the spoiler. The brute force method is probably to just call incr multiple times. I wonder if there's anything to do with the caching function to get better performance though. Will need to think about it more.
- pphysch 5y ago
  What if two clients call `mult` simultaneously, with big values?
deleted 5y ago
[deleted]
elisbce 5y ago
This is far from a great engineering interview question, let alone the best. 1. This question is heavily biased towards specific software experience and familiarity with the domain and tooling, while at the same time too shallow in terms of algorithmic and logical depth. It is like asking how to parse a CSV file in Fortran. The difficult part is not the problem per se, it is picking up an unfamiliar language and tooling on the spot. For example, someone from Windows background who wasn't familiar with the memcached / Linux environment and tooling / Messy open-source codebase might find it difficult to adapt within the time limit. That doesn't automatically make them bad coders. It is like requiring Emacs and DVORAK keyboards for coding at interviews -- In theory it's all the same, but it's cumbersome and slows you down if you are not familiar. Challenges like this during interviews can lead to high false negatives. People might get stuck on small side things and not working on the main thing enough. 2. This question takes too much time to set up and complete. Interviewers don't have three hours to wait. Interviewees can probably only do one such problem during an onsite and nothing else. Are you going to make a judgement and decision solely based on the performance of this one challenge? I don't think that's a good idea. Also the interviewer needs to explain everything. For a three-hour challenge, this is at least 15 minutes time doing nothing but setting the problem up. 3. This challenge is actually not that great to identify senior / great coders while mediocre coders with specific experience can pass easily.
- pphysch 5y ago
  The question appears to be highly relevant to the position. I would hire the candidate that completes the feature request over the 10x Programmer that cannot.
- db48x 5y ago
  > 1. This question is heavily biased towards specific software experience and familiarity with the domain and tooling Heavily biased towards the specific software experience and tooling _in use by the company who is hiring_. That seems like a win to me. If you are a Windows shop, go find an open–source C# or VB.net project to use instead. > 2. This question takes too much time to set up and complete. You might have missed the part where the candidate is given a VM to ssh into, where all the build tools and dependencies are already installed. Also, the author of the blog misremembered; they were given only one hour instead of three for this problem. There were probably other interviews, and maybe lunch, afterwards. > 3. This challenge is actually not that great to identify senior / great coders That’s true. As stated in the article, this question was intended to filter out low performers. Other interview questions would be used to judge other aspects of the candidates.
vesche 5y ago
Thanks for sharing this. It took me less than an hour to complete and I thought it was pretty fun. These sort of interview challenges that involve rapidly diving into a foreign codebase are really great, but it has to be done correctly. I once frustratingly & embarrassingly failed a live coding interview where I was asked to fix some sort of linked list issue in a fairly large C code base that wouldn't compile. I much prefer a challenge that involves implementing a new feature in a working code base (like this memcached question) as it's closer to reality and will also provide a better experience for the candidate.
sydthrowaway 5y ago
jwmoz 5y ago
Although I don't know C I really like this example as it's pretty much what happens when I start a new place as a Python dev. E.g. Hey, X, we need you to change the dates on this screen to show the timezone. - Queue me then having to drill down through the codebase to find and then fix the problem etc. Only complaint would be the amount of time it may require to do in an interview.
WC3w6pXxgGd 5y ago
hota_mazi 5y ago
The question actually appears at the end of this unnecessarily lengthy and verbose article. Here it is: > Add a mult command to memcached.
gigatexal 5y ago
I really agree with the article author. The skill of being able to get into a codebase and add an iterative feature like a multiply operation when there’s an addition is huge. It’s something I failed to do when I was a software engineer. I had to study and understand every part of the codebase before I did anything. I was paralayzed. But that understanding would have come over time if I just added features incrementally like I was asked to. I didn’t last very long at that position. Luckily I found data engineering and I much prefer that anyway.
wodenokoto 5y ago
What makes this a great question? I feel the article is missing that discussion.
trinovantes 5y ago
I think I've been reading too many interview questions lately that I originally thought this was a trick question that required implementing mult with only the provided incr I like that this was a more of an engineering question about diving into an unfamiliar codebase than a gotcha math/algorithm question
daniel-cussen 5y ago
I'm grateful for all the engineering interview questions I've gotten in person, and not for any of the ones that were homework. Dude I promised myself as a child I would never do homework after my studies, that promise is sacred, your company is not. So I never got caught out (EDIT: meaning to be clear caught out unable to solve the problem), which you would expect because in fact engineering interviews are actually no use for interviewing engineers, but in fact are great for interviewing for algorithmists, and you'd expect me to do well because I say I say I'm an algorithmist, and sure enough I did solve all of them. Nobody caught me with my mouth open, I've solved every single interview question without timing out, despite (actually because of) not getting a CS degree from Stanford (which I did attend). No advance prep, although that's bullshit, I spent every second I could on advance prep incidentally by working on algorithms. But no leetcode. I did look at it once, at one problem, and it sucks, I saw a problem regarding getting the popcount of all the numbers in a range and tons of people got the top 100 points. That's stupid, I knew a superior answer that took morally no time, and would I get more than 100 points for it? Doubtful. There's a very low cap. Not truly elitist. There can't be a max score. There is one problem that I couldn't get, and that was "how do you find a single element in a list that is not present in an otherwise identical other list?" Like tons of problems, the answer is to sort the lists and take a binary-search analog. That's the answer, but you need faster-than-state-of-the-art sorting. The thing I was supposed to say that they wanted me to say, not to be confused with "the answer", was you had to add the elements and then subtract. I instantly replied, "no, just xor them, less energy." Which was a satisfying response, and marked as a pass. I didn't get the job anyway on a gut check. Unfortunately I did terribly at the very few jobs at which I did get hired. Because I could only get hired at doomed companies, I had between one and three weeks to prove myself, do or die. Died every time. One place, Unholster, gave me one day to set up my environment, seven work days to prove myself, and after a coworker spent an hour insisting I be fired and the boss saw the stories (epics? The point system thing). Ended up saying I objectively amounted to negative-one-half-a-person, concluded "With your curriculum, nobody would give a níspero for you." Níspero is a plentiful fruit that is sweet and is nourishing, but come to think nobody eats because nobody values it. And this is NOT slander, well first off it's true and that's all you need legally, but secondly (morally) that's just what a doomed company is like, I'm actually not being negative. They had to fire like twenty people and had two employees left, and that's public information, tells you everything. Like basically all Chilean companies, they had bad Chilean debt, the Chilean bank leaves a tiny glimmer of hope, that's it, they have to grind through meat until they find the savants. Tons of companies like this, in fact I have a relatively high opinion of them considering. Really liked Unholster. They didn't cheat me out of money or time, so above average. And even when they were for sure getting nothing more out of the relationship, the boss agreed to just talking about why my career was fucked, for like fifty minutes. "I would sign [something for you] saying there's better guys at algorithms than you." "Sign what? Sign what, exactly?" They were hiring another guy, my replacement, I met him, in the time I worked there and he also thought this would be his chance. And it was, I don't know how it went for him, but the point is it is a chance, only it's very very narrow. They wanted to be cool, too, read Hacker News. Smart guys, really good to talk to in the lunch breaks, I liked all of them. They were just living in a shark tank. And truth be told, I wasn't that good at the job, sure if I got three months like everyone's supposed to it'd be easy, but that's really easy, I told them I'd be much better than that. I wanted to be. I wasn't good at the job. Just at the interviews.
croshan 5y ago
I love it--realistic example of working with the production codebase, and a chance for the interviewee to see the actual code quality + practices they'd be dealing with.
anamax 5y ago
If you're only going to use multiplication going forward (no adds), I wonder whether interpreting the output as an exponent will pass enough tests.
dave333 5y ago
Any interview question that takes 3 hours is bad by definition unless of course the 3 hours is a red herring and the interviewees who pass are the ones who do it in much less time.
- creatonez 5y ago
  The actual author of the question said elsewhere in this thread that it was actually administered with a 1 hour limit.
gorgoiler 5y ago
Type -1 candidates quit on the spot when they discover that memcached has, in the years since they last worked with it, been bastardized to do things other they GET and SET! The beauty of memcached in the 2000s was that it felt extremely opinionated about features — that is to say, it didn’t have any. That bool incr argument in the source code feels like a hint from the authors. The arithmetic feature was meant to do two things and two things only, represented by I/D = +/- = true/false. Add (pardon the pun) any more operations and here be dragons. In particular once you add MULT, some clown is going to ask for DIV, and so are you now going to convert ints to floats or force integer division on everyone? Let’s place bets on how long until we get a “bug” report that 12,999,999 DIV 10 should be 13 not 12. Maybe I’m being unfair. The integer increment thing in memcached seems useful. The useful part of it is the fact that it increments atomically, not that it does the heavy lifting of adding one for you :P If N people increment it you get a number that is N bigger. “Add 1” is also a pretty easy thing to get consistent if you have two people contending for a lock. If it were a strongly typed language it would have a type like Counter (not Number) and it makes no sense to multiply a counter. The hint is also there in the INCR and DECR commands. They are probably not called ADD and SUBTRACT for a reason. Feature rejected! Now let’s go code up MULT anyway :)
- xxs 5y ago
  I'd say MUL(T) is absolutely useless in any practical manner. I'd take some form of CAS if I want atomic operations. MUL has to deal with overflow a lot more than inc/dec. Not knowing the value prior MUL, taking any branches is another issue, CAS would solve in that case as well. Inc/dec are useful for a counter implementation (although I see no reason to have a designated dec) - even though counters should not be implemented in such a manner, e.g. each writer should have its own counter, and reader should sum them up when the value is needed. No contention, scalable. In that regard I'd consider the interview question "weak", lacking a deeper understanding - not just show us you can code.
  - gorgoiler 5y ago
    Oh it’s a great question. Code this solution without having to worry about why. If only the job was actually like that. The why is 10x the how, IRL, once you get beyond IC0.5.
agucova 5y ago
While it took me under an hour, I still wonder whether my implementation would have passed an interview. I generalized do_add_delta to three operations (incr, decr, mult) and otherwise copied code paths and extended the binary protocol.
- db48x 5y ago
  That sounds like a perfect solution to me.
amelius 5y ago
Look for the add code, copy it, replace + by *.
_andrei_ 5y ago
This is a great test (for bright folks who don't mind "doing free work at interviews"), because it directly answers the "Do you have what it takes?" question. At my first job interview, they handed me the printed documentation for a programming language they invented (2-3 pages), and a problem. I didn't have any work experience with any technology they used, but managed to solve the test, and that was enough to get my first job, and drop out of education.
throwawayHN378 5y ago
First thing I would do is download and build. Then find he incr method/interfaces/etc. Then rename all of those to mult and try and rebuild and see if mult works identically to incr. Then start working on modifying incrementing toy multiplying.
lkxijlewlf 5y ago
I would love an interview question like this. For one thing, if I was unfamiliar with the domain, it would spark a conversation that would help me decide if I wanted to work with that team or not.
minroot 5y ago
Well, if this is how interview works, I need to look for a job.
JustLurking2022 5y ago
Gonna have to disagree... waaaaaay too much setup, and a large advantage to people who are familiar with 1 specific technology.
tomtung 5y ago
My previous employer used something very similar: either adding small features or fixing known bugs of a past build of some open source software in 1-1.5hrs. We prepared one such interview question for each supported language (C++, Python, and Java), and, to help calibration, each question consisted of a series of requirements that required progressively more complex changes. Both interviewees and interviewers seemed to like these questions. I do hope this approach gets more adoption in the industry. It takes more time to prepare, but the high-quality signals it provides are worth the effort imo. Such questions are also harder to leak compared to typical whiteboard coding questions and thus more reusable too.
xaviercoll 5y ago
Nice...

893 points by db48x 5y ago

deleted 5y ago

[deleted]

db48x 5y ago

I have to agree; I think that question is better than any I have ever gotten in an interview.

camgunz 5y ago

Yeah I do love this. I think it still has some performance element to it, but I would relish it. Definitely good as an option I think.

blatherard 5y ago

Is there a filter that automatically removed "The best" from the submission title?

dang 5y ago
Yes. We glued it back.
- postingposts 5y ago
- notreallyserio 5y ago
  Thanks dang, you're
- db48x 5y ago
  That‘s actually pretty hilarious. I submitted this, made my comment, and then immediately went to make a sandwich. I never noticed the truncated title; I didn’t even look. Thank you!
kaashif 5y ago
This truly is one of the engineering questions of all time, in my opinion.

dataflow 5y ago

For a junior-ish software engineer level (say, 1-3 years out of college) who claims to have experience in C, this seems like a fine question, but for a senior software engineer (which I think is what he was interviewing for) it doesn't seem all that great, except maybe as a first-round filter [1]? Not that it's terrible, and I guess if the candidate fails it then it certainly tells you a lot—but if they complete it successfully then it doesn't really seem to tell you much about their software engineering skills or depth of understanding. It's better than some silly challenges companies give, but there's so much room to ask more probing questions or provide more challenging tasks to help you gauge where the upper limits of a candidate's abilities are, instead of just the lower limits. [1] Edit: Looks like it indeed was a filter question, as I suspected: https://news.ycombinator.com/item?id=31065819 https://news.ycombinator.com/item?id=31065819

elromulous 5y ago
This is an issue with many coding problems. I'm not sure if there is a term for this is detection / estimation theory, but it provides what I'm going to call asymmetric information. In this particular case, negatives are true, but all positives must be presumed false. If you make a problem too hard, you're faced with asymmetry in the other direction.
- dataflow 5y ago
  I think the usual solution to that is to follow it up with "what if you wanted to {introduce new/harder requirement}?" and ask them to explain & make as much of the changes as time allows. To give an example, if you were to initially ask someone to implement depth-first traversal recursively, you might follow that up with asking them why you might end up needing to do that iteratively, then you might have them do that iteratively for pre-order depth-first traversal (easier), then follow that up with post-order iterative traversal (harder), then you might ask whether/why it might perform poorly (e.g., re-visited nodes blowing up exponentially), then ask them how they might mitigate it (keeping track of visited nodes), then ask them how they might mitigate the space requirement (lots of possibilities), then ask them how they might handle parallelizing it for some particular problem... I can keep going, but the point is you can keep something like this going until they start to struggle, and that at least tells you the upper limits of their abilities on at least that axis. And you can pace it by e.g. skipping stages as you get a better sense of their skills.

kashav 5y ago

Got the same question for a new grad position around 2019. Only had an hour that time. Great question.

notreallyserio 5y ago

I'd find this question frustrating because it feels like a trap. I was looking for the trick -- was I missing something in the question that made the obvious approach untenable? No, the answer is simple, just add a way to multiply numbers with the multiply operator.

melony 5y ago
And correctly dealing with arithmetic overflow.
Syonyk 5y ago
It's "Advanced FizzBuzz." "Can you understand a complex code base, figure out how to zoom in on the section of interest, and add a function?" Far, far too many people interviewing for coding positions can't do that. Though I question how many could whip out FizzBuzz in 5 minutes and would then fail the more complex version. At some level, either you can code, or you can't. And, importantly, at some point in your career (if you go down the standard management track), you will typically lose the ability to code. If you've not done it in 5 years, you probably can't do it, in a practical sense. Not that you can't re-learn, but you can't just jump into a coding interview and ace it, either. If you find yourself at a point in your career where coding interviews are a thing, "having a side hobby project that requires coding" is a very useful way to be really quite good at coding interviews. Plus, if it's your hobby project, there's no "Hrm... can I talk about that?" sort of problems when you're asked about a time you X'd with Y. You can talk about your personal hobby project absolutely as much, and in depth, as you want.
- mikeryan 5y ago
  This isn’t an “Advanced FizzBuzz” if it takes three hours or even one hour. The whole point of a FizzBuzz question is that it’s a fast, like 2-3 minutes fast, heuristic for separating the wheat from the chaff and this isn’t that at all. At some point it’s no longer a FizzBuzz and just a coding challenge. Even the author’s answer reflects that (though he calls it a FizzBuzz) I don’t know a thing about memcached internals but, as presented here, with the potential encoding and concurrency issues means that there’s likely levels to the complexity here that will likely take someone quite a bit of time to resolve if they’re not familiar with memcached internals. But again memcached and even C++ isn’t my thing. ¯\_(ツ)_/¯

detinho 5y ago

What kind of skill does this type of test measures? I think the main ones are: * How much about the language the candidate knows. Not the control flow and data types, but how to navigate and where to look for information. And that leads to bellow * how quickly one can understand a codebase (at least part of it) and start delivering What else? If that's one of those "think out loud" tests it's possible to get a clue of how the candidate thinks, but it's not specific of this test.

teraflop 5y ago
I'd say your second bullet point encompasses a number of sub-skills, such as: * Understanding common coding patterns, and inferring design intent from them * Quickly prioritizing "critical" vs. "nice-to-have" components of a desired feature (and reprioritizing as you learn more about the implementation constraints) * Being able to take a partially-complete or slightly buggy implementation, and quickly identify what's wrong with it (e.g. by interpreting compiler errors or using a debugger) * Gaining confidence in the correctness of code by spending a reasonable amount of time on testing, without going overboard
avip 5y ago
The skill of "getting something done without breaking anything" which is basically our profession.
eyelidlessness 5y ago
If I encountered this test, here’s what I’d think (or hope) is being evaluated: - Do you understand the problem, or seek better understanding if not? (Obviously the problem here isn’t extrapolating arithmetic, but identifying the importance of an atomic operation.) - Do you recognize the value of the change, or question it if it feels non-obvious? - How do you think about approaching an unfamiliar codebase/code path? - How do you think about the challenges you encountered while working on the problem? - What else felt important to you that I’m not looking for? All of these questions provide a lot more information than “can you write code?” Especially meaningful for evaluating actual fit is reactions to walking into pre-existing code. In fact I think this has been, at least subconsciously, my best heuristic for assessing colleagues’ skill maturity. “Juniors” will stare at a problem or ask a lot of easily answered questions; “mid level” devs will bang their heads trying to answer without asking; “senior” devs will have a good intuition for which challenges are discoverable and which are best solved by asking a human or a google or whatever, or at least for recognizing the distinction after some time exploring the problem.

ankrgyl 5y ago

Author of the interview question here. I came up with the idea after working at the company for a couple months and realizing that the skill of "diving into an unfamiliar area of the code and quickly figuring it out" was very important for us. Database codebases are large and complex -- so much so that almost every new feature you work on for the first year or so feels a lot like doing the question. Over the years of evaluating with it, we learned a lot about its quirks (both false positives and false negatives). At one point, someone got so frustrated that they threw their laptop on the floor. Happy to answer any questions about it!

dataflow 5y ago
I definitely appreciate the intent of it, and think measuring along that axis is incredibly important (and rare). But as someone who continually struggles more to dive into some codebases than my peers, I can say that this task is far too easy and wouldn't help you gauge my ability to do that at all. Something that requires implementing a new feature that isn't a near-mirror of an existing one (ideally one that requires a bit of thought at the design level) would probably be a much better measure, but I don't know what that might be for something like memcached.
- deleted 5y ago
  [deleted]
- ankrgyl 5y ago
  That's a great point! We didn't actually think of this question as much of a "measure" (and I think the blog post alludes to this as well with a "Fizz Buzz" analogy). We thought of it more like a filter — if you couldn't pass, then you _may_ not be adept at navigating through a database-y codebase at the pace that was necessary (at that time) for the work we did. We had other questions (which I won't spoil) that tested more complex system-design skills, without having to implement them within an existing codebase. We felt this provided clear independent signals (ability to "hack" within a codebase and ability to reason through complex system design) vs. conflating the two. As the team grew, and we needed folks who were much better at one or the other, this helped us round out the team with folks who excelled at either skill.

avip 5y ago

I salute this dude's commitment to Chesterton's fence. If only my teammates had this mindset...

db48x 5y ago
Yea, any candidate who works Chesterton’s fence into their answer somehow gets bonus points from me.

victorronin 5y ago

Got mixed feeling - As people pointed out, it's advanced FizzBuzz (really advanced) - I can easily see a lot of people being unable to solve it, not because they can't do it, but because of a "performance anxiety". The bar here is quite high, and the result is binary (does it work or not). - I like that it is quite close to real life (that people have to read code, figure it out, write code). On another hand, again, what is not real, that you are parachuted into a unknown (quite big) codebase and expect to add a small feature in 3 hours.

theamk 5y ago
The result is not binary - you can have it fully working, or forget binary protocol, or syntax accepted but multiplication not happening; or correct code changes done, but code does not build... That is one of advantages of longer questions (compared to 5 minute one): even if task is not fully done, there are plenty of other signals.
andrewflnr 5y ago
Given that they're specifically hiring for high-performance database work in C++, I think it's a appropriate to have a pretty dang high bar. For less-intense work, I would definitely want to formulate a less intense version of this style of problem.

kcarter80 5y ago

Why couldn't a potential hire be given 5 minutes to think about a solution and then be asked what their plan is? What benefit is added by 36x-ing the time allowed that justifies putting the candidate through such pain?

ed25519FUUU 5y ago
This is the right way to handle this problem. Everything after 5-10 minutes waste both your time and the interviewer’s time. There’s nothing you can discover that’s important after walking through their plan other than to figure out if they are familiar with code bases like the one you’re using.
- db48x 5y ago
  I disagree with that. If you’re hiring a programmer, surely you want to seem them write some code. This question has the added benefit that they don’t have to write a whole program from scratch, they have to deal with a real–world program instead of a toy program created specifically for interview purposes, and they have to demonstrate that they can read and understand other people’s code. The latter seems really important to me, as apparently it was to the author of the question, because we spend so much of our time improving code that has already been written instead of writing completely new programs. Of course, I am especially good at these software–archaeology skills, so I suppose I could be biased.
- hn_throwaway_99 5y ago
  Couldn't disagree stronger with this. In fact, I love this problem, in that it gets rid of most of the complaints about engineering tech interviews: (1) it's "real world" (for the job in question), (2) can be reasonably done in the time alotted, (3) involves a task that is most common for developers (modifying existing code), (4) shouldn't require extra preparation. I have seen many times where a user can actually explain the solution to a problem, but the translation from algorithm-to-code is extremely slow, or not done correctly at all. There is real skill in being able to output code quickly, even if you already know the English-language description.
- 5y ago

a-dub 5y ago

i could do this one in my sleep. wish more were like it...

nine_k 5y ago

Just for the record. Last time I was given to code something up within 3 hours, my task was to write a Minesweeper clone (pretty graphics optional). I ended up with a workable console-mode implementation, using Python.

karlakush 5y ago

yokoprime 5y ago

There is a reason memcached does not have mult. One could argue even incr and decr should be handled by clients. Starting to add more arithmetic support is scope creep. Why stop there, why not add sqr or bitwise ops, maybe a text based adventure game? Just blindly implementing this feature without any questions should be a huge red flag

pubby 5y ago
You could say the same thing about web apps not needing fizzbuzz, but that wouldn't get you hired. As an aside, I think atomic incr/decr are useful. I don't see how you would implement them client side without some more complicated mechanism (like CAS).
- macintux 5y ago
  Absolutely. At Basho, the company learned over time that developers needed help using a distributed key/value store correctly, and started building CRDTs into the server.
- yokoprime 5y ago
  > You could say the same thing about web apps not needing fizzbuzz, but that wouldn't get you hired. fizzbuzz is not great, but its also implemented as a stand alone webapp (or console app), not added to an existing codebase. im not against incr/decr, my point is moving beyond these simple operators is pushing the product into feature bloat territory.
db48x 5y ago
I don’t completely disagree with you, but every interview question has some artificiality. This interview question cleverly moves that artificiality out of the way, so that it isn’t part of the programming task. Since the point of the question is to test the candidate’s programming ability, that is a great boon. And it’s not like you can’t chime in after you’ve done the task to say that you don’t think that the multiplication operation is a good fit for memcached’s role. You might even get extra points for that; nobody wants to hire someone who can’t think intelligently about the tasks they are given. Just don’t blow it by refusing to do the programming task on those grounds; sometimes we don’t get the opportunity to choose every task we work on. Sometimes they turn out to be a bad idea, and yet it is still to our benefit to complete those tasks to the best of our ability.

k2xl 5y ago

Somewhat got confused by the question given part 1. I didnt think we were asked to modify the source but just write a middle layer. Considering you want to stay in sync with the main codebase for memcache with updates and what not, my (potentially incorrect) solution was somewhat hacky 1. Client saves value in local memory. Then issues a command to update the number to a string version of the number (maybe with a flag telling any client to parse it as integer for get). Assuming setting a value is “safe”, this would cause other concurrent clients to error if they are trying to incr (according to example from part 1) 2. Client that did this update now multiplies that local variable. 3. Client updates value to final answer. If a failure/crash happened you could still sort of retrieve the original value since it is a string version of itself. Anyway it is hacky and sort of messes up if someone attempts to get the variable while it is a string. So I probably would fail this question.

IggleSniggle 5y ago
I went down a very similar train of thought, making the same assumption. I think this makes us “type 1” engineers: wrongly assume that we are not allowed to break the existing code contract, and attempt to use it to implement a new solution “over” the old. Extending without modifying. I took that as a kind of lesson in and of itself. I’ve certainly had to face code monstrosities and cut through multiple layers to discover the simple rewrite before. If somebody had modified instead of extended sooner, if someone assumed the existing solutions were not so sacred, maybe a monstrosity could have been avoided in the first place.
- k2xl 5y ago
  On the flip side I've also experienced the opposite. Someone modifies existing code from an open source library and then we aren't able to safely upgrade it because of tons of merge conflicts with the modifications that were added.

abhishekjha 5y ago

Why is the type1 programmer being looked down upon? I mean I am sure that I want to make sure how the lock implementation is. I also want to know if it is modular enough to be called somewhere or is it baked in the add step. I sure do want to multiple times read the add implementation and make sure there isn't a magic somewhere. And finally this problem looks a lot like can you see a pattern!! I mean sure but a lot of people who might go to interview with standards of competitive programming might take the whole 3 hours because there's a lot to rule out. I guess this is a fair question with a proper IDE and grep tools. The timing does look outrageous but if you are interviewing at a company which builds databases as a product, you better know how basic operations are provided. Redis source code is fun to read as well.

ankrgyl 5y ago
Fun fact: we tried it out with Redis, but the question worked out to be too easy because of how nicely the codebase is laid out.
- CamperBob2 5y ago
  Now that's a compliment. "We couldn't use your code as a basis for interviews because it was too obvious how it worked and how to modify it." Definitely a condition worth striving for.
- deleted 5y ago
  [deleted]
ww520 5y ago
Yes, type1 is actually better than type2. Type2 is hyper-focus and is just blindly copying and pasting the incr portion as multi. Type1 actually is curious about the code base and wants to explore it a bit, to see how the locking is working and validate its correctness, to explore different approaches to see if incr can be used to implement multi. Type1 will end up learning a lot more about the codebase and do well in the long term.
- californical 5y ago

icedchai 5y ago

So I hacked it out in under a half hour. Basically, I wound up cloning some code paths instead of modifying "add_delta". Not sure if I'd pass or fail... but it is a good question to see if someone is comfortable navigating a new code base.

jldugger 5y ago
> Basically, I wound up cloning some code paths instead of modifying "add_delta" This is more or less covered in part 2, and is indeed considered the path to passing.

soneca 5y ago

In the end of part two there is this: > ”Basically, this question is to software engineers as FizzBuzz is to programmers.” What’s the implied difference between software engineer and programmer here? I don’t get it.

macintux 5y ago
In this context I'd say: writing a piece of software (programming) vs modifying an existing, large project in a clean and correct way (engineering). I can write a pretty good standalone tool to solve a problem, but throw me into an unfamiliar code base and ask me to make a significant change that meets all the necessary style/correctness criteria is a whole 'nother level, at least in an hour interview.

dgs_sgd 5y ago

i received this question when i interviewed with them back in 2019. It’s definitely up there for my most enjoyed interview experienced

davidst 5y ago

This is a great interview question I don't want to be 'that guy' by picking at what may be a detail but there is another response to look for from a candidate. We should think very carefully about adding a multiplication command because it introduces a failure mode that may be unanticipated by the client. Code that previously worked could begin to fail after this command goes into use. Specifically, if the client needs to revert a series of operations on integers, and if the operations are transitive, there is no need for a mechanism to ensure they occur in any particular order (the usual caveat about working within the limits of precision applies.) This holds true for addition and for multiplication, in isolation, but is not true if they are combined. Change the order and the end result will change. Adding multiplication puts a burden on the client to understand this risk and be explicit in the ordering. Some people may argue that the client should already understand it and they have a point. We can't defend against every possible misunderstanding. I think there is a good discussion to be had on this question and if the candidate were to go there, it would be a favorable sign of experience on their part.

jameshart 5y ago
This is a REALLY SOLID response to the question, and I'd hope it would get some strong 'hire' points. But, to be fair, there's already technically this issue in the memcached API as described in the post, in that it supports both append and add, and "append 0" (on a value that add could also act upon) is effectively the same thing as "mult 10". If a field contains "1" and successive "add 1" and "append 0" commands come in, depending on what order they arrive in the result could be 20 or 11. So I think the interviewer would be justified in saying 'yeah, let's assume we've evaluated that risk and we plan on adding some really thorough documentation warning people about that risk, so can you just go ahead and try and implement it?' But absolutely, this was the thought that came into my mind when reading the spec too. Not enough developers think about APIs in terms of compositional, algebraic terms, and being able to see that adding 'multiply' and 'add' together at the same precedence in an API might cause trouble is a really valuable skill.
- akhmatova 5y ago
  This is a REALLY SOLID response to the question, and I'd hope it would get some strong 'hire' points. I agree -- but unfortunately the interview question doesn't filter for this level of thinking. If anything, it filters for the exact opposite: your ability and willingness to shove a random new feature into the codebase in 3 hours (or GTFO) -- stability and other consequences be damned.

mihaitodor 5y ago

To everyone praising how good this challenge is, calm down. There's nothing great about making somebody sweat for 3 hours during an interview. It's even worse if "someone got so frustrated that they threw their laptop on the floor" like the top comment claims. Yes, it would be great to avoid hiring people who have a bad temper and tend to throw tantrums when nervous, but, then again, it's beyond me why anyone would need to prove they can cope with such pressure without breaking down. We're talking about a desk job here, which requires writing good quality code according to some requirements during normal working hours, not disarming a bomb with a timer attached to it.

theamk 5y ago
If "3 hours" is "such pressure" for you, then maybe some teams are not a good fit for you? For example, I have broken the build in the past - and this prevents everyone else on your team from merging PRs. In this case the expectation was that I fix it "soon" - once I finish my current meeting/lunch/conversation. If someone is prone to breaking down from pressure given 3 hour deadline, they would have a breakdown right there.
- fzeroracer 5y ago
  Generally speaking if your build process is so fragile that one person creating a breaking build results in a chain of failures and a pressure crunch then that's a work environment problem. I would consider that sort of thing completely unreasonable and consider finding a new job immediately. If you find yourself in that sort of thing frequently then perhaps you should reconsider normalizing it and instead push for actual fixes. You look at the issue and say it's a personal issue. I look at it and see a dysfunctional development environment issue. Now this doesn't always hold true and sometimes devs have to be held to higher pressure environments (such as having on-call duties): but those kind of things are usually better compensated for and called out as part of the job duties.
  - vimwizard 5y ago
    In an ideal world, no one would ever have to work under pressure, and while some people in this industry don't, many (I would bet a majority) of people do or have in the past. This seems especially true for startups trying to break into a market with competition. Either you have been lucky with the places you've worked at, have sifted through tens of shops to find a good one, or have very little experience in the industry. I really do hope that none of my assumptions are true and that one day I can find a functional, low-pressure team to work with.

loxias 5y ago

I miss questions like this. Things that are closer to actually measuring "so, can you code?" and not if you can do CS puzzles, or how well you can shoehorn corporate "values" into answers. I "passed the shit out of this question" and have failed my last several interviews... :( I miss 2010.

CatAtHeart 5y ago

For anyone else who was initially confused like I was: On first look, I thought the solution had to involve exploiting commands over telnet to inject a mult command. It took me until looking at part 2 to realize it involved actually modifying the source code (which makes sense in retrospect). If you are looking to actually try and solve it, keep that in mind and don't look at part 2 :)

sanderjd 5y ago

Yep, I love this kind of thing and it's waaay more similar to my actual job than the little self-contained problem on a "whiteboard" style. The similar but different style question that I got (at Stripe, fwiw) in my recent job search that I also really loved was: Here's a failing test against a complex codebase; figure out what's wrong and then fix it. This flexes similar muscles I think: figure out enough of the flow of a complex codebase to understand where the problem manifests, and how to modify things to fix it without breaking other stuff. The really nice thing is: if all the tests pass at the end, you've done it. Then, time allowing, you can go work on improving the fix; making it more consistent with the architecture, etc.

cafxx 5y ago

> Via its incr and decr commands, memcached provides a built-in way to atomically add k to a number. But it doesn’t provide other arithmetic operations; in particular, there is no “atomic multiply by” operation. Your programming challenge: Add a mult command to memcached. Protip for people in junior roles: if you're interviewing for a more senior position, the correct answer would *not* be to jump in straight and waste 3 hours implementing something that then you have to maintain forever. The best answer, from an *engineering*[1] perspective, would instead be noticing (or knowing) that memcached supports a CAS command (https://github.com/memcached/memcached/wiki/Commands#cas https://github.com/memcached/memcached/wiki/Commands#cas) that would allow to implement equivalent functionality without changes in memcached, and try to confirm why that would not be a viable solution. If, and only if, CAS is confirmed not to be a viable solution (e.g. excessive contention, unacceptable pX latency, ...) then you should spend the 3 hours (+ all the maintenance effort required until the end of life for that solution). The risk, in case you jump acritically on the solution, is to show you did not spend any time trying to actually understand the problem and you did not consider the pros and cons of viable alternatives (and this is a problem both during an interview, as well and especially when you're doing actual work), something that is normally frowned upon in senior roles. Obviously, being an interview, it's all play pretend so you'll eventually have to complete the coding exercise, but if I was interviewing you for a senior position and you skipped the part above it would be a pretty big red flag (same in the unlikely case I were to ask to implement some functionality that is commonly found in the standard library of most programming languages: huge red flag if you don't ask why the functionality provided by the standard library is not a viable solution). [1]: w.r.t. software engineering being "programming integrated over time" (https://adamj.eu/tech/2021/11/03/software-engineering-is-programming-integrated-over-time/ https://adamj.eu/tech/2021/11/03/software-engineering-is-pro...)

theamk 5y ago
During interview, we are not modelling the whole process -- no one would tell a senior engineer just a single sentence "please implement mult command" -- there will be a longer preceding story: maybe the latency has to be very low, or it is a request from a stubborn customer, or you are profiling optimization... So for the reasons of time, we assume that the previous steps have been done, and it was decided that the mult command is the way to go. If I were interviewer and a candidate would mention CAS command, I'd compliment them for their memcached knowledge, and tell that the it is not a viable solution. And this would not affect my evaluation of this stage one way or another.

JohnTHaller 5y ago

I really like this sort of question since it allows a candidate to work their own way while showing their process and lets the company see if the applicant can handle the sort of thing that will be a good part of their job, at least for the first year or so. The popularity of white boards and writing binary tree traversal pseudo code is frustrating to me. I haven't done an interview where that's a part of it because the thought of it freaks my brain out a bit. Which seems odd since I regularly perform improv comedy and am considering teaching it in the near future.

hoosieree 5y ago

"Chesterton's Fence" is pragmatic for getting things done quickly, but I hope people actually come back later to decide if the fence should stay or go. Leaving fences without questioning why they exist builds tech debt.

muh_gradle 5y ago

I haven't checked the spoiler. The brute force method is probably to just call incr multiple times. I wonder if there's anything to do with the caching function to get better performance though. Will need to think about it more.

pphysch 5y ago
What if two clients call `mult` simultaneously, with big values?

deleted 5y ago

[deleted]

elisbce 5y ago

This is far from a great engineering interview question, let alone the best. 1. This question is heavily biased towards specific software experience and familiarity with the domain and tooling, while at the same time too shallow in terms of algorithmic and logical depth. It is like asking how to parse a CSV file in Fortran. The difficult part is not the problem per se, it is picking up an unfamiliar language and tooling on the spot. For example, someone from Windows background who wasn't familiar with the memcached / Linux environment and tooling / Messy open-source codebase might find it difficult to adapt within the time limit. That doesn't automatically make them bad coders. It is like requiring Emacs and DVORAK keyboards for coding at interviews -- In theory it's all the same, but it's cumbersome and slows you down if you are not familiar. Challenges like this during interviews can lead to high false negatives. People might get stuck on small side things and not working on the main thing enough. 2. This question takes too much time to set up and complete. Interviewers don't have three hours to wait. Interviewees can probably only do one such problem during an onsite and nothing else. Are you going to make a judgement and decision solely based on the performance of this one challenge? I don't think that's a good idea. Also the interviewer needs to explain everything. For a three-hour challenge, this is at least 15 minutes time doing nothing but setting the problem up. 3. This challenge is actually not that great to identify senior / great coders while mediocre coders with specific experience can pass easily.

pphysch 5y ago
The question appears to be highly relevant to the position. I would hire the candidate that completes the feature request over the 10x Programmer that cannot.
db48x 5y ago
> 1. This question is heavily biased towards specific software experience and familiarity with the domain and tooling Heavily biased towards the specific software experience and tooling _in use by the company who is hiring_. That seems like a win to me. If you are a Windows shop, go find an open–source C# or VB.net project to use instead. > 2. This question takes too much time to set up and complete. You might have missed the part where the candidate is given a VM to ssh into, where all the build tools and dependencies are already installed. Also, the author of the blog misremembered; they were given only one hour instead of three for this problem. There were probably other interviews, and maybe lunch, afterwards. > 3. This challenge is actually not that great to identify senior / great coders That’s true. As stated in the article, this question was intended to filter out low performers. Other interview questions would be used to judge other aspects of the candidates.

vesche 5y ago

Thanks for sharing this. It took me less than an hour to complete and I thought it was pretty fun. These sort of interview challenges that involve rapidly diving into a foreign codebase are really great, but it has to be done correctly. I once frustratingly & embarrassingly failed a live coding interview where I was asked to fix some sort of linked list issue in a fairly large C code base that wouldn't compile. I much prefer a challenge that involves implementing a new feature in a working code base (like this memcached question) as it's closer to reality and will also provide a better experience for the candidate.

sydthrowaway 5y ago

jwmoz 5y ago

Although I don't know C I really like this example as it's pretty much what happens when I start a new place as a Python dev. E.g. Hey, X, we need you to change the dates on this screen to show the timezone. - Queue me then having to drill down through the codebase to find and then fix the problem etc. Only complaint would be the amount of time it may require to do in an interview.

WC3w6pXxgGd 5y ago

hota_mazi 5y ago

The question actually appears at the end of this unnecessarily lengthy and verbose article. Here it is: > Add a mult command to memcached.

gigatexal 5y ago

I really agree with the article author. The skill of being able to get into a codebase and add an iterative feature like a multiply operation when there’s an addition is huge. It’s something I failed to do when I was a software engineer. I had to study and understand every part of the codebase before I did anything. I was paralayzed. But that understanding would have come over time if I just added features incrementally like I was asked to. I didn’t last very long at that position. Luckily I found data engineering and I much prefer that anyway.

wodenokoto 5y ago

What makes this a great question? I feel the article is missing that discussion.

trinovantes 5y ago

I think I've been reading too many interview questions lately that I originally thought this was a trick question that required implementing mult with only the provided incr I like that this was a more of an engineering question about diving into an unfamiliar codebase than a gotcha math/algorithm question

daniel-cussen 5y ago

I'm grateful for all the engineering interview questions I've gotten in person, and not for any of the ones that were homework. Dude I promised myself as a child I would never do homework after my studies, that promise is sacred, your company is not. So I never got caught out (EDIT: meaning to be clear caught out unable to solve the problem), which you would expect because in fact engineering interviews are actually no use for interviewing engineers, but in fact are great for interviewing for algorithmists, and you'd expect me to do well because I say I say I'm an algorithmist, and sure enough I did solve all of them. Nobody caught me with my mouth open, I've solved every single interview question without timing out, despite (actually because of) not getting a CS degree from Stanford (which I did attend). No advance prep, although that's bullshit, I spent every second I could on advance prep incidentally by working on algorithms. But no leetcode. I did look at it once, at one problem, and it sucks, I saw a problem regarding getting the popcount of all the numbers in a range and tons of people got the top 100 points. That's stupid, I knew a superior answer that took morally no time, and would I get more than 100 points for it? Doubtful. There's a very low cap. Not truly elitist. There can't be a max score. There is one problem that I couldn't get, and that was "how do you find a single element in a list that is not present in an otherwise identical other list?" Like tons of problems, the answer is to sort the lists and take a binary-search analog. That's the answer, but you need faster-than-state-of-the-art sorting. The thing I was supposed to say that they wanted me to say, not to be confused with "the answer", was you had to add the elements and then subtract. I instantly replied, "no, just xor them, less energy." Which was a satisfying response, and marked as a pass. I didn't get the job anyway on a gut check. Unfortunately I did terribly at the very few jobs at which I did get hired. Because I could only get hired at doomed companies, I had between one and three weeks to prove myself, do or die. Died every time. One place, Unholster, gave me one day to set up my environment, seven work days to prove myself, and after a coworker spent an hour insisting I be fired and the boss saw the stories (epics? The point system thing). Ended up saying I objectively amounted to negative-one-half-a-person, concluded "With your curriculum, nobody would give a níspero for you." Níspero is a plentiful fruit that is sweet and is nourishing, but come to think nobody eats because nobody values it. And this is NOT slander, well first off it's true and that's all you need legally, but secondly (morally) that's just what a doomed company is like, I'm actually not being negative. They had to fire like twenty people and had two employees left, and that's public information, tells you everything. Like basically all Chilean companies, they had bad Chilean debt, the Chilean bank leaves a tiny glimmer of hope, that's it, they have to grind through meat until they find the savants. Tons of companies like this, in fact I have a relatively high opinion of them considering. Really liked Unholster. They didn't cheat me out of money or time, so above average. And even when they were for sure getting nothing more out of the relationship, the boss agreed to just talking about why my career was fucked, for like fifty minutes. "I would sign [something for you] saying there's better guys at algorithms than you." "Sign what? Sign what, exactly?" They were hiring another guy, my replacement, I met him, in the time I worked there and he also thought this would be his chance. And it was, I don't know how it went for him, but the point is it is a chance, only it's very very narrow. They wanted to be cool, too, read Hacker News. Smart guys, really good to talk to in the lunch breaks, I liked all of them. They were just living in a shark tank. And truth be told, I wasn't that good at the job, sure if I got three months like everyone's supposed to it'd be easy, but that's really easy, I told them I'd be much better than that. I wanted to be. I wasn't good at the job. Just at the interviews.

croshan 5y ago

I love it--realistic example of working with the production codebase, and a chance for the interviewee to see the actual code quality + practices they'd be dealing with.

anamax 5y ago

If you're only going to use multiplication going forward (no adds), I wonder whether interpreting the output as an exponent will pass enough tests.

dave333 5y ago

Any interview question that takes 3 hours is bad by definition unless of course the 3 hours is a red herring and the interviewees who pass are the ones who do it in much less time.

creatonez 5y ago
The actual author of the question said elsewhere in this thread that it was actually administered with a 1 hour limit.

gorgoiler 5y ago

Type -1 candidates quit on the spot when they discover that memcached has, in the years since they last worked with it, been bastardized to do things other they GET and SET! The beauty of memcached in the 2000s was that it felt extremely opinionated about features — that is to say, it didn’t have any. That bool incr argument in the source code feels like a hint from the authors. The arithmetic feature was meant to do two things and two things only, represented by I/D = +/- = true/false. Add (pardon the pun) any more operations and here be dragons. In particular once you add MULT, some clown is going to ask for DIV, and so are you now going to convert ints to floats or force integer division on everyone? Let’s place bets on how long until we get a “bug” report that 12,999,999 DIV 10 should be 13 not 12. Maybe I’m being unfair. The integer increment thing in memcached seems useful. The useful part of it is the fact that it increments atomically, not that it does the heavy lifting of adding one for you :P If N people increment it you get a number that is N bigger. “Add 1” is also a pretty easy thing to get consistent if you have two people contending for a lock. If it were a strongly typed language it would have a type like Counter (not Number) and it makes no sense to multiply a counter. The hint is also there in the INCR and DECR commands. They are probably not called ADD and SUBTRACT for a reason. Feature rejected! Now let’s go code up MULT anyway :)

xxs 5y ago
I'd say MUL(T) is absolutely useless in any practical manner. I'd take some form of CAS if I want atomic operations. MUL has to deal with overflow a lot more than inc/dec. Not knowing the value prior MUL, taking any branches is another issue, CAS would solve in that case as well. Inc/dec are useful for a counter implementation (although I see no reason to have a designated dec) - even though counters should not be implemented in such a manner, e.g. each writer should have its own counter, and reader should sum them up when the value is needed. No contention, scalable. In that regard I'd consider the interview question "weak", lacking a deeper understanding - not just show us you can code.
- gorgoiler 5y ago
  Oh it’s a great question. Code this solution without having to worry about why. If only the job was actually like that. The why is 10x the how, IRL, once you get beyond IC0.5.

agucova 5y ago

While it took me under an hour, I still wonder whether my implementation would have passed an interview. I generalized do_add_delta to three operations (incr, decr, mult) and otherwise copied code paths and extended the binary protocol.

db48x 5y ago
That sounds like a perfect solution to me.

amelius 5y ago

Look for the add code, copy it, replace + by *.

_andrei_ 5y ago

This is a great test (for bright folks who don't mind "doing free work at interviews"), because it directly answers the "Do you have what it takes?" question. At my first job interview, they handed me the printed documentation for a programming language they invented (2-3 pages), and a problem. I didn't have any work experience with any technology they used, but managed to solve the test, and that was enough to get my first job, and drop out of education.

throwawayHN378 5y ago

First thing I would do is download and build. Then find he incr method/interfaces/etc. Then rename all of those to mult and try and rebuild and see if mult works identically to incr. Then start working on modifying incrementing toy multiplying.

lkxijlewlf 5y ago

I would love an interview question like this. For one thing, if I was unfamiliar with the domain, it would spark a conversation that would help me decide if I wanted to work with that team or not.

minroot 5y ago

Well, if this is how interview works, I need to look for a job.

JustLurking2022 5y ago

Gonna have to disagree... waaaaaay too much setup, and a large advantage to people who are familiar with 1 specific technology.

tomtung 5y ago

My previous employer used something very similar: either adding small features or fixing known bugs of a past build of some open source software in 1-1.5hrs. We prepared one such interview question for each supported language (C++, Python, and Java), and, to help calibration, each question consisted of a series of requirements that required progressively more complex changes. Both interviewees and interviewers seemed to like these questions. I do hope this approach gets more adoption in the industry. It takes more time to prepare, but the high-quality signals it provides are worth the effort imo. Such questions are also harder to leak compared to typical whiteboard coding questions and thus more reusable too.

xaviercoll 5y ago

Nice...