16 ms·
We've gone away from system design interview questions on my team. We ask people to diagram something technical they understand well and the team digs in and as
by rednerrus 4y ago
We've gone away from system design interview questions on my team. We ask people to diagram something technical they understand well and the team digs in and asks questions to understand depth and breadth of the candidates understanding. For us it works much better. It's a chance to see how well candidates do in following instructions. It gives you a chance to explore depth and breadth of their knowledge on something technical that they claim to understand. Our philosophy is how well you understand something you claim to know well is indicative of the depth and breadth of your technical knowledge in general. There are plenty of opportunities in this to ask about design and get to know how people think when it comes to design. I think it helps to eliminate false negatives and false positives.
- mdm12 4y agoI have seen this process described elsewhere as 'reverse system design', and it is my preferred approach to evaluating senior candidates as well.
- jstx1 4y agoStill sounds like a system design interview to me, just a bit less structured.
- deleted 4y ago[deleted]
- rednerrus 4y agoWe've found that candidates who better understand how things are connected make better candidates and operators. We need some way to gauge how well they understand how things are connected. This is as fair of way as we've come up with.
- zekrioca 4y agoI totally relate to that. In the university setting, I see many students more interested in learning keywords instead of “key concepts” behind such keywords. Think of preferring to learn “k8s” instead of “resource manager”. Today, very few of them knows that etcd is one of the key components in “k8s”. I guess this behavior is similar to the transformation that happened in other non-CS fields.
- vsareto 4y agoFeels like an Architect interview more than a Senior interview though
- pc86 4y agoAs a former (reformed?) non-coding architect, non-coding architects are a scourge on the industry. I know you didn't say "non-coding" but if it's a separate job, they're very likely not spending enough time coding. Seniors should be able to build large, interconnected systems. IMO that's a basic skillset required to reach that level, and part of why I roll my eyes when I see people with 5, 4, 3 years of experience claiming to be seniors.
- eternalban 4y agoI somewhat take issue with this. (3+ decades of very hands on (read: coding) architecting, including some learning experiences in orbit. I've been coding code-doodling since teenage years.) A lot of what non-coding architects traditionally brought to the table has been taken over by experts designing OSS protocols, data formats, etc. Take things like data frames that are now exploding in ML space. Seniors today, agreed, should be able to integrate (sub)systems and that may in fact be enough. But a competent systems architect (who have never touched code) should be able to also define data formats, patterns of movement of data between sub-systems (for say optimal performance), the actual computing platform considerations, etc. Also, sometimes when being too close to code, things degenerate to debates about tools, etc. Naturally my points here gain more validity as system size (or its open-ness requirements) increase.
- monsieurbanana 4y agoI always thought that non-coding architects still had at least some background in coding. What does the path of a never-had-coded architect looks like?
- projectazorian 4y agoDBA or sysadmin, I would imagine. So more “never wrote application code” than “never wrote a single line of code for anything ever.”
- slytherone 4y agoHey, Creator of the guide here Reverse system design interviews have a a lot of untapped potential. They just started getting a bit more popular in the last year or so (however, most companies haven't caught on to the trend yet.) You're one of the trendsetters @rednerrus
- moosedev 4y agoI had a "reverse" system design interview at Amazon back in the 2000s. Possibly before the modern "forward" system design interview became so popular. I also remember circa 2012-2014 being required to conduct what are now considered typical modern system design interviews (i.e. as an interviewer) while employed at Amazon, and 90% of (even senior) candidates had no idea how to even begin to approach the "design a system to do X"-type questions. I think this is partly because far fewer people in typical software jobs were building distributed systems back then anyway, and partly because all the YouTube videos and guides like yours didn't exist yet, so nobody was doing the kind of dedicated prep and rote-learning that seems increasingly expected nowadays. Back then, when the problem was too far from the candidate's real work experience, and they weren't willing to make educated guesses and ask clarifying questions in order to move forward, it was often a struggle (for both of us) to pivot the interview towards something the candidate could tackle and demonstrate their design experience. ("Tell me about a system you designed", i.e. the "reverse" approach, wasn't an acceptable alternative in our rubric.) Now, just like what happened with coding interviews vs. Leetcode, it seems that a more common challenge for the interviewer is telling the difference between a candidate who is applying real experience and understanding vs. regurgitating/performing what they read in a system design interview prep guide. But that's assuming one is actually more valuable than the other, and I don't have proof of that.
- spike021 4y agoOne place I interviewed with last year did something similar in that it was more of a BYOSD (bring your own system design). So it meant I got a chance to think through complex systems I've worked on, mock up a diagram beforehand, and then present it to the interviewer. They then drilled down on a lot of components, why decisions may have been made, etc. Sort of like what you're saying. Out of all the similar interviews I did last year for that type of round, I enjoyed that style the most. Rather than your typical "build me an {ecommerce site, social network, video streaming site, url shortener}".
- zinclozenge 4y agoSquare/Block (used to?) ask only 1 system design question and the recruiter would let you know ahead of time so you could prepare for it. Probably because it allows for deeper questioning and answering of relevant technical skills.
- blowski 4y agoIt’s hard to tell between those who know their stuff, and those who are good at memorising all the various courses that tell you what to say. By telling the candidate what system to design you make the latter’s job easier.
- pcthrowaway 4y agoIf someone can quickly learn enough about a system that they can convince a domain expert they understand it well in a deep technical discussion, they're probably a good candidate for hiring, since your business likely consists of many complex systems, and the candidate will theoretically be able to onboard faster.
- namaria 4y agoAttaining high level understanding and speaking convincingly about it are entirely different skills from knowing how to build it. Interviews are always a proxy anyway, your job will never be solely comprise of convincing people of your expertise. Sooner of later you have to do exercise it somehow. Unless you're the CEO, that is. The higher you go the easier it is to make a career out of being in the right places at the right times.
- bhvangoo 4y agohmm interesting approach, really like it. Where do you work? and are you hiring now?
- recfab 4y agoI would also like to know these things. This sounds like a type of interview that respects my time, which makes me think there is a good chance the company as a whole will respect my time and humanity as well.
- rednerrus 4y agoThis is exactly what we’re going for. It lets candidates know that we value their time and have worked as hard as we can to create an interview process that lets us gauge whether or not you’d be a good fit for the job in the least amount of time as possible.
- zabzonk 4y agobest way of doing interviews imho. apart from anything else, it gives you a chance to show that you know how to design/implement a system that is perhaps more complex than the one you are being interviewed for
- azornathogron 4y agoI like this... in theory. But I don't think I'm allowed to give you a system diagram and detailed explanation of systems I've worked on, because obviously they're proprietary. That seems like a problem. How do your candidates normally work around this? Does everyone just talk about hobby or open-source systems they've worked on?
- ok_dad 4y agoThe only really important details to keep secret are business-specific algorithms, so talking about the overall architecture of a system is usually okay, without those details. If you're building very special architectures where that actually is the business detail, then I suppose you need to figure out how to talk about something else similar or something. I just talk about the systems I worked on in generalities, so I would talk about how we get some telemetry from some devices, process it, store it, and etc., but only the mechanical details and not the meaty algorithmic parts (we use a JSON API to submit the telemetry, then we store it in a table, then we take that data in our other component and process it through our "algorithm", etc.).
- sroussey 4y agoYour ideas of what should be secret or not may not at all be what the candidate agreed to in their employment contract and NDA.
- cyc116 4y agoExactly. Not to mention high level business logic could be inferred from system architecture. Sure sounds like a good way to perform recon on a target company.
- rednerrus 4y agoWe say specifically “diagram a technical system you understand well.” It’s going to be challenging to work in technology if the only technical system you understand well is the one you’re currently working on and only the parts that are under NDA.
- onlyrealcuzzo 4y ago> It gives you a chance to explore depth and breadth of their knowledge on something technical that they claim to understand. Assuming you know enough about what they're doing to actually dig in - which, granted, maybe you're only considering interviewing people who are working things you're somewhat familiar with. Ultimately, I don't see how this is different from a technical design interview - in that you can be susceptible to hiring charlatans, and pass on people who'd rather say they don't know something than try to BS most of it on the spot to sound impressive. The two people could have the same knowledge. You're more likely to hire the charlatan. Maybe that's what you want :shrug:
- rednerrus 4y agoWe've had very few false positives. It's a conversation and we understand and appreciate when people say they don't know. We understand that really good engineers say "I don't know how that works." If all of your answers are "I don't know how that works.", it'll be red flags enough for the team. The team of interviewers has a very wide and deep knowledge of technologies (it's kind of a virtuous cycle). We call it out in the pre-interview handout that if you pick something super obscure it'll hurt your chances as the team will struggle to ask good probing questions. The nice thing about this is you're giving the candidate a choice about what they want to describe. Good engineers will choose appropriate topics.
- clucas 4y agoIs this sort of interview common in the industry? This is the first time I've seen someone describe a technical interview I know I could pass with flying colors.
- Scubabear68 4y agoIn my own experience interviewing people about systems they have worked on (as opposed to hypothetical on-the-spot design), we have been highly successful in routing out charlatans, as you put it. They very rapidly get hand-wavy and are only able to give the most shallow of answers. In a few cases they will go in-depth, but reveal truly bizarre decisions or lack of understanding of the platform. Good senior engineers can usually go fairly deep, they are honest about where they don't have as much knowledge, and are usually candid about what was good and maybe what in retrospect was a hack or a bad decision in retrospect.
- jason-phillips 4y ago> We ask people to [discuss] something technical they understand well and the team digs in and asks questions to understand depth and breadth of the candidates understanding. This is how I conduct all of my interviews. I can't stand the gotcha-centric, pitfall-laden Jeopardy contest format of interview interrogations.
- rednerrus 4y agoWe think this is the fairest way to evaluate candidates. I had one standout interview that was very gotcha centric. I didn't think the way the interview was conducted was fair or respectful of my time. I also don't think it did a good job of assessing how well or how poorly I would have done in the job. It seems indicative of poor management. I think it's very important as managers that we respect people and their time, always. This feels like as fair of a way as we've come up with so far to do that. We do two technical exercises, diagramming being one of them. We've had really great results and it takes ~1 hour. That seems fair.
- gautamdivgi 4y agoYou’re ruling out all candidates who work on classified systems and those under an NDA
- nothrowaways 4y agoIt still looks design interview
- xen2xen1 4y agoWhen doing IT people often say "Well, I don't know anything about computers", which I generally found pointless, as with about 30 seconds of talking I can tell if you know anything or not. Your story passes my smell test, as getting someone talking is 90% of telling what they know.
- rednerrus 4y agoPeople with broad and deep understanding of technology can typically gauge if someone knows there stuff or not with 30 minutes of them describing a system to you and answering questions about it.
- jahewson 4y agoI don’t like this approach because it doesn’t provide measurable or repeatable criteria for comparing candidates. It also suffers from a sort of Gell-Mann amnesia where a candidate who spouts good-sounding BS about a topic you’re not familiar with is assumed to be competent across the board.
- rednerrus 4y agoThe process is repeatable. We ask all candidates for the same role the same questions and ask them to do the same technical tasks. The other component of our technical interview has something more similar to scoring. We’ve found that this diagram exercise is actually a lot harder to BS your way through because the expectation is that we’re going to probe into your answers and it’s supposed to be something you understand well.
- bluetomcat 4y agoFew people at software companies are designing systems from scratch nowadays. If they are, they'll be gluing together third-party middleware applications like database systems, queues, proxies, load balancers, etc. A general understanding of computer architecture, programming language theory, networking and common protocols and formats is more valuable. At that fundamental level, they won't be using recent fancy buzzwords to sound cool, but could be using their brain to come up with something original.
- fdgsdfogijq 4y agoIf you havent designed a system from scratch, or designed a major change, you arent a senior engineer.
- jhp123 4y agothese "system design interviews" often focus on systems at the scale of major consumer internet companies, e.g. "design youtube", "design twitter". How many systems of that scale even exist? 100? Only the core developers of those systems would be considered "senior engineer"?
- specialp 4y agoSomeone isn't going to design one of these system or has designed some of these systems themselves or during an interview. However for something like YouTube one could say Ok to start simply I would have a frontend form to submit videos, which sends it to a video conversion API queue that calls back when it is completed or could be polled for progress. Now obviously what YouTube really does is much more complicated. But the follow up questions could be like OK your form works and now your video site becomes enormously successful and your video transcoder is overloaded, what would you do? Well I could parallelize the consumers of the queue to some large number, and scale based on load... So isn't going to be something like I would architect my own massively parallel converter database and binaries written from scratch to process all the various formats which is probably closer to what is done in reality. But senior and lead engineers should indeed understand tradeoffs, parallelization, queues, data storage concerns. They are given as questions because people know from a use case view how they generally work.
- emmanueloga_ 4y agowhere do you work?
- posharma 4y agoI don't know if you work in FAANG or no, but most FAANG interviews just don't work this way. So, as much as I like this style, it's just 1 company and definitely not the norm. The rest of us are stuck with design Uber, Whatsapp, crap...
- ramraj07 4y agoHow many rounds of interviews do you typically do? What other interviews if any? Curious what good orgs are doing in this regard!
- rednerrus 4y agoWe do phone screen, team fit, technical. Technical is two part, the diagram and a group exercise debugging something. That’s it. It’s 3.5 hours total.
- emodendroket 4y agoTo be honest, since I have to go through preparing for the standard style of interview before a job search anyway, all these novel forms seem like annoyances.
- rednerrus 4y agoWe’ve found that candidates prefer this because it doesn’t require a ton of prep. Talk about something you know well in a conversation with other engineers. Do some minor diagramming as you go to help us understand. It’s thirty minutes with no gotchas.
- angarg12 4y agoI work for a big tech co, and our interview training explicitly says to not ask candidates about systems they have built in the past, but to ask them to build brand new systems from scratch. This seems completely backwards to me. Is like saying "hey, do you know that relevant on the job experience that you have? we don't want to hear anything about it, here you have a made up scenario". Ok ok, that is a bit cynical. Asking to design novel systems can give you some good insights about the candidate, or help with people who don't have experience building systems yet. Still it seems to me that asking candidates to describe real world systems that they have actually built is much more useful at checking their skills than building imaginary systems. One argument against this is that candidates can "cheat" by preparing in depth a made up example. But how is that much different than the current approach?
- travem 4y agoI imagine a concern is that for many candidates who are under NDA with the current and past gigs, sharing detailed system design info is not something that they would be willing to do. Do you want to penalize candidates who are careful about leaking confidential IP. Given that case the fair evaluation is to start from a blank slate to see what they would build given the requirements and constraints shared in the interview.
- lightbendover 4y agoAlso big tech, we ask senior candidates both hypothetical systems and ones they have experience with.
- rednerrus 4y agoWe don’t ask people to describes systems the worked on previously. We ask them to diagram a technical system that they understand well.
- kridsdale1 4y agoIf you are an expert one one system that you have built and maintained for ten years, then that’s all you know.
- bosch_mind 4y agoThis is pretty cool. I’d probably have fun walking through some systems I’ve designed
- Jiocus 4y agoThis sounds like an enjoyable interview, even from the applicants point of view. Do you follow up on your decisions to see how it held up regarding false positives you did accept, if any? I mean as related to the heuristics, not performance review generally.