4 ms·

I find system design interviews generally provide significantly more signal than coding interviews, but still less signal than a lunch interview. It's a test o

I find system design interviews generally provide significantly more signal than coding interviews, but still less signal than a lunch interview. It's a test of breadth and depth and it takes an apt, possibly experienced interviewer to navigate the candidate's domain knowledge effectively. The goal isn't to build some elaborate, buzzwordy house of cards, but to understand where tools are appropriate and where they are not, to think on your feet and work with an interviewer to build a system that makes sense. And, just like the real world, criteria should shift and change as you flesh out the design. In particular people trip over the same things every time: reaching for things they do not understand, not understanding fundamental properties of infrastructure (CPUs, memory, networking), and cache invalidation. When I interview folks I always preface the prompt with an offer to provide advice or information, acting as if I were a trusted colleague or stakeholder. If you want to be low level, then I'd question why we're conducting a large system design interview anyways. We could certainly frame it as a small system design instead, and focus on the universe contained within the injection-molded exoskeleton of the widget. If you say "column oriented" I'm going to ask you to explain why. I'm going to challenge what the load balancer is doing or what you expect it to do, and why. Building large systems in the real world well, and watching them scale up under load with grace (and without contorting your opex to have only lunar aspirations) is somewhat akin to watching your child ride their bike for the first time after the training wheels are off. It feels good. Just like seeing your hardware in the field produce a low failure yield. There is satisfaction in doing good work, or at least their should be.

nineplay 3y ago
> If you say "column oriented" I'm going to ask you to explain why. I'm going to challenge what the load balancer is doing or what you expect it to do, and why. And I would have happily repeated something from a video I'd watched on YouTube two nights earlier. It's a cram test.
- SpicyLemonZest 3y ago
  I don't know how I could prove or verify this, but in my experience it feels very easy to detect a difference between people who understand a systems topic and people who've treated it like a cram test. I recall one interview in particular where the guy gave textbook answers to anything like "what technology would you use here" or "what are the benefits of X vs. Y", but fell apart completely whenever I scratched the surface for an implementation detail.
  - scarface_74 3y ago
    You verify it by asking them to walk through their previous experience and why they made the tradeoffs they did. You can also ask them “knowing what you know now, what would you have done differently?”. That lets them talk about practical experience and theoretical knowledge. When I use to interview infrastructure people, I could tell quickly the ones who only knew anything based on cramming with ACloudGuru. On a related note: when I work with customers consulting in cloud application development, I am quick to distinguish between what I know well where I have practical experience, what I can ramp up on quickly based on related knowledge and what I only know from watching a video.
Swizec 3y ago
> but still less signal than a lunch interview Careful with that line of thinking. There’s a significant body of research showing that people feel like a “chat about tech” interview provides the best signal, but it empirically performs the worst with a roughly 50% correlation to on-the-job performance. You’re better off flipping a coin because at least then you’re not biased. source: https://en.wikipedia.org/wiki/Noise:_A_Flaw_in_Human_Judgment https://en.wikipedia.org/wiki/Noise:_A_Flaw_in_Human_Judgmen...
- eschneider 3y ago
  Lunch interview is often a great way to get some signal about the company on your potential coworkers. It's not so much asking questions and getting answers, but you often get a chance to see the team interacting _with each other_ and sometimes you can get a view into what sort of issues they are really dealing with.
  - deleted 3y ago
    [deleted]
- pedrosorio 3y ago
  > but it empirically performs the worst with a roughly 50% correlation to on-the-job performance. You’re better off flipping a coin because at least then you’re not biased. I was going to point out that a correlation of 50% is pretty good (specially for predicting job performance from a single interview), whereas flipping a coin has 0% correlation with anything that is independent of the coin flip (such as job performance). You probably meant to say the probability you rank job-performance of two random candidates correctly based on an interview is about 50% (what your source calls "percent concordant"), a correlation of 0%. Out of curiosity: do you remember which section of the book looks at "chat about tech" interviews compared to other kinds of tech interviews in regards to their job performance prediction capabilities?
  - Swizec 3y ago
    You are right, I got my stats terms mixed up. The "Improving Judgements" portion uses interviews as a case study and builds up from "just have a chat" to the typical multi-round panel interview with pre-defined rubrics that we see in tech these days. When done correctly, the book suggests this is the best we can do short of hiring everyone and firing low performers soon after. I remember they specifically mention Google as one of the companies where they ran a study linking interview practices/decisions to on-the-job performance.

nineplay 3y ago

> If you say "column oriented" I'm going to ask you to explain why. I'm going to challenge what the load balancer is doing or what you expect it to do, and why. And I would have happily repeated something from a video I'd watched on YouTube two nights earlier. It's a cram test.

SpicyLemonZest 3y ago
I don't know how I could prove or verify this, but in my experience it feels very easy to detect a difference between people who understand a systems topic and people who've treated it like a cram test. I recall one interview in particular where the guy gave textbook answers to anything like "what technology would you use here" or "what are the benefits of X vs. Y", but fell apart completely whenever I scratched the surface for an implementation detail.
- scarface_74 3y ago
  You verify it by asking them to walk through their previous experience and why they made the tradeoffs they did. You can also ask them “knowing what you know now, what would you have done differently?”. That lets them talk about practical experience and theoretical knowledge. When I use to interview infrastructure people, I could tell quickly the ones who only knew anything based on cramming with ACloudGuru. On a related note: when I work with customers consulting in cloud application development, I am quick to distinguish between what I know well where I have practical experience, what I can ramp up on quickly based on related knowledge and what I only know from watching a video.

Swizec 3y ago

> but still less signal than a lunch interview Careful with that line of thinking. There’s a significant body of research showing that people feel like a “chat about tech” interview provides the best signal, but it empirically performs the worst with a roughly 50% correlation to on-the-job performance. You’re better off flipping a coin because at least then you’re not biased. source: https://en.wikipedia.org/wiki/Noise:_A_Flaw_in_Human_Judgment https://en.wikipedia.org/wiki/Noise:_A_Flaw_in_Human_Judgmen...

eschneider 3y ago
Lunch interview is often a great way to get some signal about the company on your potential coworkers. It's not so much asking questions and getting answers, but you often get a chance to see the team interacting _with each other_ and sometimes you can get a view into what sort of issues they are really dealing with.
- deleted 3y ago
  [deleted]
pedrosorio 3y ago
> but it empirically performs the worst with a roughly 50% correlation to on-the-job performance. You’re better off flipping a coin because at least then you’re not biased. I was going to point out that a correlation of 50% is pretty good (specially for predicting job performance from a single interview), whereas flipping a coin has 0% correlation with anything that is independent of the coin flip (such as job performance). You probably meant to say the probability you rank job-performance of two random candidates correctly based on an interview is about 50% (what your source calls "percent concordant"), a correlation of 0%. Out of curiosity: do you remember which section of the book looks at "chat about tech" interviews compared to other kinds of tech interviews in regards to their job performance prediction capabilities?
- Swizec 3y ago
  You are right, I got my stats terms mixed up. The "Improving Judgements" portion uses interviews as a case study and builds up from "just have a chat" to the typical multi-round panel interview with pre-defined rubrics that we see in tech these days. When done correctly, the book suggests this is the best we can do short of hiring everyone and firing low performers soon after. I remember they specifically mention Google as one of the companies where they ran a study linking interview practices/decisions to on-the-job performance.