4 ms·

It's possible in those cases that the candidates were really good. The main tells are a mismatch between apparent practical experience and apparent knowledge o

It's possible in those cases that the candidates were really good. The main tells are a mismatch between apparent practical experience and apparent knowledge of the design space. I always dig into a few specific technical aspects of the design as far as possible to see how deep the candidate can go. For example, if there's a queue I'll ask what would happen if in production the queue starts to fill, and how to mitigate. Most candidates reflexively suggest making the queue bigger, some suggest adding more downstream capacity to drain faster. People with real experience will have encountered this problem and know that you have to spend time investigating to find the root cause of the backpressure, otherwise you could add capacity to random things without reducing the queue occupancy. Are your queue consumers getting throttled by some downstream service? Are your workers CPU bound? The "right" answer here is basically "we should figure out why the queue is filling up before changing anything", but very few candidates answer that way. Another kind of mismatch is the red flag I listed, where candidates will basically list buzzwords or ideas regardless of whether or not they are related to the question I asked. For example if I ask a question where the user-facing API is very simple (ie, write opaque data to a log) then they start talking about the tradeoffs between graphql vs REST, that's a bad sign.

jefQuery 4y ago
It feels cool to me that my nontraditional degree for software (ChemE) drilled this into us and now it comes up in my current job: Rate of input - rate of output = rate of accumulation It’d be fine to just up the queue capacity if the input is just inconsistent/spikey but averages to the input, but under steady state (or steadily increasing growth) higher input, you must increase the output rate or you will outgrow whatever capacity the system has
yunwal 4y ago
> People with real experience will have encountered this problem and know that you have to spend time investigating to find the root cause of the backpressure, otherwise you could add capacity to random things without reducing the queue occupancy. Are your queue consumers getting throttled by some downstream service? Are your workers CPU bound? The "right" answer here is basically "we should figure out why the queue is filling up before changing anything", but very few candidates answer that way. This is the opposite of what "people with real experience" do, at least in production-down situations. The first step is to mitigate, the second step is to root cause. If there's some no-brainer step that has a chance of alleviating the issue while you root-cause, and is unlikely to make things worse, you should take it.
- apsurd 4y ago
  Your point is good, but in making it, you sound like an adversarial interviewer. You and OP are actually agreeing with "the correct answer is it depends and now let's discuss context". This echoes my experience as interviewer too. It's a red flag when the candidate responds with "the correct answer". That's what OP is calling out. I'm replying here because I get the impression you're looking for "the right answer" as you see it: "the first step is to mitigate then do root cause". You're right! But it also could be too adversarial.
  - bradlys 4y ago
    > Your point is good, but in making it, you sound like an adversarial interviewer. Most interviewers are adversarial. Let's be honest - if an interviewer wants you to pass a system design interview, they'll make it work. I see this with particular candidates all the time. If we want the person to make it through - we'll let them get through. If we don't want them to get through - no amount of correct and behaviorally appropriate answers are gonna make them get through.
    - siggen 4y ago
      Pretty much this. You can have experience and also perform well in a system design interview. However, passing some of these rounds involves chance. Some interviewers are looking for specific answers. For example, some interviewers are looking for you to be knowledgeable about recent papers specific to the heavy-hitters problem. Or the two ways to best support lynchpin objects. It’s not so much as dropping well-known building blocks to construct a system but rather the interviewer asking you to design a system requiring specific characteristics you might not have any specialization in building.
    - deleted 4y ago
      [deleted]
- mgraczyk 4y ago
  This might be true of SRE roles, but not really true for most engineers, particularly not where I currently work. I wasn't completely clear but what I'd be asking for here is how to predict and design for bottlenecks, not how to mitigate active disasters. Also making a queue bigger is a bad reflexive response to queues being full. I was very involved in the SEV review (postmortem) process at Facebook and witnessed lots of cases where bad situations were made much worse by misguided hasty responses.
nonethewiser 4y ago
What do you expect them to say instead? I take your word for it that they’re trying to trick you or something - it’s not my story. But it kinda comes off like you’re denigrating people for not knowing something.
- mgraczyk 4y ago
  If they bring up graphql, I expect them to say something like "none of the advantages of graphql are applicable in this case, so it may make sense to use if it's already well supported. All else being equal it's probably not a good fit" The key is that the response should reflect the actual design we're discussing, it should be more specific than something that could be said in any interview.
- kevinmchugh 4y ago
  In an interview it is a negative to not know something. It's a bigger negative to not know something and try to hide that, or to be ignorant of the ignorance. "I don't know" is a fine answer in an interview. "I don't know, here's where I would look" is better

jefQuery 4y ago

It feels cool to me that my nontraditional degree for software (ChemE) drilled this into us and now it comes up in my current job: Rate of input - rate of output = rate of accumulation It’d be fine to just up the queue capacity if the input is just inconsistent/spikey but averages to the input, but under steady state (or steadily increasing growth) higher input, you must increase the output rate or you will outgrow whatever capacity the system has

yunwal 4y ago

> People with real experience will have encountered this problem and know that you have to spend time investigating to find the root cause of the backpressure, otherwise you could add capacity to random things without reducing the queue occupancy. Are your queue consumers getting throttled by some downstream service? Are your workers CPU bound? The "right" answer here is basically "we should figure out why the queue is filling up before changing anything", but very few candidates answer that way. This is the opposite of what "people with real experience" do, at least in production-down situations. The first step is to mitigate, the second step is to root cause. If there's some no-brainer step that has a chance of alleviating the issue while you root-cause, and is unlikely to make things worse, you should take it.

apsurd 4y ago
Your point is good, but in making it, you sound like an adversarial interviewer. You and OP are actually agreeing with "the correct answer is it depends and now let's discuss context". This echoes my experience as interviewer too. It's a red flag when the candidate responds with "the correct answer". That's what OP is calling out. I'm replying here because I get the impression you're looking for "the right answer" as you see it: "the first step is to mitigate then do root cause". You're right! But it also could be too adversarial.
- bradlys 4y ago
  > Your point is good, but in making it, you sound like an adversarial interviewer. Most interviewers are adversarial. Let's be honest - if an interviewer wants you to pass a system design interview, they'll make it work. I see this with particular candidates all the time. If we want the person to make it through - we'll let them get through. If we don't want them to get through - no amount of correct and behaviorally appropriate answers are gonna make them get through.
  - siggen 4y ago
    Pretty much this. You can have experience and also perform well in a system design interview. However, passing some of these rounds involves chance. Some interviewers are looking for specific answers. For example, some interviewers are looking for you to be knowledgeable about recent papers specific to the heavy-hitters problem. Or the two ways to best support lynchpin objects. It’s not so much as dropping well-known building blocks to construct a system but rather the interviewer asking you to design a system requiring specific characteristics you might not have any specialization in building.
  - deleted 4y ago
    [deleted]
mgraczyk 4y ago
This might be true of SRE roles, but not really true for most engineers, particularly not where I currently work. I wasn't completely clear but what I'd be asking for here is how to predict and design for bottlenecks, not how to mitigate active disasters. Also making a queue bigger is a bad reflexive response to queues being full. I was very involved in the SEV review (postmortem) process at Facebook and witnessed lots of cases where bad situations were made much worse by misguided hasty responses.

nonethewiser 4y ago

What do you expect them to say instead? I take your word for it that they’re trying to trick you or something - it’s not my story. But it kinda comes off like you’re denigrating people for not knowing something.

mgraczyk 4y ago
If they bring up graphql, I expect them to say something like "none of the advantages of graphql are applicable in this case, so it may make sense to use if it's already well supported. All else being equal it's probably not a good fit" The key is that the response should reflect the actual design we're discussing, it should be more specific than something that could be said in any interview.
kevinmchugh 4y ago
In an interview it is a negative to not know something. It's a bigger negative to not know something and try to hide that, or to be ignorant of the ignorance. "I don't know" is a fine answer in an interview. "I don't know, here's where I would look" is better