5 ms·

>I feel like I can easily see through interview practice or coaching How do you spot it? I've had candidates with stellar performance on system design intervie

>I feel like I can easily see through interview practice or coaching How do you spot it? I've had candidates with stellar performance on system design interviews, and I've never had any suspect of them (and the resumé also seemed pretty strong in those cases). But now I'm curious if I missed something.

moandcompany 4y ago
One of the "heavy practice/coaching/leetcode grinding" signals is when the interviewee immediately dives into proposing a solution and discusses facets of that solution in great detail without bothering to ask clarifying questions or discussing assumptions and the risks/consequences of those assumptions.
- ditonal 4y ago
  I think the opposite is true. An experienced engineer not as versed in interview prep might hear the question to design something, and go ahead and show how they would design it based on their experience doing it in the real world. The leetcode grinder would almost certainly ask clarifying questions since it’s heavily emphasized as part of the rubric in pretty much all system design interview prep material. Of course, the non leetcode grinder might ask those questions, but even the social cues that you’re allowed to ask clarifying questions might not be understood. In my experience, I’ve seen a lot of people say that systems design test the seniority of a candidate, but then stick to a rubric that the leetcode grinder has memorized and the senior engineer might miss several important points on despite having the experience because they didn’t understand all the unwritten rules of the song and dance.
  - apsurd 4y ago
    It's funny cuz the leetcode grinder also learns that "asking clarifying questions" is part of the right answer structure too. So the cat-&-mouse game becomes more nuanced, but I do agree that it usually does become apparent that less-experienced candidates are _blindly_ following a strategy they learned somewhere. Interestingly this is not _inherently_ wrong since if you think about it, don't experienced people do the same thing? playbooks and checklists etc. But TLDR: the hollowness of each step of the approach does become apparent in my experience.
  - leonidasv 4y ago
    The business goal of interviews is not to select the best one, but to narrow down candidates with an acceptable fit for the role. An interview system with a high false-negative rate is not seem as problematic if the false-positive rate is also low. The cost of not hiring a very good candidate is way lower than the cost of hiring a bad one (even considering the opportunity cost in most cases).
mgraczyk 4y ago
It's possible in those cases that the candidates were really good. The main tells are a mismatch between apparent practical experience and apparent knowledge of the design space. I always dig into a few specific technical aspects of the design as far as possible to see how deep the candidate can go. For example, if there's a queue I'll ask what would happen if in production the queue starts to fill, and how to mitigate. Most candidates reflexively suggest making the queue bigger, some suggest adding more downstream capacity to drain faster. People with real experience will have encountered this problem and know that you have to spend time investigating to find the root cause of the backpressure, otherwise you could add capacity to random things without reducing the queue occupancy. Are your queue consumers getting throttled by some downstream service? Are your workers CPU bound? The "right" answer here is basically "we should figure out why the queue is filling up before changing anything", but very few candidates answer that way. Another kind of mismatch is the red flag I listed, where candidates will basically list buzzwords or ideas regardless of whether or not they are related to the question I asked. For example if I ask a question where the user-facing API is very simple (ie, write opaque data to a log) then they start talking about the tradeoffs between graphql vs REST, that's a bad sign.
- jefQuery 4y ago
  It feels cool to me that my nontraditional degree for software (ChemE) drilled this into us and now it comes up in my current job: Rate of input - rate of output = rate of accumulation It’d be fine to just up the queue capacity if the input is just inconsistent/spikey but averages to the input, but under steady state (or steadily increasing growth) higher input, you must increase the output rate or you will outgrow whatever capacity the system has
- yunwal 4y ago
  > People with real experience will have encountered this problem and know that you have to spend time investigating to find the root cause of the backpressure, otherwise you could add capacity to random things without reducing the queue occupancy. Are your queue consumers getting throttled by some downstream service? Are your workers CPU bound? The "right" answer here is basically "we should figure out why the queue is filling up before changing anything", but very few candidates answer that way. This is the opposite of what "people with real experience" do, at least in production-down situations. The first step is to mitigate, the second step is to root cause. If there's some no-brainer step that has a chance of alleviating the issue while you root-cause, and is unlikely to make things worse, you should take it.

moandcompany 4y ago

One of the "heavy practice/coaching/leetcode grinding" signals is when the interviewee immediately dives into proposing a solution and discusses facets of that solution in great detail without bothering to ask clarifying questions or discussing assumptions and the risks/consequences of those assumptions.

ditonal 4y ago
I think the opposite is true. An experienced engineer not as versed in interview prep might hear the question to design something, and go ahead and show how they would design it based on their experience doing it in the real world. The leetcode grinder would almost certainly ask clarifying questions since it’s heavily emphasized as part of the rubric in pretty much all system design interview prep material. Of course, the non leetcode grinder might ask those questions, but even the social cues that you’re allowed to ask clarifying questions might not be understood. In my experience, I’ve seen a lot of people say that systems design test the seniority of a candidate, but then stick to a rubric that the leetcode grinder has memorized and the senior engineer might miss several important points on despite having the experience because they didn’t understand all the unwritten rules of the song and dance.
- apsurd 4y ago
  It's funny cuz the leetcode grinder also learns that "asking clarifying questions" is part of the right answer structure too. So the cat-&-mouse game becomes more nuanced, but I do agree that it usually does become apparent that less-experienced candidates are _blindly_ following a strategy they learned somewhere. Interestingly this is not _inherently_ wrong since if you think about it, don't experienced people do the same thing? playbooks and checklists etc. But TLDR: the hollowness of each step of the approach does become apparent in my experience.
- leonidasv 4y ago
  The business goal of interviews is not to select the best one, but to narrow down candidates with an acceptable fit for the role. An interview system with a high false-negative rate is not seem as problematic if the false-positive rate is also low. The cost of not hiring a very good candidate is way lower than the cost of hiring a bad one (even considering the opportunity cost in most cases).

mgraczyk 4y ago

It's possible in those cases that the candidates were really good. The main tells are a mismatch between apparent practical experience and apparent knowledge of the design space. I always dig into a few specific technical aspects of the design as far as possible to see how deep the candidate can go. For example, if there's a queue I'll ask what would happen if in production the queue starts to fill, and how to mitigate. Most candidates reflexively suggest making the queue bigger, some suggest adding more downstream capacity to drain faster. People with real experience will have encountered this problem and know that you have to spend time investigating to find the root cause of the backpressure, otherwise you could add capacity to random things without reducing the queue occupancy. Are your queue consumers getting throttled by some downstream service? Are your workers CPU bound? The "right" answer here is basically "we should figure out why the queue is filling up before changing anything", but very few candidates answer that way. Another kind of mismatch is the red flag I listed, where candidates will basically list buzzwords or ideas regardless of whether or not they are related to the question I asked. For example if I ask a question where the user-facing API is very simple (ie, write opaque data to a log) then they start talking about the tradeoffs between graphql vs REST, that's a bad sign.

jefQuery 4y ago
It feels cool to me that my nontraditional degree for software (ChemE) drilled this into us and now it comes up in my current job: Rate of input - rate of output = rate of accumulation It’d be fine to just up the queue capacity if the input is just inconsistent/spikey but averages to the input, but under steady state (or steadily increasing growth) higher input, you must increase the output rate or you will outgrow whatever capacity the system has
yunwal 4y ago
> People with real experience will have encountered this problem and know that you have to spend time investigating to find the root cause of the backpressure, otherwise you could add capacity to random things without reducing the queue occupancy. Are your queue consumers getting throttled by some downstream service? Are your workers CPU bound? The "right" answer here is basically "we should figure out why the queue is filling up before changing anything", but very few candidates answer that way. This is the opposite of what "people with real experience" do, at least in production-down situations. The first step is to mitigate, the second step is to root cause. If there's some no-brainer step that has a chance of alleviating the issue while you root-cause, and is unlikely to make things worse, you should take it.