6 ms·

A side topic: how can a company find people like the author, who can really articulate how to build a system? I've been interviewing senior engineers for years,

A side topic: how can a company find people like the author, who can really articulate how to build a system? I've been interviewing senior engineers for years, and more than 95% of them, if not more, are really box drawers. They throw terms around confidently but fail miserably the moment I ask for specifics. No, I don't ask for details like how Raft works. That level of details probably would fail all the candidates. Just simple things like how you organize the data, how you route the traffic, how you manage metadata, or how your data path keeps metadata in sync. Really basic ones, yet most interviewers fail. Indeed, the more senior they are, the more likely they lose touch of the details -- even those who claim to work on distributed databases.

qznc 3y ago
You find them via their blogs.
bombela 3y ago
Where do you find the jobs where people actually care about engineering? Because real work is mostly careful plumbing without breaking what works, but in interviews you must leetcode. Real systems only work when kept reasonably simple, but system design interviews are all about drawing boxes.
BossingAround 3y ago
I recently interviewed with a company, and the interviewer asked me "how do you organize data." I wasn't sure if they wanted to talk about classes, modules, databases, k-v stores, hashing data and routing distributed requests to the same pods. I asked, and they answered "I mean in general, how do you organize data." After talking for a bit about pretty much all of the above, the interviewer asked "have you used dictionaries?" The reason I'm telling the story is, if a lot of your candidates fail to answer your questions, the problem might be in the question.
- no_wizard 3y ago
  I would have halted for a moment and asked something like: organize data for what purpose? At which point I would expect there to be some amount of clarification. As it sounds to me like they're talking about organizing some set of data for lookup, as opposed to organizing data around how it flows through an application, for example
  - aranw 3y ago
    I often find when you are given questions like that no matter how you probe you get a really defensive interviewer who doesn’t want to give away the answer they are expecting. I’ve been in very similar situations with vague questions and I’ve tried to probe for more details and just been met with defensiveness with an attitude that says they expect me to know exactly what they mean
    - d0mine 3y ago
      It is worse: your attempt to clarify the question may be regarded by some interviewers as a sign you are not senior dev (they think [erroneously in my opinion] that "senior" means that you must figure out fine details yourself).
      - no_wizard 3y ago
        This seems odd to me, though its entirely possible my experience thus far hasn't been this. I've been told at the last 3 places I worked that one of the reasons I was hired was how inquisitive I was during the interview stage, asking lots of questions before coming up with solutions I am most definitely not ruling out what you're saying may be common case and I am just very lucky, but I do want to act as a sort of counter point to see if others can weigh in so we can get more of an industry sense around this
        zimpenfish 3y ago
        Anecdatum: my experience has been the same as others - asking questions about the interview questions generally leads to a bad outcome. In 25 years, I reckon it's less than a handful of times where asking questions of the interviewers has actually got a positive response. Hell, even when in jobs, asking questions about projects I'm assigned has sometimes got a negative response...
- pavluha 3y ago
  There are plenty of different types of dictionaries, organizing their data differently.
- skippyboxedhero 3y ago
  I have had that exact experience before. It is fine, you realise the company is toasted and you move on (the place that asked me this had just got rinsed by an offshore consulting firm, ecomm site built with a graph database, comedic stuff).
- maximinus_thrax 3y ago
  > After talking for a bit about pretty much all of the above, the interviewer asked "have you used dictionaries?" Hahaha classic. I was once in an interview (which I failed) and was asked a problem related to some sort of monotonic queue. I wrote the solution and was going through it, when the interviewer asked: 'what CS concept are you using in this solution'? I didn't really understand what he was asking for and in turn I asked for more clarification like 'Are you referring to the data structures I used? Or the technique (it was some sort of greedy)?' 'no'. After a couple of back-and-forth questions and answers, he finally tells me he was looking for me to say 'a state machine'. Seriously?
  - jerf 3y ago
    "I was once in an interview (which I failed)" You didn't fail that interview. They did.
bastardoperator 3y ago
Maybe you're the problem? Looking at your questions, you're asking for opinions, not answers or solutions. Putting people on the spot is bad form in my opinion. If I have a technical challenge I want you to solve, I'll give you the time for that so I can actually see your work and discuss it with you. I've seen people that can talk the talk but not walk the walk and vice versa, they can't articulate well, but their approach and work speaks for itself.
- hintymad 3y ago
  Maybe I am. For a senior enough role, my questions are open ended. I'd expect the candidate to drive conversations and examine alternatives. I ask follow-up questions based on what the candidate mentions. Yeah, such may be indeed their opinions, but at least they should be opinions derived from assumptions, constraints, and fundamentals.
  - darth_avocado 3y ago
    That’s a terrible way to interview. Interviews are high stress, time constrained settings where you’re talking to people you don’t know. If you want thoughtful answers, you need to be precise in what you’re asking. You can’t be like “tell me about yourself” and then complain I didn’t get from the candidate whether steak is their favorite food.
    - sriram_malhar 3y ago
      I don't think that it is a terrible way to interview to all, for a senior position. A generic question like "how do you organise data in this situation" is open-ended enough that it merits followups that the interviewee has to drive. If I was being interviewed, I'd ask "what are the operations you want on the data (random lookup vs scan, append, lifo access etc), what performance guarantees you want on those operations, what persistence and distributed fault-tolerance guarantees if any, etc. It is no different from having to be in front of a client and teasing out what the client really wants. The interviewee has to guide the questions and answer them in a satisfactory way. Reminds of Prof. Manuel Blum (then at Berkeley), whose class project would be "think of a problem and solve it, and you'll be graded on both the quality of the problem and the solution".
convolvatron 3y ago
I have worked in distributed databases. I can't seem to find any host organization that's actually building anything and really cares about those details. I know several other senior systems engineers in the same boat. where are you guys hiding?
- cmrdporcupine 3y ago
  I think there are plenty of companies building stuff like this, and hiring for it. The problem is going to be that the founding engineers have already hammered down (or attempted to) the bulk of the initial interesting engineering problems and what they want is foot soldiers to complete, fix, patch up, and maintain their design. That's ok, because that's what the software engineering profession is, on the whole, for better or for worse. But... the interview process should be clear about that, and the questions reflect that. Rather than "how would you build a high performance distributed database" the questions in those cases need to revolve around identifying problems in a design and how benchmarking, diagnostics, bug fixing, etc. would go. Even at the height of VC fever a couple years ago, being blessed with "Hey, here's some $$, go write a [database|operating system|game engine|programming language|other sexy thing]" doesn't happen really (and probably for good reasons.) So, yeah, GP commenter is a bit disingenuous. There really are only a scant number of jobs which would involve actually being able to do these things, let alone answer how to do them in interview on demand. The jobs that are there are fixing someone else's thing where already they did this stuff. Said interviewer would likely find a candidate who can answer all those questions... and then hire them to go write some microservices in C# for an insurance company, or join an on-call shift diagnosing production problems with an off the shelf database replication process or something. Of course, not telling you what you don't already know, just ranting :-)
pavluha 3y ago
Try to interview more senior people, who still remember their C course in an University, or even know what is assembler. Currently, most graduates (from my experience) don't know how an integer is stored in memory.
- mydogcanpurr 3y ago
  I don't even know what a good answer to that is. Some integers are stored in registers. Some are stored on the stack. Some are stored on the heap. Or you could be asking how integers are represented, with two's complement being the usual way. Did I pass your ambiguous question? Probably not.
skippyboxedhero 3y ago
Do you understand the difference between writing a blog post about something and being able to talk about it in an interview? The majority of work in software is not being an "expert" on any one thing because the one thing that employers require is changing quite frequently. Rather it is being able to build up experience and carrying knowledge forward to new areas. Most technical interviews are conducted in a completely inane way where the goal is to memorize pieces of information. If that was how software worked, the best engineer would just be Google. 95% of people who know enough to build this system would likely be unable to answer your questions because, more than likely, they do not currently work in a job that requires them to use this information every day. This does not mean they do not understand it or are unable to build it. The reason why companies can't find people like this is a combination of not understanding what software development is (and not understanding people, it is really the blind leading the blind but technical people are often as bad as interviewing). You are asking completely wrong questions. You aren't starting from the right place.
- hintymad 3y ago
  > Rather it is being able to build up experience and carrying knowledge forward to new areas. Then I'd expect the candidate could articulate her experience and knowledge in some way, no? Otherwise, how would I know the candidate has the built-up expertise? Of course, I assume we can only have interviews. Otherwise, we can have other means, like mini-project, onsite project, or a writeup. Some candidates do like the alternatives more, and some not. > where the goal is to memorize pieces of information I disagree. The goal is to see if a candidate does have what the candidate herself claims to know. I find it hard to imagine that a candidate claims to be an expert in a field yet couldn't articulate even one thing in depth for hours if not days, let alone 30 minutes of interview time. Note this is not about any specific details, but the general picture and insights that an expert can convey. This is like a PhD oral defense. The candidate talks about the topic that the candidate is familiar with, and the professors dive in on such topics. I don't see what's wrong with that.
  - skippyboxedhero 3y ago
    Candidates aren't defending a PHd. The interviewer is nowhere near competent enough for this. > The goal is to see if a candidate does have what the candidate herself claims to know. The process you are using does not tell you that. You will notice that your explanation is full of things that you expect, not based in an understanding of how reality actually works. And the result is, unsurprisingly, one where you do not understand the output.
nitwit005 3y ago
You seem confident you wouldn't have failed the author of this blog post, but I wouldn't be so certain. Interviews are a high pressure situation where people want an immediate answer, without much of any time to think, let alone research options or run an experiment. You get people's absolute worst possible results.
deleted 3y ago
[deleted]

qznc 3y ago

You find them via their blogs.

bombela 3y ago

Where do you find the jobs where people actually care about engineering? Because real work is mostly careful plumbing without breaking what works, but in interviews you must leetcode. Real systems only work when kept reasonably simple, but system design interviews are all about drawing boxes.

BossingAround 3y ago

I recently interviewed with a company, and the interviewer asked me "how do you organize data." I wasn't sure if they wanted to talk about classes, modules, databases, k-v stores, hashing data and routing distributed requests to the same pods. I asked, and they answered "I mean in general, how do you organize data." After talking for a bit about pretty much all of the above, the interviewer asked "have you used dictionaries?" The reason I'm telling the story is, if a lot of your candidates fail to answer your questions, the problem might be in the question.

no_wizard 3y ago
I would have halted for a moment and asked something like: organize data for what purpose? At which point I would expect there to be some amount of clarification. As it sounds to me like they're talking about organizing some set of data for lookup, as opposed to organizing data around how it flows through an application, for example
- aranw 3y ago
  I often find when you are given questions like that no matter how you probe you get a really defensive interviewer who doesn’t want to give away the answer they are expecting. I’ve been in very similar situations with vague questions and I’ve tried to probe for more details and just been met with defensiveness with an attitude that says they expect me to know exactly what they mean
  - d0mine 3y ago
    It is worse: your attempt to clarify the question may be regarded by some interviewers as a sign you are not senior dev (they think [erroneously in my opinion] that "senior" means that you must figure out fine details yourself).
    - no_wizard 3y ago
      This seems odd to me, though its entirely possible my experience thus far hasn't been this. I've been told at the last 3 places I worked that one of the reasons I was hired was how inquisitive I was during the interview stage, asking lots of questions before coming up with solutions I am most definitely not ruling out what you're saying may be common case and I am just very lucky, but I do want to act as a sort of counter point to see if others can weigh in so we can get more of an industry sense around this
      - zimpenfish 3y ago
        Anecdatum: my experience has been the same as others - asking questions about the interview questions generally leads to a bad outcome. In 25 years, I reckon it's less than a handful of times where asking questions of the interviewers has actually got a positive response. Hell, even when in jobs, asking questions about projects I'm assigned has sometimes got a negative response...
pavluha 3y ago
There are plenty of different types of dictionaries, organizing their data differently.
skippyboxedhero 3y ago
I have had that exact experience before. It is fine, you realise the company is toasted and you move on (the place that asked me this had just got rinsed by an offshore consulting firm, ecomm site built with a graph database, comedic stuff).
maximinus_thrax 3y ago
> After talking for a bit about pretty much all of the above, the interviewer asked "have you used dictionaries?" Hahaha classic. I was once in an interview (which I failed) and was asked a problem related to some sort of monotonic queue. I wrote the solution and was going through it, when the interviewer asked: 'what CS concept are you using in this solution'? I didn't really understand what he was asking for and in turn I asked for more clarification like 'Are you referring to the data structures I used? Or the technique (it was some sort of greedy)?' 'no'. After a couple of back-and-forth questions and answers, he finally tells me he was looking for me to say 'a state machine'. Seriously?
- jerf 3y ago
  "I was once in an interview (which I failed)" You didn't fail that interview. They did.

bastardoperator 3y ago

Maybe you're the problem? Looking at your questions, you're asking for opinions, not answers or solutions. Putting people on the spot is bad form in my opinion. If I have a technical challenge I want you to solve, I'll give you the time for that so I can actually see your work and discuss it with you. I've seen people that can talk the talk but not walk the walk and vice versa, they can't articulate well, but their approach and work speaks for itself.

hintymad 3y ago
Maybe I am. For a senior enough role, my questions are open ended. I'd expect the candidate to drive conversations and examine alternatives. I ask follow-up questions based on what the candidate mentions. Yeah, such may be indeed their opinions, but at least they should be opinions derived from assumptions, constraints, and fundamentals.
- darth_avocado 3y ago
  That’s a terrible way to interview. Interviews are high stress, time constrained settings where you’re talking to people you don’t know. If you want thoughtful answers, you need to be precise in what you’re asking. You can’t be like “tell me about yourself” and then complain I didn’t get from the candidate whether steak is their favorite food.
  - sriram_malhar 3y ago
    I don't think that it is a terrible way to interview to all, for a senior position. A generic question like "how do you organise data in this situation" is open-ended enough that it merits followups that the interviewee has to drive. If I was being interviewed, I'd ask "what are the operations you want on the data (random lookup vs scan, append, lifo access etc), what performance guarantees you want on those operations, what persistence and distributed fault-tolerance guarantees if any, etc. It is no different from having to be in front of a client and teasing out what the client really wants. The interviewee has to guide the questions and answer them in a satisfactory way. Reminds of Prof. Manuel Blum (then at Berkeley), whose class project would be "think of a problem and solve it, and you'll be graded on both the quality of the problem and the solution".

convolvatron 3y ago

I have worked in distributed databases. I can't seem to find any host organization that's actually building anything and really cares about those details. I know several other senior systems engineers in the same boat. where are you guys hiding?

cmrdporcupine 3y ago
I think there are plenty of companies building stuff like this, and hiring for it. The problem is going to be that the founding engineers have already hammered down (or attempted to) the bulk of the initial interesting engineering problems and what they want is foot soldiers to complete, fix, patch up, and maintain their design. That's ok, because that's what the software engineering profession is, on the whole, for better or for worse. But... the interview process should be clear about that, and the questions reflect that. Rather than "how would you build a high performance distributed database" the questions in those cases need to revolve around identifying problems in a design and how benchmarking, diagnostics, bug fixing, etc. would go. Even at the height of VC fever a couple years ago, being blessed with "Hey, here's some $$, go write a [database|operating system|game engine|programming language|other sexy thing]" doesn't happen really (and probably for good reasons.) So, yeah, GP commenter is a bit disingenuous. There really are only a scant number of jobs which would involve actually being able to do these things, let alone answer how to do them in interview on demand. The jobs that are there are fixing someone else's thing where already they did this stuff. Said interviewer would likely find a candidate who can answer all those questions... and then hire them to go write some microservices in C# for an insurance company, or join an on-call shift diagnosing production problems with an off the shelf database replication process or something. Of course, not telling you what you don't already know, just ranting :-)

pavluha 3y ago

Try to interview more senior people, who still remember their C course in an University, or even know what is assembler. Currently, most graduates (from my experience) don't know how an integer is stored in memory.

mydogcanpurr 3y ago
I don't even know what a good answer to that is. Some integers are stored in registers. Some are stored on the stack. Some are stored on the heap. Or you could be asking how integers are represented, with two's complement being the usual way. Did I pass your ambiguous question? Probably not.

skippyboxedhero 3y ago

Do you understand the difference between writing a blog post about something and being able to talk about it in an interview? The majority of work in software is not being an "expert" on any one thing because the one thing that employers require is changing quite frequently. Rather it is being able to build up experience and carrying knowledge forward to new areas. Most technical interviews are conducted in a completely inane way where the goal is to memorize pieces of information. If that was how software worked, the best engineer would just be Google. 95% of people who know enough to build this system would likely be unable to answer your questions because, more than likely, they do not currently work in a job that requires them to use this information every day. This does not mean they do not understand it or are unable to build it. The reason why companies can't find people like this is a combination of not understanding what software development is (and not understanding people, it is really the blind leading the blind but technical people are often as bad as interviewing). You are asking completely wrong questions. You aren't starting from the right place.

hintymad 3y ago
> Rather it is being able to build up experience and carrying knowledge forward to new areas. Then I'd expect the candidate could articulate her experience and knowledge in some way, no? Otherwise, how would I know the candidate has the built-up expertise? Of course, I assume we can only have interviews. Otherwise, we can have other means, like mini-project, onsite project, or a writeup. Some candidates do like the alternatives more, and some not. > where the goal is to memorize pieces of information I disagree. The goal is to see if a candidate does have what the candidate herself claims to know. I find it hard to imagine that a candidate claims to be an expert in a field yet couldn't articulate even one thing in depth for hours if not days, let alone 30 minutes of interview time. Note this is not about any specific details, but the general picture and insights that an expert can convey. This is like a PhD oral defense. The candidate talks about the topic that the candidate is familiar with, and the professors dive in on such topics. I don't see what's wrong with that.
- skippyboxedhero 3y ago
  Candidates aren't defending a PHd. The interviewer is nowhere near competent enough for this. > The goal is to see if a candidate does have what the candidate herself claims to know. The process you are using does not tell you that. You will notice that your explanation is full of things that you expect, not based in an understanding of how reality actually works. And the result is, unsurprisingly, one where you do not understand the output.

nitwit005 3y ago

You seem confident you wouldn't have failed the author of this blog post, but I wouldn't be so certain. Interviews are a high pressure situation where people want an immediate answer, without much of any time to think, let alone research options or run an experiment. You get people's absolute worst possible results.

deleted 3y ago

[deleted]