9 ms·

I like this... in theory. But I don't think I'm allowed to give you a system diagram and detailed explanation of systems I've worked on, because obviously they'

I like this... in theory. But I don't think I'm allowed to give you a system diagram and detailed explanation of systems I've worked on, because obviously they're proprietary. That seems like a problem. How do your candidates normally work around this? Does everyone just talk about hobby or open-source systems they've worked on?

ok_dad 4y ago
The only really important details to keep secret are business-specific algorithms, so talking about the overall architecture of a system is usually okay, without those details. If you're building very special architectures where that actually is the business detail, then I suppose you need to figure out how to talk about something else similar or something. I just talk about the systems I worked on in generalities, so I would talk about how we get some telemetry from some devices, process it, store it, and etc., but only the mechanical details and not the meaty algorithmic parts (we use a JSON API to submit the telemetry, then we store it in a table, then we take that data in our other component and process it through our "algorithm", etc.).
- sroussey 4y ago
  Your ideas of what should be secret or not may not at all be what the candidate agreed to in their employment contract and NDA.
  - cyc116 4y ago
    Exactly. Not to mention high level business logic could be inferred from system architecture. Sure sounds like a good way to perform recon on a target company.
    - rednerrus 4y ago
      We say specifically “diagram a technical system you understand well.” It’s going to be challenging to work in technology if the only technical system you understand well is the one you’re currently working on and only the parts that are under NDA.
      - cauthon 4y ago
        I have the opposite question from other people in the thread - this seems pretty open-ended. Just as an example, I do bioinformatics work, and "technical system" for someone I'm interviewing (as I understand your question) might encompass any of: - a workflow manager: how it works under the hood, common patterns for modularity/reuse or configuration - a specific pipeline: the stages of cleaning and quantifying raw data, the rationale for various stages and the tools chosen to execute them (including how to benchmark various methods against each other) - a specific third-party tool: theoretical details of the algorithm, practical considerations for when to apply one over another - familiarity with common general-purpose packages or APIs (e.g. pandas, numpy, scikit-learn) - QC: common metrics for QC'ing various experimental data, how to decide if data is "good enough", how to troubleshoot biological vs technical (lab) vs technical (computational) sources of error - biology: technical details of an experiment, the technologies generating the data, or the underlying biology - data analysis: how to choose and make relevant figures, any of many data-science/ML topics (e.g. clustering), connecting data to relevant domain questions - devops: data storage/management on HPC or cloud, HPC job schedulers, any of many cloud topics (e.g. IAC, setting up a database or cloud execution of a pipeline) I'm more curious about how you guide a candidate towards selecting a system that gives you the most relevant perspective on their thinking and skillset - do you provide any suggestions, or are there typically pretty evident choices based on the role?
        rednerrus 4y ago
        For our more junior roles, we leave it more open ended because candidates may not have a similar knowledge bases to the panel. For more senior roles we advise that if they pick technologies that are pertinent to the role and likely to be understood by senior/lead/staff engineers, the process will go better for candidates and panel members. If I were in your position, one of understanding many complex systems, I would advise you to pick where you felt both strong and the panel was likely to have some entry point into.
      - ikiris 4y ago
        I literally can't diagram 90% of the stacks I've worked on in the last 8 years because they're highly NDA.
- deleted 4y ago
  [deleted]
rednerrus 4y ago
We call this out in our pre-interview handout. If you've got an NDA, choose something that isn't included in your NDA or figure out a way to describe it at a high level. We caution candidates that if we dig in and we run into something blocked by the NDA it creates challenges with our evaluation. It doesn't necessarily have to be something that you're working on. It just has to be something that you understand well enough to describe the design of and how things are inter-connected.
- operatingthetan 4y ago
  >We caution candidates that if we dig in and we run into something blocked by the NDA it creates challenges with our evaluation. So you're penalizing those under NDA and unecessarily limiting your candidate pool.
  - rednerrus 4y ago
    There’s no penalty. There are plenty of technologies that you’re working with that aren’t under NDA. We’re not asking you to diagram what you’re working on, we asking you to diagram something technical that you understand well.
    - Salgat 4y ago
      Being able to use something you've intimately worked on as part of your job is a huge advantage since you're so familiar with it.
      - rednerrus 4y ago
        There have to be parts of what you work on that are not covered by NDA. Solving problems like this would be indicative of someone we would want on our team.
        kridsdale1 4y ago
        When I worked at Apple they made it abundantly clear that EVERYTHING was under NDA. Even things like “what version of the standard library do you use”. Or “what are computers for”. And that any minuscule transgression of the NDA would result in me Being immediately deported, divorced, and likely put in front of a firing squad.
eschneider 4y ago
98% of what you'd be inclined to diagram out and explain for a system design interview isn't remotely proprietary. People are generally looking for a high-level overview of a system you've worked on to see a) that you've really worked on it and b) you can explain it such that it makes sense. Sure, folks will ask you to drill down here and there for more detailed info, but if someone pokes on something where the details really _ARE_ proprietary, it's fine to just say that and offer to drill down somewhere else.
- toolz 4y ago
  I'm not a lawyer, but wouldn't anything you or anyone else developed on company time at another company be owned by that company and thus proprietary? In practice I can't imagine the spirit of any law would be violated by doing an architecture diagram, but it seems likely (to me, at least) the word of the law would be violated.
  - jfarina 4y ago
    How could you ever interview anywhere if you couldn't provide details of the projects that you worked on?
    - deleted 4y ago
      [deleted]
    - lelandfe 4y ago
      I have a relative who works on very secretive stuff for big companies as a chemist. His interviews are humorous. Those leading the interviews can't tell him what he'll be working on, and he can't tell them what he's done in the past. The questions tend to be very theoretical, instead.
    - hirvi74 4y ago
      Leetcode questions? (Don't kill me, you all. I'm joking).
  - eschneider 4y ago
hadlock 4y ago
System diagram of a CRUD SAAS isn't going to be a critical business secret in the same way it might be for a company that does stream processing of marketing/advertising data off of twitter or video/radar analytics. Or a real-time fintech trading company. Many companies are a Python/Django monolith on Postgres with redis & || memcached. That system diagram isn't going to let you disrupt the online medical services industry but it's what a lot of them use.
yieldcrv 4y ago
At this point I just build on top of other take-home projects I've done and pretend like its my portfolio.
- Yoric 4y ago
  That, I must admit, is extremely tempting.
mlhpdx 4y ago
Show me a system you’ve worked on that wasn’t put together from knowledge from previous education and jobs (yours and others), documentation from vendors, stack overflow, code generators , etc. I would hate to be a stakeholder in a company whose value depends on employees not talking about and reusing the incremental skills and knowledge gained while working there.
- bqmjjx0kac 4y ago
  This is a pretty common situation in the US for folks who work on government contracts.
  - alistairSH 4y ago
    Yup. I’m outside DC. Somewhere near half my software engineer (and adjacent fields) friends are under contract to either the military or NSA/CIA. Most of them with TS/poly clearances. Ask one of the what they’re working on and the most you’ll get is something like “dig data on platform X”. If platform X is something public, they could talk about the platform. But, talking about any project specifics, implementation details, etc would be a felony.
    - fatneckbeard 4y ago
      [flagged]
- tourist2d 4y ago
  NDAs don't care where you sourced your knowledge from.
  - deleted 4y ago
    [deleted]
- 8note 4y ago
  Im pretty sure I learned a full set of new skills at my current job. I don't think knowledge on any particular api is particularly useful compare to the more generic "can you read docs and use apis" skill, which is directly applicable from something like high level hard ware work making an ftdi USB chip go brrr to making dynamoDB work nicely, but I definitely didn't know anything about dynamodb before doing a server/web job Async distributed systems and FPGA vhdl have some overlap in how you build things, but doing an interview based on my old FPGA skills likely won't show much of my servers and databases systems design skills
l8again 4y ago
In my first semester as a freshman in an American university, I took a quiz in the Calculus class that had a question based on American football. Having never played the sports or even knowing the rules all that well, I aced the exam. Yes, it is not that hard to keep the proprietary business logic separate from the system design.
flashgordon 4y ago
See the reason ops thing works is this can even be applied to something you haven't worked on but purely as a passion project on the side. If you are Saying "oh what if I don't have side projects" you do realize people spend/waste 3+ months going these interviews + tons of mock interviews with companies they have no intention of joining. What does it say about if you are telling me you'd rather waste valuable personal time that way instead of building something fun? I'm fact I'd even say that even if you haven't built a side project, I'd still use op's technique and ask the candidate to talk about and white board a system they would like to build and then dive deeper from there. Heck I'd even tell this to them before an interview!
- CardenB 4y ago
  I’ll tell you precisely what it says about me: I put in 60 hour weeks at my current job, except for a month before I change jobs, where I put in an additional 2 hours/night to prep for interviews, because that’s what will benefit me for every other company. I care about my career and my family, so I don’t have the luxury of spending it on passion projects outside of work. I’d rather spend the precious little time I have left on my real hobbies and family/friends. It doesn’t communicate that I’m a bad engineer. On the contrary it communicates that I give my all at work.
  - flashgordon 4y ago
    But I never said you had to spend all your life on passion projects. How is leet code fair game but asking how they would build something they would to even if they never did or do somehow an indication that I think you are a bad engineer? Because I see today's state of system design question interviewing pretty much like leetcode no (asking this as a faang interviewer)? Having said that what you said about spending 60 hours at work tells others you will be a much better employee. Which is probably the signal they are looking for?
    - djur 4y ago
      Leetcode isn't fair game either. It's a despicable practice.
joenot443 4y ago
What is the vector you’re concerned about here? You sketch out a diagram so well that the company you’re interviewing with takes a photo and secretly integrates it into their stack? Then your past employer catches wind of this, realizes you were the only person in the world to understand this system, so they go after you for damages? Sorry man, I don’t see it. I understand that you see your past work as “proprietary”, but in the real world the vast, vast majority of us do not work on systems shrouded in such secrecy. There’s nothing interesting or proprietary about CRUD infrastructure and I genuinely can’t think of a situation where you’d be exposing yourself to any real risk by explaining it to a new employer.

ok_dad 4y ago

The only really important details to keep secret are business-specific algorithms, so talking about the overall architecture of a system is usually okay, without those details. If you're building very special architectures where that actually is the business detail, then I suppose you need to figure out how to talk about something else similar or something. I just talk about the systems I worked on in generalities, so I would talk about how we get some telemetry from some devices, process it, store it, and etc., but only the mechanical details and not the meaty algorithmic parts (we use a JSON API to submit the telemetry, then we store it in a table, then we take that data in our other component and process it through our "algorithm", etc.).

sroussey 4y ago
Your ideas of what should be secret or not may not at all be what the candidate agreed to in their employment contract and NDA.
- cyc116 4y ago
  Exactly. Not to mention high level business logic could be inferred from system architecture. Sure sounds like a good way to perform recon on a target company.
  - rednerrus 4y ago
    We say specifically “diagram a technical system you understand well.” It’s going to be challenging to work in technology if the only technical system you understand well is the one you’re currently working on and only the parts that are under NDA.
    - cauthon 4y ago
      I have the opposite question from other people in the thread - this seems pretty open-ended. Just as an example, I do bioinformatics work, and "technical system" for someone I'm interviewing (as I understand your question) might encompass any of: - a workflow manager: how it works under the hood, common patterns for modularity/reuse or configuration - a specific pipeline: the stages of cleaning and quantifying raw data, the rationale for various stages and the tools chosen to execute them (including how to benchmark various methods against each other) - a specific third-party tool: theoretical details of the algorithm, practical considerations for when to apply one over another - familiarity with common general-purpose packages or APIs (e.g. pandas, numpy, scikit-learn) - QC: common metrics for QC'ing various experimental data, how to decide if data is "good enough", how to troubleshoot biological vs technical (lab) vs technical (computational) sources of error - biology: technical details of an experiment, the technologies generating the data, or the underlying biology - data analysis: how to choose and make relevant figures, any of many data-science/ML topics (e.g. clustering), connecting data to relevant domain questions - devops: data storage/management on HPC or cloud, HPC job schedulers, any of many cloud topics (e.g. IAC, setting up a database or cloud execution of a pipeline) I'm more curious about how you guide a candidate towards selecting a system that gives you the most relevant perspective on their thinking and skillset - do you provide any suggestions, or are there typically pretty evident choices based on the role?
      - rednerrus 4y ago
        For our more junior roles, we leave it more open ended because candidates may not have a similar knowledge bases to the panel. For more senior roles we advise that if they pick technologies that are pertinent to the role and likely to be understood by senior/lead/staff engineers, the process will go better for candidates and panel members. If I were in your position, one of understanding many complex systems, I would advise you to pick where you felt both strong and the panel was likely to have some entry point into.
    - ikiris 4y ago
      I literally can't diagram 90% of the stacks I've worked on in the last 8 years because they're highly NDA.
deleted 4y ago
[deleted]

rednerrus 4y ago

We call this out in our pre-interview handout. If you've got an NDA, choose something that isn't included in your NDA or figure out a way to describe it at a high level. We caution candidates that if we dig in and we run into something blocked by the NDA it creates challenges with our evaluation. It doesn't necessarily have to be something that you're working on. It just has to be something that you understand well enough to describe the design of and how things are inter-connected.

operatingthetan 4y ago
>We caution candidates that if we dig in and we run into something blocked by the NDA it creates challenges with our evaluation. So you're penalizing those under NDA and unecessarily limiting your candidate pool.
- rednerrus 4y ago
  There’s no penalty. There are plenty of technologies that you’re working with that aren’t under NDA. We’re not asking you to diagram what you’re working on, we asking you to diagram something technical that you understand well.
  - Salgat 4y ago
    Being able to use something you've intimately worked on as part of your job is a huge advantage since you're so familiar with it.
    - rednerrus 4y ago
      There have to be parts of what you work on that are not covered by NDA. Solving problems like this would be indicative of someone we would want on our team.
      - kridsdale1 4y ago
        When I worked at Apple they made it abundantly clear that EVERYTHING was under NDA. Even things like “what version of the standard library do you use”. Or “what are computers for”. And that any minuscule transgression of the NDA would result in me Being immediately deported, divorced, and likely put in front of a firing squad.

eschneider 4y ago

98% of what you'd be inclined to diagram out and explain for a system design interview isn't remotely proprietary. People are generally looking for a high-level overview of a system you've worked on to see a) that you've really worked on it and b) you can explain it such that it makes sense. Sure, folks will ask you to drill down here and there for more detailed info, but if someone pokes on something where the details really _ARE_ proprietary, it's fine to just say that and offer to drill down somewhere else.

toolz 4y ago
I'm not a lawyer, but wouldn't anything you or anyone else developed on company time at another company be owned by that company and thus proprietary? In practice I can't imagine the spirit of any law would be violated by doing an architecture diagram, but it seems likely (to me, at least) the word of the law would be violated.
- jfarina 4y ago
  How could you ever interview anywhere if you couldn't provide details of the projects that you worked on?
  - deleted 4y ago
    [deleted]
  - lelandfe 4y ago
    I have a relative who works on very secretive stuff for big companies as a chemist. His interviews are humorous. Those leading the interviews can't tell him what he'll be working on, and he can't tell them what he's done in the past. The questions tend to be very theoretical, instead.
  - hirvi74 4y ago
    Leetcode questions? (Don't kill me, you all. I'm joking).
- eschneider 4y ago

hadlock 4y ago

System diagram of a CRUD SAAS isn't going to be a critical business secret in the same way it might be for a company that does stream processing of marketing/advertising data off of twitter or video/radar analytics. Or a real-time fintech trading company. Many companies are a Python/Django monolith on Postgres with redis & || memcached. That system diagram isn't going to let you disrupt the online medical services industry but it's what a lot of them use.

yieldcrv 4y ago

At this point I just build on top of other take-home projects I've done and pretend like its my portfolio.

Yoric 4y ago
That, I must admit, is extremely tempting.

mlhpdx 4y ago

Show me a system you’ve worked on that wasn’t put together from knowledge from previous education and jobs (yours and others), documentation from vendors, stack overflow, code generators , etc. I would hate to be a stakeholder in a company whose value depends on employees not talking about and reusing the incremental skills and knowledge gained while working there.

bqmjjx0kac 4y ago
This is a pretty common situation in the US for folks who work on government contracts.
- alistairSH 4y ago
  Yup. I’m outside DC. Somewhere near half my software engineer (and adjacent fields) friends are under contract to either the military or NSA/CIA. Most of them with TS/poly clearances. Ask one of the what they’re working on and the most you’ll get is something like “dig data on platform X”. If platform X is something public, they could talk about the platform. But, talking about any project specifics, implementation details, etc would be a felony.
  - fatneckbeard 4y ago
    [flagged]
tourist2d 4y ago
NDAs don't care where you sourced your knowledge from.
- deleted 4y ago
  [deleted]
8note 4y ago
Im pretty sure I learned a full set of new skills at my current job. I don't think knowledge on any particular api is particularly useful compare to the more generic "can you read docs and use apis" skill, which is directly applicable from something like high level hard ware work making an ftdi USB chip go brrr to making dynamoDB work nicely, but I definitely didn't know anything about dynamodb before doing a server/web job Async distributed systems and FPGA vhdl have some overlap in how you build things, but doing an interview based on my old FPGA skills likely won't show much of my servers and databases systems design skills

l8again 4y ago

In my first semester as a freshman in an American university, I took a quiz in the Calculus class that had a question based on American football. Having never played the sports or even knowing the rules all that well, I aced the exam. Yes, it is not that hard to keep the proprietary business logic separate from the system design.

flashgordon 4y ago

See the reason ops thing works is this can even be applied to something you haven't worked on but purely as a passion project on the side. If you are Saying "oh what if I don't have side projects" you do realize people spend/waste 3+ months going these interviews + tons of mock interviews with companies they have no intention of joining. What does it say about if you are telling me you'd rather waste valuable personal time that way instead of building something fun? I'm fact I'd even say that even if you haven't built a side project, I'd still use op's technique and ask the candidate to talk about and white board a system they would like to build and then dive deeper from there. Heck I'd even tell this to them before an interview!

CardenB 4y ago
I’ll tell you precisely what it says about me: I put in 60 hour weeks at my current job, except for a month before I change jobs, where I put in an additional 2 hours/night to prep for interviews, because that’s what will benefit me for every other company. I care about my career and my family, so I don’t have the luxury of spending it on passion projects outside of work. I’d rather spend the precious little time I have left on my real hobbies and family/friends. It doesn’t communicate that I’m a bad engineer. On the contrary it communicates that I give my all at work.
- flashgordon 4y ago
  But I never said you had to spend all your life on passion projects. How is leet code fair game but asking how they would build something they would to even if they never did or do somehow an indication that I think you are a bad engineer? Because I see today's state of system design question interviewing pretty much like leetcode no (asking this as a faang interviewer)? Having said that what you said about spending 60 hours at work tells others you will be a much better employee. Which is probably the signal they are looking for?
  - djur 4y ago
    Leetcode isn't fair game either. It's a despicable practice.

joenot443 4y ago

What is the vector you’re concerned about here? You sketch out a diagram so well that the company you’re interviewing with takes a photo and secretly integrates it into their stack? Then your past employer catches wind of this, realizes you were the only person in the world to understand this system, so they go after you for damages? Sorry man, I don’t see it. I understand that you see your past work as “proprietary”, but in the real world the vast, vast majority of us do not work on systems shrouded in such secrecy. There’s nothing interesting or proprietary about CRUD infrastructure and I genuinely can’t think of a situation where you’d be exposing yourself to any real risk by explaining it to a new employer.