5 ms·

> While I'm confident that an experienced senior engineer could find a solution over a weekend, What they mean to say is ".. an experienced senior engineer *wi

> While I'm confident that an experienced senior engineer could find a solution over a weekend, What they mean to say is ".. an experienced senior engineer *with my exact skill set* could ..." Senior engineers vary wildly in their skills. Would you expect a senior engineer with experience with (to pick a similar set) Ansible, Rust and AWS to implement this over the weekend using Golang, Terraform and GCP? Obviously not and, just as obviously, you don't need such a wide difference to quickly bleed from "over a weekend" to "over a week". These sorts of bizarrely complex and specific code challenges would at least serve to greatly reduce the number of resumes you have to review. So that is one upside. Unfortunately what that selects for is probably not as obvious as it would seem as many of their candidates would spend that full week on the problem and I wouldn't apply for fear of a workplace culture that values their employees time that little.

gafferongames 3y ago
What I mean to say is that we're hiring Golang engineers. I mean, is that OK? To have a technology stack that you are hiring for (Google Cloud, Golang, Terraform etc), and to look for candidates with experience with that stack, or who would consider learning that stack? ps. This interview question was designed to also be a great question to learn that stack with, but I would not expect a senior programmer would be able to do so in one weekend -- hence I gave programmers as much time as they wanted, in many cases, several months to play around with the stack and see if they enjoyed it in their spare time. Some engineers even asked for some budget to play around with it in Google Cloud and I provided it. Or should we just code each service in Rust or Elixir or Node or F# or whatever language stack a candidate prefers that day?
- eikenberry 3y ago
  It's not the tech they are using but the size of the project. This project requires simple tasks but spread over multiple technologies such that most developers won't have experiences with all (or even many) of them. The key thing here is the time. Having just done a ton of interviewing I can tell you that almost every code challenge has some form of this issue and so you learn to just multiply the time they say by 3-4 (as a rough average). Thus taking ones that say 'a few hours', which means a day or two at most, are feasible but the ones that say 'a day or two' or 'a weekend' are not. Also remember candidate are usually not only applying to your job. They will easily have 3-4 of these going in parallel and with each taking up that much time is untenable.
  - gafferongames 3y ago
    That's a completely fair point
- waynesonfire 3y ago
  > I mean, is that OK? To have a technology stack that you are hiring for You're aware that a senior engineer will take a trivial amount of time to get up-to speed in a programming language paradigm they are familiar with. And, an entry level engineer will have the same learning curve--their core source of inexperience is not the programming language. So, no, the technology stack would be a very low signal for me if I were hiring.
  - gafferongames 3y ago
    That's great, we're trying to filter out engineers who can't learn new things quickly, and who lie about their experience level and the sort of work they can do. People do this a lot, unfortunately. So... we actually ask them to demonstrate to us that they can learn new stacks quickly. That's the point of the question. If somebody doesn't know Golang or the stack directly, that's awesome. They can learn it quickly and show us.
    - cryptonector 3y ago
      > That's great, we're trying to filter out engineers who can't learn new things quickly, and who lie about their experience level and the sort of work they can do. People do this a lot, unfortunately. I feel for you. It does happen a lot. A portfolio matters. References matter. But it's really nice that you give unknowns a chance too, and to do that you really do need a filter. And it's a fairly trivial filter, since you're asking the applicant to build a proxy. I don't think you'll get too many applicants who don't already know the stack though because... think it of from their perspective: if someone is applying who knows that stack already, then they can beat the others to the punch, so taking 2x as long to learn the stack is likely to be a waste of time. I'm not sure how to make this "fairer" so that you really do measure learning time rather than comfort level. Your filter is likely better at finding experienced Golang devs than experienced devs who can learn Golang quickly. In the short term this is probably very good for you, but the latter are probably better in the long run because people who learn quickly and well are more valuable than people who know a tool well.
AnthonyMouse 3y ago
> These sorts of bizarrely complex and specific code challenges would at least serve to greatly reduce the number of resumes you have to review. So that is one upside. Unfortunately what that selects for is probably not as obvious as it would seem as many of their candidates would spend that full week on the problem and I wouldn't apply for fear of a workplace culture that values their employees time that little. It also adversely selects for something else. In general it doesn't matter what specific technologies someone has used in the past, if they do the same thing. When you hire someone their first few weeks/months are going to be less productive because they have to learn your internal systems and even if you use AWS and they've used AWS, that doesn't mean it will be the same subset of AWS. Whereas by the time they've been there a year they'll have learned the relevant systems regardless of whether it's cosmetically different than the one they used before. Selecting for the exact set of technologies your company currently uses is going to filter out attractive candidates. Worse, it selects for a monoculture. The last thing you want is to spend two years building something in Golang that exists in the Rust standard library because none of your engineers are aware of that. And it's simple enough to ask the same question without explicitly specifying specific technologies.
- deleted 3y ago
  [deleted]
- gafferongames 3y ago
  If somebody applied, and had strong opinions about a particular stack or approach, I would offer that they could choose to use their existing stack to solve the problem, if they wish.
  - cryptonector 3y ago
    Do you review their submissions? Do you use the problem to probe the applicant's thinking? Your blog's focus on the details of the task make it sound like you're more interested in using the task as a filter than anything else, but surely there's more to it.
    - 3y ago
cryptonector 3y ago
Besides, if you're interviewing senior engineers their CVs/portfolios probably speak to their ability to implement, and what you're really after is their ability to reason. Here the key elements of the problem are: the need to scale horizontally and the need to cost out the result -- everything about Golang and Terraform is just implementation details. TFA seems much more interested in fetch-a-rock as a filter than the specific problem as a way to gauge how the applicant thinks.
- gafferongames 3y ago
  You might be underestimating how much thinking is required on the part of the applicant to fetch-a-rock at 60M requests per-second.
  - cryptonector 3y ago
    In your setup the applicant is asked to build a proxy, and that proxy should scale horizontally just fine. When you get to C10M you start thinking about unikernel-style or user-land networking stack style solutions, but that's not going to be a weekend project, and the conversation will get interesting. If you really expect a weekend project here then the answer to scaling the proxy is bound to be: horizontally (meaning: not that interesting).
    - gafferongames 3y ago
      I would argue that both vertical AND horizontal scaling are required, because the winning solution in the end at the absolute highest level of this question, is one that gets the work done for significantly less $$$ than my golang implementation, in whatever language, stack, cloud or bare metal you deem appropriate.
      - cryptonector 3y ago
        The vertical component is about efficiency. It's very hard to get past a certain point with a general purpose OS and Golang. You'll need NCPU threads/processes and no context switching, so you're definitely talking about: - unikernel or user-land NIC driver and network stack - async/await or continuation passing style to reduce the per-client/request memory footprint I'm not applying though, but I'm saying that "Golang + super-efficient" is definitely not something I'd assign as an interview weekend project -- that's something to be discussed in detail during the interview, and I'd expect no more than a horizontally scalable solution over a weekend. Getting the vertical efficiency tuning done will take a lot of time and cannot be expected to be a weekend or week project by an unpaid applicant, but you can expect the applicant to tell you how they'd go about it.
cchance 3y ago
Honestly this entire article was a red flag, and the fact that thats a quesiton they expect everyone to go home and build out for an INTERVIEW is a massive MASSIVE red flag, like are you kidding me you want a prospective employee to basically give you a solution for one of the main challenges of your company lol.
- gafferongames 3y ago
  Strong no hire

gafferongames 3y ago

What I mean to say is that we're hiring Golang engineers. I mean, is that OK? To have a technology stack that you are hiring for (Google Cloud, Golang, Terraform etc), and to look for candidates with experience with that stack, or who would consider learning that stack? ps. This interview question was designed to also be a great question to learn that stack with, but I would not expect a senior programmer would be able to do so in one weekend -- hence I gave programmers as much time as they wanted, in many cases, several months to play around with the stack and see if they enjoyed it in their spare time. Some engineers even asked for some budget to play around with it in Google Cloud and I provided it. Or should we just code each service in Rust or Elixir or Node or F# or whatever language stack a candidate prefers that day?

eikenberry 3y ago
It's not the tech they are using but the size of the project. This project requires simple tasks but spread over multiple technologies such that most developers won't have experiences with all (or even many) of them. The key thing here is the time. Having just done a ton of interviewing I can tell you that almost every code challenge has some form of this issue and so you learn to just multiply the time they say by 3-4 (as a rough average). Thus taking ones that say 'a few hours', which means a day or two at most, are feasible but the ones that say 'a day or two' or 'a weekend' are not. Also remember candidate are usually not only applying to your job. They will easily have 3-4 of these going in parallel and with each taking up that much time is untenable.
- gafferongames 3y ago
  That's a completely fair point
waynesonfire 3y ago
> I mean, is that OK? To have a technology stack that you are hiring for You're aware that a senior engineer will take a trivial amount of time to get up-to speed in a programming language paradigm they are familiar with. And, an entry level engineer will have the same learning curve--their core source of inexperience is not the programming language. So, no, the technology stack would be a very low signal for me if I were hiring.
- gafferongames 3y ago
  That's great, we're trying to filter out engineers who can't learn new things quickly, and who lie about their experience level and the sort of work they can do. People do this a lot, unfortunately. So... we actually ask them to demonstrate to us that they can learn new stacks quickly. That's the point of the question. If somebody doesn't know Golang or the stack directly, that's awesome. They can learn it quickly and show us.
  - cryptonector 3y ago
    > That's great, we're trying to filter out engineers who can't learn new things quickly, and who lie about their experience level and the sort of work they can do. People do this a lot, unfortunately. I feel for you. It does happen a lot. A portfolio matters. References matter. But it's really nice that you give unknowns a chance too, and to do that you really do need a filter. And it's a fairly trivial filter, since you're asking the applicant to build a proxy. I don't think you'll get too many applicants who don't already know the stack though because... think it of from their perspective: if someone is applying who knows that stack already, then they can beat the others to the punch, so taking 2x as long to learn the stack is likely to be a waste of time. I'm not sure how to make this "fairer" so that you really do measure learning time rather than comfort level. Your filter is likely better at finding experienced Golang devs than experienced devs who can learn Golang quickly. In the short term this is probably very good for you, but the latter are probably better in the long run because people who learn quickly and well are more valuable than people who know a tool well.

AnthonyMouse 3y ago

> These sorts of bizarrely complex and specific code challenges would at least serve to greatly reduce the number of resumes you have to review. So that is one upside. Unfortunately what that selects for is probably not as obvious as it would seem as many of their candidates would spend that full week on the problem and I wouldn't apply for fear of a workplace culture that values their employees time that little. It also adversely selects for something else. In general it doesn't matter what specific technologies someone has used in the past, if they do the same thing. When you hire someone their first few weeks/months are going to be less productive because they have to learn your internal systems and even if you use AWS and they've used AWS, that doesn't mean it will be the same subset of AWS. Whereas by the time they've been there a year they'll have learned the relevant systems regardless of whether it's cosmetically different than the one they used before. Selecting for the exact set of technologies your company currently uses is going to filter out attractive candidates. Worse, it selects for a monoculture. The last thing you want is to spend two years building something in Golang that exists in the Rust standard library because none of your engineers are aware of that. And it's simple enough to ask the same question without explicitly specifying specific technologies.

deleted 3y ago
[deleted]
gafferongames 3y ago
If somebody applied, and had strong opinions about a particular stack or approach, I would offer that they could choose to use their existing stack to solve the problem, if they wish.
- cryptonector 3y ago
  Do you review their submissions? Do you use the problem to probe the applicant's thinking? Your blog's focus on the details of the task make it sound like you're more interested in using the task as a filter than anything else, but surely there's more to it.
  - 3y ago

cryptonector 3y ago

Besides, if you're interviewing senior engineers their CVs/portfolios probably speak to their ability to implement, and what you're really after is their ability to reason. Here the key elements of the problem are: the need to scale horizontally and the need to cost out the result -- everything about Golang and Terraform is just implementation details. TFA seems much more interested in fetch-a-rock as a filter than the specific problem as a way to gauge how the applicant thinks.

gafferongames 3y ago
You might be underestimating how much thinking is required on the part of the applicant to fetch-a-rock at 60M requests per-second.
- cryptonector 3y ago
  In your setup the applicant is asked to build a proxy, and that proxy should scale horizontally just fine. When you get to C10M you start thinking about unikernel-style or user-land networking stack style solutions, but that's not going to be a weekend project, and the conversation will get interesting. If you really expect a weekend project here then the answer to scaling the proxy is bound to be: horizontally (meaning: not that interesting).
  - gafferongames 3y ago
    I would argue that both vertical AND horizontal scaling are required, because the winning solution in the end at the absolute highest level of this question, is one that gets the work done for significantly less $$$ than my golang implementation, in whatever language, stack, cloud or bare metal you deem appropriate.
    - cryptonector 3y ago
      The vertical component is about efficiency. It's very hard to get past a certain point with a general purpose OS and Golang. You'll need NCPU threads/processes and no context switching, so you're definitely talking about: - unikernel or user-land NIC driver and network stack - async/await or continuation passing style to reduce the per-client/request memory footprint I'm not applying though, but I'm saying that "Golang + super-efficient" is definitely not something I'd assign as an interview weekend project -- that's something to be discussed in detail during the interview, and I'd expect no more than a horizontally scalable solution over a weekend. Getting the vertical efficiency tuning done will take a lot of time and cannot be expected to be a weekend or week project by an unpaid applicant, but you can expect the applicant to tell you how they'd go about it.

cchance 3y ago

Honestly this entire article was a red flag, and the fact that thats a quesiton they expect everyone to go home and build out for an INTERVIEW is a massive MASSIVE red flag, like are you kidding me you want a prospective employee to basically give you a solution for one of the main challenges of your company lol.

gafferongames 3y ago
Strong no hire