7 ms·

I completed that exercise along with a Nomad one w ELK stack. And submitted it. And never got a response ever besides we will look at it ASAP. (3 months ago.).

I completed that exercise along with a Nomad one w ELK stack. And submitted it. And never got a response ever besides we will look at it ASAP. (3 months ago.). It was billed as a "2 hour" exercise, but to even get close to that you would need to be completely fluent with Fly, Nomad, ELK, Wireguard, Linux networking stack. I thought it was a great take home but you would at least expect a response for completing it successfully.

sosodev 4y ago
That's even worse than my experience https://news.ycombinator.com/item?id=34523227 https://news.ycombinator.com/item?id=34523227 I'm pretty skeptical of these hiring challenges at this point. It seems that they often require more time than the traditional model and provide worse outcomes for the candidates. For me it was particularly frustrating because I can typically write off the traditional model as irrational but I was drawn to fly.io because they make it seem so much better than that.
- whiplash451 4y ago
  If I were looking for a job, I would trade the usual stupid 1h leetcode interview for this take-home in an eyeblink. This one is much more time-consuming _up front_, but at least it gives you a sense of the job to be done _and_ doing the exercise lets you think through whether you really want that job after all.
  - paxys 4y ago
    Well in that 1 hour leetcode session at least the interviewer is wasting the same amount of time as you. In this case you can spend all weekend working on the exercise and never even get a response.
    - deleted 4y ago
      [deleted]
    - whiplash451 4y ago
      This is an adversarial perspective of interviewing that does not get you anywhere because indeed, your time is always more precious than any company’s time. By 1h leetcode, I meant the whole yak shaving of classical interviewing, which costs candidate many hours actually (in interview time and prep time).
  - jb3689 4y ago
    A take home without actual feedback is a complete waste of everyone’s time. Who cares if you get a taste for the job in the process. You spend a weekend, the company spends 10 minutes.
    - tptacek 4y ago
      That's an important knock against processes that do both interviews and WSTs ("take homes"). We don't: we exclusively use WSTs. We calibrate the amount of time our WSTs should take against the amount of time a typical interview process takes. There's a lot of stuff we can't do --- like expansive programming tests, or, for that matter, sharing our answer rubrics --- that we wish we could, precisely because we're fitting the whole process into a tight time budget. We don't time our WSTs; it adds cortisol to the process that defeats some of the purpose of simulating what actually working here is like. So, it is the case that people can end up blowing way past our expected time budget working on things. We're not going to stop people from doing that; it would be hard to do, and we're also excited to get candidates who teach themselves stuff as they go. We're up front about this. From my vantage point, this process is strictly better than conventional interviews. It demands the same amount of time from candidates, but allows the candidates to pick where and when to spend that time. There is a lot of understandable enmity built up against "take homes" because firms have added them to their existing interview loop, so they become just another hurdle candidates have to clear before running through the same interviews they always have. With respect to feedback: at some point in our process, we do start giving feedback. It has not been my experience that it is as appreciated by candidates as people on message boards seem to think it is. We've gotten to the point where we try to do a decent job of explaining what the best submissions have in common, and leave it at that. This kind of feedback is more than I'd ever come to expect from conventional interviews, so I'm at peace with it. Finally, at to again be candid: we got a huge flood of candidates for several roles, and while for the most part we kept up and gave timely responses to people who submitted challenge responses, there have definitely been ticket mishaps that caused people not to get responses --- in other words, we've ghosted people. I fucking hated getting ghosted when I was applying for jobs and am mortified by the fact that we did it too. What I can say there is: we've never done that deliberately, and when we've caught that happening, we've gotten detailed feedback out quickly. So if the premise of your comment is: "submitting a code challenge and then never hearing anything back at all, even so much as a pass/fail, is bullshit", then yes, I agree, it's total bullshit. Not OK at all. (With respect to the coding challenges we're talking about today: the public ones aren't a part of our hiring process at all! They're just there for people who are interested in them. We do have a related challenge Kyle and Ben worked on, as a post-L3 leveler. It's not a small challenge. But: any candidate that gets that challenge already knows they're getting an offer from us, so we're comfortable making it ambitious).
andrewfromx 4y ago
hehe yes the Nomad one w ELK stack took me longer than 2 hours for sure but was a lot of fun! I'll bet the fly people are just swamped. I loved the language in their email though. I've been on so many 1 hour live coding interviews where I can't shine like I want to shine this approach really made me go yes! A new way to interview.
- tptacek 4y ago
  We are swamped. We kept up as long as we could, and now we're making some changes: 1. We have someone running infra now that also owns infra hiring, instead of me owning it. Infra is an ultra-important role here and we're still hiring for it. 2. I'm still on the hook for platform hiring, and we've paused it for a couple months (if you submitted a challenge, we're still moving forward! but we're not taking new applications). I know people on HN don't know these details, but knowing it myself makes it especially goofy to see people calling these challenges an elaborate scheme to recruit people for platform development (which is where these challenges apply).
tptacek 4y ago
I don't have a better response for this than that we got a huge flood of candidates for that role, are doing our best to keep up, and have done an imperfect job. We're a small company. You're more than welcome to reach out directly to find out what happened with your application (it's entirely possible that what happened is that the ticket got buried somewhere, and that we'd have been excited to hire you had we not screwed something up on our end). As for the scope of the challenge: that's deliberate, and we're up front about it. If the challenge is going to take you many hours to complete, and doing that work isn't something that lights you up, then it's the wrong role for you. There are candidates that breeze through it, and there are candidates who wrestle with it for a couple evenings just because they're nerdsniped and enjoy doing that stuff. Both those kinds of candidates are super interesting to us! But we're very comfortable with the idea that the job (in this case: infra engineering) isn't a perfect fit for everybody.
- specialp 4y ago
  I really liked the take home. It is just that if you have someone doing this, you are having that person commit hours of their time to interviewing without the same investment on your end. So if you do not have the means or time to process these submissions then it is not fair to have someone spend their time on it. I think your hiring process is great otherwise. And yes if someone isn't very familiar with every intersection of stacks there it will take them more time. That is fine too. It shows the person can learn on the spot.
  - tptacek 4y ago
    I agree. We came to that conclusion about platform just a week or so ago and have paused hiring because we can't meet the commitment we've made about responsiveness. We haven't paused infra; instead, we've hired someone to help keep up with it. I can't mitigate any bad experience you've had. I fucking hate getting ghosted, and I get a little nauseous when I think about us having ghosted people. But ghost people we have! All I can say is that it's not deliberate; it's just been a lot to keep up with. For everyone else: my sense is that the majority of people in our process have gotten timely (if not as fast as I'd like) responses from us, but if it's been weeks and you haven't heard back from me, you can hit me up directly. We don't ghost people "communicatively"; that's just not how we convey decisions.
deleted 4y ago
[deleted]

sosodev 4y ago

That's even worse than my experience https://news.ycombinator.com/item?id=34523227 https://news.ycombinator.com/item?id=34523227 I'm pretty skeptical of these hiring challenges at this point. It seems that they often require more time than the traditional model and provide worse outcomes for the candidates. For me it was particularly frustrating because I can typically write off the traditional model as irrational but I was drawn to fly.io because they make it seem so much better than that.

whiplash451 4y ago
If I were looking for a job, I would trade the usual stupid 1h leetcode interview for this take-home in an eyeblink. This one is much more time-consuming _up front_, but at least it gives you a sense of the job to be done _and_ doing the exercise lets you think through whether you really want that job after all.
- paxys 4y ago
  Well in that 1 hour leetcode session at least the interviewer is wasting the same amount of time as you. In this case you can spend all weekend working on the exercise and never even get a response.
  - deleted 4y ago
    [deleted]
  - whiplash451 4y ago
    This is an adversarial perspective of interviewing that does not get you anywhere because indeed, your time is always more precious than any company’s time. By 1h leetcode, I meant the whole yak shaving of classical interviewing, which costs candidate many hours actually (in interview time and prep time).
- jb3689 4y ago
  A take home without actual feedback is a complete waste of everyone’s time. Who cares if you get a taste for the job in the process. You spend a weekend, the company spends 10 minutes.
  - tptacek 4y ago
    That's an important knock against processes that do both interviews and WSTs ("take homes"). We don't: we exclusively use WSTs. We calibrate the amount of time our WSTs should take against the amount of time a typical interview process takes. There's a lot of stuff we can't do --- like expansive programming tests, or, for that matter, sharing our answer rubrics --- that we wish we could, precisely because we're fitting the whole process into a tight time budget. We don't time our WSTs; it adds cortisol to the process that defeats some of the purpose of simulating what actually working here is like. So, it is the case that people can end up blowing way past our expected time budget working on things. We're not going to stop people from doing that; it would be hard to do, and we're also excited to get candidates who teach themselves stuff as they go. We're up front about this. From my vantage point, this process is strictly better than conventional interviews. It demands the same amount of time from candidates, but allows the candidates to pick where and when to spend that time. There is a lot of understandable enmity built up against "take homes" because firms have added them to their existing interview loop, so they become just another hurdle candidates have to clear before running through the same interviews they always have. With respect to feedback: at some point in our process, we do start giving feedback. It has not been my experience that it is as appreciated by candidates as people on message boards seem to think it is. We've gotten to the point where we try to do a decent job of explaining what the best submissions have in common, and leave it at that. This kind of feedback is more than I'd ever come to expect from conventional interviews, so I'm at peace with it. Finally, at to again be candid: we got a huge flood of candidates for several roles, and while for the most part we kept up and gave timely responses to people who submitted challenge responses, there have definitely been ticket mishaps that caused people not to get responses --- in other words, we've ghosted people. I fucking hated getting ghosted when I was applying for jobs and am mortified by the fact that we did it too. What I can say there is: we've never done that deliberately, and when we've caught that happening, we've gotten detailed feedback out quickly. So if the premise of your comment is: "submitting a code challenge and then never hearing anything back at all, even so much as a pass/fail, is bullshit", then yes, I agree, it's total bullshit. Not OK at all. (With respect to the coding challenges we're talking about today: the public ones aren't a part of our hiring process at all! They're just there for people who are interested in them. We do have a related challenge Kyle and Ben worked on, as a post-L3 leveler. It's not a small challenge. But: any candidate that gets that challenge already knows they're getting an offer from us, so we're comfortable making it ambitious).

andrewfromx 4y ago

hehe yes the Nomad one w ELK stack took me longer than 2 hours for sure but was a lot of fun! I'll bet the fly people are just swamped. I loved the language in their email though. I've been on so many 1 hour live coding interviews where I can't shine like I want to shine this approach really made me go yes! A new way to interview.

tptacek 4y ago
We are swamped. We kept up as long as we could, and now we're making some changes: 1. We have someone running infra now that also owns infra hiring, instead of me owning it. Infra is an ultra-important role here and we're still hiring for it. 2. I'm still on the hook for platform hiring, and we've paused it for a couple months (if you submitted a challenge, we're still moving forward! but we're not taking new applications). I know people on HN don't know these details, but knowing it myself makes it especially goofy to see people calling these challenges an elaborate scheme to recruit people for platform development (which is where these challenges apply).

tptacek 4y ago

I don't have a better response for this than that we got a huge flood of candidates for that role, are doing our best to keep up, and have done an imperfect job. We're a small company. You're more than welcome to reach out directly to find out what happened with your application (it's entirely possible that what happened is that the ticket got buried somewhere, and that we'd have been excited to hire you had we not screwed something up on our end). As for the scope of the challenge: that's deliberate, and we're up front about it. If the challenge is going to take you many hours to complete, and doing that work isn't something that lights you up, then it's the wrong role for you. There are candidates that breeze through it, and there are candidates who wrestle with it for a couple evenings just because they're nerdsniped and enjoy doing that stuff. Both those kinds of candidates are super interesting to us! But we're very comfortable with the idea that the job (in this case: infra engineering) isn't a perfect fit for everybody.

specialp 4y ago
I really liked the take home. It is just that if you have someone doing this, you are having that person commit hours of their time to interviewing without the same investment on your end. So if you do not have the means or time to process these submissions then it is not fair to have someone spend their time on it. I think your hiring process is great otherwise. And yes if someone isn't very familiar with every intersection of stacks there it will take them more time. That is fine too. It shows the person can learn on the spot.
- tptacek 4y ago
  I agree. We came to that conclusion about platform just a week or so ago and have paused hiring because we can't meet the commitment we've made about responsiveness. We haven't paused infra; instead, we've hired someone to help keep up with it. I can't mitigate any bad experience you've had. I fucking hate getting ghosted, and I get a little nauseous when I think about us having ghosted people. But ghost people we have! All I can say is that it's not deliberate; it's just been a lot to keep up with. For everyone else: my sense is that the majority of people in our process have gotten timely (if not as fast as I'd like) responses from us, but if it's been weeks and you haven't heard back from me, you can hit me up directly. We don't ghost people "communicatively"; that's just not how we convey decisions.

deleted 4y ago

[deleted]