14 ms·

This. 100% this. As many others are saying, I find that this is the most comfortable way for me as an interviewee to demonstrate that I do in fact know what I'

This. 100% this. As many others are saying, I find that this is the most comfortable way for me as an interviewee to demonstrate that I do in fact know what I'm talking about. Put me in front of a room full of people and a whiteboard and throw something at me, and I'll freeze. Freezing in that situation and being a bad engineer are wholly separate things. Anecdotally, when I get one of these problems, I spent 20 minutes reading the prompt, looking at resources and making an outline of my code. Then I step away for the afternoon and don't think about it. Let the subconscious internalize the problem for a while and when I come back, I've got a much better mindset to approach it. Solving new problems on the spot is just a skill I don't have. I prefer to think that this is what makes you valuable as an employee and engineer, solving a problem by the end of the day. Not while under scrutiny. Personally, my resume and experience don't get me far interviewing by the big companies, but on the occasions that I get take home code assignments, I tend to get pleasantly surprised and pleased reviewers and it takes me to the final rounds (where I face the dreaded white board and actually-there-is-a-correct-answer situationals). My two cents, this just really resonated with me. Does anybody else happen to solve problems this way?

erikbye 10y ago
If you can't solve a new problem "on the spot", or at least quickly, then one is at least not suited for SRE work.
throwaway4job 10y ago
This isn't a viable approach for much of the hiring pool, as a majority of working engineers have contracts which explicitly forbid work for contract for other employers. As such, if you hire any of those, you're hiring somebody with a proven willingness to ignore their contract, which is a strong anti-pattern.
- zaroth 10y ago
  IANAL, but that's not what this is. "Work for contract for other employers" could literally mean I can't help my neighbor mow his lawn in exchange for a beer. In this case what it really means is a non-trivial amount of work done for meaningful profit. If I fix a friends computer in exchange for a nice meal, I am using my relevant skills to perform work, but I'm not actually violating a contract forbidding outside work in a way that could ever possibly result in damages or firing for cause. For a potential employee to be so rule-stricken as to concern themselves with performing a coding interview, that would be the potential red flag for me. So this could actually be another positive outcome of this approach.
  - nolite 10y ago
    You perfectly verbalized what was bothering me about this reply
  - mcpherrinm 10y ago
    I fear deportation, being unable to ever enter - never mind work in this country again. You can call that a red flag, but it is the reality I live in. Maybe most of your candidates don't have that problem, but I would definitely be eliminated.
    - zaroth 10y ago
      I mean, you could always just ask that the payment be donated to a charity instead, right? But since almost everything we do is in some way technically illegal, it is important for everyone to be able to function under that premise. I mean, just to cut the check technically the employer would need a W-9, a work for hire contract, and who knows what else. Let's call the lawyer and spend $5k deciding whether we can do this, and then decide 6 months later it's too risky.... Or, we do it, get better hires, execute on our plans better, meet revenue targets, and succeed in the market, by not following all the rules, just the rules that matter. Knowing the rules to follow and the rules to forget is the hardest part, but a crucial skill in life and business. The more you push the envelope, the faster you can run, and the more likely you are to combust. See Zenefits...
    - st3v3r 10y ago
      You are an edge case. There should be a workaround for you, but just because something doesn't work for you doesn't mean it doesn't work in general.
  - FireBeyond 10y ago
    Ever hear of tortuous interference? When one party induces another to break their contract with a third party. For as realistic as your example is (no employer is going to sue because you mowed your neighbors lawn), plenty of employers can, have and will sue when they realize you have hired away one of "their" people, regardless of your rights and theirs. Learning of your little deal to pay them for software development work will be the stick they use to hit you with that interference suit.
- sillysaurus3 10y ago
  It's possible to not pay them. Most candidates will do this for free, as it's far better than traditional ineffective hiring practices. The thing is, there are always reasons why this method isn't viable. It's not till you try it that you notice huge gains.
  - rizzom5000 10y ago
    That only makes sense if you're talking about people who are unemployed. As someone who is already employed full-time, I simply don't have the time to solve your problems for free just to see if you'd like to invite me over for an interview. I avoid these problems like the plague and very much prefer to whiteboard. If you pay me consultant rates, I can more easily justify the effort.
    - thinkmassive 10y ago
      Even when I was between jobs a couple months ago I neglected to complete a take-home interview exercise, because the opportunity cost still exists. The advantage to interviewing in person is that the conversation goes both ways. I was interviewing daily, so I still valued my spare time for personal activities. Edit: I most likely would have completed it if I were offered some type of compensation, even below market. They claimed it was an exercise and they wouldn't use it, but this was a startup with about 10 employees, and the deliverable was definitely something they could have used. That alone rubbed me the wrong way.
    - developer2 10y ago
      >> much prefer to whiteboard Who would have thought? Maybe - just maybe - not everyone has the same strengths and weaknesses. The real answer here is obvious: present the interviewee with options. Forcing all potential hires to whiteboard is a terrible idea. Forcing all potential hires to do a take-home project is a terrible idea. It's extremely short-sighted and a little pompous to assume that any single interview format one chooses is going to magically sort everyone into neat little buckets of "good" and "bad". If you force the whiteboarding approach, you are only going to wind up hiring social butterflies who have absolutely no nerves standing up in front of complete strangers and having all the answers on the spot. If you force the take-home project approach, you're turning off a lot of people who can nail a first impression presenting themselves and their skillset in person. People are different. Applying the same interview type to everyone is going to target a specific set of strengths, and a specific set of weaknesses. And frankly, no business should be composed entirely of one type of person. Some diversity does wonders when assessing the overall strength of a team.
- wpietri 10y ago
  Will they? I've seen contracts saying you can't work for competitors, but I'm not sure I've ever seen one that forbids all other employment. In California, it's illegal to prevent employees from moonlighting: https://californiaemploymentlaw.foxrothschild.com/2015/03/articles/wageandhour/can-california-employers-prohibit-moonlighting/ https://californiaemploymentlaw.foxrothschild.com/2015/03/ar...
- elchief 10y ago
  Or a $200 donation to their fave charity
  - RexetBlell 10y ago
    This is a great idea, no legal issues. The important thing is that since the candidate loses 2 hours of his time working on the problem, the employer has to lose something equivalent of that too. I would be much more willing to do the project if I knew the employer would lose $200. Among other things, this shows that I'm valued.
  - laksjd 10y ago
    That's a great idea! Not only does it just 'feel right', it also prevents some of the legal mess people were seeing with effectively moonlighting for the hiring company.
  - arethuza 10y ago
    That is a really good idea.
- inopinatus 10y ago
  IANAL but I took business law 101, which taught me that courts almost universally lean strongly towards the employee when considering restraint clauses. Moreover, if you have a current employer that is sufficiently disrespectful of their employees that they'd even consider enforcing an anti-moonlighting clause in this circumstance, with the preposterous assertion that it qualifies as employment - run away now. The law is not a programming language.
  - vostok 10y ago
    It is a requirement of my country's regulators that I ask for my employer's approval before I do any other work for compensation.
    - moyta 10y ago
      Damn, what country is that? Sounds like a very controlled, restrictive environment.
      - AtomicOrbital 10y ago
        I worked at a top investment bank in NY which had/has that policy
  - deleted 10y ago
    [deleted]
  - kabdib 10y ago
    If you're involved in the court system while looking for employment, you've already lost.
- moron4hire 10y ago
  >> a majority of working engineers have contracts which explicitly forbid work for contract for other employers. Well that was very, very dumb of them. Maybe we engineers ain't so smart after all.
  - walshemj 10y ago
    Custom and practice - you need to read your company handbook /contract some time.
    - moron4hire 10y ago
      Work agreements are really easy to modify before you sign them. And I don't have an employee handbook. I own my own business.
      - walshemj 10y ago
        Only if your Beyoncé and can demand a bucket of kittens in you dressing room before a show - the rest of us less so?
        deleted 10y ago
        [deleted]
- quicklime 10y ago
  On top of this, many candidates have visa-related restrictions that would prevent them from doing this legally.
- dimino 10y ago
  I don't agree that it's a string anti-pattern whatsoever. It's a clear signal that the person can interpret the meaning of a rule and adhere to it in a way that makes sense for the person who made the rule as well as the contractor.
- majormajor 10y ago
  I don't understand the paid part at all. If I'm working full time, $200 isn't going to swing me one way or the other on deciding if I want to do your homework assignment. And if you come up with a ten-hour problem, then we're talking more of a chunk of cash, but you're also asking me to burn a lot of free time. I'd much rather travel and meet you in person and do stuff there, but if you don't have the budget for it, we could figure other stuff out (shared docs, skype, whatever, I've done a few of these on both ends with some luck). "Since the candidate is getting paid" seems like far weaker motivation than "since the candidate wants the job you're offering," which is what's going to determine my level of effort. The "not a real problem" thing is key, though. I've tried testing out a few different "this is one of our real problem" things but there's almost always more hidden business rules or assumptions in there than you think.
  - ghaff 10y ago
    >I don't understand the paid part at all. There's a concept around a lot of preliminary engagements between two entities called "skin in the game." The idea is that when something is completely free the other party will take advantage of it without any real serious intent to follow through on anything. Mostly this seems a reaction to the homework-type assignments where candidates are expected to spend a lot of time on some interview assignment with very little real cost to the company--which raises the possibility that it's effectively a cattle call.
    - majormajor 10y ago
      I get this in theory, but the numbers sound too low for that to make a difference still. I could run 50 candidates through a $200 problem for 10K. That's still a pretty large mismatch between "amount of work done by the candidate" and "amount of work done by the interviewer," and is cost-of-doing-business money for recruiting for a lot of companies currently. Compare that to the cost of me flying people out (which is still done in this approach), or even the cost of spending an hour of mine or someone good on my team's time on the phone with them. I guess it's just down to the difference between trying to hire fresh-out-of-college (or still in) free-time-to-spare junior devs and experienced people. That's actually a topic I should write a blog about somewhere myself, one day - I pushed pretty hard at my current company for moving towards a different process for industry candidates, and are extremely pleased with some of the people it's helped us hire.
- elcct 10y ago
  I think person who is either very well compensated for that or with slave mentality would sign such a contract. Either way I don't think those people are a target group of this post.
- st3v3r 10y ago
  Non-competes are worthless clauses that should never have existed in the first place. I am perfectly ok with someone ignoring that blatantly anti-employee clause.
blazespin 10y ago
This works, but make sure they spend the 2 hours in your office and not 12 hours at home solving a 2 hour problem.. The other problem is that when you pay them you generate all sorts of nightmaris legal liabilities.
- jpalomaki 10y ago
  2 hour deadline and forcing the task to be done at the office turns it to a different kind test. This becomes more like how do you perform under stress and how well can you propose things out of your head. These are valuable skills as well, but I think having the weekend to spend with the problem more accurately models the actual development work.
  - thinkmassive 10y ago
    Your point is valid, although technically it would be under pressure as opposed to stress. The difference might seem minor but it's rather significant, and it's a characteristic many employers would intentionally want to measure. I highly recommend the book Performing Under Pressure, by Weisinger & Pawliw-Fry.
  - clarry 10y ago
    Pressure comes in many forms. Certainly, being able to handle pressure is a good thing. But how often is it needed and what kind of pressure is it? What kind of relief have you got? As I understand it, the more common form of pressure would come from having to deliver things in a tight schedule, and perhaps in certain kinds of meetings and presentations. Not from being forced to work in a foreign environment with foreign computers & tools and unknown people hanging around, watching and JUDGING you. If a company puts an engineer in the latter position very often, it might be doing something rather wrong. Whereas in the former situation, employers can find relief in at least having colleagues they are familiar with and an environment they're used to & comfortable with.
- davidbanham 10y ago
  Absolutely agree with the sentiment here. I have used this technique for a couple of years now, except that I kept the task to 20 minutes (and didn't pay people, since it's a short amount of time). To save dragging people into an office for that time period, I set up a custom git server that marks the timestamps of their pulls and pushes. The instructions for the task are contained in the README of the repo, so as soon as they pull it they've started. https://github.com/Prismatik/codescreen https://github.com/Prismatik/codescreen It's critical to me that the limit is adhered to so that over time I can compare candidate responses on a level playing field.
stillworks 10y ago
I don't care about getting paid. I don't care if I am doing it at home or in the interviewer's premises. Don't let me connect to internet. I'll also leave my phone with you just to make sure I don't have access to the internet. I don't mind taking the time away from current employment to go through the process. BUT PLEASE LEAVE ME ALONE THOUGH WHILE I AM AT IT. FOR GOD'S SAKE DON"T SIT NEXT TO ME AND TALK TO ME OR EXPECT ME TO TALK TO YOU WHILE I AM SOLVING THE PROBLEM !!! Let's discuss the solution or the non-solution AFTER I have spent some alone time with the problem. I don't mind getting rejected AFTER I have made a serious attempt at it. It is simply beyond me why people think it's all right for a complete stranger to sit next to a programmer while he is trying to solve a problem under the significant cognitive load of a new environment, new people possibly a laptop/keyboard he has never used. This did sound like a rant :-/ but what I wanted to say was that the process in the article seems fair enough. Interviewees should be given the alone time to solve the problem. Most people are decent and sociable enough that once they are one board and are comfortable and familiar with the rest of the team they will be able to function and be reasonable in pair programming envoirnment or other technical discussions as well. BUT not when they are in an interview. No matter how it is sugar coated, interview is where a programmer is getting JUDGED and that by default leads to pressure.
- blowski 10y ago
  Sounds like I'm completely the opposite of you! I'm fine with someone sitting next to me - I love pair programming. But cutting off internet access means no StackOverflow, no online documentation. The test should be very similar to normal working conditions.
  - stillworks 10y ago
    Interesting to note how different people are indeed. I am fine with white board type situations, don't worry about writing code without IDEs etc (in my very first job vi editor was the only thing we had to write code so I still have instinctive memory to be resourceful without an IDE but I love IntelliJ for most things). Happy to discuss/draw out ideas/solutions on whiteboards. But find it very seriously annoying talking or being expected to talk while I am writing code ! I think the internet-access requirement depends on the problem. If I am expected to solve a problem by first-principles on a core CS topic then I guess internet is of no use ? Online Java Docs for e.g. may not serve a purpose here. If the problem requires using many different tools/APIs for the solution then I guess you are right, internet access adds value.
z3t4 10y ago
There's a big difference acting on instinct and heuristics, versus actually thinking witch is very slow.
SamUK96 10y ago
> actually-there-is-a-correct-answer situationals Ugh. The worst step of interview process. Remember that guy who wrote a post about Google's hiring process, where they outsourced the asking of technical questions to untrained call-centre staff. "Standard way to allocate memory in C?" "malloc" "Incorrect. Actually there's this cool library that does it for you with additional checks. Sorry, you have failed the interview" "Wut?" I've always found that those kind of interview processes put far too much stress on being correct to their "ideals" rather than be consistent with your own and your team's. I.e. I don't care if you name your column "domain_id", but for christ's sake name the other column "account_id".
- mathw 10y ago
  FWIW, my experience of Google's hiring process wasn't like that at all. Sure, it was hard, and they did do whiteboarding problems in the interviews (all seven of them), which is in its own way terrible, but it was all done with people who really did know the stuff they were asking me about. I didn't get the job, but it was an interesting experience. Certainly selects for a certain kind of engineer, and would not be my preference for how to hire people if I was setting up my own operation.
- expertentipp 10y ago
  > Remember that guy who wrote a post about Google's hiring process, where they outsourced the asking of technical questions to untrained call-centre staff. Hey, I had this experience with Amazon when applying for a software related position based in Europe. For the entry screening I got a call from Indian phone number and a person with Indian accent was clearly going through a list of various quirks an features of a programming language. Wasn't so bad though... the second round with MBA product managers was much worse:)
SonicSoul 10y ago
i do love this idea but not sure if doing it remotely is a good idea. We do this in house because I am not sure how to be certain the candidate did not get help. I mean i believe people are generally honest but there will be cases when they are not, and I'd rather not find out 3 months after hiring them. Also there are many indicators along the way, i.e. which parts the candidate got stuck at, did it take them 1 or 3 hrs to get to the same point (i.e. basic wiring of DB calls). I think all that would be lost if they did it on their own time.
- Lord_Zero 10y ago
  > I am not sure how to be certain the candidate did not get help Pressing them hard for information on Monday will expose this.
- sidlls 10y ago
  Do employees at the place you work at never ask their colleagues for help?
naterr 10y ago
I'm a subconscious problem solver also, I do it on purpose alot of the time, I can solve some things immediately but most of the time I get it later in the day when I go back to it or let the answer come to me.
- md81544 10y ago
  Yes, me too. I often find far better solutions to problems after backing off and deliberately not thinking about the issue.
hinkley 10y ago
We tend to think in a one size fits all manner and in a team environment I find that counterproductive or even destructive. Every team needs a couple of people who can solve problems on the spot. They're going to fix your production issues. You need a couple of people who want to know why all the time. Someone with a big picture view. Someone with good aesthetics. Someone schedule driven, and a process nerd. But they don't have to be the same couple of people. Diversity is more than fine, it's essential to avoid echo chamber syndrome. Some of the worst projects I've worked on had little to no dissent in the ranks. The code was self consistent but entirely wrong (bad info architecture, slow, brittle, impossible to learn after the fact). But everyone tries to hire people that think exactly like they do. If we weren't so bad at determining that we'd probably never make anything interesting at all.
Florin_Andrei 10y ago
> I spent 20 minutes reading the prompt, looking at resources and making an outline of my code. Then I step away for the afternoon and don't think about it. Let the subconscious internalize the problem for a while and when I come back, I've got a much better mindset to approach it. That's definitely a thing, and I believe it applies to some extent to everyone. But here's the other side of the coin - the ability to operate efficiently when the world is burning. I've been in the industry for a while, and this sort of thing definitely happens. But it's very hard to simulate in an interview. You don't know who the adrenalin junkies are on your team until PagerDuty lights up glowing red across the board. The only indication I've seen is the correlation with a preference for risk-taking activities, such as sky diving, etc. But it's quite circumstantial. And there's definitely also the true reverse of it: some of the car racers and bull fighters out there are not good team players in times of peace. Well, if at least a few key players can handle pressure gracefully, I guess you'll be fine, since it's a skill rarely used. You don't need everyone to be able to switch to catastrophe mode at a moment's notice.

erikbye 10y ago

If you can't solve a new problem "on the spot", or at least quickly, then one is at least not suited for SRE work.

throwaway4job 10y ago

This isn't a viable approach for much of the hiring pool, as a majority of working engineers have contracts which explicitly forbid work for contract for other employers. As such, if you hire any of those, you're hiring somebody with a proven willingness to ignore their contract, which is a strong anti-pattern.

zaroth 10y ago
IANAL, but that's not what this is. "Work for contract for other employers" could literally mean I can't help my neighbor mow his lawn in exchange for a beer. In this case what it really means is a non-trivial amount of work done for meaningful profit. If I fix a friends computer in exchange for a nice meal, I am using my relevant skills to perform work, but I'm not actually violating a contract forbidding outside work in a way that could ever possibly result in damages or firing for cause. For a potential employee to be so rule-stricken as to concern themselves with performing a coding interview, that would be the potential red flag for me. So this could actually be another positive outcome of this approach.
- nolite 10y ago
  You perfectly verbalized what was bothering me about this reply
- mcpherrinm 10y ago
  I fear deportation, being unable to ever enter - never mind work in this country again. You can call that a red flag, but it is the reality I live in. Maybe most of your candidates don't have that problem, but I would definitely be eliminated.
  - zaroth 10y ago
    I mean, you could always just ask that the payment be donated to a charity instead, right? But since almost everything we do is in some way technically illegal, it is important for everyone to be able to function under that premise. I mean, just to cut the check technically the employer would need a W-9, a work for hire contract, and who knows what else. Let's call the lawyer and spend $5k deciding whether we can do this, and then decide 6 months later it's too risky.... Or, we do it, get better hires, execute on our plans better, meet revenue targets, and succeed in the market, by not following all the rules, just the rules that matter. Knowing the rules to follow and the rules to forget is the hardest part, but a crucial skill in life and business. The more you push the envelope, the faster you can run, and the more likely you are to combust. See Zenefits...
  - st3v3r 10y ago
    You are an edge case. There should be a workaround for you, but just because something doesn't work for you doesn't mean it doesn't work in general.
- FireBeyond 10y ago
  Ever hear of tortuous interference? When one party induces another to break their contract with a third party. For as realistic as your example is (no employer is going to sue because you mowed your neighbors lawn), plenty of employers can, have and will sue when they realize you have hired away one of "their" people, regardless of your rights and theirs. Learning of your little deal to pay them for software development work will be the stick they use to hit you with that interference suit.
sillysaurus3 10y ago
It's possible to not pay them. Most candidates will do this for free, as it's far better than traditional ineffective hiring practices. The thing is, there are always reasons why this method isn't viable. It's not till you try it that you notice huge gains.
- rizzom5000 10y ago
  That only makes sense if you're talking about people who are unemployed. As someone who is already employed full-time, I simply don't have the time to solve your problems for free just to see if you'd like to invite me over for an interview. I avoid these problems like the plague and very much prefer to whiteboard. If you pay me consultant rates, I can more easily justify the effort.
  - thinkmassive 10y ago
    Even when I was between jobs a couple months ago I neglected to complete a take-home interview exercise, because the opportunity cost still exists. The advantage to interviewing in person is that the conversation goes both ways. I was interviewing daily, so I still valued my spare time for personal activities. Edit: I most likely would have completed it if I were offered some type of compensation, even below market. They claimed it was an exercise and they wouldn't use it, but this was a startup with about 10 employees, and the deliverable was definitely something they could have used. That alone rubbed me the wrong way.
  - developer2 10y ago
    >> much prefer to whiteboard Who would have thought? Maybe - just maybe - not everyone has the same strengths and weaknesses. The real answer here is obvious: present the interviewee with options. Forcing all potential hires to whiteboard is a terrible idea. Forcing all potential hires to do a take-home project is a terrible idea. It's extremely short-sighted and a little pompous to assume that any single interview format one chooses is going to magically sort everyone into neat little buckets of "good" and "bad". If you force the whiteboarding approach, you are only going to wind up hiring social butterflies who have absolutely no nerves standing up in front of complete strangers and having all the answers on the spot. If you force the take-home project approach, you're turning off a lot of people who can nail a first impression presenting themselves and their skillset in person. People are different. Applying the same interview type to everyone is going to target a specific set of strengths, and a specific set of weaknesses. And frankly, no business should be composed entirely of one type of person. Some diversity does wonders when assessing the overall strength of a team.
wpietri 10y ago
Will they? I've seen contracts saying you can't work for competitors, but I'm not sure I've ever seen one that forbids all other employment. In California, it's illegal to prevent employees from moonlighting: https://californiaemploymentlaw.foxrothschild.com/2015/03/articles/wageandhour/can-california-employers-prohibit-moonlighting/ https://californiaemploymentlaw.foxrothschild.com/2015/03/ar...
elchief 10y ago
Or a $200 donation to their fave charity
- RexetBlell 10y ago
  This is a great idea, no legal issues. The important thing is that since the candidate loses 2 hours of his time working on the problem, the employer has to lose something equivalent of that too. I would be much more willing to do the project if I knew the employer would lose $200. Among other things, this shows that I'm valued.
- laksjd 10y ago
  That's a great idea! Not only does it just 'feel right', it also prevents some of the legal mess people were seeing with effectively moonlighting for the hiring company.
- arethuza 10y ago
  That is a really good idea.
inopinatus 10y ago
IANAL but I took business law 101, which taught me that courts almost universally lean strongly towards the employee when considering restraint clauses. Moreover, if you have a current employer that is sufficiently disrespectful of their employees that they'd even consider enforcing an anti-moonlighting clause in this circumstance, with the preposterous assertion that it qualifies as employment - run away now. The law is not a programming language.
- vostok 10y ago
  It is a requirement of my country's regulators that I ask for my employer's approval before I do any other work for compensation.
  - moyta 10y ago
    Damn, what country is that? Sounds like a very controlled, restrictive environment.
    - AtomicOrbital 10y ago
      I worked at a top investment bank in NY which had/has that policy
- deleted 10y ago
  [deleted]
- kabdib 10y ago
  If you're involved in the court system while looking for employment, you've already lost.
moron4hire 10y ago
>> a majority of working engineers have contracts which explicitly forbid work for contract for other employers. Well that was very, very dumb of them. Maybe we engineers ain't so smart after all.
- walshemj 10y ago
  Custom and practice - you need to read your company handbook /contract some time.
  - moron4hire 10y ago
    Work agreements are really easy to modify before you sign them. And I don't have an employee handbook. I own my own business.
    - walshemj 10y ago
      Only if your Beyoncé and can demand a bucket of kittens in you dressing room before a show - the rest of us less so?
      - deleted 10y ago
        [deleted]
quicklime 10y ago
On top of this, many candidates have visa-related restrictions that would prevent them from doing this legally.
dimino 10y ago
I don't agree that it's a string anti-pattern whatsoever. It's a clear signal that the person can interpret the meaning of a rule and adhere to it in a way that makes sense for the person who made the rule as well as the contractor.
majormajor 10y ago
I don't understand the paid part at all. If I'm working full time, $200 isn't going to swing me one way or the other on deciding if I want to do your homework assignment. And if you come up with a ten-hour problem, then we're talking more of a chunk of cash, but you're also asking me to burn a lot of free time. I'd much rather travel and meet you in person and do stuff there, but if you don't have the budget for it, we could figure other stuff out (shared docs, skype, whatever, I've done a few of these on both ends with some luck). "Since the candidate is getting paid" seems like far weaker motivation than "since the candidate wants the job you're offering," which is what's going to determine my level of effort. The "not a real problem" thing is key, though. I've tried testing out a few different "this is one of our real problem" things but there's almost always more hidden business rules or assumptions in there than you think.
- ghaff 10y ago
  >I don't understand the paid part at all. There's a concept around a lot of preliminary engagements between two entities called "skin in the game." The idea is that when something is completely free the other party will take advantage of it without any real serious intent to follow through on anything. Mostly this seems a reaction to the homework-type assignments where candidates are expected to spend a lot of time on some interview assignment with very little real cost to the company--which raises the possibility that it's effectively a cattle call.
  - majormajor 10y ago
    I get this in theory, but the numbers sound too low for that to make a difference still. I could run 50 candidates through a $200 problem for 10K. That's still a pretty large mismatch between "amount of work done by the candidate" and "amount of work done by the interviewer," and is cost-of-doing-business money for recruiting for a lot of companies currently. Compare that to the cost of me flying people out (which is still done in this approach), or even the cost of spending an hour of mine or someone good on my team's time on the phone with them. I guess it's just down to the difference between trying to hire fresh-out-of-college (or still in) free-time-to-spare junior devs and experienced people. That's actually a topic I should write a blog about somewhere myself, one day - I pushed pretty hard at my current company for moving towards a different process for industry candidates, and are extremely pleased with some of the people it's helped us hire.
elcct 10y ago
I think person who is either very well compensated for that or with slave mentality would sign such a contract. Either way I don't think those people are a target group of this post.
st3v3r 10y ago
Non-competes are worthless clauses that should never have existed in the first place. I am perfectly ok with someone ignoring that blatantly anti-employee clause.

blazespin 10y ago

This works, but make sure they spend the 2 hours in your office and not 12 hours at home solving a 2 hour problem.. The other problem is that when you pay them you generate all sorts of nightmaris legal liabilities.

jpalomaki 10y ago
2 hour deadline and forcing the task to be done at the office turns it to a different kind test. This becomes more like how do you perform under stress and how well can you propose things out of your head. These are valuable skills as well, but I think having the weekend to spend with the problem more accurately models the actual development work.
- thinkmassive 10y ago
  Your point is valid, although technically it would be under pressure as opposed to stress. The difference might seem minor but it's rather significant, and it's a characteristic many employers would intentionally want to measure. I highly recommend the book Performing Under Pressure, by Weisinger & Pawliw-Fry.
- clarry 10y ago
  Pressure comes in many forms. Certainly, being able to handle pressure is a good thing. But how often is it needed and what kind of pressure is it? What kind of relief have you got? As I understand it, the more common form of pressure would come from having to deliver things in a tight schedule, and perhaps in certain kinds of meetings and presentations. Not from being forced to work in a foreign environment with foreign computers & tools and unknown people hanging around, watching and JUDGING you. If a company puts an engineer in the latter position very often, it might be doing something rather wrong. Whereas in the former situation, employers can find relief in at least having colleagues they are familiar with and an environment they're used to & comfortable with.
davidbanham 10y ago
Absolutely agree with the sentiment here. I have used this technique for a couple of years now, except that I kept the task to 20 minutes (and didn't pay people, since it's a short amount of time). To save dragging people into an office for that time period, I set up a custom git server that marks the timestamps of their pulls and pushes. The instructions for the task are contained in the README of the repo, so as soon as they pull it they've started. https://github.com/Prismatik/codescreen https://github.com/Prismatik/codescreen It's critical to me that the limit is adhered to so that over time I can compare candidate responses on a level playing field.

stillworks 10y ago

I don't care about getting paid. I don't care if I am doing it at home or in the interviewer's premises. Don't let me connect to internet. I'll also leave my phone with you just to make sure I don't have access to the internet. I don't mind taking the time away from current employment to go through the process. BUT PLEASE LEAVE ME ALONE THOUGH WHILE I AM AT IT. FOR GOD'S SAKE DON"T SIT NEXT TO ME AND TALK TO ME OR EXPECT ME TO TALK TO YOU WHILE I AM SOLVING THE PROBLEM !!! Let's discuss the solution or the non-solution AFTER I have spent some alone time with the problem. I don't mind getting rejected AFTER I have made a serious attempt at it. It is simply beyond me why people think it's all right for a complete stranger to sit next to a programmer while he is trying to solve a problem under the significant cognitive load of a new environment, new people possibly a laptop/keyboard he has never used. This did sound like a rant :-/ but what I wanted to say was that the process in the article seems fair enough. Interviewees should be given the alone time to solve the problem. Most people are decent and sociable enough that once they are one board and are comfortable and familiar with the rest of the team they will be able to function and be reasonable in pair programming envoirnment or other technical discussions as well. BUT not when they are in an interview. No matter how it is sugar coated, interview is where a programmer is getting JUDGED and that by default leads to pressure.

blowski 10y ago
Sounds like I'm completely the opposite of you! I'm fine with someone sitting next to me - I love pair programming. But cutting off internet access means no StackOverflow, no online documentation. The test should be very similar to normal working conditions.
- stillworks 10y ago
  Interesting to note how different people are indeed. I am fine with white board type situations, don't worry about writing code without IDEs etc (in my very first job vi editor was the only thing we had to write code so I still have instinctive memory to be resourceful without an IDE but I love IntelliJ for most things). Happy to discuss/draw out ideas/solutions on whiteboards. But find it very seriously annoying talking or being expected to talk while I am writing code ! I think the internet-access requirement depends on the problem. If I am expected to solve a problem by first-principles on a core CS topic then I guess internet is of no use ? Online Java Docs for e.g. may not serve a purpose here. If the problem requires using many different tools/APIs for the solution then I guess you are right, internet access adds value.

z3t4 10y ago

There's a big difference acting on instinct and heuristics, versus actually thinking witch is very slow.

SamUK96 10y ago

> actually-there-is-a-correct-answer situationals Ugh. The worst step of interview process. Remember that guy who wrote a post about Google's hiring process, where they outsourced the asking of technical questions to untrained call-centre staff. "Standard way to allocate memory in C?" "malloc" "Incorrect. Actually there's this cool library that does it for you with additional checks. Sorry, you have failed the interview" "Wut?" I've always found that those kind of interview processes put far too much stress on being correct to their "ideals" rather than be consistent with your own and your team's. I.e. I don't care if you name your column "domain_id", but for christ's sake name the other column "account_id".

mathw 10y ago
FWIW, my experience of Google's hiring process wasn't like that at all. Sure, it was hard, and they did do whiteboarding problems in the interviews (all seven of them), which is in its own way terrible, but it was all done with people who really did know the stuff they were asking me about. I didn't get the job, but it was an interesting experience. Certainly selects for a certain kind of engineer, and would not be my preference for how to hire people if I was setting up my own operation.
expertentipp 10y ago
> Remember that guy who wrote a post about Google's hiring process, where they outsourced the asking of technical questions to untrained call-centre staff. Hey, I had this experience with Amazon when applying for a software related position based in Europe. For the entry screening I got a call from Indian phone number and a person with Indian accent was clearly going through a list of various quirks an features of a programming language. Wasn't so bad though... the second round with MBA product managers was much worse:)

SonicSoul 10y ago

i do love this idea but not sure if doing it remotely is a good idea. We do this in house because I am not sure how to be certain the candidate did not get help. I mean i believe people are generally honest but there will be cases when they are not, and I'd rather not find out 3 months after hiring them. Also there are many indicators along the way, i.e. which parts the candidate got stuck at, did it take them 1 or 3 hrs to get to the same point (i.e. basic wiring of DB calls). I think all that would be lost if they did it on their own time.

Lord_Zero 10y ago
> I am not sure how to be certain the candidate did not get help Pressing them hard for information on Monday will expose this.
sidlls 10y ago
Do employees at the place you work at never ask their colleagues for help?

naterr 10y ago

I'm a subconscious problem solver also, I do it on purpose alot of the time, I can solve some things immediately but most of the time I get it later in the day when I go back to it or let the answer come to me.

md81544 10y ago
Yes, me too. I often find far better solutions to problems after backing off and deliberately not thinking about the issue.

hinkley 10y ago

We tend to think in a one size fits all manner and in a team environment I find that counterproductive or even destructive. Every team needs a couple of people who can solve problems on the spot. They're going to fix your production issues. You need a couple of people who want to know why all the time. Someone with a big picture view. Someone with good aesthetics. Someone schedule driven, and a process nerd. But they don't have to be the same couple of people. Diversity is more than fine, it's essential to avoid echo chamber syndrome. Some of the worst projects I've worked on had little to no dissent in the ranks. The code was self consistent but entirely wrong (bad info architecture, slow, brittle, impossible to learn after the fact). But everyone tries to hire people that think exactly like they do. If we weren't so bad at determining that we'd probably never make anything interesting at all.

Florin_Andrei 10y ago

> I spent 20 minutes reading the prompt, looking at resources and making an outline of my code. Then I step away for the afternoon and don't think about it. Let the subconscious internalize the problem for a while and when I come back, I've got a much better mindset to approach it. That's definitely a thing, and I believe it applies to some extent to everyone. But here's the other side of the coin - the ability to operate efficiently when the world is burning. I've been in the industry for a while, and this sort of thing definitely happens. But it's very hard to simulate in an interview. You don't know who the adrenalin junkies are on your team until PagerDuty lights up glowing red across the board. The only indication I've seen is the correlation with a preference for risk-taking activities, such as sky diving, etc. But it's quite circumstantial. And there's definitely also the true reverse of it: some of the car racers and bull fighters out there are not good team players in times of peace. Well, if at least a few key players can handle pressure gracefully, I guess you'll be fine, since it's a skill rarely used. You don't need everyone to be able to switch to catastrophe mode at a moment's notice.