9 ms·
Ask HN: A coding test or working for free?
I recently interviewed for a company called Rxdata.net here in New York.
The task was a data ETL problem transforming natural language text into specific parts of data. I opted to use pyparse to create a working recursive descent parser to pick out the relevant bits of text.
At this point I was wary of completing the parser for all cases found in the database table, so I decided to build it only for a few base cases which I threw in to a unit test. I reasoned I could still demonstrate my skills without giving them much free work.
The solution worked well, and I submitted it to Joe, the CTO, for review. A few hours later Joe contacted me complaining that the code crapped out after trying to run the code against their database table. He then offered me a github branch to continue work on the problem. I sent them the nicest e-mail I could muster explaining that I felt the code adequately demonstrated my skills and that the error is due to the parser not handling all cases of the problem text.
Joe's responded with "I am only trying to evaluate your work, not on just how the code looks, which is clean and well organized, but that it works correctly on the data as well. I asked advice since the program crashed on the 5th package, which didn't give me enough data to verify, and I wasn't sure if it was just a system-related issue or something quickly fixed."
Lastly I responded with "Sorry for any confusion, I was just giving you a sample of my work. The system error you mentioned is a result of the code being a sample. The cases that do work are found in the test."
Naturally they haven't gotten back to me, even though my code more than adequately met the challenge. Perhaps Joe the CTO did not understand that the other cases would be trivial to implement. In that case, perhaps I've dodged a bullet. A CTO should know better.
I'm wondering if anyone else has had similar experiences. How did you handle it? How did the interviewers respond?
- creyes123 11y agoAny programming test that takes over an hour to complete is highly suspect. If it took less than that to complete the code (you did not specify), then IMHO I can understand why they were expecting a more complete answer. Regardless, my hunch is that your instincts are right about this one.
- alphanumeric0 11y agoThanks for the feedback. It took several hours. After an hour of examination of the data and research into the problem at their office I opted to make it a 'take home' test. They agreed and I worked on it the next day (hence the e-mail conversation).
- mcv 11y agoI don't agree about the hour. I've done a programming test that took way more time (partially because it involved learning an entirely new framework). It was at a company where everybody gets the same test (and everybody solves it in a different way), and then presents their code to the assembled programmers of the company. I still think that is the best way to interview (after a regular interview to weed out the real idiots). I think the real thing that's suspect, is simply having to send in your code, rather than presenting your solution to a group of programmers.
- bjourne 11y agoI disagree. I think coding tests are a great way to judge candidates. But only if they are given proper weight. Nothing worse than submitting and acing a big coding challenge and then being told they don't think you're a good culture fit. Some companies employ such hiring practices and I think it is very unethical.
- fsk 11y agoThat's why I've soured on coding tests. After lots of employers didn't give me an interview after a coding test that a KNOW I aced, that's poisoned the well for me. Now if someone gives me a pre-interview coding test, I mentally translate it to the employer saying "I don't respect your time."
- PopeOfNope 11y agoSame here. I've been through the same song and dance. If something happens over and over and it results in my not getting an interview, much less the job, I'm not going to repeat it. Especially if it's time consuming. If the company asks for a coding test, I politely decline, but I offer to provide them with a code sample instead. Very rarely has that not resulted in a phone interview and I didn't have to spend hours of my time on it[0]. [0]: per company. Write once, give to many scales better than coding tests.
- kstenerud 11y agoDid he even ask you to describe how you would likely tackle the problem while you were in their office? Things like design ideas, architectures, technologies, pros and cons, etc? If all he did was ask you to "do this" at their office without picking your brains as to your thought process, then regardless of the legitimacy of the offer, I would think twice about working there. The fact that he's asking for a solution that works on all of the data rather than a sample of it is also highly suspect. Either he is not being frank with you, or he doesn't have a firm grasp of scope (most likely the latter). Either way, he's probably not the sort of person you'd want to work for.
- alphanumeric0 11y agoI went through a couple of cases with him where a regular expression wouldn't work, how I might test my solutions, pros and cons of my proposed solutions, how it could scale in the future, etc. That conversation was there. After we spoke he left me to work.
- jcwilde 11y agoAm I to understand from your explanation that they asked you to solve a real business problem with real business data as an unpaid "coding test"? If so, I'd say they were clearly in the (legal) wrong even before asking for the refined version.
- bweitzman 11y agoI had an interview at a company once which was basically pair programming with a few different engineers for a full day on real code written for real clients. The best part was the NDA they asked me to sign 6 months later.
- deleted 11y ago[deleted]
- feedjoelpie 11y agoI haven't run into this on the interviewee end, but my company has applicants work on real projects, and here's how we deal with it: Pay them.
- maratd 11y agoThis is a simple solution to the problem. Paying someone for an hour or two is absolutely nothing for a healthy company. So if the company wants to have you write code to prove your mettle, have them pay you for the time.
- xsace 11y agoExcept you cannot do that in every country. You know, Labour right.
- hndl 11y agoI'm not sure I understand. Care to elaborate? Pointers?
- juandopazo 11y agoIn my country the potential employee would have to be registered as an independent contractor before being able to charge for the provided service. And then if he wanted to become a full time employee he'd have to unregister as a contractor.
- xsace 11y agoYes. In some countries, laws are strongly in favor of employee protection (for a good reason). That's the case here. One employer could severely abuse paying candidates for some quick job without any contract being even mentioned between the 2.
- jschulenklopper 11y agoWell, in case labour laws prevent people being paid for some labour -- or if the applicant needs to be registered as an independent contractor in order to receive payment for the work -- can't the company just pay for the expenses (and a little extra) and thus compensate the applicant sufficiently? It should be possible to pay for travel, stay, lunch or other 'inconveniences' in case the labour/deliverables can't be paid for directly, right?
- mrcold 11y agoIt's the future. Crowdsourcing through job candidates. I had a similar experience with a company in Berlin, Germany. They didn't care about the code or my skills. Only if the submitted solution worked against their existing tests. I think it's just a sleazy way of getting things built for free. Basically, write only a test for the functionality you want. And give the actual task to job candidates. If someone succeeds, you have both the algorithm and its test for pennies. It's based on P = NP. Testing the result is easy. Designing a solution is the hard part. I now refuse both technical interviews and take at home tests. Limits my choices a lot. But the ones that remain are usually top shelf.
- swampthing 11y agoIs it possible the German company was trying to see how you react to failing tests? I just ask because it seems like trying to get work done for free through code exercises is much more trouble than it's worth.
- rrss1122 11y agoYou would stand a good chance of waiting a lot longer to get that work done, but as mrcold said, testing a solution doesn't cost much. If you're willing to wait for the solution to a problem, getting work done through code exercises can be worth it.
- swampthing 11y agoYea, I guess I was thinking more that it would take a lot of time to unnaturally break down a problem into portions small enough to pass as code exercises.
- mapleoin 11y agoWait, you refuse technical interviews and you say the rest are top shelf? I'm curious to know what your ideal interview looks like. Or do you mean you just get hired through networking?
- 11y ago
- GabrielF00 11y agoIt's probably not a good idea to post the name and title of the person you spoke with when it's a small company.
- mkagenius 11y agoTrue. OP could have asked for advice without naming names.
- overgard 11y agoHonest question: is tricking coders into doing work for free a thing that actually happens? I'm not saying it doesn't, I've just never heard of it. From their end, if that was a thing they regularly did, it would be such a hassle. After all, you'd have to find a new person to do that work every time, find a way to split up the work into very small chunks, and who knows if the result from a person off the street would be usable? My guess would be that he was trying to evaluate how you respond to bug fixes and criticism. (Still sounds like a weird interview strategy though)
- alphanumeric0 11y agoThis is a valid point. I agree it doesn't seem like an efficient way of conducting a business in the long-term, but for a one-off problem that they happen to be stuck on, it makes more sense. The feeling I got was that they weren't developers by trade (the code quality I mentioned was a red flag), very small, early-stage startup with no money and they needed expert help, fast.
- jcwilde 11y agoMost companies that do this (unintentionally, I believe) are not benefiting from the actual work product, but from the consulting regarding their business problems. How much do you think a consultant would charge to come in for a few hours and discuss their needs, not to mention put some working code together for their problem?
- k__ 11y agoMost of the time I faced only absurd coding challenges that yielded stuff the companies couldn't use for anything. I still paid it with my time.
- brudgers 11y agoDiplomatically for Joe's sake: your standards of professionalism do not align with Joe's vision of company culture. Apparently, someone has mistaken cost-savings for a core company product line. Good luck.
- sidarok 11y agoIt has to be a throw-away scenario and a throw-away code. Anything else is wildly unethical.
- giaour 11y agoI think this is why problems that feature singly-linked lists are so popular: you can tell from the start that the problem is limited in scope and has nothing to do with any system currently in production.
- sidarok 11y agoIndeed. On the other hand the interviewing company should avoid such situations too because of the information secrecy - unless they are mandating an NDA with each and every candidate.
- rmc 11y agoI wonder about the copyright of such a thing. If you aren't an empployee and don't sign over rights, are they even allowed use it?
- bjourne 11y agoWhy don't you post your sample code on github? Maybe the CTO just wanted free work from you, but maybe he just wanted a candidate that was able to complete the assignment. He could have thought that you cherry-picked the easy cases and were unable to solve the tougher ones. Coding tests unfortunately aren't just about demonstrating skill. Then people like me could just link to their github QED. It's also about completing assignments and doing stuff because you are being instructed to.
- jschwartzi 11y agoMaybe the CTO should post his database on Github. If he wants the test code to function under certain conditions, he needs to publish those exact conditions. Not specifications, but the actual data that the test will be run against. If he's not comfortable doing that because it's restricted intellectual property, I might wonder what the real purpose of the test was.
- dmuth 11y agoIf they're running your code against actual production data, that crosses the line into questionable territory. I've been given programming tests before, but they were always for hypothetical problems that were open-ended. That way, each candidate solved the problem in a different way which was when discussed during the interview process. In your case, it sounds like they wanted you to solve a specific real world problem they are dealing with.
- cldellow 11y agoThat smells a bit fishy. I think you've dodged a bullet. I have to be a bit cautious here, because our company does something similar. We have a little programming challenge that we ask candidates to submit. We don't use those submissions in production. We find it gives people with non-traditional backgrounds a chance to show their chops, and gives us some meat to discuss during an interview. The challenge probably takes 4-8 hours, though. Part of me does think it's a little asymmetric for us as employers to be demanding this of candidates. We've also done week-long work tests, where we pay the person market rate to work with us on real-life code. Anecdotally, people view the paid week-long trial as demeaning, but the (unpaid) challenge as exciting and fun. We live in a bizarre world.
- swampthing 11y agoWhat we've found is that the best folks want a (paid) trial - not because they want us to trial them, but because they want to trial us. Works out for everyone.
- cldellow 11y agoThat's a good insight. We should make sure we frame the trial the right way so both sides see the benefit of it. One big downside of trials is they exclude people who are already employed...ah well, can't win 'em all.
- theothermkn 11y ago> Anecdotally, people view the paid week-long trial as demeaning, but the (unpaid) challenge as exciting and fun. We live in a bizarre world. That's because it is demeaning. It represents a large chunk of someone's life invested in proving that they're good enough for you. It's also inherently asymmetric in that the company you work for could not possibly be economically harmed by a week-long trial period, but a single person could be financially devastated by it. Additionally, if the person discovers an illness during that period, do you provide health insurance? What about employees coming from other employers? Do they have to quit their prior jobs or use up vacation time to complete your "interview" process? A four or eight hour programming exercise skirts some really stupid aspects of interviews and the hiring process. Of course that's welcome. A week-long commitment to work is a sizable tenure as a second-class citizen undergoing prolonged scrutiny. Your company should be deeply ashamed of this practice.
- davimack 11y agoAfter having been asked this several times, I've begun to make it clear that I will only engage in these tests if I'm compensated for the time. It's a delicate thing to do, really, telling someone that, no, you're not going to work for free, just on the hope that you might get a job. But it doesn't only happen in this industry: my niece is a graphic designer & has been asked to provide samples of her work on projects ... to the interviewer's specification. She's done it, because she really needed the work, but felt terribly taken advantage of, and I'd agree: it's predatory. I have the luxury of telling people that, no, I won't take their 1) IQ test, 2) Personality Test, or 3) their Coding Challenges. Not everybody does, certainly. The appropriate thing to do is to either state up front that you'd need to be compensated for your time, or acknowledge that you're giving away work as a way to possibly get in the door & just write the best solution you can write - not leaving it incomplete, necessarily, either, as that is a sort-of halfway position that makes you look bad.
- alphanumeric0 11y agoCompleteness is subjective here, but I understand where you're coming from.
- OpenDrapery 11y agoCurious. What is your objection to the personality tests? I took one for a new job once, and I felt uneasy about it, but can't really articulate why.
- RogerL 11y agoI don't know what one you took, but the one I was told to take (and vociferously declined) was somebody's unscientific hack of the MMBI. You know, the test that tests for mental illness. No one's business. Sorry, you don't get to know how many friends I have (a question), if I have nagging doubts (a question), and so on. It doesn't matter if those are problem spots for me or not, it is the principle. Also, that HR person is not a trained doctor. They have no business administering any kind of test of this type, and certainly have no business trying to interpret the results. It is just woo nonsense. Might as well try to do handwriting analysis, or read my tea leaves. You are not allowed (in the states) to ask interview questions about health. Yet, somehow, the courts have found these 'personality' tests legal. I don't understand, but I will not participate. I don't take drugs, but I won't take a pee test either.* * these results might vary depending on how close I am to foreclosure and how low my saving are. Principles are costly.
- agounaris 11y agoI have seen a lot of such cases as well, always the best is not to accept it in the first place and offer giving a test on an open source project for example.
- moogatronic 11y agoSomething similar happened to me, but I was never provided with the "programming test". My interview for a freelance gig was wrapping up, and the COO asked if I would be willing to "fix a few bugs in the system" to see if I were a "good fit" for the project. I asked point blank if they were asking me to do work for free. He responded that no, he was not, and that the task would take no more than 30 minutes to an hour. I said that I would be happy to take a programming test, or provide some other proof of competency, but that it's pretty non-standard to ask me to fix bugs in a system for free, especially for a limited hour freelance gig. Up until that point, we were discussing project time lines, handing over of logins, and my number of hours of availability. I got an email the next day saying that they were going to go another direction. Likely, trying to fund some other suckers to fix their bugs for free.
- cdnsteve 11y agoThis is like asking a carpenter to prove their skills by building you a deck, in a day, for free. Good luck. Any work performed should be paid, period.
- webtards 11y agoI have done at home prescreener tasks, as long as they were sub one hour, clearly academic, and patently not related to the business. These are not paid. I have also done on-site dev tasks and tests which were squarely related to the core business, and have invoiced my day rate each time, which was paid. If a company asks for more than an hour, for tasks related to their business, and don't want to pay, run, don't walk away, and better still, spread the word so people become aware and avoid them in the future.
- lexi-mono 11y agoYou working on a commercial project means that they can implement your solution and get money from it, so it'd be fair that you'd be compensated for your work. I'd ask to be paid for the amount of work that I'd be doing on this project or the option to get my skills assessed on some other, non-commercial test project that they wouldn't benefit from.
- fsk 11y agoI remember the guy who asked me, as a coding test, to write a program that connected to Interactive Brokers and executed VWAP trades. He got very angry when I offered to do it as a paid consulting project. If I did it for free, he wouldn't need to hire me at all!
- gtk40 11y agoI was recently applying for work after graduation. One entry level position and a small company (with a vague job posting) wanted me to complete a pretty extensive problem which would involve access to their code base and take "5 to 10 hours." I also would need to use a Mac and Objective C, even though these were not on my resume or the job posting. (I work in a .NET shop and had mentioned using Xamarin for Android development, and they were mainly doing iOS and OS X work, but extending into being cross platform). I asked if they had a problem I could do that would be on more familiar tooling, noting that even if I could borrow an OS X machine, it would take me a while to get familiar enough to complete the task they sent me. I also expressed concern at the length. I ended up declining to participate and they moved on. (Note: I have never used OS X besides briefly using someone else's computer. Everything I have used has been Windows and Linux, with a couple of weekends of trying some BSD-based distros thrown in.
- roneesh 11y agoIt sounds like a miscommunication between you two, nothing more than that. I would view this as a blessing though. For me, interviews are a chance to evaluate how well we can communicate. If I can't be relaxed, make jokes and just feel free to talk, then I won't feel that way working for you during the next year, and that to me, is a no go. I'd say indirectly you dodged a bullet, sounds like you two might be a high risk for constant miscommunication.
- steven2012 11y agoDon't bother continuing the interview. An employer should be more considerate of the interviewee's time. Imagine you worked for this person?
- edoceo 11y agoWhen I interview candidate I make them fix a problem in our real system. First by just talking it out (interview, unpaid) then I have them build it. Or what they can build in the time allotted. I pay for this time. Its basically hiring through short interview then real work on 1099style. Interviews you should do for free but fixing my real problems I pay for. I can decide who to hire with less time invested and know more about my new hire. #winning. Built a web-service/ATS to help me manage the while things
- theaccordance 11y agoShady code test or not, I don't agree with the manner in which you handled the situation. You agreed to do the test, submitted it partially completed, and then replied that it should be sufficient (knowing full well you didn't meet the requirements). I doubt the CTO didn't understand that the other cases would be trivial to implement; he simply sees a half-finished submission and a candidate who didn't correct the mistake when given the chance. Had this been a college exam and not a code test, would you expect a professor to award you a passing grade based on the entirety of the test, or the questions you chose to answer? One thing to keep in mind is that while you may view the code test as bad, it might not having malicious intent; some managers simply lack experience in putting together tests and fall back on old specs. There's two things I do during interviews prior to writing a single line of code: 1. Every code test I have taken has been brought up during a discussion. I use this opportunity to ask questions about the test. If you're interviewing with a recruiter who's not familiar with the test, they'll forward your questions to the manager who's evaluating the test for clarification. The manager's response will come either prior to accepting the assignment, or with the assignment attached to it. 2. Immediately upon receiving the code test, I review the spec in its entirety. If I have concerns with the spec, I raise those right away, as to not give the impression that I'm incapable of completing the spec if I bring up these concerns at a later time.
- herval 11y agoI interviewed at a rather fast growing startup in SF (hear about it here on hn pretty often). The interview was simply 'sit on that corner and fix this bug'. I said I don't work for free, so they said 'oh sure, we'll pay for that!'. Finished the bug fix, expecting a follow-up review or anything, so their CTO responded with 'oh, that was too fast. Hm, let me get you another bug.'. At that point I realised their git log must have at least a hundred different names on it. I politely refused and said if there wasn't no interview, I'd just leave. They gave me a check and that was that. One week later, the CEO emails me saying they'd love to hire me, but I'd have to quit my job first and they'd evaluate my case after 2 months with them (I was an h1b at the time). 'No thanks.'
- kasey_junk 11y agoLet me argue in favor of coding problems. Unpaid coding problems have a couple of very appealing properties: 1) They are available to all applicants (not all applicants have github profiles, or can show previous work samples, or are available for short term contracting work). 2) They do not cause problems with typical IP agreements or extra curricular work restrictions. 3) They can be standardized so all candidates receive the same process. 4) They are a closer proxy to the "real work" of development jobs than many other alternatives (and there is lots of research that indicates this leads to better results). Those properties are very important and cannot be found in hardly any other style of hiring filter. For this reason, its my belief that coding problems are central to a good hiring pipeline. I believe this so much that I'm willing to replace nearly every other part of the hiring pipeline with just the programming challenge (though that is a hard sell with management often). The problem with coding problems is that it is very hard to come up with good ones. They should take less than 4 hours to solve (everyone severely underestimates how long their challenges are), they should be indicative of the kind of work you will you do, and they should be, if not fun, engaging. So normally when I encounter bad programming problems I chalk it up to that instead of malice. It is everyones decision to not do programming challenges of course, and I completely understand people who will not do them. But to me the standardized hiring pipeline is the most important part and I'm willing to lose a few good candidates to get it. For this reason, I view a programming challenge as a good sign for a potential employer, and the lack of one to be a serious red flag. If someone is using a coding challenge to get free work, that is at the least unethical and likely against the law depending on your jurisdiction.