There is barely any supervision: they are given a problem, and asked for a solution which matches certain criteria. You can't find the solution on Github. What'

There is barely any supervision: they are given a problem, and asked for a solution which matches certain criteria. You can't find the solution on Github. What's important is that it isn't a test of someone's intelligence or personality, but rather their pragmatism and ability to actually produce code. It really depends on what values you are looking for, though. An example problem is: parse this non-standard log file and report back some numbers/answer these questions (e.g. show a visitor funnel, based on uniques, etc.). The more encapsulated and defined the problem, the better. It's not a theory type question, such as build a btree search of this data: it's something which you could face while working at the company and you can solve it however you want. Just like in real life, there are a range of solutions and edge cases. There is a soft time limit of a few hours, but nobody is counting. However, that time limit is what changes the difficulty. They can either use their own machine or a company machine, and any language they want (with some recommendations). They can of course ask questions anytime they want, but other than that they are left on their own. At the end, they are asked to walk through the solution and tell us why they chose to do what they did, what issues they had, if they see alternative ways to do something, how they might change it if various conditions changed, etc. It's the opening to a conversation. Afterwards we rarely try to actually run the program, but different people on the team are asked what they think of the solution (kind of like a code review). Since everybody has seen it many times before, there is general consensus on red flags and everybody more or less has an idea of how a particular solution fits in with other ones we've seen, but sometimes there is debate on merit or style. So it isn't that time consuming from our point of view.