2 ms·
Ooh, that sounds fun. In terms of process design, I think you could take a hint or two from the Netflix guys here. Currently you're distributing a single test
by patio11 17y ago
Ooh, that sounds fun.
In terms of process design, I think you could take a hint or two from the Netflix guys here. Currently you're distributing a single test case. That takes very little work to accomplish: good first step! Consider creating two test cases: one we'll call the probe case, and one we'll call the real case. The real case is kept on your servers and never gets exposed directly to the participants.
Make a really freaking simple way for you to check your work on the probe case automatically, without needing to speak to you guys. For example, you could make a simple REST API or even a web page with an upload/submit button. Then run whatever your evaluation function is against the submitted solution for the probe case, and tell folks how good their submission was. For an extra ten minutes of work you can add a leaderboard. (Add a leaderboard!) You can optionally rate-limit this if you're worried about people overfitting to the probe via taking a lot of measurements.
This has a better user experience and will get you more viral exposure because a) you've now totally decoupled your ability to run code from people's ability to participate in the contest and b) people can now get immediate feedback that their participation is valued and producing results. Including a leaderboard also gives folks an emotional hook both for participating and for spreading word of your contest: "Oh cripes guys, the top submission for this problem was done in Ruby. Let's show them the awesome power of functional programming. For the greater glory of Lisp!"
Of course, you can feel free on putting whatever restrictions you want on code which you individually invite for offline exposure to the real test case.
- threepointone 17y agoNow you have 2 problems :)