2 ms·
Interesting, I was also going to suggest some kind of alternative review system. I was invited to review applications to CORFO (the government program behind S
by GFischer 11y ago
Interesting, I was also going to suggest some kind of alternative review system.
I was invited to review applications to CORFO (the government program behind Startup Chile), and since they crowdsource reviews between unrelated startup founders, they have a "points" system to determine whether submissions were reasonably graded (they look for big differences between graders).
Your system sounds better :) , but I think they want to promote individual discussion of submissions too.
- zaguios 11y agoYea, a point system would work decently, but as you mentioned there is the problem where certain reviewers give higher and lower ratings than one another. With a large enough sample size you should still get fairly accurate ratings. A 1 vs 1 system based on elo eliminates all of these issues though and helps you find accurate ratings much faster. The system doesn't need to stop individual discussion, you should be allowed to post responses to the applications as your are reviewing them to vote on. You should also be allowed to directly view and comment on an application by going to it directly as long as you aren't allowed to do anything that affects its rating.