As I said, the problem is writing the tests and knowing what to measure. How do you discern between a bunch of tests and measurements that are poorly-designed,

As I said, the problem is writing the tests and knowing what to measure. How do you discern between a bunch of tests and measurements that are poorly-designed, and a bunch that are well-designed? You gather a bunch of people with good taste, and trust their gut. But at that point, why not just cut out the middleman, forget the tests, and let the artists be artists? See also: Footnote 3 of http://www.paulgraham.com/colleges.html#f3n http://www.paulgraham.com/colleges.html#f3n