2 ms·
Sorry, but I write tests and I have absolutely no idea how you propose to measure these things objectively and automatically, without producing exactly the kind
by knzhou 7y ago
Sorry, but I write tests and I have absolutely no idea how you propose to measure these things objectively and automatically, without producing exactly the kind of "convergent" test you said you didn't like. For example, take "decision capacity". I don't even know what you mean by that, and I don't know how this will be assessed automatically without fixing a pre-designed correct answer.
Incidentally, the GRE has a system for grading essay composition automatically. It also absolutely sucks. It has no idea what your arguments are actually saying, so points are allocated based on irrelevant features like average sentence length or the total number of paragraphs. Researchers have succeeded in getting perfect scores by just copy-pasting the same sentence 25 times. In my book, this is worse than multiple choice.
- Animats 7y agoFor example, take "decision capacity". For that, there's an "inbox test", a set of incoming messages to be dealt with in order. Such tests are widely used to screen candidates for manager jobs.
- knzhou 7y agoWow, I just learned something new. These look legit and well-matched to the actual work. It does seem hard to make a "standard" decision test though, it seems each would have to be tuned to the job it was made for.