I recently took an artificial intelligence class that required writing an agent to answer non-verbal IQ test questions. One of the greatest challenges was that

I recently took an artificial intelligence class that required writing an agent to answer non-verbal IQ test questions. One of the greatest challenges was that some of the questions had ambiguous answers. For example, an axial symmetry in the example frame could be reproduced with either a rotation or a mirror operation in the output frame, and both options were available as candidate solutions. Naturally, the question came up: how do we define the "right" answer when there are multiple valid choices? As it turns out, the folks who write the test determine the correct answer to these types of question based on the most common response from test takers who get the highest scores on non-ambiguous questions. The thinking goes that if folks who get most of the other questions right largely agree about a preference for one answer over the other, that must be the most correct answer. The most interesting part of that discussion to me was when a classmate observed that we use human intelligence as a baseline for measuring human intelligence, and then marvel at the extent of our own intelligence; if a machine used more sophisticated logic than a human was capable of to choose a different answer, we would look down on the machine for being less intelligent than man. It reminded me very much of Plato and the shadows in the cave.