Unfortunately, the ways AI researchers measure and compare their results to human performance are often rather weird and favor algorithms via.. let's say "unusu

Unfortunately, the ways AI researchers measure and compare their results to human performance are often rather weird and favor algorithms via.. let's say "unusual" experiment setup. For example, face recognition performance is often measured by comparing two random faces taken from a sample set. In real life, the "problem" is almost always to match a random face against the entire set of faces you remember. Performance in these two tasks is not at all equivalent. I've seen similarly weird problem formulations in many other AI research papers.