To add to it: a recent attempt at designing a psychometric test encoding human cognitive priors: https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.015

To add to it: a recent attempt at designing a psychometric test encoding human cognitive priors: https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 - the ARC dataset. https://github.com/google/BIG-bench https://github.com/google/BIG-bench - the most recent kitchen sink of a cognitive test, which includes the previous test as a subtask. Apparently there are large language models (Chinchilla) that perform comparably to a median human level on this second test.