Well, I would say we are learning how such a test really has to look. In a sense it‘s like we are learning what it should look like at a time. Additionally, h

Well, I would say we are learning how such a test really has to look. In a sense it‘s like we are learning what it should look like at a time. Additionally, humans make mistakes, too, especially when they are too lazy to think. Currently the test is still good enough. But in the long term, we should change it to test for transfer learning: Given a new task that is designed to be sufficiently complex and different from known tasks, how well does the AI do? It will require creativity to design those tests. But we‘re still far from that.