2 ms·
The point of this "IQ Test" is to set a relatively low-bar for passing the IQ test question so that even intellectually lazy people can get an intuitive feel fo
by jabowery 3y ago
The point of this "IQ Test" is to set a relatively low-bar for passing the IQ test question so that even intellectually lazy people can get an intuitive feel for the limitation of Transformer models. This limitation has been pointed out formally by the DeepMind paper "Neural Networks and the Chomsky Hierarchy".
https://arxiv.org/abs/2207.02098 https://arxiv.org/abs/2207.02098
The general principle may be understood in terms of the approximation of Solomonoff Induction by natural intelligence during the activity known as "data driven science" aka "The Unreasonable Effectiveness of Mathematics In the Natural Sciences". Basically, if your learning model is incapable of at least context sensitive grammars in the Chomsky hierarchy, it isn't capable of inducing dynamical algorithmic models of the world. If it can't do that, then it can't model causality and is therefore going to go astray when it comes to understanding what "is" and therefore can't be relied upon when it comes to alignment of what it "ought" to be doing.
PS: You never bothered to say whether the program you provided was from an LLM or from yourself. Why not?