4 ms·
>Training transformers to predict the next item in sequences eventually forces them to learn a function that approximates a world model There is absolutely no
by theGnuMe 3y ago
>Training transformers to predict the next item in sequences eventually forces them to learn a function that approximates a world model
There is absolutely no proof for this statement.
- H8crilA 3y agoHave you used ChatGPT? It can do (at least) simple reasoning, for example simple spatial reasoning or simple human behavior reasoning.
- theGnuMe 3y agoI suggest you start here: https://ahtiahde.medium.com/limits-of-turing-machines-and-algorithmic-finger-prints-f33803ecbb43 https://ahtiahde.medium.com/limits-of-turing-machines-and-al... If you don't have a CS background, I would suggest reading the wikipedia entries referenced in the medium article as well.