Indeed. What has caught researchers off guard is the way system 2 properties seem to appear as emergent phenomena in LLMs. This is also what has prompted people

Indeed. What has caught researchers off guard is the way system 2 properties seem to appear as emergent phenomena in LLMs. This is also what has prompted people like Hinton and Sutskever to make this condensed point about statistical modelling and understanding (hypothetically) being simply matter of a spectrum.