3 ms·
Solving this puzzle is an excellent example of what Ilya Sutskever said recently in the Lunar Society Podcast ..... "It seems predicting the next token well me
by jimmySixDOF 4y ago
Solving this puzzle is an excellent example of what Ilya Sutskever said recently in the Lunar Society Podcast ..... "It seems predicting the next token well means that you understand the underlying reality that led to the creation of that token"
It is the phase shift increases at this meta associative layer (which nobody seems to have seen coming from LLMs or so soon) that are responsible such feats of apparent comprehension of the question even when the answer provided at the end is wrong. The question now is if bigger training sets et al will lead to more reliable answers. TBD.
- uoaei 4y agoUltimately it seems that is the case. However a 1D model of the world is much less useful than an N-D model of the world that is subsequently projected to 1D. Until we build architectures that allow for more "reason space" in the model, we will continue to see obvious blunders.