3 ms·
Super cool! Maybe I missed it, but it didn’t seem to me like the authors presented any reasoning about what makes this capability particular to transformers… e
by szvsw 2y ago
Super cool!
Maybe I missed it, but it didn’t seem to me like the authors presented any reasoning about what makes this capability particular to transformers… especially on the relatively small HMMs, it’s not clear to me that we wouldn’t see the same behavior with some other model performing prediction, but maybe I missed that.
Obviously it makes sense to focus on transformers given their ubiquity in SotA research, but it would be interesting to see how “special” this property is given that there is an implicit suggestion that it will continue to be true on larger problems which currently are primarily only solved with transformers.
On the other hand, we have things like selective state space models (most notably, Mamba) which very explicitly of course represent system state within the architecture of the model itself, and are promising alternatives to transformers.
Anyways, very cool post!
- gwern 2y ago> Maybe I missed it, but it didn’t seem to me like the authors presented any reasoning about what makes this capability particular to transformers… especially on the relatively small HMMs, it’s not clear to me that we wouldn’t see the same behavior with some other model performing prediction, but maybe I missed that. I don't believe they are claiming that? There is no reason to think that that would be the case (Transformers are in general grossly overrated in terms of magic pixie dust), the fundamental theory here implies that any predictor achieving high performance is going to have to do this, and in the comments, people point out examples elsewhere like with RNNs where researchers have read off the beliefs from the hidden state and found the same thing: https://www.lesswrong.com/posts/gTZ2SxesbHckJ3CkF/transformers-represent-belief-state-geometry-in-their?commentId=mMvq339M52KBG7q2s https://www.lesswrong.com/posts/gTZ2SxesbHckJ3CkF/transforme...
- int_19h 2y ago"A final theoretical note about Computational Mechanics and the theory presented here: because Computational Mechanics is not contingent on the specifics of transformer architectures and is a well-developed first-principles framework, we can apply this framework to any optimal predictor, not just transformers."