2 ms·
After skimming the original paper[1] there are some things that I expected, but found missing. Further study (as one does here before posting) finds that there
by mikewarot 4y ago
After skimming the original paper[1] there are some things that I expected, but found missing. Further study (as one does here before posting) finds that there is a new term used to describe surprise, one of the basic states that we are very aware of, and very important to learning.
In the section describing Joint Embedding Predictive Architecture (JEPA), "prediction error in representation space" appears to me, at least, to match up with surprise.
I think most early childhood learning is about avoiding surprise, and thus coalescing and then fine-tuning models of the world.
The one thing I'd add to his model is the feedback we all have in sensing our energy levels, becoming tired, etc. I think even in a completely artificial environment it might be helpful for an intelligence to be able to gage how much effort is required in thinking, and moving, etc.
[1] https://openreview.net/pdf?id=BZ5a1r-kVsf https://openreview.net/pdf?id=BZ5a1r-kVsf