JEPA is trying to learn real world regularities by predicting internal perceptual representations (using a transformer). It seems a good direction to go in, b

JEPA is trying to learn real world regularities by predicting internal perceptual representations (using a transformer). It seems a good direction to go in, but tbh he doesn't seem to have taken it very far, and it is the transformer doing all the heavy lifting.