3 ms·
JEPA is not an alternative to transformers, it is built out of transformers.
by Legend2440 1y ago
JEPA is not an alternative to transformers, it is built out of transformers.
- HarHarVeryFunny 1y agoJEPA is trying to learn real world regularities by predicting internal perceptual representations (using a transformer). It seems a good direction to go in, but tbh he doesn't seem to have taken it very far, and it is the transformer doing all the heavy lifting.