Some points that stood out: - Hierarchical learning will be needed for AGI. The intelligence to decide whether to turn the steering wheel left or right based o

Some points that stood out: - Hierarchical learning will be needed for AGI. The intelligence to decide whether to turn the steering wheel left or right based on visual stimuli is at a different level of abstraction (and requires different input representations) to what's needed to plan and execute a long trip from A to B. Those levels need to be combined somehow and we have little idea how. - RL is misguided because it tries to do too much with sparse rewards. Representation learning should be done first followed by the RL step once the representations are learned. - JEPA tries to get an encoder that generates the same representations from masked inputs without needing to have a contrastive loss function. It works better than masked autoencoder which has failed to beat supervised learning at image recognition. V-JEPA is the video version and can be used on get a world model by masking future video frames and adding action pairs, and then predicting those future representations. The discussion around open source was underwhelming. No mention of any points any detractors make, as if he doesn't even understand what it is he's arguing against.