I disagree that gradient descent brute force extracts the training set. That "overfitting" kind of thing has been shown to be false many times. Transformers lea

I disagree that gradient descent brute force extracts the training set. That "overfitting" kind of thing has been shown to be false many times. Transformers learn predictive models of their input, beyond what their training set contains. But in general I agree about active inference. Clearly there is something missing there. Doing alpha-go style MCTS would be interesting but how would you approach training the policy and value net? It's not like we can take snapshots of people's thought processes as they read text in the same way you can perform arbitrary rollouts of your game engine.