The vast majority of GPT4's learning (in terms of compute time) isn't reinforcement learning, it's training the transformer to predict the next token in the seq

The vast majority of GPT4's learning (in terms of compute time) isn't reinforcement learning, it's training the transformer to predict the next token in the sequence. Doing this with sufficient accuracy requires it to build an internal model of the world. We don't know exactly how human brains learn internally, but if we treat it as a black box, then like GPT4 it takes in words/texts and learns to output words/text that make correct inferences about the world.