That seems unlikely to me. Models are multi-modal now, so they'll likely train on video and audio content too. Once that's tapped out they can learn better worl

That seems unlikely to me. Models are multi-modal now, so they'll likely train on video and audio content too. Once that's tapped out they can learn better world models by not passively ingesting data but interacting with it, e.g. via simulated environments or capturing human-bot interactions that are flagged as interesting and incorporating those in the next run.