It's possible. We're talking about pretraining meaningfully larger models past the point at which they plateau, only to see if they can improve beyond that plat

It's possible. We're talking about pretraining meaningfully larger models past the point at which they plateau, only to see if they can improve beyond that plateau with RL. Call it option (3). No one knows if it would work, and it would be very expensive, so only the largest players can try it, but why the heck not?