PPO for RL. Plus a lot of the people behind the innovations you mention are now at OAI - Lukas who was one of the Transformer authors, Bert paper author, chain

PPO for RL. Plus a lot of the people behind the innovations you mention are now at OAI - Lukas who was one of the Transformer authors, Bert paper author, chain of thought prompting author, etc. But ya their strong point is execution and doing the hundreds of little things that make the model do well and it turns out that that's more important than "novel" ideas