There's pretraining which is just raw text from the internet but there's also supervised preference data sourced from users. Right now the edge is in acquiring

There's pretraining which is just raw text from the internet but there's also supervised preference data sourced from users. Right now the edge is in acquiring the latter which OpenAI has a slight lead in