It's the latter. The current approach is akin to throwing the entire internet at the foundational model, train it for an epoch or two, and that's that. Afterwar

It's the latter. The current approach is akin to throwing the entire internet at the foundational model, train it for an epoch or two, and that's that. Afterwards, the finetuning with curated training material, and techniques like RLHF (Reinforcement Learning from Human Feedback) takes place. The sources of the training material are... questionable, to say the least. There's a reason the training dataset for GPT 3.5 and 4 remains undisclosed.