Well, the transformer was invented at Google, language models were decades old. But scaling it was not done before, and preparing the dataset at this size, and

Well, the transformer was invented at Google, language models were decades old. But scaling it was not done before, and preparing the dataset at this size, and babysitting the model so its loss doesn't explode during training - all were innovations that required lots of work and money to be achieved, so we can just copy the same formula without redoing all the steps.