> Unlike BERT, GPT models are unidirectional, and their main advantage is the magnitude of data they were pretrained on: GPT-3, the third-generation GPT model,

> Unlike BERT, GPT models are unidirectional, and their main advantage is the magnitude of data they were pretrained on: GPT-3, the third-generation GPT model, was trained on 175 billion parameters, about 10 times the size of previous models. Although the number of parameters of a Transformer correlates with the amount of training data, this statement is misleasing. Precisely, the model wasn't trained on parameters but its parameters were trained on data.