I was collecting examples of models trained on single GPU or with very low cost. A number of projects used Bert or GPT to since the implementations were very si

I was collecting examples of models trained on single GPU or with very low cost. A number of projects used Bert or GPT to since the implementations were very simple with with some components optimized. There’s also a lot of projects that have trained Bert in GPT two models which make for more scientific comparisons. With no other information, those would be my guesses as to why one would use a GPT2 model.