"Fortunately, Uber AI’s Plug and Play Language Model allows researchers to make use of the few pretrained models out there: rather than requiring everyone to tr

"Fortunately, Uber AI’s Plug and Play Language Model allows researchers to make use of the few pretrained models out there: rather than requiring everyone to train their own wooly mammoth, PPLM lets users combine small attribute models with an LM to steer its generation. Attribute models can be 100,000 times smaller than the LM and still be effective in steering it, like a mouse sitting atop our wooly mammoth friend and telling it where to go (Figure 1)." Essentially, Uber put some attribute models on top of GPT-2 to steer how text is generated. This is better than finetuning the entire model which at the moment requires access to expensive TPUs or 32GB vRAM GPUs which are available to select research institutions/I think AWS may have them at a high cost. Finetuning is great when you want to specifically create poems or songs instead of some jumbled up string of words which may not be what you're aiming to create.