3 ms·
The Colab notebook linked in the article (which I made) is designed to make it as user-friendly as possible to finetune GPT-2 and generate text from it. Unfortu
by minimaxir 7y ago
The Colab notebook linked in the article (which I made) is designed to make it as user-friendly as possible to finetune GPT-2 and generate text from it. Unfortunately "as possible" is load-bearing in this case, as GPT-2 is slightly more difficult to work with.
In the case of short-form content like this, using a single-column CSV with each document as a row will automatically prepend/append the texts with appropriate start/stop tokens, which you can then force the model to generate text within those bounds and regex out the text you want. (this is what gpt-2-simple does with the prefix, truncate, and include_prefix parameters to generate())