3 ms·
As it happens, nshepperd ran his finetuning GPT-2-small on our IRC channel. I'd tried it before with char-RNN back in 2015 or so, and I have to say, GPT-2-small
by gwern 8y ago
As it happens, nshepperd ran his finetuning GPT-2-small on our IRC channel. I'd tried it before with char-RNN back in 2015 or so, and I have to say, GPT-2-small trained way faster and better than my IRC char-RNN did.
The samples also looked a lot better than OP's. I assume that's because he ran it for more like a day on a few hundred MB of chat logs.
- deleted 8y ago[deleted]
- gwern 8y agoI didn't run on IRC because my GPUs were busy with poetry, FWIW: https://www.gwern.net/RNN-metadata#finetuning-the-gpt-2-small-transformer-for-english-poetry-generation https://www.gwern.net/RNN-metadata#finetuning-the-gpt-2-smal... https://slatestarcodex.com/2019/03/14/gwerns-ai-generated-poetry/ https://slatestarcodex.com/2019/03/14/gwerns-ai-generated-po... http://sevensecularsermons.org/on-the-significance-of-gwerns-poem-generator/ http://sevensecularsermons.org/on-the-significance-of-gwerns...
- deleted 8y ago[deleted]