3 ms·

>To that end, I'd like to see what you get if you first fine tune on RapGenius to learn general things like song structure, rhyme, etc, then fine tune even furt

>To that end, I'd like to see what you get if you first fine tune on RapGenius to learn general things like song structure, rhyme, etc, then fine tune even further on the Beatles corpus. I suspect you'd get much nicer, less memorized songs. OT: Is that how fine-tuning actually works with GPT-2? It makes sense that it'd just be strengthening connections on the most-recently-fine-tuned corpus, with previous fine-tunes still around in some way. Should you expect that first fine tune to pick up and solidify song structure, rhyme, etc, and the second fine tune to keep those concepts in place while muddying up other aspects like the specific lyrics used? (Hope this doesn't come off as "you're wrong" or too off topic -- I'm just very interested and would love to read more about how all this works. :) )

eugenhotaj 7y ago
I would expect it to (but I haven't thought about it too deeply so I could be extremely wrong). My thinking is as follows: At the end of the day, all we're doing is maximum likelihood estimation. So we're trying to find model parameters which define a probability distribution where our observed data is the most probable. In the original GPT-2, this observed data is the text from quality outgoing links on Reddit. Since this data is so diverse, there will not really be any special structure that the model can pick up on, besides whatever structure exists in the English language. However, when we fine-tune on RapGenius, the observed data is now songs. These songs have a certain structure to them such as stanzas, rhyming, etc. In order to maximize the likelihood of this data, the model must learn to model the structure. Finally, if we further fine-tune on Beatles lyrics, the model is again trying to find parameters which maximize the likelihood of the data. So the model will try to match both the lyrics and the structure of Beatles songs. It's likely that the structure of Beatles songs is pretty similar to the other songs from RapGenius, so mostly what will change are the lyrics. Also, changing the lyrics seems to be the most straightforward way to maximize the likelihood since by definition we want these particular lyrics to be the most likely. That being said, this is all just conjecture. It would be interesting to try out both methods and see if you get better results doing this two step fine-tuning vs the original fine tuning (or just fine tuning on RapGenius then conditionally sampling Beatles songs as @gwern suggested).

eugenhotaj 7y ago

I would expect it to (but I haven't thought about it too deeply so I could be extremely wrong). My thinking is as follows: At the end of the day, all we're doing is maximum likelihood estimation. So we're trying to find model parameters which define a probability distribution where our observed data is the most probable. In the original GPT-2, this observed data is the text from quality outgoing links on Reddit. Since this data is so diverse, there will not really be any special structure that the model can pick up on, besides whatever structure exists in the English language. However, when we fine-tune on RapGenius, the observed data is now songs. These songs have a certain structure to them such as stanzas, rhyming, etc. In order to maximize the likelihood of this data, the model must learn to model the structure. Finally, if we further fine-tune on Beatles lyrics, the model is again trying to find parameters which maximize the likelihood of the data. So the model will try to match both the lyrics and the structure of Beatles songs. It's likely that the structure of Beatles songs is pretty similar to the other songs from RapGenius, so mostly what will change are the lyrics. Also, changing the lyrics seems to be the most straightforward way to maximize the likelihood since by definition we want these particular lyrics to be the most likely. That being said, this is all just conjecture. It would be interesting to try out both methods and see if you get better results doing this two step fine-tuning vs the original fine tuning (or just fine tuning on RapGenius then conditionally sampling Beatles songs as @gwern suggested).