Interesting! Are you able to share what you came up with? It seems like at minimum you could verify that the sentiment analysis for your generated text matches

Interesting! Are you able to share what you came up with? It seems like at minimum you could verify that the sentiment analysis for your generated text matches that of the input prompt. I've been working on getting GPT2 to write poetry and I've made a fair amount of progress by fine-tuning on a poetic corpus. Then during generation I constrain tokens that would break the style of the previously generated text. It has already produced a few samples that are surprisingly moving. Here's one of my favorite cherry-picked samples which was generated with a prompt of "The sea grew": "The sea grew angry, and turned to fire, And all the stars did mourn for earth and sky. The wind did moan, as when one who hath lost Love through some evil counsellor, hears A lamentable voice that sobs and sighs; Or when some old sorrow's semblance doth appear" It makes me think of someone describing the meteor strike that killed off the dinosaurs. To achieve this, I simply perform analysis on the tokens as they are generated, then turn off the logits for tokens that would break the current flow. I'm currently working on enforcing meter and rhyme using a similar approach. I've also had success with creating checkpoints by pickling the model state at a good stopping point, like at the end of a line, then letting the model do its thing and reverting back to the checkpoint if it produces something that doesn't match what I'm looking for.