I wrote a whole paper about ways to "fix" tokenization in a plug-and-play fashion for poetry generation: Filter the vocabulary before decoding. https://papersw

I wrote a whole paper about ways to "fix" tokenization in a plug-and-play fashion for poetry generation: Filter the vocabulary before decoding. https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai-1 https://paperswithcode.com/paper/most-language-models-can-be...