Well, yes, but the tokenization schemes are causing these models to struggle on actually following syntactic rules in poetry, e.g. syllable counts in hiakus, rh

Well, yes, but the tokenization schemes are causing these models to struggle on actually following syntactic rules in poetry, e.g. syllable counts in hiakus, rhymes. I actually wrote an entire paper which gwern cited about how to make LLMs always follow these kind of constraints with no mistakes. https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai https://paperswithcode.com/paper/most-language-models-can-be... If you don't believe this is a problem, try getting ChatGPT to write a paragraph of correct English which omits words which use the letter "e" in it. Too bad you can't use my technique on ChatGPT since they don't expose their output probability distribution...