6 ms·Byte-Level Tokenizers Unavoidably Enable LLMs to Generate Ill-Formed UTF-82 points by PaulHoule 1y agorini17 1y agoYou can enforce grammar. Llama.cpp can do that, without any need to change models.