I wondered about that. My understanding is that the models were trained to look for letter shapes, not words. And that the models couldn't produce known words u

I wondered about that. My understanding is that the models were trained to look for letter shapes, not words. And that the models couldn't produce known words unless they were trained on the language. If it wasn't trained on a substantial text body, a model producing letter sequences that form known words means it found something and didn't hallucinate.