I attended the ACL talk where he presented this. While the method was reasonably well-justified, it seems clear that the data was flawed, leading to a flawed mo

I attended the ACL talk where he presented this. While the method was reasonably well-justified, it seems clear that the data was flawed, leading to a flawed model, leading to a false result. The author failed to ask himself the question "what if this didn't work?". Here's what I would identify as the main flaw: To identify the language of a centuries-old document, he trained his model on a modern multilingual text -- the Universal Declaration of Human Rights. (A questioner in the audience asked "If you needed a document in hundreds of languages, why didn't you use the Bible?", to which he had no real answer.) So he ran this somewhat anachronistic model and it told him that the text was a good statistical match for being in... Esperanto. He threw this prediction out on the basis that it makes absolutely no sense. You'd think this would be the first sign that the model did not fit the data. The #2 prediction was Modern Hebrew, which of course did not exist in the 15th century, so he said "that's close enough to biblical Hebrew" and claimed that as the result. He asked a Hebrew speaker he knew to decipher a couple of sentences based on the model. The Hebrew speaker could not actually decipher them. The author pressed on and the Hebrew speaker obligingly produced some unsatisfying guesses at sentences, which sound remarkably like sentences from unsuccessful attempts to decipher Linear B texts. And I don't understand why anyone would so confidently report a modification of the #2-ranked prediction of the model. We might as well go ahead and say "AI confirms: the Voynich Manuscript is in ancient Esperanto!" The much more realistic interpretation is that the Voynich text matches the statistics of none of the language samples that the model was trained to recognize, and the model is overconfident in the probabilities it outputs. I think the fact that the method was cool must have been part of the paper's acceptance at ACL. But it does not give us a real answer about the Voynich manuscript.