3 ms·
I disagree. If I tell you that argoubr + vliugo - viyt = plqtv, do you now understand what any of those words mean? If I say that "gye" must always precede "tur
by YeGoblynQueenne 8y ago
I disagree. If I tell you that argoubr + vliugo - viyt = plqtv, do you now understand what any of those words mean? If I say that "gye" must always precede "turo", do you know what "gye turo" means, or why it doesn't make sense to say "turo" without "gye"? What kind of meaning is captured by the relations between these words that these rules I just described, define? Would it help if I gave you all the possible ways that every word in the language those words are drawn from are to be articulated together, without telling you what any of those wors mean? And how would I explain those words' meaning without recourse to words whose meaning you already know?
Also, that's what I mean by "structure" - the context in which tokens are found in text. I don't want to say "syntax" or "semantics" because it seems to me those are loaded terms that have very specific, er, meaning and I am not a linguist to be able to unpick it.
- yorwba 8y ago> Would it help if I gave you all the possible ways that every word in the language those words are drawn from are to be articulated together, without telling you what any of those wors mean? And how would I explain those words' meaning without recourse to words whose meaning you already know? That's why I linked the two papers on unsupervised machine translation. Given enough text in an unknown language, it is already possible to turn those unknown words into words which humans understand.
- YeGoblynQueenne 8y agoBut that's humans. Humans already understand language. I don't disagree that humans can extract meaning from language models. But that only speaks to the ability of humans to extract meaning from even very badly formed language. Not to the ability of those models to represent meaning. Edit: OK, I don't think I understand what you, er, mean. I think you're saying that language models essentially have a kind of understanding of language, and not just the ability to reproduce it without understanding it. Is that what you are saying?