3 ms·

Not sure how? As a thought experiment, say I created two languages completely from scratch : symbols, words, sentence order, etc. One of these languages, I've

Not sure how? As a thought experiment, say I created two languages completely from scratch : symbols, words, sentence order, etc. One of these languages, I've given meaning to the words, the other is random. However I create massive corpuses for each of hundreds of thousands of documents. Now go ahead and machine learn the hell out of both languages. First question : which one is the 'random' language? Second question : what is the meaning of the non-random language texts? Assume I have not based my 'real' language on any existing example ...

nopinsight 5y ago
If the machine or human is sufficiently intelligent and there is enough data and time, it is possible to do so as long as the non-random language is useful for human communication and the language sample describes some experiences or things we understand. There are principles underlying all human languages. For example, they are used to describe the world or communicate intention and they must be understandable by a human. Given that the copious data of the non-random language should describe something we understand about the world or social dynamics, we can try to match them with world/domain models and find patterns of plausible interpretations that fit. The patterns should also satisfy the principle of communication economy and the limits of human cognition, such as the size of working memory. These constraints help narrow down the possibilities. The human/AI system can test and figure out the most likely patterns first and use them to decipher more patterns. All above is possible in principle but beyond the current state-of-the-art.
- rvense 5y ago
  This is true, but you're forgetting that the sign (ie. pairing of an expression and its meaning) is arbitrary. There's no particular reason that the sound 'cow' (or the squiggles c-o-w when we write) refers to cows, and you can see that quite clearly by the fact that it only does so in one out of the Earth's 6-7000 current languages. Note that this doesn't mean that relationships between words are arbitrary, they are of course highly systematic, else language wouldn't work. It also doesn't mean that language doesn't have history, the changes are also systematic. But if you take a random surface form (sound or writing), there is absolutely no way to relate it with any certainty to any content without a context that allows you to infer it.
  - eru 5y ago
    You are being too pessimistic. Yes, it's hard to figure out cow just from a big corpus; though you might learn enough about 'cow' to form meaningful sentences and answer questions. That's what GPT-3 is doing after all, and language models are still improving quickly. And we do have context! Just knowing that a language is used by humans gives lots and lots of context. Human cultures and languages have more in common than you might realize. For example, have a look at Deixis (https://en.wikipedia.org/wiki/Deixis https://en.wikipedia.org/wiki/Deixis): > In linguistics, deixis (/ˈdaɪksɪs/, /ˈdeɪksɪs/)[1] is the use of general words and phrases to refer to a specific time, place, or person in context, e.g., the words tomorrow, there, and they. Words are deictic if their semantic meaning is fixed but their denoted meaning varies depending on time and/or place. Words or phrases that require contextual information to be fully understood—for example, English pronouns—are deictic. Deixis is closely related to anaphora. Although this article deals primarily with deixis in spoken language, the concept is sometimes applied to written language, gestures, and communication media as well. In linguistic anthropology, deixis is treated as a particular subclass of the more general semiotic phenomenon of indexicality, a sign "pointing to" some aspect of its context of occurrence. > Although this article draws examples primarily from English, deixis is believed to be a feature (to some degree) of all natural languages. I suspect with a large enough corpus you can most likely figure out the deixis of the given language. See https://linguistics.stackexchange.com/a/4046 https://linguistics.stackexchange.com/a/4046 for some more structure shared with all human languages: > However, all languages have some sort of a clause-type thing allowing them to express predication, attribution, etc. See Dixon's Basic Linguistic Theory: Basic Linguistic Theory Volume 1: Methodology . All languages also must have means of expressing cohesion and coherence (texture) although this is much less studied in cross linguistic perspective. Punctuated sentences are a kind of cohesive device. (A comment on this answer points out that the reality is probabilistic, of course:) > While I agree with you, it seems to me that linguists who study languages with a strong written tradition often talk about 'sentences', even when their examples are not from writing. Those of us who work on previously unwritten languages (I think) tend to talk about 'texts' or 'utterances'. I think this is an important issue as it relates to how some linguists have ignored the true messiness that is often found in examples of human language. Of course, all of this refers to natural human languages. Not totally arbitrary constructs.
thom 5y ago
I don't think it would be very hard to detect which of the two is random, because meanings are likely to be somewhat correlated. Obviously you could model your 'random' language to be statistically similar to the meaningful one though, if you were feeling particularly evil, but then you'd be imbuing it with some small degree of meaning, arguably. Whether you can go from this statistical analysis to anything with absolute meaning, I don't know. I suspect _some_ extremely common concepts (pronouns, verbs like to be or to have) might yield to this sort of direction.
- rvense 5y ago
  But the meaning is not in the text (text-as-artifact, the ink on the page), it is strictly in the mind of the reader, accessed by reference to the speaker's experience with the language in question. If you don't have something to ground at least part of the symbols in, there is nothing to work from. Any word can be assigned any meaning, and there would be infinite possibilities. No amount of statistics will allow you to work out whether a given word or part of a word refers to a rock, a rabbit or Noam Chomsky's granddaughter. How could it? To be able to do this is called knowing a language, and if all the people who had that skill in the soft tissue of their brain have kicked the bucket, tough luck, you can't get that knowledge back. All (bar none) decipherments of unknown writing systems have been done by reference to related languages, especially the existence of bilingual texts. Computers and statistics are definitely able to help us do that, but if all you have is a page of text with no context, then the information you're looking for is gone.
  - thom 5y ago
    I agree to a very high degree! But I still think that you have greater than zero chance (but not much greater) to make some correct guesses about some very, very basic grammatical forms based on frequencies within other languages. You'd never be able to confirm this, and it's not a very important point. However, I do strongly disagree with the other point in the comment I was responding to, that you could not tell a real language with meaningful words from one with random words. That (entirely made up and not very helpful) problem does seem like it would be amenable to basis statistical analysis.
mcswell 5y ago
The first question is--if I understand what you're proposing--(relatively) easy. The second is much harder, perhaps impossible. The reason the first question is relatively easy is because of syntax (sentence order, but also the order of words in phrases). If it's a real language, that order is not random; the English word "the" shows up before "man" much more often than it does before "listens", and so for other words. This is true even in so-called free word order languages, which are really free phrase order languages. The other thing that could make the first question relatively easy is morphology. If you've decided to make a language that has inflectional morphology (in English, suffixes like -ed and -ing), that also helps determine whether the language is "real", because in a real language not all words will take the same affixes--you can't say 'theseing' (these-ing) in English. In other words, the affixes that you find on a given word are not random. Of course if you decide not to give your language inflectional morphology, this won't help--but if you do that, then your language's syntax will have to be more rigid: in a language with case marking affixes, like Latin, you don't need word order to tell which noun is subject and which (if any) is object; but you need a more or less fixed word order in English, because (apart from pronouns) we don't have any case marking.

nopinsight 5y ago

If the machine or human is sufficiently intelligent and there is enough data and time, it is possible to do so as long as the non-random language is useful for human communication and the language sample describes some experiences or things we understand. There are principles underlying all human languages. For example, they are used to describe the world or communicate intention and they must be understandable by a human. Given that the copious data of the non-random language should describe something we understand about the world or social dynamics, we can try to match them with world/domain models and find patterns of plausible interpretations that fit. The patterns should also satisfy the principle of communication economy and the limits of human cognition, such as the size of working memory. These constraints help narrow down the possibilities. The human/AI system can test and figure out the most likely patterns first and use them to decipher more patterns. All above is possible in principle but beyond the current state-of-the-art.

rvense 5y ago
This is true, but you're forgetting that the sign (ie. pairing of an expression and its meaning) is arbitrary. There's no particular reason that the sound 'cow' (or the squiggles c-o-w when we write) refers to cows, and you can see that quite clearly by the fact that it only does so in one out of the Earth's 6-7000 current languages. Note that this doesn't mean that relationships between words are arbitrary, they are of course highly systematic, else language wouldn't work. It also doesn't mean that language doesn't have history, the changes are also systematic. But if you take a random surface form (sound or writing), there is absolutely no way to relate it with any certainty to any content without a context that allows you to infer it.
- eru 5y ago
  You are being too pessimistic. Yes, it's hard to figure out cow just from a big corpus; though you might learn enough about 'cow' to form meaningful sentences and answer questions. That's what GPT-3 is doing after all, and language models are still improving quickly. And we do have context! Just knowing that a language is used by humans gives lots and lots of context. Human cultures and languages have more in common than you might realize. For example, have a look at Deixis (https://en.wikipedia.org/wiki/Deixis https://en.wikipedia.org/wiki/Deixis): > In linguistics, deixis (/ˈdaɪksɪs/, /ˈdeɪksɪs/)[1] is the use of general words and phrases to refer to a specific time, place, or person in context, e.g., the words tomorrow, there, and they. Words are deictic if their semantic meaning is fixed but their denoted meaning varies depending on time and/or place. Words or phrases that require contextual information to be fully understood—for example, English pronouns—are deictic. Deixis is closely related to anaphora. Although this article deals primarily with deixis in spoken language, the concept is sometimes applied to written language, gestures, and communication media as well. In linguistic anthropology, deixis is treated as a particular subclass of the more general semiotic phenomenon of indexicality, a sign "pointing to" some aspect of its context of occurrence. > Although this article draws examples primarily from English, deixis is believed to be a feature (to some degree) of all natural languages. I suspect with a large enough corpus you can most likely figure out the deixis of the given language. See https://linguistics.stackexchange.com/a/4046 https://linguistics.stackexchange.com/a/4046 for some more structure shared with all human languages: > However, all languages have some sort of a clause-type thing allowing them to express predication, attribution, etc. See Dixon's Basic Linguistic Theory: Basic Linguistic Theory Volume 1: Methodology . All languages also must have means of expressing cohesion and coherence (texture) although this is much less studied in cross linguistic perspective. Punctuated sentences are a kind of cohesive device. (A comment on this answer points out that the reality is probabilistic, of course:) > While I agree with you, it seems to me that linguists who study languages with a strong written tradition often talk about 'sentences', even when their examples are not from writing. Those of us who work on previously unwritten languages (I think) tend to talk about 'texts' or 'utterances'. I think this is an important issue as it relates to how some linguists have ignored the true messiness that is often found in examples of human language. Of course, all of this refers to natural human languages. Not totally arbitrary constructs.

thom 5y ago

I don't think it would be very hard to detect which of the two is random, because meanings are likely to be somewhat correlated. Obviously you could model your 'random' language to be statistically similar to the meaningful one though, if you were feeling particularly evil, but then you'd be imbuing it with some small degree of meaning, arguably. Whether you can go from this statistical analysis to anything with absolute meaning, I don't know. I suspect _some_ extremely common concepts (pronouns, verbs like to be or to have) might yield to this sort of direction.

rvense 5y ago
But the meaning is not in the text (text-as-artifact, the ink on the page), it is strictly in the mind of the reader, accessed by reference to the speaker's experience with the language in question. If you don't have something to ground at least part of the symbols in, there is nothing to work from. Any word can be assigned any meaning, and there would be infinite possibilities. No amount of statistics will allow you to work out whether a given word or part of a word refers to a rock, a rabbit or Noam Chomsky's granddaughter. How could it? To be able to do this is called knowing a language, and if all the people who had that skill in the soft tissue of their brain have kicked the bucket, tough luck, you can't get that knowledge back. All (bar none) decipherments of unknown writing systems have been done by reference to related languages, especially the existence of bilingual texts. Computers and statistics are definitely able to help us do that, but if all you have is a page of text with no context, then the information you're looking for is gone.
- thom 5y ago
  I agree to a very high degree! But I still think that you have greater than zero chance (but not much greater) to make some correct guesses about some very, very basic grammatical forms based on frequencies within other languages. You'd never be able to confirm this, and it's not a very important point. However, I do strongly disagree with the other point in the comment I was responding to, that you could not tell a real language with meaningful words from one with random words. That (entirely made up and not very helpful) problem does seem like it would be amenable to basis statistical analysis.

mcswell 5y ago

The first question is--if I understand what you're proposing--(relatively) easy. The second is much harder, perhaps impossible. The reason the first question is relatively easy is because of syntax (sentence order, but also the order of words in phrases). If it's a real language, that order is not random; the English word "the" shows up before "man" much more often than it does before "listens", and so for other words. This is true even in so-called free word order languages, which are really free phrase order languages. The other thing that could make the first question relatively easy is morphology. If you've decided to make a language that has inflectional morphology (in English, suffixes like -ed and -ing), that also helps determine whether the language is "real", because in a real language not all words will take the same affixes--you can't say 'theseing' (these-ing) in English. In other words, the affixes that you find on a given word are not random. Of course if you decide not to give your language inflectional morphology, this won't help--but if you do that, then your language's syntax will have to be more rigid: in a language with case marking affixes, like Latin, you don't need word order to tell which noun is subject and which (if any) is object; but you need a more or less fixed word order in English, because (apart from pronouns) we don't have any case marking.