5 ms·

If you have lots of text and nothing else, deep contemplation can lead to understanding. This is far from the case here though, so you are right.

rvense 5y ago
I don't understand how you could say that it ever would. If you don't have anything at all (a related language, partial translations, accompanying illustrations, anything) to compare to, what can you do? You're trying to extract something (meaning) from an object that just does not contain it (text).
- simiones 5y ago
  Theoretically, given huge amounts of text, you can try to match it with possible human communication, based on the experience of being human - but you would likely need huge amounts of guesses, so it's unlikely to be truly doable with plausible computation resources.
- petters 5y ago
  GPT-3 proves that it is possible. It can reason about and answer novel questions just by looking at huge amounts of text.
  - rvense 5y ago
    GPT-3 does not understand language like you and I do with reference to the world. It has no senses, so it is not possible for it to do so. It is also to my knowledge not able to translate anything, so even if we were able to make it "learn" Harrappan writing from these seals, it would not be able to generate English text from them, because the information required to do so does not exist. To reply to your other comment, rabbit and rocks would occur in different contexts, but you have no way of knowing which is which. You can get numbers, probabilities and mutual information scores and vectors and what have you, but if you can't ground them in anything, you have no way of getting to meaning, and therefore no way of translating.
    - eru 5y ago
      > You can get numbers, probabilities and mutual information scores and vectors and what have you, but if you can't ground them in anything, you have no way of getting to meaning, and therefore no way of translating. But you have a grounding! You know it's a language used by humans.
    - petters 5y ago
      > It is also to my knowledge not able to translate anything It translates very well actually. I've tried with English/French and English/German with very good results. I think it is quite well understood that deciphering a language without external information is possible? But GPT-3 was a recent example that I think proves it. There are probably tons of literature about this question. I think aliens would be able to understand lots of things about Earth just by listening in. > To reply to your other comment, rabbit and rocks would occur in different contexts, but you have no way of knowing which is which. You can get numbers, probabilities and mutual information scores and vectors and what have you, but if you can't ground them in anything, you have no way of getting to meaning, and therefore no way of translating. Well, GPT-3 translates very well, so this argument can not be correct. I don't see why it would be like that? It seems quite intuitive that it would be possible to extract meaning.
      - rvense 5y ago
        GPT-3 trained on only English can produce French text? Citation please! I've looked for one, but I can't find it. I can, however, find more than a little information suggesting that GPT-3 is trained on multilingual data. > I think it is quite well understood that deciphering a language without external information is possible? I don't think this is understood at all, and I would have to ask you for just one example of it. I would, in fact, say it is categorically impossible, if by "no external information" you mean that we have only text or sound. The reason that it is not possible to extract meaning is that meaning is a substance that is not present in the text. It is more than the relations between words, in that it is connected to the world.
        rvense 5y ago
gilleain 5y ago
Not sure how? As a thought experiment, say I created two languages completely from scratch : symbols, words, sentence order, etc. One of these languages, I've given meaning to the words, the other is random. However I create massive corpuses for each of hundreds of thousands of documents. Now go ahead and machine learn the hell out of both languages. First question : which one is the 'random' language? Second question : what is the meaning of the non-random language texts? Assume I have not based my 'real' language on any existing example ...
- nopinsight 5y ago
  If the machine or human is sufficiently intelligent and there is enough data and time, it is possible to do so as long as the non-random language is useful for human communication and the language sample describes some experiences or things we understand. There are principles underlying all human languages. For example, they are used to describe the world or communicate intention and they must be understandable by a human. Given that the copious data of the non-random language should describe something we understand about the world or social dynamics, we can try to match them with world/domain models and find patterns of plausible interpretations that fit. The patterns should also satisfy the principle of communication economy and the limits of human cognition, such as the size of working memory. These constraints help narrow down the possibilities. The human/AI system can test and figure out the most likely patterns first and use them to decipher more patterns. All above is possible in principle but beyond the current state-of-the-art.
  - rvense 5y ago
    This is true, but you're forgetting that the sign (ie. pairing of an expression and its meaning) is arbitrary. There's no particular reason that the sound 'cow' (or the squiggles c-o-w when we write) refers to cows, and you can see that quite clearly by the fact that it only does so in one out of the Earth's 6-7000 current languages. Note that this doesn't mean that relationships between words are arbitrary, they are of course highly systematic, else language wouldn't work. It also doesn't mean that language doesn't have history, the changes are also systematic. But if you take a random surface form (sound or writing), there is absolutely no way to relate it with any certainty to any content without a context that allows you to infer it.
anovikov 5y ago
deep contemplation combined with the lack of information may lead to completely wrong conclusion. example is medieval interpretation of Egyptian hieroglyphics - completely and laughably wrong, but coherent and making sense, just based on limited amount of inputs.
- eru 5y ago
  Presumably your deep contemplation would take the risk of overfitting into account.

rvense 5y ago

I don't understand how you could say that it ever would. If you don't have anything at all (a related language, partial translations, accompanying illustrations, anything) to compare to, what can you do? You're trying to extract something (meaning) from an object that just does not contain it (text).

simiones 5y ago
Theoretically, given huge amounts of text, you can try to match it with possible human communication, based on the experience of being human - but you would likely need huge amounts of guesses, so it's unlikely to be truly doable with plausible computation resources.
petters 5y ago
GPT-3 proves that it is possible. It can reason about and answer novel questions just by looking at huge amounts of text.
- rvense 5y ago
  GPT-3 does not understand language like you and I do with reference to the world. It has no senses, so it is not possible for it to do so. It is also to my knowledge not able to translate anything, so even if we were able to make it "learn" Harrappan writing from these seals, it would not be able to generate English text from them, because the information required to do so does not exist. To reply to your other comment, rabbit and rocks would occur in different contexts, but you have no way of knowing which is which. You can get numbers, probabilities and mutual information scores and vectors and what have you, but if you can't ground them in anything, you have no way of getting to meaning, and therefore no way of translating.
  - eru 5y ago
    > You can get numbers, probabilities and mutual information scores and vectors and what have you, but if you can't ground them in anything, you have no way of getting to meaning, and therefore no way of translating. But you have a grounding! You know it's a language used by humans.
  - petters 5y ago
    > It is also to my knowledge not able to translate anything It translates very well actually. I've tried with English/French and English/German with very good results. I think it is quite well understood that deciphering a language without external information is possible? But GPT-3 was a recent example that I think proves it. There are probably tons of literature about this question. I think aliens would be able to understand lots of things about Earth just by listening in. > To reply to your other comment, rabbit and rocks would occur in different contexts, but you have no way of knowing which is which. You can get numbers, probabilities and mutual information scores and vectors and what have you, but if you can't ground them in anything, you have no way of getting to meaning, and therefore no way of translating. Well, GPT-3 translates very well, so this argument can not be correct. I don't see why it would be like that? It seems quite intuitive that it would be possible to extract meaning.
    - rvense 5y ago
      GPT-3 trained on only English can produce French text? Citation please! I've looked for one, but I can't find it. I can, however, find more than a little information suggesting that GPT-3 is trained on multilingual data. > I think it is quite well understood that deciphering a language without external information is possible? I don't think this is understood at all, and I would have to ask you for just one example of it. I would, in fact, say it is categorically impossible, if by "no external information" you mean that we have only text or sound. The reason that it is not possible to extract meaning is that meaning is a substance that is not present in the text. It is more than the relations between words, in that it is connected to the world.
      - rvense 5y ago

gilleain 5y ago

Not sure how? As a thought experiment, say I created two languages completely from scratch : symbols, words, sentence order, etc. One of these languages, I've given meaning to the words, the other is random. However I create massive corpuses for each of hundreds of thousands of documents. Now go ahead and machine learn the hell out of both languages. First question : which one is the 'random' language? Second question : what is the meaning of the non-random language texts? Assume I have not based my 'real' language on any existing example ...

nopinsight 5y ago
If the machine or human is sufficiently intelligent and there is enough data and time, it is possible to do so as long as the non-random language is useful for human communication and the language sample describes some experiences or things we understand. There are principles underlying all human languages. For example, they are used to describe the world or communicate intention and they must be understandable by a human. Given that the copious data of the non-random language should describe something we understand about the world or social dynamics, we can try to match them with world/domain models and find patterns of plausible interpretations that fit. The patterns should also satisfy the principle of communication economy and the limits of human cognition, such as the size of working memory. These constraints help narrow down the possibilities. The human/AI system can test and figure out the most likely patterns first and use them to decipher more patterns. All above is possible in principle but beyond the current state-of-the-art.
- rvense 5y ago
  This is true, but you're forgetting that the sign (ie. pairing of an expression and its meaning) is arbitrary. There's no particular reason that the sound 'cow' (or the squiggles c-o-w when we write) refers to cows, and you can see that quite clearly by the fact that it only does so in one out of the Earth's 6-7000 current languages. Note that this doesn't mean that relationships between words are arbitrary, they are of course highly systematic, else language wouldn't work. It also doesn't mean that language doesn't have history, the changes are also systematic. But if you take a random surface form (sound or writing), there is absolutely no way to relate it with any certainty to any content without a context that allows you to infer it.

anovikov 5y ago

deep contemplation combined with the lack of information may lead to completely wrong conclusion. example is medieval interpretation of Egyptian hieroglyphics - completely and laughably wrong, but coherent and making sense, just based on limited amount of inputs.

eru 5y ago
Presumably your deep contemplation would take the risk of overfitting into account.