4 ms·

This reminds me of Searle's Chinese Room Argument[0]: imagine you have a particularly dreary job where you sit in a room filled with boxes of symbols written on

by sgentle 10y ago

This reminds me of Searle's Chinese Room Argument[0]: imagine you have a particularly dreary job where you sit in a room filled with boxes of symbols written on paper. Every now and again someone comes in and hands you some new symbols. You look through your rulebook and, depending on what it says, hand them some symbols back. It turns out that these rules actually implement a conversational program in Chinese. And if you can implement those rules and not understand Chinese, why would you think a computer program, implementing its own rules, could understand anything? The common "Systems Reply" response to this is that you're looking at the wrong layer of abstraction. The computer hardware (or the person in the room) doesn't understand Chinese, the computer plus the rules plus the data forms a system that understands Chinese. Searle's answer to this is that, well, what if you memorised the rules and the database? You might know all the rules, you might be able to follow them, but you wouldn't understand Chinese. What I think is fascinating about this is that it's vulnerable to Bayesian Judo[1]: if you have a strong belief that computers aren't capable of true understanding because of the Chinese Room Argument, then building an actual Chinese Room-style computer and having it show understanding should be a fairly strong blow to that belief. Now, it's easy to quibble about what true understanding actually means, but one version (used by Searle's answer) is this: "[..] he would not know the meaning of the Chinese word for hamburger. He still cannot get semantics from syntax." But this news is exactly that! A computer translation of the same semantic concept from one syntax to another without ever having been taught the rules connecting them. In other words, this is semantics from syntax implemented by nothing but a computer, a database, and a set of rules. So, by the reverse Chinese Room Argument, I would say this system exhibits a kind of understanding. Not a very sophisticated kind, mind you, but something that should still spook you if you believe computers are categorically incapable of thinking like us. [0] http://plato.stanford.edu/entries/chinese-room/ http://plato.stanford.edu/entries/chinese-room/ [1] http://lesswrong.com/lw/i5/bayesian_judo/ http://lesswrong.com/lw/i5/bayesian_judo/

timr 10y ago
"But this news is exactly that! A computer translation of the same semantic concept from one syntax to another without ever having been taught the rules connecting them." By that standard, statistical translation approaches were "understanding" a long time ago. The new thing here isn't that systems aren't being taught "the rules" (that wasn't happening in statistical MT either), the new thing is that there's a different kind of classifier in the "middle" now, which is representing a hidden state. This classifier is more flexible in a lot of ways, but also more of a black box, and takes a lot more effort to train without overfitting. It's cool that you can translate between language pairs that have never been explicitly trained, but let's not overstate the meaning of it. The blog post makes this rather breathless speculation: "Within a single group, we see a sentence with the same meaning but from three different languages. This means the network must be encoding something about the semantics of the sentence rather than simply memorizing phrase-to-phrase translations. We interpret this as a sign of existence of an interlingua in the network." This is...a fun story, but not much else. First off, you can make dimensionality reduction plots that "show" a lot of things. Even ignoring that issue, in translations of short sentences involving specific concepts (i.e. the example about the stratosphere), is it really surprising that you'd find clusters? The words in that sentence are probably unique enough that they'd form a distinct cluster in mappings from any translation system. Folks get caught up in the "neural" part of neural networks, and assume that some magical quasi-human thought is happening. If the tech were called "highly parameterized reconfigurable weighted networks of logistic classifiers", there'd be less loopy speculation.
- sgentle 10y ago
  Don't worry, I'm not being bamboozled by the word "neural". My argument that there is a definition of understanding that you can derive from a well-known thought experiment that looks like it is met by this implementation of "highly parameterized reconfigurable weighted networks of logistic classifiers". I don't see any particular difference between training a classifier and teaching rules; the rules are just encoded in the parameters of the classifier. If it helps, you can just replace "taught" with "trained on" and "rules" with "data", but there's no version of the Chinese Room Argument where you're sitting in a room with boxes full of unsupervised learning datasets and a book of sigmoid functions. Perhaps this system works similarly to previous ones, but not having been taught (trained on) any rules (data) about the specific language pairs in question seems to be a strong argument for some kind of semantic representation of language. You might have seen that before, but I haven't and the article seems to imply that it's new. Again, I'm talking specifically about the similarity between this result and an example of something "machines can't do". The point is that the non-magical argument goes both ways. If a brain is just a complicated and meaty computer, then we should expect sophisticated enough programs on powerful enough hardware to start displaying things we might recognise as intelligent. That's not going to look particularly impressive – our machine translator isn't going to develop a conscience or try to unionise – but it might do something that qualifies for some definition of understanding.
  - timr 10y ago
    But you are getting into magical thinking, in that there is no reasonable definition of "understanding" that this system meets. It cannot reason or make deductions. It can't re-write sentences to use completely different words/structures but imply the same meaning. In fact, there is literally no "conceptual" representation here -- there is a vector of numbers that gets passed between encoder and decoder, but it is no more a form of intelligence than the "hidden" state that is maintained by an HMM. "not having been taught (trained on) any rules (data) about the specific language pairs in question seems to be a strong argument for some kind of semantic representation of language." Well, yeah, there's a representation of language. But it isn't "semantic" -- it's vector of language-independent parameters for a decoder, which can then output symbols in a second language. Could you theoretically imagine some huge magical network of logistic classifiers that uses this as the first of a (far larger) processing machine that enables something like human intelligence? Maybe. But this is not it. This is bigger, far more complicated/flexible version of a machine that is purpose-built to map between sequences of text. (That said, I really don't want to go down the rabbit hole of "what is AGI, anyway?", which is about as productive/interesting as hitting the bong and wondering if maybe we all live in a computer simulation after all. I'm merely observing that this is not an intelligent machine.)
wyager 10y ago
Aaronson has an interesting proposal to address the Chinese room problem that I think makes a lot of sense. The idea is that the Chinese Room intuitively doesn't exhibit understanding because it's a constant-time exponential-memory algorithm (a lookup table), whereas the algorithm that generated the entries in the Chinese Room table (a human) is super-constant time, sub-exponential memory algorithm, which introduces a place for consciousness to emerge. So the only reason the Chinese Room problem is philosophically confounding is that it adds a layer of indirection (a cache table) that obviously can't be conscious over the algorithm that actually might be conscious and that generates the table entries. http://www.scottaaronson.com/papers/philos.pdf http://www.scottaaronson.com/papers/philos.pdf
d--b 10y ago
I don't think you can say that this is a refutation of the Chinese room argument. Here the system doesn't know what the semantics are. For instance, it wouldn't be able to use some kind of logic inference to disambiguate between the two meanings of the same word. It may use some statistical inference (ie. if we talk about meat elsewhere in the sentence, it's likely that Hamburger refers to the sandwich rather than to the inhabitant of Hamburg), but it wouldn't be able to understand the content of the text to provide a translation that derives from this meaning. However, what this article seems to point out is that there is an intermediate representation that is being used within the neural network. This intermediate representation would correspond to a Chinese room where symbols (like hamburger) are first matched to something common to all language (let's call it X), and then translated back to specific languages. What this means is neither the Chinese room argument is true or false, it means that it's more complicated than that. It means that "understanding" has various degrees of depth, and that the first degree of understanding really is just syntactic conceptualization.
SonicSoul 10y ago
it is also possible to think of humans are pre-programmed systems that think have a consciousness and understanding but are in fact following a set of rules they don't full understand yet. Perhaps creating "artificial" systems that can evolve their own intelligence could be a way for us to understand our own programming better.

by sgentle 10y ago

timr 10y ago

"But this news is exactly that! A computer translation of the same semantic concept from one syntax to another without ever having been taught the rules connecting them." By that standard, statistical translation approaches were "understanding" a long time ago. The new thing here isn't that systems aren't being taught "the rules" (that wasn't happening in statistical MT either), the new thing is that there's a different kind of classifier in the "middle" now, which is representing a hidden state. This classifier is more flexible in a lot of ways, but also more of a black box, and takes a lot more effort to train without overfitting. It's cool that you can translate between language pairs that have never been explicitly trained, but let's not overstate the meaning of it. The blog post makes this rather breathless speculation: "Within a single group, we see a sentence with the same meaning but from three different languages. This means the network must be encoding something about the semantics of the sentence rather than simply memorizing phrase-to-phrase translations. We interpret this as a sign of existence of an interlingua in the network." This is...a fun story, but not much else. First off, you can make dimensionality reduction plots that "show" a lot of things. Even ignoring that issue, in translations of short sentences involving specific concepts (i.e. the example about the stratosphere), is it really surprising that you'd find clusters? The words in that sentence are probably unique enough that they'd form a distinct cluster in mappings from any translation system. Folks get caught up in the "neural" part of neural networks, and assume that some magical quasi-human thought is happening. If the tech were called "highly parameterized reconfigurable weighted networks of logistic classifiers", there'd be less loopy speculation.

sgentle 10y ago
Don't worry, I'm not being bamboozled by the word "neural". My argument that there is a definition of understanding that you can derive from a well-known thought experiment that looks like it is met by this implementation of "highly parameterized reconfigurable weighted networks of logistic classifiers". I don't see any particular difference between training a classifier and teaching rules; the rules are just encoded in the parameters of the classifier. If it helps, you can just replace "taught" with "trained on" and "rules" with "data", but there's no version of the Chinese Room Argument where you're sitting in a room with boxes full of unsupervised learning datasets and a book of sigmoid functions. Perhaps this system works similarly to previous ones, but not having been taught (trained on) any rules (data) about the specific language pairs in question seems to be a strong argument for some kind of semantic representation of language. You might have seen that before, but I haven't and the article seems to imply that it's new. Again, I'm talking specifically about the similarity between this result and an example of something "machines can't do". The point is that the non-magical argument goes both ways. If a brain is just a complicated and meaty computer, then we should expect sophisticated enough programs on powerful enough hardware to start displaying things we might recognise as intelligent. That's not going to look particularly impressive – our machine translator isn't going to develop a conscience or try to unionise – but it might do something that qualifies for some definition of understanding.
- timr 10y ago
  But you are getting into magical thinking, in that there is no reasonable definition of "understanding" that this system meets. It cannot reason or make deductions. It can't re-write sentences to use completely different words/structures but imply the same meaning. In fact, there is literally no "conceptual" representation here -- there is a vector of numbers that gets passed between encoder and decoder, but it is no more a form of intelligence than the "hidden" state that is maintained by an HMM. "not having been taught (trained on) any rules (data) about the specific language pairs in question seems to be a strong argument for some kind of semantic representation of language." Well, yeah, there's a representation of language. But it isn't "semantic" -- it's vector of language-independent parameters for a decoder, which can then output symbols in a second language. Could you theoretically imagine some huge magical network of logistic classifiers that uses this as the first of a (far larger) processing machine that enables something like human intelligence? Maybe. But this is not it. This is bigger, far more complicated/flexible version of a machine that is purpose-built to map between sequences of text. (That said, I really don't want to go down the rabbit hole of "what is AGI, anyway?", which is about as productive/interesting as hitting the bong and wondering if maybe we all live in a computer simulation after all. I'm merely observing that this is not an intelligent machine.)

wyager 10y ago

Aaronson has an interesting proposal to address the Chinese room problem that I think makes a lot of sense. The idea is that the Chinese Room intuitively doesn't exhibit understanding because it's a constant-time exponential-memory algorithm (a lookup table), whereas the algorithm that generated the entries in the Chinese Room table (a human) is super-constant time, sub-exponential memory algorithm, which introduces a place for consciousness to emerge. So the only reason the Chinese Room problem is philosophically confounding is that it adds a layer of indirection (a cache table) that obviously can't be conscious over the algorithm that actually might be conscious and that generates the table entries. http://www.scottaaronson.com/papers/philos.pdf http://www.scottaaronson.com/papers/philos.pdf

d--b 10y ago

I don't think you can say that this is a refutation of the Chinese room argument. Here the system doesn't know what the semantics are. For instance, it wouldn't be able to use some kind of logic inference to disambiguate between the two meanings of the same word. It may use some statistical inference (ie. if we talk about meat elsewhere in the sentence, it's likely that Hamburger refers to the sandwich rather than to the inhabitant of Hamburg), but it wouldn't be able to understand the content of the text to provide a translation that derives from this meaning. However, what this article seems to point out is that there is an intermediate representation that is being used within the neural network. This intermediate representation would correspond to a Chinese room where symbols (like hamburger) are first matched to something common to all language (let's call it X), and then translated back to specific languages. What this means is neither the Chinese room argument is true or false, it means that it's more complicated than that. It means that "understanding" has various degrees of depth, and that the first degree of understanding really is just syntactic conceptualization.

SonicSoul 10y ago

it is also possible to think of humans are pre-programmed systems that think have a consciousness and understanding but are in fact following a set of rules they don't full understand yet. Perhaps creating "artificial" systems that can evolve their own intelligence could be a way for us to understand our own programming better.