4 ms·

> Just as you can make infinitely many arithmetic equations by combining a few mathematical symbols and following a small set of rules, you can make infinitely

by namuol 8y ago

> Just as you can make infinitely many arithmetic equations by combining a few mathematical symbols and following a small set of rules, you can make infinitely many sentences by combining a modest set of words and a modest set of rules. A genuine, human-level A.I. will need to be able to cope with all of those possible sentences, not just a small fragment of them. The issue is less about understanding language and much more about creating models of "the world" through observations, and then applying those models to perform tasks or answer questions. Parsing language is actually pretty easy; "knowing" how it relates to a generalized model of "the world" is the hard part. > No matter how much data you have and how many patterns you discern, your data will never match the creativity of human beings or the fluidity of the real world. This is _firmly_ in the Opinion section of the NY Times... What Google seemed to do with Duplex might seem like a babystep, but it doesn't take much imagination to recognize very real possibility for so-called "genuine, human-level A.I." to -- gradually -- emerge.

azinman2 8y ago
Uhh, it does. Being able to have a fun date night conversation has little in common with making an appointment for a hair cut. Duplex is a codified set of rules — essentially a set of blanks that need to get filled in and an elegant way of pulling together a bunch of components to do so. That’s nothing like free form conversion which requires world knowledge, perspective (gained from experience), personality, creativity, emotional connection, etc etc. There’s nothing new going on in AI vs the machine learning “pragmatic downgrade” of the 90s, just the accuracy is higher.
- Retric 8y ago
  Parsing the language at some level is required for translation beyond simple 1:1 word mapping. Understanding is really a separate task that sits outside of language and would be just as hard with a very limited language designed to be easy to decode.
  - uryga 8y ago
    Correct me if I'm wrong – are you saying that one doesn't need to understand something to be able to translate it? Because that'd be a strange argument to make, looking at the state of Google Translate.
    - robkop 8y ago
      > looking at the state of Google Translate last time I checked Google Translate was a seq2seq based model with symbol level embeddings. The way symbol embeddings are generated doesn't lead to any real understanding of the sentence/ word/ language. It only really leads to understanding what is normally around that symbol in different situations. You could also very easily argue that the seq2seq model doesn't provide understanding, it only learns to encode the general meaning of the sequence in a fixed compressed format. It's possible to argue that being able to compress a sequence into a fixed length vector requires understanding but I would argue that understanding requires more than just being able to do lossy compression on a sequence. In my view entity modelling and related problems are much closer to achieving some level of understanding. They at least are able to use context to figure out what parts of the sequence have what meaning and the relationship between separate parts of the sequence. I'd be interested to hear your view of how Google Translate has understanding.
      - YeGoblynQueenne 8y ago
        I think the parent actually argues that GT doesn't have understanding.
      - uryga 8y ago
        I didn't mean to say GT has understanding, quite the opposite ;) I meant to use it as an example of how translating without understanding doesn't go too well in many cases. I think I might have misunderstood your original comment though, as we seem to mostly be on the same side of the issue. That being said, could you expand on this?: > the seq2seq model doesn't provide understanding, it only learns to encode the general meaning of the sequence Maybe I'm nitpicking, but "encoding the general meaning" sounds a lot like a form of "understanding", and I wouldn't say seq2seq does any of that. (That's getting pretty philosophical though...)
sullyj3 8y ago
The reason language is infinite is that it's structured recursively. It's the same reason mathematical expression are infinite, because you can replace a number with a bracketed expression. But computers can understand arbitrarily nested and large mathematical expressions given enough memory. So I think that it's misguided to argue that infinitude and recursiveness are the main obstacle to understanding natural language. The real issue most of the time is ambiguity and a lack of contextual knowledge, and in the general case, contextual knowledge requires general intelligence.

by namuol 8y ago

azinman2 8y ago

Uhh, it does. Being able to have a fun date night conversation has little in common with making an appointment for a hair cut. Duplex is a codified set of rules — essentially a set of blanks that need to get filled in and an elegant way of pulling together a bunch of components to do so. That’s nothing like free form conversion which requires world knowledge, perspective (gained from experience), personality, creativity, emotional connection, etc etc. There’s nothing new going on in AI vs the machine learning “pragmatic downgrade” of the 90s, just the accuracy is higher.

Retric 8y ago
Parsing the language at some level is required for translation beyond simple 1:1 word mapping. Understanding is really a separate task that sits outside of language and would be just as hard with a very limited language designed to be easy to decode.
- uryga 8y ago
  Correct me if I'm wrong – are you saying that one doesn't need to understand something to be able to translate it? Because that'd be a strange argument to make, looking at the state of Google Translate.
  - robkop 8y ago
    > looking at the state of Google Translate last time I checked Google Translate was a seq2seq based model with symbol level embeddings. The way symbol embeddings are generated doesn't lead to any real understanding of the sentence/ word/ language. It only really leads to understanding what is normally around that symbol in different situations. You could also very easily argue that the seq2seq model doesn't provide understanding, it only learns to encode the general meaning of the sequence in a fixed compressed format. It's possible to argue that being able to compress a sequence into a fixed length vector requires understanding but I would argue that understanding requires more than just being able to do lossy compression on a sequence. In my view entity modelling and related problems are much closer to achieving some level of understanding. They at least are able to use context to figure out what parts of the sequence have what meaning and the relationship between separate parts of the sequence. I'd be interested to hear your view of how Google Translate has understanding.
    - YeGoblynQueenne 8y ago
      I think the parent actually argues that GT doesn't have understanding.
    - uryga 8y ago
      I didn't mean to say GT has understanding, quite the opposite ;) I meant to use it as an example of how translating without understanding doesn't go too well in many cases. I think I might have misunderstood your original comment though, as we seem to mostly be on the same side of the issue. That being said, could you expand on this?: > the seq2seq model doesn't provide understanding, it only learns to encode the general meaning of the sequence Maybe I'm nitpicking, but "encoding the general meaning" sounds a lot like a form of "understanding", and I wouldn't say seq2seq does any of that. (That's getting pretty philosophical though...)

sullyj3 8y ago

The reason language is infinite is that it's structured recursively. It's the same reason mathematical expression are infinite, because you can replace a number with a bracketed expression. But computers can understand arbitrarily nested and large mathematical expressions given enough memory. So I think that it's misguided to argue that infinitude and recursiveness are the main obstacle to understanding natural language. The real issue most of the time is ambiguity and a lack of contextual knowledge, and in the general case, contextual knowledge requires general intelligence.