4 ms·

> Yes, it's cool that you can identify pictures or have a program read your handwriting, but none of that gets us any closer to a program that behaves like a hu

> Yes, it's cool that you can identify pictures or have a program read your handwriting, but none of that gets us any closer to a program that behaves like a human when I tell it to do something Yes it does. Humans didn't just emerge out of the aether. We evolved from animals that first had to perform simple tasks like that. Our brains are still heavily based on networks of simple pattern recognition neurons. We just don't know how they are structured, or have enough computing power. But it's definitely progress. As proof, we are starting to get neural networks that can do complex language understanding tasks. RNNs are the state of the art at tasks like predicting the next word, and make interesting chatbots. Obviously they aren't human level, but they only have 1,000 neurons. Humans have billions! They don't even have episodic memory yet, it's amazing they can do language at all. But research on that is just starting. Give it time.

YeGoblynQueenne 10y ago
>> we are starting to get neural networks that can do complex language understanding tasks. Ah, please, don't say things like that. We have neural networks that can build complex models of language. But those complex models are nowhere near performing well in "understanding tasks", whatever those are (processing meaning, you mean basically). The use of ANNs in Natural Language Processing has not led to the same advances it did in, say, image and speech processing, and... and, well, image and speech processing. There's a ton of things about language that go well beyond building a model of language. Neural networks are nowhere near addressing meaning and context, or the way those are conveyed through language but are not the same as language. In a way, NLP today, with ANNs or not, is still stuck in the same rut we were ages ago, where we assumed that syntax was the biggest part of language and solving that would solve language [1]. We got syntax down pat and it didn't do us much good. Neural Networks simply build more complex models of language than what we were able to build by hand-crafting grammars thirty years ago- but they're still just complex models of syntax. But syntax is just the surface of language and it's not even all of the surface. No we're not starting to get nets that can do "complex understanding tasks". Not even ones that can do simple ones. No AI technique does "understanding" yet, except in very limited ways. [1] Did we really assume that? I'm not even so sure anymore.
jeffmcmahan 10y ago
>> it's amazing they can do language at all. But research on that is just starting. Give it time. Not so. Academic work in natural language processing has been very active for decades. Check out NL SOAR, which is an awesome attempt from the early 90s by top CS/AI scholars: http://www.cs.cmu.edu/afs/cs/project/soar/utc/nl/doc/nl-soar.html http://www.cs.cmu.edu/afs/cs/project/soar/utc/nl/doc/nl-soar... In general, language processing has been an important motivator and funding source for AI work. Remember that US Navy Signals Intelligence funded Noam Chomsky's early work on syntax. When early advances in basic linguistic research alongside early advances in computer hardware made machine translation seem plausible, money poured in. And it's pouring in again (now from private industry too).
dmreedy 10y ago
This is a sound argument, but I think it's operating a level abstract enough to gloss over some of the niggling details that might make the parent's argument valid. - We don't know that our brains are heavily based on networks of simple pattern recognition neurons. Some parts of them certainly seem to be; especially those parts that seem to correlate to certain tasks that our current ANNs are really good at solving (i.e., basic-to-intermediate image processing). The fundamental mechanism of the brain, if there is one, seems to involve neurons pretty significantly, but we're still not entirely sure what the whole picture is. - The evolutionary argument is interesting and two-sided; yes it implies that any progress can be 'part of the evolutionary process', so to speak, but it doesn't guarantee that the particular route of the tree being discussed terminates in the desired outcome. It doesn't even guarantee it's on the same tree. It may be the case that image recognition as a task was fundamental to the evolution of high-level human behaviors. It may be the case that it works as a transitionary state, assuming other sets of solved problems unified under the same model, but when treated as foundational, it leads to an evolutionary dead end. - Following on the 'humans didn't emerge out of the aether' argument from another angle, there's another important consideration to be made. State-of-the-art research is often starting with 'randomly'-wired nets, maybe some lightly-informed initial weights from previous models. It's a kind of tabula rasa, "if you wish to make an apple pie from scratch" picture. The human brain, however, seems to acquire a very specific structure (down to a pretty low level), that is pretty universal amongst, well, humans. So it's not entirely out of the question that some of the things we do aren't strictly learnable from a base, blank initial state; the evolutionary process may well have hardcoded structures pre-wired to know or learn these things. This, of course, is the fundamental principle behind Chomskyian universal grammars. - I take issue with saying that we are starting to get NNs capable of complex natural language understanding tasks. We really aren't; they are basic-to-intermediate at best, with no real evidence to suggest that they are (A) doing anything interesting with respect to semantics beyond the already well-established distributional semantics strategies, and (B) are not going to dead-end in the near future, like the GOFAI techniques their advocates never tire of feeling superior to.

YeGoblynQueenne 10y ago

>> we are starting to get neural networks that can do complex language understanding tasks. Ah, please, don't say things like that. We have neural networks that can build complex models of language. But those complex models are nowhere near performing well in "understanding tasks", whatever those are (processing meaning, you mean basically). The use of ANNs in Natural Language Processing has not led to the same advances it did in, say, image and speech processing, and... and, well, image and speech processing. There's a ton of things about language that go well beyond building a model of language. Neural networks are nowhere near addressing meaning and context, or the way those are conveyed through language but are not the same as language. In a way, NLP today, with ANNs or not, is still stuck in the same rut we were ages ago, where we assumed that syntax was the biggest part of language and solving that would solve language [1]. We got syntax down pat and it didn't do us much good. Neural Networks simply build more complex models of language than what we were able to build by hand-crafting grammars thirty years ago- but they're still just complex models of syntax. But syntax is just the surface of language and it's not even all of the surface. No we're not starting to get nets that can do "complex understanding tasks". Not even ones that can do simple ones. No AI technique does "understanding" yet, except in very limited ways. [1] Did we really assume that? I'm not even so sure anymore.

jeffmcmahan 10y ago

>> it's amazing they can do language at all. But research on that is just starting. Give it time. Not so. Academic work in natural language processing has been very active for decades. Check out NL SOAR, which is an awesome attempt from the early 90s by top CS/AI scholars: http://www.cs.cmu.edu/afs/cs/project/soar/utc/nl/doc/nl-soar.html http://www.cs.cmu.edu/afs/cs/project/soar/utc/nl/doc/nl-soar... In general, language processing has been an important motivator and funding source for AI work. Remember that US Navy Signals Intelligence funded Noam Chomsky's early work on syntax. When early advances in basic linguistic research alongside early advances in computer hardware made machine translation seem plausible, money poured in. And it's pouring in again (now from private industry too).

dmreedy 10y ago

This is a sound argument, but I think it's operating a level abstract enough to gloss over some of the niggling details that might make the parent's argument valid. - We don't know that our brains are heavily based on networks of simple pattern recognition neurons. Some parts of them certainly seem to be; especially those parts that seem to correlate to certain tasks that our current ANNs are really good at solving (i.e., basic-to-intermediate image processing). The fundamental mechanism of the brain, if there is one, seems to involve neurons pretty significantly, but we're still not entirely sure what the whole picture is. - The evolutionary argument is interesting and two-sided; yes it implies that any progress can be 'part of the evolutionary process', so to speak, but it doesn't guarantee that the particular route of the tree being discussed terminates in the desired outcome. It doesn't even guarantee it's on the same tree. It may be the case that image recognition as a task was fundamental to the evolution of high-level human behaviors. It may be the case that it works as a transitionary state, assuming other sets of solved problems unified under the same model, but when treated as foundational, it leads to an evolutionary dead end. - Following on the 'humans didn't emerge out of the aether' argument from another angle, there's another important consideration to be made. State-of-the-art research is often starting with 'randomly'-wired nets, maybe some lightly-informed initial weights from previous models. It's a kind of tabula rasa, "if you wish to make an apple pie from scratch" picture. The human brain, however, seems to acquire a very specific structure (down to a pretty low level), that is pretty universal amongst, well, humans. So it's not entirely out of the question that some of the things we do aren't strictly learnable from a base, blank initial state; the evolutionary process may well have hardcoded structures pre-wired to know or learn these things. This, of course, is the fundamental principle behind Chomskyian universal grammars. - I take issue with saying that we are starting to get NNs capable of complex natural language understanding tasks. We really aren't; they are basic-to-intermediate at best, with no real evidence to suggest that they are (A) doing anything interesting with respect to semantics beyond the already well-established distributional semantics strategies, and (B) are not going to dead-end in the near future, like the GOFAI techniques their advocates never tire of feeling superior to.