5 ms·

The problem with all approaches to machine learning I see today is that they only focus on grouping and separating concepts based on certain characteristics. Th

The problem with all approaches to machine learning I see today is that they only focus on grouping and separating concepts based on certain characteristics. They seem to be all fundamentally statistical. None of them seem to work towards a fundamental understanding of what the concepts mean. I'm not sure how that could be accomplished though. Is there even a meaningful distinction to be made between beeing able to identify a concept and understanding what a concept actually means?

pavpanchekha 12y ago
This is in fact a classic criticism of modern AI. Classic approaches tried to replicate human reasoning mechanisms. The work was very cool, but far harder to understand and scale, compared to modern statistical algorithms. To some extent, you are faced with a choice: a small palette of mediocre techniques whose workings are beautiful theories of the human mind; or, a broadly-applicable, incredibly practical set of tools with little motivation or spirit.
- PurplePanda 12y ago
  I understand that other being hard to scale in a machine efficiency sense, the old techniques were also rather limited in applicable domain. They were often based on unjustified models of whatever the author decided was a good model of thought/reasoning, whether "frames", predicate calculus, constraint propagation or whatever. It seems to me that although you say statistical methods have little motivation, the motivation of the alternative - heuristics - is very questionable. Can anyone correct me on this?
  - pavpanchekha 12y ago
    That's actually what I meant by scale—that they were slow isn't really a problem if you're interested in AI for the raw fun of it. While justifications for using logic or constraints or probability theory were often lacking from classical techniques, do understand that logic itself was originally developed as a crystallization of proper human thought. And many of the classic AI folks did care deeply about understanding how humans achieved certain results. For example, take a look at Marvin Minsky's work. To the contrary, modern techniques aren't interested in replicating human thought; they are simply interested in replicating human results.
- tegeek 12y ago
  I think one of source of intelligence in humans is "imagination". IMO "Imagination" itself isn't intelligence but an input to the last "intelligent" activity. Computers can be taught to exhibit different "intelligent" activities and that is AI & Machine learning is all about. Human imagination works without any knowledge skill or learning (it is just my understanding, and i might be wrong). Producing rough and wild-west imagination isn't problem of AI.
  - pavpanchekha 12y ago
    Whatever your opinions on what "intelligence" really is and the role of "imagination" in producing it, I can assure you that many computer scientists would like to replicate human creativity, heuristics, and problem solving techniques. Many of the early AI pioneers were as interested in understanding human thought and understanding machine thought (source: conversations with some of them). Modern techniques simply have a different aim.
baddox 12y ago
I don't think there is a meaningful distinction. Do chess computers have a "fundamental understanding" of chess, which humans traditionally considered a benchmark of human intelligence/strategy? Analogous to the philosophical zombie thought experiment, I think that "real intelligence/understanding" is indistinguishable from simply being able to perform actions to accomplish the same tasks that humans traditionally consider to require intelligence. Of course, that's one of the PR problems that AI has always had: once a computer can outperform a human at some task, that task is no longer considered to be something that requires "true intelligence." Most people would consider someone who can multiply numbers together to be intelligent, but when computers do that (incomprehensibly faster and more reliably), few people consider even for a moment that it's AI. Same with more advanced mathematics, like computer algebra and automated theorem proving. Same with facial and voice recognition. And I'm sure it will be the same with self-driving cars.
- thret 12y ago
  If fictional media is anything to go by, the single defining aspect of human intelligence is love. This is the last bastion of human understanding that is incomprehensible to evil, machines and aliens.
  - herendin 12y ago
    Humans can already fall in love with computers and virtual personalities, so we're halfway there
  - aaronem 12y ago
    Depends on your choice of fictional media; try reading Peter Watts, some time when you're already not in a good mood.
    - thret 12y ago
      Thanks, I'll place Blindsight by Peter Watts in my queue.
      - 12y ago
PurplePanda 12y ago
I don't think that anyone can answer that categorically. It's a question for philosophy or metaphysics. Almost merely a matter of opinion. It's hard to criticize software for not working towards a "fundamental understanding of what the concepts mean" if we admit that we don't really even agree on whether that is a valid sentence or what it might mean.
c0g 12y ago
Machine Learning is more of an engineering discipline than a philosophical one. The aim (mostly, with exceptions) is to make machines that learn, not to research the nature of "understanding" or "concepts". There are people who are looking at broader ideas of "what is intelligence", sometimes even the same people doing the more statistical research, but it's an open ended problem. The main reason science/math/statistics doesn't spend time worrying about philosophy is because you can't really specify the problem cleanly - and having a clear idea of the problem is 90% of the way to finding the solution. [edit]: If you want brain-inspired ML stuff, it's worth looking at what neuroscientists and cognitive scientists are doing. From what I've seen of some of their work they're developing ML algos as minimal tests for understanding how parts of the brain or consciousness works.
gtremper 12y ago
This reminds me of one of my favorite Dijkstra quotes. "The question of whether machines can think... is about as relevant as the question of whether submarines can swim"
- deleted 12y ago
  [deleted]
  - knome 12y ago
    Do you happen to natively speak Russian? I've heard that in Russian the word used for people swimming is the same word used for ships sailing. In English, swimming specifically refers to the motions living organisms use to move in water. Ships instead are said to "sail", even though most do not use sails for power.
opium_tea 12y ago
I'm of the impression that concepts only have fundamental meaning to us because we can relate them to sensory input. For example, initially a child is taught what a boat is by seeing pictures, it eventually learns that boats, cars and bicycles share common characteristics and can be grouped together as vehicles. These common characteristics are all understood and identified by seeing, hearing, touching or smelling. It would be impossible completely describe a concept such as "horse" to a person with no knowledge of animal physiology, without recourse to pointing at a a horse or impersonating a horse noise. Resorting to analogies with other concepts - "a horse is like a cow" - doesn't work because the person has no knowledge of the other concepts. This person could be a young child or an untrained computer, and they can be taught fundamental meanings of concepts by feeding them information in the form of sights, sounds, smells etc. I haven't thought about this very deeply but these are the thoughts that always come into my mind when this topic comes up. Concepts have no meaning without relating them to sensory input. Humans learn by connecting the two. Why can the same not be applied to computers?
- Houshalter 12y ago
  There is no question that additional data would be potentially useful to an AI. But it's a common fallacy that it's the only way to get "meaning". Data from a digital camera isn't fundamentally different than data from text. The only benefit is that visual data is far less compressed, more redundant. It's easier to find patterns in it. Natural language is a highly compressed description of things in the real world. That said, it doesn't at all mean you can't learn anything meaningful from it. For example, you build a machine learning system to predict the missing words in a sentence. "A horse is similar to other farm animals like ____." Machines are getting better at this kind of thing, though still far from human level. Google's word2vec for example can take a word like "horse" and list the words that it is most similar to. You can subtract the representation for "man" from "king", add "woman" and it outputs "queen".
goblin89 12y ago
> Is there even a meaningful distinction to be made between beeing able to identify a concept and understanding what a concept actually means? How would you define “understanding” and “actually means” in your question? See also: http://en.wikipedia.org/wiki/Chinese_room http://en.wikipedia.org/wiki/Chinese_room
hyp0 12y ago
have you seen chomsky and norvig's debate on this? http://norvig.com/chomsky.html http://norvig.com/chomsky.html

pavpanchekha 12y ago

This is in fact a classic criticism of modern AI. Classic approaches tried to replicate human reasoning mechanisms. The work was very cool, but far harder to understand and scale, compared to modern statistical algorithms. To some extent, you are faced with a choice: a small palette of mediocre techniques whose workings are beautiful theories of the human mind; or, a broadly-applicable, incredibly practical set of tools with little motivation or spirit.

PurplePanda 12y ago
I understand that other being hard to scale in a machine efficiency sense, the old techniques were also rather limited in applicable domain. They were often based on unjustified models of whatever the author decided was a good model of thought/reasoning, whether "frames", predicate calculus, constraint propagation or whatever. It seems to me that although you say statistical methods have little motivation, the motivation of the alternative - heuristics - is very questionable. Can anyone correct me on this?
- pavpanchekha 12y ago
  That's actually what I meant by scale—that they were slow isn't really a problem if you're interested in AI for the raw fun of it. While justifications for using logic or constraints or probability theory were often lacking from classical techniques, do understand that logic itself was originally developed as a crystallization of proper human thought. And many of the classic AI folks did care deeply about understanding how humans achieved certain results. For example, take a look at Marvin Minsky's work. To the contrary, modern techniques aren't interested in replicating human thought; they are simply interested in replicating human results.
tegeek 12y ago
I think one of source of intelligence in humans is "imagination". IMO "Imagination" itself isn't intelligence but an input to the last "intelligent" activity. Computers can be taught to exhibit different "intelligent" activities and that is AI & Machine learning is all about. Human imagination works without any knowledge skill or learning (it is just my understanding, and i might be wrong). Producing rough and wild-west imagination isn't problem of AI.
- pavpanchekha 12y ago
  Whatever your opinions on what "intelligence" really is and the role of "imagination" in producing it, I can assure you that many computer scientists would like to replicate human creativity, heuristics, and problem solving techniques. Many of the early AI pioneers were as interested in understanding human thought and understanding machine thought (source: conversations with some of them). Modern techniques simply have a different aim.

baddox 12y ago

I don't think there is a meaningful distinction. Do chess computers have a "fundamental understanding" of chess, which humans traditionally considered a benchmark of human intelligence/strategy? Analogous to the philosophical zombie thought experiment, I think that "real intelligence/understanding" is indistinguishable from simply being able to perform actions to accomplish the same tasks that humans traditionally consider to require intelligence. Of course, that's one of the PR problems that AI has always had: once a computer can outperform a human at some task, that task is no longer considered to be something that requires "true intelligence." Most people would consider someone who can multiply numbers together to be intelligent, but when computers do that (incomprehensibly faster and more reliably), few people consider even for a moment that it's AI. Same with more advanced mathematics, like computer algebra and automated theorem proving. Same with facial and voice recognition. And I'm sure it will be the same with self-driving cars.

thret 12y ago
If fictional media is anything to go by, the single defining aspect of human intelligence is love. This is the last bastion of human understanding that is incomprehensible to evil, machines and aliens.
- herendin 12y ago
  Humans can already fall in love with computers and virtual personalities, so we're halfway there
- aaronem 12y ago
  Depends on your choice of fictional media; try reading Peter Watts, some time when you're already not in a good mood.
  - thret 12y ago
    Thanks, I'll place Blindsight by Peter Watts in my queue.
    - 12y ago

PurplePanda 12y ago

I don't think that anyone can answer that categorically. It's a question for philosophy or metaphysics. Almost merely a matter of opinion. It's hard to criticize software for not working towards a "fundamental understanding of what the concepts mean" if we admit that we don't really even agree on whether that is a valid sentence or what it might mean.

c0g 12y ago

Machine Learning is more of an engineering discipline than a philosophical one. The aim (mostly, with exceptions) is to make machines that learn, not to research the nature of "understanding" or "concepts". There are people who are looking at broader ideas of "what is intelligence", sometimes even the same people doing the more statistical research, but it's an open ended problem. The main reason science/math/statistics doesn't spend time worrying about philosophy is because you can't really specify the problem cleanly - and having a clear idea of the problem is 90% of the way to finding the solution. [edit]: If you want brain-inspired ML stuff, it's worth looking at what neuroscientists and cognitive scientists are doing. From what I've seen of some of their work they're developing ML algos as minimal tests for understanding how parts of the brain or consciousness works.

gtremper 12y ago

This reminds me of one of my favorite Dijkstra quotes. "The question of whether machines can think... is about as relevant as the question of whether submarines can swim"

deleted 12y ago
[deleted]
- knome 12y ago
  Do you happen to natively speak Russian? I've heard that in Russian the word used for people swimming is the same word used for ships sailing. In English, swimming specifically refers to the motions living organisms use to move in water. Ships instead are said to "sail", even though most do not use sails for power.

opium_tea 12y ago

I'm of the impression that concepts only have fundamental meaning to us because we can relate them to sensory input. For example, initially a child is taught what a boat is by seeing pictures, it eventually learns that boats, cars and bicycles share common characteristics and can be grouped together as vehicles. These common characteristics are all understood and identified by seeing, hearing, touching or smelling. It would be impossible completely describe a concept such as "horse" to a person with no knowledge of animal physiology, without recourse to pointing at a a horse or impersonating a horse noise. Resorting to analogies with other concepts - "a horse is like a cow" - doesn't work because the person has no knowledge of the other concepts. This person could be a young child or an untrained computer, and they can be taught fundamental meanings of concepts by feeding them information in the form of sights, sounds, smells etc. I haven't thought about this very deeply but these are the thoughts that always come into my mind when this topic comes up. Concepts have no meaning without relating them to sensory input. Humans learn by connecting the two. Why can the same not be applied to computers?

Houshalter 12y ago
There is no question that additional data would be potentially useful to an AI. But it's a common fallacy that it's the only way to get "meaning". Data from a digital camera isn't fundamentally different than data from text. The only benefit is that visual data is far less compressed, more redundant. It's easier to find patterns in it. Natural language is a highly compressed description of things in the real world. That said, it doesn't at all mean you can't learn anything meaningful from it. For example, you build a machine learning system to predict the missing words in a sentence. "A horse is similar to other farm animals like ____." Machines are getting better at this kind of thing, though still far from human level. Google's word2vec for example can take a word like "horse" and list the words that it is most similar to. You can subtract the representation for "man" from "king", add "woman" and it outputs "queen".

goblin89 12y ago

> Is there even a meaningful distinction to be made between beeing able to identify a concept and understanding what a concept actually means? How would you define “understanding” and “actually means” in your question? See also: http://en.wikipedia.org/wiki/Chinese_room http://en.wikipedia.org/wiki/Chinese_room

hyp0 12y ago

have you seen chomsky and norvig's debate on this? http://norvig.com/chomsky.html http://norvig.com/chomsky.html