5 ms·

How do we develop intelligent systems when we don't know how to measure intelligence?

Houshalter 13y ago
Who says we don't know how to measure intelligence? The simplest way would be just to select a problem that requires intelligence and measure how good it does at that problem.
- rectangletangle 13y ago
  But what is a problem that specifically requires intelligence to solve? How do we know we're measuring intelligence, and not the programs aptitude for the particular problem?
  - Houshalter 13y ago
    Why does it matter? If if solves the problem then it's as good as if it was fully intelligent. Who cares what goes on inside the black box? If you worried about it overfitting to a specific problem, give it lots of problems and weight the solutions by complexity. So you heavily favor simple algorithms that can learn to solve a large class of problems, over ones that are more adapted for those specific problems.
    - benched 13y ago
      All I'm learning from this thread is that some people see infinite regression where others see tautology. edit: what I mean: Person 1: "What if we're just measuring the color of the sky? What if we're just measuring blueness? What if we're just measuring a wavelength of light? ... ..." Person 2: "The sky looks blue."
      - Houshalter 13y ago
        What do you mean?
    - rectangletangle 13y ago
      It definitely could still be useful. However it may not posses "general intelligence," and therefore may not be applicable to as many scenarios as human like intelligence is.
    - mannykannot 13y ago
      "Who cares what goes on inside the black box?" That's the most interesting and informative issue presented by this scenario. The problem with the sort of test you propose is that just because a human uses intelligence to solve a problem, it does not follow that the task requires intelligence. For example, playing chess. From an engineering point of view, your black box may be as good as the real thing, though you couldn't really trust it beyond the areas of its demonstrated competence. Knowing how it works, however, would be the most significant achievement. Furthermore, it's going to be hard to build one of these black boxes without a reasonably good idea of how it is going to work. There is also the risk that your weighting scheme will rule out the only algorithms that have a chance of succeeding, because I bet they are pretty complex.
- seanalltogether 13y ago
  That's just it. We know how to measure the efficiency and accuracy of a given task, but we don't know how to measure "intelligence"
  - Houshalter 13y ago
    That's what intelligence is, being good at a given task.
    - seanalltogether 13y ago
      My current laptop has 100s of programs capable of performing 1000s of tasks but I wouldn't say it's intelligent. If we do want to call that intelligence, then we need a new term to describe the search for machines that can mimic human decisions.
rectangletangle 13y ago
This is a serious issue. The notion of "intelligence" is so abstract and intangible, that giving it a concrete definition is nearly redefining it. Likely what we're calling intelligence, is an oversimplification of reality; somewhat akin to the species problem in biology.
- benched 13y ago
  It is not universally believed that intelligence is ineffable. The most cursory Googling turns up lots of serviceable definitions articulated by smart people. Unfortunately, it is just complex enough to let people argue about it forever.
nl 13y ago
Shane Legg (one founder of DeepMind, purchased by Google) has written a number of good papers around the "measuring machine intelligence" problem. http://www.vetta.org/documents/Benelearn-UniversalIntelligence-Talk.pdf http://www.vetta.org/documents/Benelearn-UniversalIntelligen... is a pretty good place to start.

Who says we don't know how to measure intelligence? The simplest way would be just to select a problem that requires intelligence and measure how good it does at that problem.

rectangletangle 13y ago
But what is a problem that specifically requires intelligence to solve? How do we know we're measuring intelligence, and not the programs aptitude for the particular problem?
- Houshalter 13y ago
  Why does it matter? If if solves the problem then it's as good as if it was fully intelligent. Who cares what goes on inside the black box? If you worried about it overfitting to a specific problem, give it lots of problems and weight the solutions by complexity. So you heavily favor simple algorithms that can learn to solve a large class of problems, over ones that are more adapted for those specific problems.
  - benched 13y ago
    All I'm learning from this thread is that some people see infinite regression where others see tautology. edit: what I mean: Person 1: "What if we're just measuring the color of the sky? What if we're just measuring blueness? What if we're just measuring a wavelength of light? ... ..." Person 2: "The sky looks blue."
    - Houshalter 13y ago
      What do you mean?
  - rectangletangle 13y ago
    It definitely could still be useful. However it may not posses "general intelligence," and therefore may not be applicable to as many scenarios as human like intelligence is.
  - mannykannot 13y ago
    "Who cares what goes on inside the black box?" That's the most interesting and informative issue presented by this scenario. The problem with the sort of test you propose is that just because a human uses intelligence to solve a problem, it does not follow that the task requires intelligence. For example, playing chess. From an engineering point of view, your black box may be as good as the real thing, though you couldn't really trust it beyond the areas of its demonstrated competence. Knowing how it works, however, would be the most significant achievement. Furthermore, it's going to be hard to build one of these black boxes without a reasonably good idea of how it is going to work. There is also the risk that your weighting scheme will rule out the only algorithms that have a chance of succeeding, because I bet they are pretty complex.
seanalltogether 13y ago
That's just it. We know how to measure the efficiency and accuracy of a given task, but we don't know how to measure "intelligence"
- Houshalter 13y ago
  That's what intelligence is, being good at a given task.
  - seanalltogether 13y ago
    My current laptop has 100s of programs capable of performing 1000s of tasks but I wouldn't say it's intelligent. If we do want to call that intelligence, then we need a new term to describe the search for machines that can mimic human decisions.

rectangletangle 13y ago

This is a serious issue. The notion of "intelligence" is so abstract and intangible, that giving it a concrete definition is nearly redefining it. Likely what we're calling intelligence, is an oversimplification of reality; somewhat akin to the species problem in biology.

benched 13y ago
It is not universally believed that intelligence is ineffable. The most cursory Googling turns up lots of serviceable definitions articulated by smart people. Unfortunately, it is just complex enough to let people argue about it forever.

nl 13y ago

Shane Legg (one founder of DeepMind, purchased by Google) has written a number of good papers around the "measuring machine intelligence" problem. http://www.vetta.org/documents/Benelearn-UniversalIntelligence-Talk.pdf http://www.vetta.org/documents/Benelearn-UniversalIntelligen... is a pretty good place to start.