4 ms·

IBM Watson won Jeopardy years ago, was it intelligent?

by aprilthird2021 1y ago

perching_aix 1y ago
> Rather than being given questions, contestants are instead given general knowledge clues in the form of answers and they must identify the person, place, thing, or idea that the clue describes, phrasing each response in the form of a question. [0] Doesn't sound like a test of intelligence to me, so no. [0] https://en.wikipedia.org/wiki/Jeopardy https://en.wikipedia.org/wiki/Jeopardy!
- aprilthird2021 1y ago
  Why? Computers also won chess years ago, but they're not intelligent either? Why is winning a math competition intelligent but a trivia competition or a chess competition not intelligent?
  - perching_aix 1y ago
    I don't wish to join you in framing intelligence as a step function. I think winning a Go or a chess competition does demonstrate intelligence. And winning a math competition does even more so. I do not think a trivia competition like Jeopardy demonstrates intelligence much at all, however. Specifically because it reads like it's not about intelligence, but about knowledge: it tests for association and recall, not for performing complex logical transformations. This isn't to say I consider these completely independent. Most smart people are both knowledgeable and intelligent. It's just that they are distinct dimensions in my opinion. You wouldn't say something tastes bad because its texture feels weird in your mouth, would you?
    - tjr 1y ago
      I might even think that a symbolic chess program is in some sense more intelligent than a modern LLM. It has a concrete model of the world it operates in along with representation what it can, cannot, and is trying to, do. When LLMs get the right answer, it seems more like... highly-optimized chance, rather than coming from any sort of factual knowledge.
    - aDyslecticCrow 1y ago
      > I think winning a Go or a chess competition does demonstrate intelligence. Chess is a simple alfa beta pruned minmax seaech tree. If that's intelligent then a drone flight controller or a calculator is aswell. > association and recall, not for performing complex logical transformations. By that definition humans doing chess aren't as intelligent as a computer doing chess, since high level chess is heavily reliant on memory and recall of moves and progressions. So your definition falls appart.
      - perching_aix 1y ago
        > So your definition falls apart. I did not share any definitions, only vague opinions. Not that I'd know what it means for a definition to "fall apart". And the specific bit you cite is barely even a vague opinion; it is my interpretation of the show "Jeopardy!" based on the Wiki article (I've never seen a single episode, wasn't really a thing where I'm from): > Specifically because it reads like it's about (...) knowledge: it tests for association and recall (...) Also: > By that definition humans doing chess aren't as intelligent as a computer doing chess, since high level chess is heavily reliant on memory and recall of moves and progressions. Yes, I did find this really quite disappointing and disillusioning when I first learned about it. A colleague of mine even straight up quit competitive chess over it.
        aDyslecticCrow 1y ago
        > it is my interpretation of the show "Jeopardy!" based on the Wiki article You are spot on though. I mostly wanted to argue that no decent distinction can be made here. > I did find this really quite disappointing and disillusioning when I first learned about it ye... same here. --- I'm personally in the camp that "intelligence" is a human concept. A metric to compare humans. Applying it to computers makes us anthropomorphism computers and think of them as people. Thinking of LLMs as people makes us trust them with bad things. So we should call them impressive, fluent, fast, useful, good at tasks. Computers already beat us at most math, statistics, searching for information, spacial visualization, information recollection, lossless communication. LLMs just adds to that list, but does nothing new to make the word "intelligent" applicable. Even if we reach the AGI singularity; thinking of them as humans or using human terminology to describe them is a fatal error. (Destroying earth to make paperclips is arguably the least intelligent thing you could ever do.)
  - im3w1l 1y ago
    None of these things are enough by itself. It's rather that they have now solved so many things that the sum total has (arguably) crossed the threshold. As for solving math problems, that is an important part of recursive self improvement. If it can come up with better algorithms and turn them into code, that will translate into raising it's own intelligence.
  - School-Cotton 1y ago
    Math and chess are similar in the sense that for humans, both require creativity, logical problem solving, etc. But they are not at all similar for computers. Chess has a constrained small set of rules and it is pretty straightforward to make a machine that beats humans by brute force computation. Pre-Leela chess programs were just tree search, a hardcoded evaluation function, and lots of pruning heuristics. So those programs are really approaching the game in a fundamentally different way from strong humans, who rely much more on intuition and pattern-recognition rather than calculation. It just turns out the computer approach is actually better than the human one. Sort of like how a car can move faster than a human even though cars don’t do anything much like walking. Math is not analogous: there’s no obvious algorithm for discovering mathematical proofs or solving difficult problems that could be implemented in a classical, pre-Gen AI computer program.
    - aDyslecticCrow 1y ago
      > there’s no obvious algorithm for discovering mathematical proofs or solving difficult problems that could be implemented in a classical, pre-Gen AI computer program. Fundamentally opposite. Computer algorithms have been part of math research since they where invented, and mathematical proof algorithms are widespread and excellent. The llms that are now "intelligent enough to do maths" are just trained to rephrase questions into prolog code.
      - School-Cotton 1y ago
        > The llms that are now "intelligent enough to do maths" are just trained to rephrase questions into prolog code. Do you have a source that talks about this?
        aDyslecticCrow 1y ago
        3b1b just posted something very relevant, if you're curious. https://www.youtube.com/watch?v=4NlrfOl0l8U https://www.youtube.com/watch?v=4NlrfOl0l8U
        aDyslecticCrow 1y ago

IBM Watson won Jeopardy years ago, was it intelligent?

> Rather than being given questions, contestants are instead given general knowledge clues in the form of answers and they must identify the person, place, thing, or idea that the clue describes, phrasing each response in the form of a question. [0] Doesn't sound like a test of intelligence to me, so no. [0] https://en.wikipedia.org/wiki/Jeopardy https://en.wikipedia.org/wiki/Jeopardy!

aprilthird2021 1y ago
Why? Computers also won chess years ago, but they're not intelligent either? Why is winning a math competition intelligent but a trivia competition or a chess competition not intelligent?
- perching_aix 1y ago
  I don't wish to join you in framing intelligence as a step function. I think winning a Go or a chess competition does demonstrate intelligence. And winning a math competition does even more so. I do not think a trivia competition like Jeopardy demonstrates intelligence much at all, however. Specifically because it reads like it's not about intelligence, but about knowledge: it tests for association and recall, not for performing complex logical transformations. This isn't to say I consider these completely independent. Most smart people are both knowledgeable and intelligent. It's just that they are distinct dimensions in my opinion. You wouldn't say something tastes bad because its texture feels weird in your mouth, would you?
  - tjr 1y ago
    I might even think that a symbolic chess program is in some sense more intelligent than a modern LLM. It has a concrete model of the world it operates in along with representation what it can, cannot, and is trying to, do. When LLMs get the right answer, it seems more like... highly-optimized chance, rather than coming from any sort of factual knowledge.
  - aDyslecticCrow 1y ago
    > I think winning a Go or a chess competition does demonstrate intelligence. Chess is a simple alfa beta pruned minmax seaech tree. If that's intelligent then a drone flight controller or a calculator is aswell. > association and recall, not for performing complex logical transformations. By that definition humans doing chess aren't as intelligent as a computer doing chess, since high level chess is heavily reliant on memory and recall of moves and progressions. So your definition falls appart.
    - perching_aix 1y ago
      > So your definition falls apart. I did not share any definitions, only vague opinions. Not that I'd know what it means for a definition to "fall apart". And the specific bit you cite is barely even a vague opinion; it is my interpretation of the show "Jeopardy!" based on the Wiki article (I've never seen a single episode, wasn't really a thing where I'm from): > Specifically because it reads like it's about (...) knowledge: it tests for association and recall (...) Also: > By that definition humans doing chess aren't as intelligent as a computer doing chess, since high level chess is heavily reliant on memory and recall of moves and progressions. Yes, I did find this really quite disappointing and disillusioning when I first learned about it. A colleague of mine even straight up quit competitive chess over it.
      - aDyslecticCrow 1y ago
        > it is my interpretation of the show "Jeopardy!" based on the Wiki article You are spot on though. I mostly wanted to argue that no decent distinction can be made here. > I did find this really quite disappointing and disillusioning when I first learned about it ye... same here. --- I'm personally in the camp that "intelligence" is a human concept. A metric to compare humans. Applying it to computers makes us anthropomorphism computers and think of them as people. Thinking of LLMs as people makes us trust them with bad things. So we should call them impressive, fluent, fast, useful, good at tasks. Computers already beat us at most math, statistics, searching for information, spacial visualization, information recollection, lossless communication. LLMs just adds to that list, but does nothing new to make the word "intelligent" applicable. Even if we reach the AGI singularity; thinking of them as humans or using human terminology to describe them is a fatal error. (Destroying earth to make paperclips is arguably the least intelligent thing you could ever do.)
- im3w1l 1y ago
  None of these things are enough by itself. It's rather that they have now solved so many things that the sum total has (arguably) crossed the threshold. As for solving math problems, that is an important part of recursive self improvement. If it can come up with better algorithms and turn them into code, that will translate into raising it's own intelligence.
- School-Cotton 1y ago
  Math and chess are similar in the sense that for humans, both require creativity, logical problem solving, etc. But they are not at all similar for computers. Chess has a constrained small set of rules and it is pretty straightforward to make a machine that beats humans by brute force computation. Pre-Leela chess programs were just tree search, a hardcoded evaluation function, and lots of pruning heuristics. So those programs are really approaching the game in a fundamentally different way from strong humans, who rely much more on intuition and pattern-recognition rather than calculation. It just turns out the computer approach is actually better than the human one. Sort of like how a car can move faster than a human even though cars don’t do anything much like walking. Math is not analogous: there’s no obvious algorithm for discovering mathematical proofs or solving difficult problems that could be implemented in a classical, pre-Gen AI computer program.
  - aDyslecticCrow 1y ago
    > there’s no obvious algorithm for discovering mathematical proofs or solving difficult problems that could be implemented in a classical, pre-Gen AI computer program. Fundamentally opposite. Computer algorithms have been part of math research since they where invented, and mathematical proof algorithms are widespread and excellent. The llms that are now "intelligent enough to do maths" are just trained to rephrase questions into prolog code.
    - School-Cotton 1y ago
      > The llms that are now "intelligent enough to do maths" are just trained to rephrase questions into prolog code. Do you have a source that talks about this?
      - aDyslecticCrow 1y ago
        3b1b just posted something very relevant, if you're curious. https://www.youtube.com/watch?v=4NlrfOl0l8U https://www.youtube.com/watch?v=4NlrfOl0l8U
      - aDyslecticCrow 1y ago