6 ms·

Why not? If the only thing that can solve problem X is AGI (e.g. humans), and something else comes along that solves it, then rationally that should be evidence

Why not? If the only thing that can solve problem X is AGI (e.g. humans), and something else comes along that solves it, then rationally that should be evidence that the something else is AGI right? Unless you have strong prior beliefs (like "computers can't be AGI") or something else that's problem specific ("these problems can be solved by these techniques which don't count as AGI"). So I guess that's my real question.

lucianbr 2y ago
That makes no sense at all. Any problem is initially only solvable by humans, until some technology is developed to solve it. Calculating a logarithm was at some point only doable by humans, and then digital computers came along. This would be in your view evidence that digital computers are AGI!? As in, an 8086 with some math code is AGI. We've had it for decades now, only nobody noticed :)
- bubblyworld 2y ago
  It's just Bayes theorem - there are basically two variables that control how strong the evidence is: * How likely you think AGI is in general. * How solvable you think the problem is, independently of what's solving it. In the cases you've brought up that latter probability is very high, which means that they are extremely weak evidence that computers are AGI. So we agree! In this case the latter probability seems to be quite low - attempts to solve it with computers have largely failed so far!
  - lucianbr 2y ago
    We don't agree. You're now saying anything is evidence of anything, which just makes the word "evidence" meaningless. In real life, when people say "A is evidence of B" they mean strong evidence, or even overwhelming evidence. You just backpedalled by redefining evidence to mean anything and nothing, so you can salvage an obviously false claim. Nobody in the real world says "rain is evidence of aliens" with the implicit assumption that it's just extremely weak evidence. The way English is used by people makes that sentence simply false, as is yours that anything previously not solved is evidence of AGI.
    - bubblyworld 2y ago
      We're talking about a specific problem here - the competition in the OP. Not aliens in the rain.
educasean 2y ago
This flies directly in the face of technologies such as Deep Blue and AlphaGo. They excel in tiny domains previously thought to be the pinnacle of intelligence, and now they dominate humans. Are they AGI in your definition?
- bubblyworld 2y ago
  See my response to the other commenter. In these cases as well I would conclude it's very weak evidence of AGI, so I don't think we disagree. Edit: I think maybe the disagreement here is about the nature of evidence. I think there can be evidence that something is AGI even if it isn't, in fact, AGI. You seem to believe that if there's any evidence that something is AGI, it must be AGI, I think?
  - educasean 2y ago
    I personally don't find this line of rhetoric useful or relevant. Let's agree to disagree.
    - bubblyworld 2y ago
      Okay, that's fair. But to be clear - this is a theorem of probability theory, not rhetoric.
nl 2y ago
> If the only thing that can solve problem X is AGI (e.g. humans), and something else comes along that solves it, then rationally that should be evidence that the something else is AGI right? No. Because there might undiscovered ways to solve these problems that no one claims is AGI. The definition of AGI is notoriously fuzzy, but non-the-less if there was a 10 line python program (with no external dependencies or data) that could solve it then few would argue that was AGI. So perhaps there is an algorithm that solves these puzzles 100% of the time and can be easily expressed. So I agree that only being able to solve these problems doesn't define AGI.
- bubblyworld 2y ago
  I think I agree with you, but consider these two cases: 1. Only humans are known to have solved problem X, and we've spent no time looking for alternative solutions. 2. Only humans are known to have solved problem X, and we've spent hundreds of thousands of hours looking for alternative solutions and failed. Now suppose something solves the problem. I feel like in case 2 we are justified in saying there's evidence that something is a human-like AGI. In case 1 we probably aren't justified in saying that. To me this seems evident regardless of what the problem actually is! Because if it's hard enough that thousands of human hours cannot find a simple/algorithmic solution it's probably something like an "AGI-complete" problem?
  - nl 2y ago
    Maybe (2). But it took ~50 years work to build systems that can beat people at poker and I don't think people argue poker bots are AGI. To be clear, I think we have AGI (LLMs with tool use are generalized enough) and we are currently finding edge cases that they fail at.
    - HarHarVeryFunny 2y ago
      > I think we have AGI That seems a pretty extreme position! What's your definition of AGI ?
      - 2y ago
HarHarVeryFunny 2y ago
Humans can do infinitely many things because we have general intelligence. Testing whether an AI can play chess or solve Chollet's ARC problems, or some other set of narrow skills, doesn't prove generality. If you want to test for generality, then you either have to: 1) Have a huge and very broad test suite, covering as many diverse human-level skills as possible. and/or, 2) Reductively understand what human intelligence is, and what combination of capabilities it provides, then test for all of those capabilities both individually and in combination. As Chollet notes, a crucial part of any AGI test is solving novel problems that are not just templated versions (or shallow combinatins) of things the wanna-be AGI has been trained on, so for both of above tests this is key.
- bubblyworld 2y ago
  I suspect trying to reductively understand intelligence is a bit like trying to reductively understand biology - every level of abstraction is causally influenced by every other level of abstraction, so there just aren't simple primitives you can break everything down into.
  - daveguy 2y ago
    You can get pretty far with an understanding of biochemical reaction cycles, genetic theory, and protein molecular interactions.
    - bubblyworld 2y ago
      Trying to express the high-level behaviour of an organism in those reductive terms is way beyond science right now, if it's even possible at all. Like have a look at a chart of human metabolic pathways - it's absolute insanity. And those are extremely simplified already!
HarHarVeryFunny 2y ago
A implies B, doesn't mean than B implies A. That's a basic logical fallacy. AGI can add 1+1 correctly, but an ability to do that is not a test for AGI.
- bubblyworld 2y ago
  This is not what I'm saying. Consider the following statement: "Absence of evidence is evidence of absence." Presumably you would call this a simple logical fallacy for the same reason, but a little reflection would show that in many cases such a statement is true! It depends on context, in this case your estimate of how well your search covered the possible search space. Evidence is a continuous variable - things can be weak evidence, strong evidence... There's a whole spectrum. I just take issue with statements like "X is zero evidence of Y" because often you can do a lot better than that with the information at hand.
  - HarHarVeryFunny 2y ago
    We know that computers are capable of things that humans can't - anything related to brute force computation, search and memory for example. So, just because a human can't do something, or struggles to do it, doesn't mean that the task requires a huge IQ or generality - it may just require a lot of compute/memory, such as DeepBlue playing chess. In the case in point of these ARC puzzles, they are easy for a human, so "absense of evidence" doesn't even apply, and it's worth noting that one could also brute force solve them by trying all applicable solution techniques (as indicated by the examples and challenge description) in combinatorial fashion, or just (as Chollet notes) generate a massive training set and train an LLM on it, and solve them via recall rather than active inference, which again proves nothing about AGI. The point of the ARC challenge is to encourage advances in active inference (i.e. reasoning/problem solving), which is what LLMs lack. It's HOW you solve them that matters if you want to show general intelligence. Even in the realm of static inference, which is what they are built for, LLMs are really closer to DeepBlue than something intelligent - they brute force extract the training set rules using gradient descent. The interesting thing is that they have any learning ability at all (in-context learning) at inference time, but it's clearly no match for a human and they are also architecturally missing all the machinery such as working memory and looping/iteration to perform any meaningful try/fail/backtrack/try-again (while learning the whole time) active inference. It'll be interesting to see to what extent pre-trained transformers can be combined with other components (maybe some sort of DeepBlue/AlphaGo MCTS?) to get closer towards human-level problem solving ability, but IMO it's really the wrong architecture. We need to stop using gradient descent and find a learning algorithm that can be used at inference time too.

lucianbr 2y ago

That makes no sense at all. Any problem is initially only solvable by humans, until some technology is developed to solve it. Calculating a logarithm was at some point only doable by humans, and then digital computers came along. This would be in your view evidence that digital computers are AGI!? As in, an 8086 with some math code is AGI. We've had it for decades now, only nobody noticed :)

bubblyworld 2y ago
It's just Bayes theorem - there are basically two variables that control how strong the evidence is: * How likely you think AGI is in general. * How solvable you think the problem is, independently of what's solving it. In the cases you've brought up that latter probability is very high, which means that they are extremely weak evidence that computers are AGI. So we agree! In this case the latter probability seems to be quite low - attempts to solve it with computers have largely failed so far!
- lucianbr 2y ago
  We don't agree. You're now saying anything is evidence of anything, which just makes the word "evidence" meaningless. In real life, when people say "A is evidence of B" they mean strong evidence, or even overwhelming evidence. You just backpedalled by redefining evidence to mean anything and nothing, so you can salvage an obviously false claim. Nobody in the real world says "rain is evidence of aliens" with the implicit assumption that it's just extremely weak evidence. The way English is used by people makes that sentence simply false, as is yours that anything previously not solved is evidence of AGI.
  - bubblyworld 2y ago
    We're talking about a specific problem here - the competition in the OP. Not aliens in the rain.

educasean 2y ago

This flies directly in the face of technologies such as Deep Blue and AlphaGo. They excel in tiny domains previously thought to be the pinnacle of intelligence, and now they dominate humans. Are they AGI in your definition?

bubblyworld 2y ago
See my response to the other commenter. In these cases as well I would conclude it's very weak evidence of AGI, so I don't think we disagree. Edit: I think maybe the disagreement here is about the nature of evidence. I think there can be evidence that something is AGI even if it isn't, in fact, AGI. You seem to believe that if there's any evidence that something is AGI, it must be AGI, I think?
- educasean 2y ago
  I personally don't find this line of rhetoric useful or relevant. Let's agree to disagree.
  - bubblyworld 2y ago
    Okay, that's fair. But to be clear - this is a theorem of probability theory, not rhetoric.

nl 2y ago

> If the only thing that can solve problem X is AGI (e.g. humans), and something else comes along that solves it, then rationally that should be evidence that the something else is AGI right? No. Because there might undiscovered ways to solve these problems that no one claims is AGI. The definition of AGI is notoriously fuzzy, but non-the-less if there was a 10 line python program (with no external dependencies or data) that could solve it then few would argue that was AGI. So perhaps there is an algorithm that solves these puzzles 100% of the time and can be easily expressed. So I agree that only being able to solve these problems doesn't define AGI.

bubblyworld 2y ago
I think I agree with you, but consider these two cases: 1. Only humans are known to have solved problem X, and we've spent no time looking for alternative solutions. 2. Only humans are known to have solved problem X, and we've spent hundreds of thousands of hours looking for alternative solutions and failed. Now suppose something solves the problem. I feel like in case 2 we are justified in saying there's evidence that something is a human-like AGI. In case 1 we probably aren't justified in saying that. To me this seems evident regardless of what the problem actually is! Because if it's hard enough that thousands of human hours cannot find a simple/algorithmic solution it's probably something like an "AGI-complete" problem?
- nl 2y ago
  Maybe (2). But it took ~50 years work to build systems that can beat people at poker and I don't think people argue poker bots are AGI. To be clear, I think we have AGI (LLMs with tool use are generalized enough) and we are currently finding edge cases that they fail at.
  - HarHarVeryFunny 2y ago
    > I think we have AGI That seems a pretty extreme position! What's your definition of AGI ?
    - 2y ago

HarHarVeryFunny 2y ago

Humans can do infinitely many things because we have general intelligence. Testing whether an AI can play chess or solve Chollet's ARC problems, or some other set of narrow skills, doesn't prove generality. If you want to test for generality, then you either have to: 1) Have a huge and very broad test suite, covering as many diverse human-level skills as possible. and/or, 2) Reductively understand what human intelligence is, and what combination of capabilities it provides, then test for all of those capabilities both individually and in combination. As Chollet notes, a crucial part of any AGI test is solving novel problems that are not just templated versions (or shallow combinatins) of things the wanna-be AGI has been trained on, so for both of above tests this is key.

bubblyworld 2y ago
I suspect trying to reductively understand intelligence is a bit like trying to reductively understand biology - every level of abstraction is causally influenced by every other level of abstraction, so there just aren't simple primitives you can break everything down into.
- daveguy 2y ago
  You can get pretty far with an understanding of biochemical reaction cycles, genetic theory, and protein molecular interactions.
  - bubblyworld 2y ago
    Trying to express the high-level behaviour of an organism in those reductive terms is way beyond science right now, if it's even possible at all. Like have a look at a chart of human metabolic pathways - it's absolute insanity. And those are extremely simplified already!

HarHarVeryFunny 2y ago

A implies B, doesn't mean than B implies A. That's a basic logical fallacy. AGI can add 1+1 correctly, but an ability to do that is not a test for AGI.

bubblyworld 2y ago
This is not what I'm saying. Consider the following statement: "Absence of evidence is evidence of absence." Presumably you would call this a simple logical fallacy for the same reason, but a little reflection would show that in many cases such a statement is true! It depends on context, in this case your estimate of how well your search covered the possible search space. Evidence is a continuous variable - things can be weak evidence, strong evidence... There's a whole spectrum. I just take issue with statements like "X is zero evidence of Y" because often you can do a lot better than that with the information at hand.
- HarHarVeryFunny 2y ago
  We know that computers are capable of things that humans can't - anything related to brute force computation, search and memory for example. So, just because a human can't do something, or struggles to do it, doesn't mean that the task requires a huge IQ or generality - it may just require a lot of compute/memory, such as DeepBlue playing chess. In the case in point of these ARC puzzles, they are easy for a human, so "absense of evidence" doesn't even apply, and it's worth noting that one could also brute force solve them by trying all applicable solution techniques (as indicated by the examples and challenge description) in combinatorial fashion, or just (as Chollet notes) generate a massive training set and train an LLM on it, and solve them via recall rather than active inference, which again proves nothing about AGI. The point of the ARC challenge is to encourage advances in active inference (i.e. reasoning/problem solving), which is what LLMs lack. It's HOW you solve them that matters if you want to show general intelligence. Even in the realm of static inference, which is what they are built for, LLMs are really closer to DeepBlue than something intelligent - they brute force extract the training set rules using gradient descent. The interesting thing is that they have any learning ability at all (in-context learning) at inference time, but it's clearly no match for a human and they are also architecturally missing all the machinery such as working memory and looping/iteration to perform any meaningful try/fail/backtrack/try-again (while learning the whole time) active inference. It'll be interesting to see to what extent pre-trained transformers can be combined with other components (maybe some sort of DeepBlue/AlphaGo MCTS?) to get closer towards human-level problem solving ability, but IMO it's really the wrong architecture. We need to stop using gradient descent and find a learning algorithm that can be used at inference time too.