5 ms·

I dont have a strong opinion about it, but when I read parents post I immediately thought of Gödel s works and how there are limits of what you can know within

I dont have a strong opinion about it, but when I read parents post I immediately thought of Gödel s works and how there are limits of what you can know within a system (and in Mathematics no less). Thinking that something similar exists in intelligence does not seem so far fetched. As a hypothesis, of course.

jameshart 3y ago
And yet we can use mathematics to know that that limit exists. It’s a powerful thing, mathematics. Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.
- mejutoco 3y ago
  It is an analogy to mention that is not a far fetched idea that there are limits. As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.
ncallaway 3y ago
The reason it seems far fetched is we have one example of a mechanistic, physics based machine that we know can perform the computations necessary to produce intelligence. I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI. Since we do have an existing machine that can produce AGI, we would have to suppose it does something: - outside of physics to achieve its results - performs some operation that is impossible for us to understand or replicate Both of those seem… unlikely to me.
- jprete 3y ago
  It seems like an article of faith to believe that the universe is a computer in the first place. Without that assumption, we don't even know that intelligence is computation.
  - ncallaway 3y ago
    The article of faith is not that the universe is a computer. The "article of faith" is that the universe follows physical rules. I also don't think it's fair to call it an article of faith, since we have strong evidence to show that _to date_ the universe has followed predictable physical rules. That could, obviously, change at any time, but it seems at least a reasonable prior to assume that the physical rules that we've studied in the past will continue to operate into the future. "The sun will rise tomorrow" is, I _guess_ an article of faith, but I think it's more fair to say it's a reasonable and well founded prediction based on a well studied model of the solar system and the physical laws we've observed the universe follow in the past. So, my beliefs are: - The universe is mechanistic, and follows physical rules - Human beings are also mechanistic, and follow the same physical rules as the universe - Human beings are intelligent - Human beings are at minimum turing complete computers (since you could give me a roll of paper, a set of op-codes, and I could perform calcuations) So, it seems to me a reasonable starting assumption that intelligence is a result of the mechanistic universe. Do we have any evidence for something outside the physical, mechanistic universe impacting human cognition? I'm not aware of any. But we have lots of evidence of the physical, mechanistic universe impacting human cognition and intelligence. I'm willing to acknowledge that we don't know that intelligence is Turing Computable, but I'd argue that intelligence is likely to be a mechanical process that's compatible with the physical rules of the universe. Can we be certain of that? No, we can not be 100% certain. But it seems a much more reasonable and evidenced hypothesis than something which asserts there is a metaphysical process that produces intelligence, but which we have no evidence for. So, no, I don't think it's reasonable to say that assuming a "mechanistic universe" is an article of faith. I think it's a reasonable belief, based on the evidence that we have. What would be an article of faith is asserting that it could _only_ be a mechanistic universe, and refusing to accept any evidence to the contrary. I have not done that.
    - jprete 3y ago
      I think a mechanistic universe is not necessarily a computation, so purely physical intelligence is also not necessarily a computation. But I'm not prepared at the moment to go dig up the alternatives to "computation" and "metaphysical process", or to figure out how a deterministic (or even stochastic) process could be uncomputable.
saltcured 3y ago
Right. I think a charitable reading of the post is something like, "it probably takes a mind superior to a human's in order to willfully develop a human-rivaling AGI". There's a lot of ground to cover between AI as increasingly elaborate magic 8-ball toys and a real human-rivaling AGI. That is, one capable of observing an environment, identifying goals and problems, planning, acting, and reacting. In these much longer (stateful) cognitive chains, there are more opportunities for pathological failure modes and less opportunity for a toy user to charitably excuse the misbehavior. This is not some kind of dualist metaphysical argument about the possibility of AGI in the abstract. Merely a doubt that we can blindly scale up the complexity of a synthetic mind to meet or surpass our own. Consider that we still can't even begin to understand our own minds in enough detail to reliably predict, repair, or augment them. I am outside this field and so may have too much of a layman's perspective. But it seems to me that contemporary AGI believers conflate training and evolution. That nature did it in eons doesn't argue that we can do it in practical product development cycles, unless we can simulate these evolutionary processes to follow a similar search in a compressed time scale. As a crude analogy, I think todays LLM products are a bit like horoscope generators. Clever arrangements of words that attract a charitable or gullible reader. But AGI use cases are more like wanting a life partner who will understand and willingly assist ones efforts.
Vecr 3y ago
That's wrong, see See Gödel, Escher, Bach [1979, pp. 452–456] If you get rid of the infinities and approximate (including cutting recursions short), you can (approximately) compute a whole lot. Not everything, but everything your brain can, and more.
- mejutoco 3y ago
  Sorry, what exactly is wrong? Are there no limits to what you can know within a system?
  - Vecr 3y ago
    It's wrong that an AI system is more limited than a human brain. Obviously there's computability and Busy Beaver stuff, as well as other earlier practical limits to computation, but that's not a limit to "general intelligence" if you mean the level of a human brain. Human brains approximates and hardly want to recurse at all, and there's no reason an AI can't do the same thing.
    - mejutoco 3y ago
      If I understand correctly you state that it is wrong to say a computer is more limited than a human brain. IMO that is far from clear. In some aspects (multiplying matrixes) it might be more powerful. In others (power consumption) the human brain wins by a wide gap.
      - PoignardAzur 3y ago
        No GP's point is that Gödel's theorem has nothinhg to do with this debate because the things that Gödel says are impossible aren't the things GPT-9 would need to do to reach a human-level intelligence. The point that brains are more efficient than computers is valid, but that doesn't tell us much about eg whether we'll be able to make a computer do everything a human brain can do within 50 years.

jameshart 3y ago

And yet we can use mathematics to know that that limit exists. It’s a powerful thing, mathematics. Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.

mejutoco 3y ago
It is an analogy to mention that is not a far fetched idea that there are limits. As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.

ncallaway 3y ago

The reason it seems far fetched is we have one example of a mechanistic, physics based machine that we know can perform the computations necessary to produce intelligence. I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI. Since we do have an existing machine that can produce AGI, we would have to suppose it does something: - outside of physics to achieve its results - performs some operation that is impossible for us to understand or replicate Both of those seem… unlikely to me.

jprete 3y ago
It seems like an article of faith to believe that the universe is a computer in the first place. Without that assumption, we don't even know that intelligence is computation.
- ncallaway 3y ago
  The article of faith is not that the universe is a computer. The "article of faith" is that the universe follows physical rules. I also don't think it's fair to call it an article of faith, since we have strong evidence to show that _to date_ the universe has followed predictable physical rules. That could, obviously, change at any time, but it seems at least a reasonable prior to assume that the physical rules that we've studied in the past will continue to operate into the future. "The sun will rise tomorrow" is, I _guess_ an article of faith, but I think it's more fair to say it's a reasonable and well founded prediction based on a well studied model of the solar system and the physical laws we've observed the universe follow in the past. So, my beliefs are: - The universe is mechanistic, and follows physical rules - Human beings are also mechanistic, and follow the same physical rules as the universe - Human beings are intelligent - Human beings are at minimum turing complete computers (since you could give me a roll of paper, a set of op-codes, and I could perform calcuations) So, it seems to me a reasonable starting assumption that intelligence is a result of the mechanistic universe. Do we have any evidence for something outside the physical, mechanistic universe impacting human cognition? I'm not aware of any. But we have lots of evidence of the physical, mechanistic universe impacting human cognition and intelligence. I'm willing to acknowledge that we don't know that intelligence is Turing Computable, but I'd argue that intelligence is likely to be a mechanical process that's compatible with the physical rules of the universe. Can we be certain of that? No, we can not be 100% certain. But it seems a much more reasonable and evidenced hypothesis than something which asserts there is a metaphysical process that produces intelligence, but which we have no evidence for. So, no, I don't think it's reasonable to say that assuming a "mechanistic universe" is an article of faith. I think it's a reasonable belief, based on the evidence that we have. What would be an article of faith is asserting that it could _only_ be a mechanistic universe, and refusing to accept any evidence to the contrary. I have not done that.
  - jprete 3y ago
    I think a mechanistic universe is not necessarily a computation, so purely physical intelligence is also not necessarily a computation. But I'm not prepared at the moment to go dig up the alternatives to "computation" and "metaphysical process", or to figure out how a deterministic (or even stochastic) process could be uncomputable.

saltcured 3y ago

Right. I think a charitable reading of the post is something like, "it probably takes a mind superior to a human's in order to willfully develop a human-rivaling AGI". There's a lot of ground to cover between AI as increasingly elaborate magic 8-ball toys and a real human-rivaling AGI. That is, one capable of observing an environment, identifying goals and problems, planning, acting, and reacting. In these much longer (stateful) cognitive chains, there are more opportunities for pathological failure modes and less opportunity for a toy user to charitably excuse the misbehavior. This is not some kind of dualist metaphysical argument about the possibility of AGI in the abstract. Merely a doubt that we can blindly scale up the complexity of a synthetic mind to meet or surpass our own. Consider that we still can't even begin to understand our own minds in enough detail to reliably predict, repair, or augment them. I am outside this field and so may have too much of a layman's perspective. But it seems to me that contemporary AGI believers conflate training and evolution. That nature did it in eons doesn't argue that we can do it in practical product development cycles, unless we can simulate these evolutionary processes to follow a similar search in a compressed time scale. As a crude analogy, I think todays LLM products are a bit like horoscope generators. Clever arrangements of words that attract a charitable or gullible reader. But AGI use cases are more like wanting a life partner who will understand and willingly assist ones efforts.

Vecr 3y ago

That's wrong, see See Gödel, Escher, Bach [1979, pp. 452–456] If you get rid of the infinities and approximate (including cutting recursions short), you can (approximately) compute a whole lot. Not everything, but everything your brain can, and more.

mejutoco 3y ago
Sorry, what exactly is wrong? Are there no limits to what you can know within a system?
- Vecr 3y ago
  It's wrong that an AI system is more limited than a human brain. Obviously there's computability and Busy Beaver stuff, as well as other earlier practical limits to computation, but that's not a limit to "general intelligence" if you mean the level of a human brain. Human brains approximates and hardly want to recurse at all, and there's no reason an AI can't do the same thing.
  - mejutoco 3y ago
    If I understand correctly you state that it is wrong to say a computer is more limited than a human brain. IMO that is far from clear. In some aspects (multiplying matrixes) it might be more powerful. In others (power consumption) the human brain wins by a wide gap.
    - PoignardAzur 3y ago
      No GP's point is that Gödel's theorem has nothinhg to do with this debate because the things that Gödel says are impossible aren't the things GPT-9 would need to do to reach a human-level intelligence. The point that brains are more efficient than computers is valid, but that doesn't tell us much about eg whether we'll be able to make a computer do everything a human brain can do within 50 years.