4 ms·

> Personally, I prefer to think about it in terms of basic computability theory: Gödel's incompleteness theorem applies to computing. I'm sure you're familiar

> Personally, I prefer to think about it in terms of basic computability theory: Gödel's incompleteness theorem applies to computing. I'm sure you're familiar with the Halting Problem. Gödel's applies to any axiomatic system. The trouble is, it's very hard to make a system without axioms. They are sneaky and it's different than any logic you're probably familiar with. And don't forget Church-Turing, Gödel Numbers, and all the other stuff. Programming is math and Gödel did essential work on the theory of computation. It would be weird NOT to include his work in this conversation. > are we at most just Turing machines or something else? But this is a great question. Many believe no. Personally I'm unsure, but lean no. Penrose is a clear no but he has some wacky ideas. Problem is, it's hard to tell a bad wacky idea from a good wacky idea. Rephrasing Clarke's Second Law: Genius is nearly indistinguishable from insanity. The only way to tell is with time. But look into things like NARS and Super Turing machines (Hypercomputation). There's a whole world of important things that are not often discussed when it comes to the discussion of AGI. But for those that don't want to dig deep into the math, pick up some Sci-Fi and suspend your disbelief. Star Trek, The Orville and the like have holographic simulations and I doubt anyone would think they're conscious, despite being very realistic. But The Doctor in Voyager or Isaac in The Orville are good examples of the contrary. The Doctor is an entity you see become conscious. It's fiction, but that doesn't mean there aren't deep philosophical questions. Even if they're marked by easy to digest entertainment. Good stories are like good horror, they get under your skin, infect you, and creep in Edit: I'll leave you with another question. Regardless of our Turing or Super-Turing status; is a Turing machine sufficient for consciousness to arise?

red75prime 2y ago
> Regardless of our Turing or Super-Turing status; is a Turing machine sufficient for consciousness to arise? Another question. How do you go about detecting whether consciousness has arisen?
- godelski 2y ago
  It's a great question. Best we got is ~~RLHF~~ Potter's definition: I know it when I see it
eru 2y ago
There's no evidence that hypercomputation is anything that happens in our world, is there? I'm fairly confident of the weaker claim that there's no evidence of hypercomputation in any biological system. (Who know what spinning, charged black holes etc are doing?) > Regardless of our Turing or Super-Turing status; is a Turing machine sufficient for consciousness to arise? A Turing machine can in principle beat the Turing test. But so can a giant lookup table, if there's any finite time limit (however generous) placed on the test. The 'magic' would in the implementation of the table (or the Turing machine) into something that can answer in a reasonable amount of time and be physically realised in a reasonable amount of space. Btw, that's an argument from Scott Aaronson's https://www.scottaaronson.com/papers/philos.pdf https://www.scottaaronson.com/papers/philos.pdf
- godelski 2y ago
  > There's no evidence that hypercomputation I'm not the right person to ask this tbh. As far as I know it doesn't even well in theory. (Though I don't think they would make it useless to study) > can in principle beat the Turing test What's the relevance of the Turing test? It's been beaten for over half a century.
  - eru 2y ago
    > What's the relevance of the Turing test? It's been beaten for over half a century. I would be very interested if you have any sources on anyone beating the Turing test in anything close to Turing's original adversarial formulation.
    - godelski 2y ago
      ELIZA PARRY Eugene Goostman Mitsuku (Kuki AI) Or see the Loebner Prize where judges aren't just average people but experts (so harder). The Turing Test was never about intelligence or thinking, even said so by Turing himself. He made the test because those words are too vague. He specifically wanted to shift the conversation to task completion, since that is actually testable. Great! That's how science works! (Looking at you String Theory...) But science also progresses. We know more about what these things intelligence and thinking are. These still are not testable in a concrete sense, but we have better proxies than the Turing Test now. The problem is that knowledge percolates through society slowly and with a lag. We've advanced a lot since then. I'm sure you are likely willing to believe that most LLMs these days can pass it. The Turing Test was a great start. You gotta start somewhere. But to think we came up with good tests at the same time we invented electronic computers, should give you surprise. Because it would require us to have been much smarter then than we are now.
red75prime 2y ago
> Regardless of our Turing or Super-Turing status; is a Turing machine sufficient for consciousness to arise? In addition to the detectability problem, I wrote in the adjacent comment, this question can be further refined. A Turing machine is an abstract concept. Do we need to take into account material/organizational properties of its physical realization? Do we need to take into account computational complexity properties of its physical realization? Quantum mechanics without Penrose's Orch OR is Turing computable, but its runtime on classical hardware is exponential in, roughly, the number of interacting particles. So, theoretically, we can simulate all there is to simulate about a given person. But to get the initial state of the simulation we need to either measure the person's quantum state (thus losing some information) or teleport his/her quantum state into a quantum computer (the no-cloning theorem doesn't allow to copy it). The quantum computer in this case is a physical realization of an abstract Turing machine, but we can't know its initial state. The quantum computer will simulate everything there are to simulate, but the interaction of a physical human with the initial state of the Universe via photons of the cosmic microwave background. Which may deprive the simulated one of "free will" (see "The Ghost in the Quantum Turing Machine" by Scott Aaronson). Or maybe we can simulate those photons too, I'm not sure about it. Does all of it have anything to do with consciousness? Yeah, those are interesting questions.
- godelski 2y ago
  Yeah I think that's a good point, and my issue with Penrose. His point sounds more profound but it is like dropping everything in complexity analysis (Big O). An approximation only goes so far, and sometimes the specifics matter. More importantly, when they matter, they usually matter A LOT. I think people underestimate how many things we approximate, and approximate with only first or second order terms. But last I checked, most things are not simple harmonic oscillators (springs)[0] [0] See the Taylor Series expansion. This is an only physics joke

red75prime 2y ago

> Regardless of our Turing or Super-Turing status; is a Turing machine sufficient for consciousness to arise? Another question. How do you go about detecting whether consciousness has arisen?

godelski 2y ago
It's a great question. Best we got is ~~RLHF~~ Potter's definition: I know it when I see it

eru 2y ago

There's no evidence that hypercomputation is anything that happens in our world, is there? I'm fairly confident of the weaker claim that there's no evidence of hypercomputation in any biological system. (Who know what spinning, charged black holes etc are doing?) > Regardless of our Turing or Super-Turing status; is a Turing machine sufficient for consciousness to arise? A Turing machine can in principle beat the Turing test. But so can a giant lookup table, if there's any finite time limit (however generous) placed on the test. The 'magic' would in the implementation of the table (or the Turing machine) into something that can answer in a reasonable amount of time and be physically realised in a reasonable amount of space. Btw, that's an argument from Scott Aaronson's https://www.scottaaronson.com/papers/philos.pdf https://www.scottaaronson.com/papers/philos.pdf

godelski 2y ago
> There's no evidence that hypercomputation I'm not the right person to ask this tbh. As far as I know it doesn't even well in theory. (Though I don't think they would make it useless to study) > can in principle beat the Turing test What's the relevance of the Turing test? It's been beaten for over half a century.
- eru 2y ago
  > What's the relevance of the Turing test? It's been beaten for over half a century. I would be very interested if you have any sources on anyone beating the Turing test in anything close to Turing's original adversarial formulation.
  - godelski 2y ago
    ELIZA PARRY Eugene Goostman Mitsuku (Kuki AI) Or see the Loebner Prize where judges aren't just average people but experts (so harder). The Turing Test was never about intelligence or thinking, even said so by Turing himself. He made the test because those words are too vague. He specifically wanted to shift the conversation to task completion, since that is actually testable. Great! That's how science works! (Looking at you String Theory...) But science also progresses. We know more about what these things intelligence and thinking are. These still are not testable in a concrete sense, but we have better proxies than the Turing Test now. The problem is that knowledge percolates through society slowly and with a lag. We've advanced a lot since then. I'm sure you are likely willing to believe that most LLMs these days can pass it. The Turing Test was a great start. You gotta start somewhere. But to think we came up with good tests at the same time we invented electronic computers, should give you surprise. Because it would require us to have been much smarter then than we are now.

red75prime 2y ago

> Regardless of our Turing or Super-Turing status; is a Turing machine sufficient for consciousness to arise? In addition to the detectability problem, I wrote in the adjacent comment, this question can be further refined. A Turing machine is an abstract concept. Do we need to take into account material/organizational properties of its physical realization? Do we need to take into account computational complexity properties of its physical realization? Quantum mechanics without Penrose's Orch OR is Turing computable, but its runtime on classical hardware is exponential in, roughly, the number of interacting particles. So, theoretically, we can simulate all there is to simulate about a given person. But to get the initial state of the simulation we need to either measure the person's quantum state (thus losing some information) or teleport his/her quantum state into a quantum computer (the no-cloning theorem doesn't allow to copy it). The quantum computer in this case is a physical realization of an abstract Turing machine, but we can't know its initial state. The quantum computer will simulate everything there are to simulate, but the interaction of a physical human with the initial state of the Universe via photons of the cosmic microwave background. Which may deprive the simulated one of "free will" (see "The Ghost in the Quantum Turing Machine" by Scott Aaronson). Or maybe we can simulate those photons too, I'm not sure about it. Does all of it have anything to do with consciousness? Yeah, those are interesting questions.

godelski 2y ago
Yeah I think that's a good point, and my issue with Penrose. His point sounds more profound but it is like dropping everything in complexity analysis (Big O). An approximation only goes so far, and sometimes the specifics matter. More importantly, when they matter, they usually matter A LOT. I think people underestimate how many things we approximate, and approximate with only first or second order terms. But last I checked, most things are not simple harmonic oscillators (springs)[0] [0] See the Taylor Series expansion. This is an only physics joke