5 ms·

I feel like Penrose presupposes the human mind is non computable. Perhaps he and other true geniuses can understand things transcendently. Not so for me. My th

I feel like Penrose presupposes the human mind is non computable. Perhaps he and other true geniuses can understand things transcendently. Not so for me. My thoughts are serialized and obviously countable. And in any case: any kind of theorem or idea communicated to another mathematician needs to be serialized into language which would make it computable. So I’m not convinced I could be convinced without a computable proof. And finally just like computable numbers are dense in the reals, maybe computable thoughts are dense in transcendence.

pfortuny 2y ago
I think (but may be wrong) that you are thinking metamathematics is a part of mathematics, which (to my knowledge) it is not.
cwillu 2y ago
He explicitly believes that, yes.
ForTheKidz 2y ago
This is accurate from his Emperor's New Mind. Penrose essentially takes for granted that human brains can reason about or produce results that are otherwise uncomputable. Of course, you can reduce all (historical) human reasoning to a computable heuristic as it is finite, but for some reason he just doesn't see this. His intent at the time was to open a physical explanation for free will by taking the recourse to quantum nano-tubules magnifying true randomness to the level of human cognition. As much as I'm also skeptical that this actually moves the needle on whether or not we have free will (...vs occasionally having access to statistically-certain nondeterminism? Ok...) the computable stuff was just in service of this end. I strongly suspect he just hasn't grasped how powerful heuristics are at overcoming general restrictions on computation. Either that or this is an ideological commitment. Kind of sad—penrose tilings hold a special place in my heart.
- drivebyhooting 2y ago
  If stories are to be believed real geniuses can tap into God’s mind. (See Ramanujan) If so then it really comes down to believing something not because you can prove it but because it is true. I’m just a mediocre mathematician with rigor mortis. So I won’t be too hard on Penrose.
  - Lerc 2y ago
    Those stories are not to be believed.
- tbrownaw 2y ago
  > His intent at the time was to open a physical explanation for free will by taking the recourse to quantum nano-tubules magnifying true randomness to the level of human cognition. As much as I'm also skeptical that this actually moves the needle on whether or not we have free will (...vs occasionally having access to statistically-certain nondeterminism? Ok...) the computable stuff was just in service of this end. Free will is a useful abstraction. Just like life and continuity of self are. > I strongly suspect he just hasn't grasped how powerful heuristics are at overcoming general restrictions on computation. Allowing approximations or "I don't know" is what's helpful. The bpf verifier can work despite the halting problem being unsolvable, not because it makes guesses (uses heuristics) but because it's allowed to lump in "I don't know" with "no".
  - throwup238 2y ago
    > Free will is a useful abstraction. Just like life and continuity of self are. I think it’s more useful to think of them as language games (in the Wittgenstein sense) than abstractions.
    - tbrownaw 2y ago
      I suppose if you really wanted to you could view condensing useful abstractions out of a highly detailed system as a kind of language game, but I'm not convinced that that's useful in the context of investigating particular abstractions rather than investigating the nature of the process of making abstractions.
      - throwup238 2y ago
        I think the very concept of an abstraction is a language game, one that speaks to the old Platonic ideals of Greek philosophy - with all the good and bad that implies. This specific language game takes on a very concrete meaning to programmers that can be quantitatively analyzed by a compiler but epistemologically it’s just another abstract concept (I hate philosophical inception).
    - sunshowers 2y ago
- Animats 2y ago
  > Penrose essentially takes for granted that human brains can reason about or produce results that are otherwise uncomputable. That's Penrose's old criticism. We're past that. It's the wrong point now. Generative AI systems are quite creative. Better than the average human at art. LLMs don't have trouble blithering about advanced abstract concepts. It's concrete areas where these systems have trouble, such as arithmetic. Common sense is still tough. Hallucinations are a problem. Lying is a problem. None of those areas are limited by computability. It's grounding in the real world that's not working well. (A legit question to ask today is this: We now know how much compute it takes to get to the Turing test level of faking intelligence. How do biological brains, with such a slow clock rate, do it? That was part of the concept behind "nanotubules". Something in there must be running fast, right?)
  - lowbloodsugar 2y ago
    > It's concrete areas where these systems have trouble, such as arithmetic. Common sense is still tough. Hallucinations are a problem. Lying is a problem Gestures broadly at humanity
  - Tuna-Fish 2y ago
    > Something in there must be running fast, right? Nah. It just needs to be really wide. This is a very fuzzy comparison, but a human brain has ~100 trillion synaptic connections, which are the closest match we have to "parameters" in AI models. The largest such models currently have on the order of ~2 trillion parameters. (edit to add: and this is a low end estimate of the differences between them. There might be more stuff in neurons that effectively acts as parameters, and should be counted as such in a comparison.) So AI models are still at least two orders of magnitude off from humans in pure width. In contrast, they run much, much faster.
  - gizajob 2y ago
    They’re better than your average human at producing jpegs. I think if you put any of your average humans in a closed room with nothing but canvases, paint, and a mirror, within a month or two they’d be producing pretty interesting paintings without having been fed an image of every single artwork humanity has ever made.
sampo 2y ago
> I feel like Penrose presupposes the human mind is non computable. Yes. He has also written books about it. https://en.wikipedia.org/wiki/Roger_Penrose#Consciousness https://en.wikipedia.org/wiki/Roger_Penrose#Consciousness
thrance 2y ago
What is "understanding transcendently"? Just because Penrose is an authority on some subjects in theoretical physics doesn't mean he is a universal genius and that his ideas on consciousness or AI hold any value. We gotta stop making infaillible super heroes/geniuses of people. In this particular case, Penrose is a convinced dualist and his theories are unscientific. There are very good reasons to not be a dualist, a minority view in philosophy, which I would encourage anyone to seek if they want to better understand Penrose's position and where it came from.
- sarchertech 2y ago
  He’s written and cowritten 5 books on the subject, going back nearly 40 years. I think he can as much as anyone can be considered an “authority” on something as inherently hard to observe or develop falsifiable theories as subjective conscious experience. This isn’t an example of physicist stumbling into a new field for the first time and saying “oh that’s an easy problem. you just need to…” The ideas of a very smart person who has spent decades thinking about a problem tend to be very valuable even if you don’t agree with them.
saulpw 2y ago
> My thoughts are serialized and obviously countable. You might want to consider doing a bit of meditation...anyone who describes their thoughts as 'serialized' and 'obviously countable' has not much time actually looking at their thoughts.
- int_19h 2y ago
  Any kind of self-observation like this is too contaminated to prove anything. For example, you might be perceiving thoughts as simultaneous even though they are actually serialized (or the other way around).
  - saulpw 2y ago
    But then there comes a moment when you recognize perception without thoughts, and it becomes apparent that awareness is different from thought. You might 'think' this observation would be contaminated by thought too, but when there's a direct experience uncontaminated by thought, it's a real a-ha moment. Verified by a great many people across many traditions and over millenia.
    - int_19h 2y ago
      How do you know that you're perceiving "without thoughts", though?
      - saulpw 2y ago
        How does one 'know' anything? Centuries of philosophical inquiry have been devoted to answering this question, but meditative practice is what actually shows you.
AlexCoventry 2y ago
I watched half of the video. He keeps appealing to the idea that Goedel applies to AI because AI doesn't understand what it's doing. But I seriously doubt that we humans really know what we're doing, either. IIRC, his Goedel argument against AI is that someone could construct a Goedel proposition for an intelligent machine which that machine could reason its way through to hit a contradiction. But, at least by default, humans don't base their epistemology on such reasoning, and I don't see why a conscious machine would either. It's not ideal, but frankly, when most humans hit a contradiction, they usually just ignore whichever side of the contradiction is most inconvenient for them.
- enugu 2y ago
  The argument does not need to involve typical human behaviour with its faults. Because if a computer can simulate humans, it can also simulate humans with error correction and verification mechanisms. So, the computer should also be able to simulate a process where a group of humans write down initial deductions and then verify it extensively for logical errors using both computers and other humans. Most of the objections have been covered in his book "Shadows of the Mind". Also, the fact that most human behaviour is not about deducing theorems isn't relevant as that is used as a counterexample which attacks the 'computers can simulate humans' hypothesis. This particular behaviour is chosen, as it is easy to make reflective arguments precise.
cowl 2y ago
> any kind of theorem or idea communicated to another mathematician needs to be serialized into language which would make it computable. This is a fallacy. Just because you need to serialize a concept to communicate it doesnt mean the concept itself is computable. This is established and well proven: https://en.wikipedia.org/wiki/List_of_undecidable_problems https://en.wikipedia.org/wiki/List_of_undecidable_problems The fact that we can come up with this kind of uncumputable problems is a big plus in supprt of Penrose's Idea that consciousnes is not computable and goes way beyond compatability.
- drivebyhooting 2y ago
  Deciding an undecidable problem is well, undecidable. But describing it is clearly not. Otherwise we would not have been able to write about it.
  - cowl 2y ago
    well that is the "beyond computable". we are somehow able to say that this function will not halt, and we wouldnt be able to do that if we only had computable power to simulate it because that would proove that the probem was decidable in the first place. how you comunicate it does not alter the nature of the problem.
- einrealist 2y ago
  That's how I understood Penrose's reasoning too. He differentiated between the computer and whatever is going on in our brain. Computers are just "powerful" enough to encode something that mimics intelligence on the surface (the interviewer tried to pin him on that "something new"), but is still the result of traditional computation, without the involvement of consciousness (his requirement for intelligence).
gizajob 2y ago
From where do those serialised thoughts arise?
Lerc 2y ago
If they can imagine things transcendently they can only assert those things are true. They can't prove it. If they prove it then they have either shown that the idea is not transcendent or that Gödel's theorum is false. That's the same as saying "I know the answer, when you are speculating"
- enugu 2y ago
  Penrose does not require transcendent insights, but merely the ability to examine a finite knowably correct system to arrive at a correct statement which is not provable by the system. In fact, the construction of a Godel statement is mechanical, but this does not mean that the original system can see it. It is a bit like given a supposed finite list of all primes, we can construct a new prime by multiplying them and adding 1. This construction is a simple computation.
  - Lerc 2y ago
    You can always arrive at a correct statement. A random expression generator can occasionally do that. You just can't tell if it is true. A simple integer counter can generate the Gödel statement, it just doesn't have the ability to identify it. You could take a guess, which is what people are doing when they say they they understand, they have applied hurisics to convince themselves, the problem is either decidable, or they are simply wrong.
    - enugu 2y ago
      > You just can't tell if it is true. But in the Penrose argument, we can start from a true system and use reflection to arrive at another true statement which is not deducible from the original system. This is important to the argument as one starts with a proposed program which can perform mathematical reasoning correctly and is not just a random generator. Then, the inability to see the new statement is a genuine limitation.
      - drivebyhooting 2y ago
        How can you see something is true without proving it is true? Many an obvious truth turns out to be mendacious. See various counter examples in analysis.
throwawaymaths 2y ago
> I feel like Penrose presupposes the human mind is non computable i would be a bit more aggressive: Penrose asserts without evidence
- TheOtherHobbes 2y ago
  So do devotees of computability. Remember - there is no such thing as an objective consciousness meter. Emulating the behaviours we associate with consciousness - something that still hasn't been achieved - solves the problem of emulation, not the problem of identity. The idea that an emulation is literally identical to the thing it emulates in this instance only is a very strange belief. Nowhere else in science is a mathematical model of something considered physically identical and interchangeable with the entity being modelled.
  - throwawaymaths 2y ago
    > Nowhere else in science is a mathematical model of something considered physically identical and interchangeable with the entity being modelled you can make the argument that everything in science is a mathematical model... if you measure a basketball arcing through the sky, you are not actually privy to any existential sensing of the ball, you are proxying the essence of the basketball using photons, and even collecting those photons is not really "collecting those photons", etc.
xanderlewis 2y ago
> any kind of theorem or idea communicated to another mathematician needs to be serialized into language which would make it computable Are you aware of how little of modern mathematics has been formalised? As in, properly formalised on a computer. Not just written up into a paper that other mathematics can read and nod along to. Mathematics might seem very formal and serialised (and it is, compared to most other human endeavours) but that’s actually quite far from the truth. Really, it all exists in the mind of the mathematician and a lot of it is hard, if not currently impossible, to pin down precisely enough to enter into a formal system. I think you probably do understand some things ‘transcendently’! Almost by definition they’re the things you’re least aware of understanding.
77pt77 2y ago
> I feel like Penrose presupposes the human mind is non computable. He has been very clear in that he claims exactly that and that there is Quantum Mechanics (wave function collapse in particular) involved. I personally think he's probably wrong, but who really knows?
- amanaplanacanal 2y ago
  Isn't there quantum mechanics in every particle interaction though?
  - 77pt77 2y ago
    Yes, but common wisdom is that in macroscopic systems it all decoheres and it "averages out". He and some other guy claim microtubles maintain coherence in the brain.
sethev 2y ago
Experience is what's hard to square with computability. David Chalmer's calls this the hard problem. As long as you're taking about producing speech or other behaviors, it's easy to see how that might be a computation (and nothing more). It's harder (for me) to see how it's possible to say that pain is just a way of describing things, i.e. that there's in principle no difference between feeling pain and computing a certain function.
enugu 2y ago
This is not a presupposition for Penrose, but a conclusion. The argument for the conclusion is the subject of several of his books. Secondly, the issue is not being a genius, but an ability to reflect. What can be shown, uncontroversially, is that a formal computer system which is knowably correct, a human (or indeed a machine apart which is not the original system) can know something(like a mathematical theorem) which is not accesible to the system. This is due to a standard diagonalization argument used in logic and computability. The important qualifier is 'knowably correct' which doesn't apply to LLMs which are famous for their hallucinations. But, this is not a solid argument for LLMs being able to do everything that humans can do. Because correctness need not refer to immediate outputs, but outputs which are processed through several verification systems.
- lukebuehler 2y ago
  would you mind summarizing what the main argument is? I've watched several of his interviews (but not read the books), and I don't really understand why he concludes that consciousness is not computable.
  - enugu 2y ago
    The core of the argument is simple diagonalization. We know that the problem of deciding if a program halts on some input is undecidable via computation. But suppose we have a partial program P(C,I), to decide halting/not-halting of a program P on an input I, with errors, including input not being of right type, counting as a halt. For instance, P sees a loop at the start of the program for which there is no exit or it sees that the program is trying to find a prime which ends with 4. It marks these cases as non-halting. Halting cases can always be marked by P as halting by just running the code in one thread and waiting to see if it halts. The third option, non-halting programs which are not seen to be so, in this case P itself runs forever. Now given P, it is possible to construct P' a program which detects all the non-halting programs that P does, but atleast one more non-halting case also. To do this consider R, with R(C)=not(P(C,C)) which takes an input program C, asks P 'does C halt with input the source of C itself'. Then R reverses P's decision. If P says halt, then R does not halt on C. If P says not-halt, R halts and finally if P is undecided(so P doesn't stop), R again doesn't halt. What about R(R), R applied to itself. P(R,R) can't be decisive as we will get a contradiction. But in fact R(R) doesn't halt, as all halts are detected by P. So, P' is basically P with the extra detection - R(R) doesn't halt. This proves the weaker claim of Penrose 'No knowably correct program P, can simulate human behaviour' as we can see that if P is right, P' is also right. The 'knowably correct' might seem like an escape hatch, but it is actually hard to split the difference. Simulating humans necessarily means simulating a group of humans who are also doing verification of their initial results with each other and proof-assistants. Also, going from P to P' is a computable process, but if you try to add this to P, you get a new program with a new limitation.
ragle 2y ago
> Perhaps he and other true geniuses can understand things transcendently. Not so for me. My thoughts are serialized and obviously countable. You needn't be a genius. Go on a few vipassana meditation retreats and your perception of all this may shift a bit. > any kind of theorem or idea communicated to another mathematician needs to be serialized into language which would make it computable Hence the suggestion by all mystical traditions that truth can only be experienced, not explained. It may be possible for an AI to have access to the same experiences of consciousness that humans have (around thought, that make human expressions of thought what they are) - but we will first need to understand the parts of the mind / body that facilitate this and replicate them (or a sufficient subset of them) such that AI can use them as part of its computational substrate.

pfortuny 2y ago

I think (but may be wrong) that you are thinking metamathematics is a part of mathematics, which (to my knowledge) it is not.

cwillu 2y ago

He explicitly believes that, yes.

ForTheKidz 2y ago

This is accurate from his Emperor's New Mind. Penrose essentially takes for granted that human brains can reason about or produce results that are otherwise uncomputable. Of course, you can reduce all (historical) human reasoning to a computable heuristic as it is finite, but for some reason he just doesn't see this. His intent at the time was to open a physical explanation for free will by taking the recourse to quantum nano-tubules magnifying true randomness to the level of human cognition. As much as I'm also skeptical that this actually moves the needle on whether or not we have free will (...vs occasionally having access to statistically-certain nondeterminism? Ok...) the computable stuff was just in service of this end. I strongly suspect he just hasn't grasped how powerful heuristics are at overcoming general restrictions on computation. Either that or this is an ideological commitment. Kind of sad—penrose tilings hold a special place in my heart.

drivebyhooting 2y ago
If stories are to be believed real geniuses can tap into God’s mind. (See Ramanujan) If so then it really comes down to believing something not because you can prove it but because it is true. I’m just a mediocre mathematician with rigor mortis. So I won’t be too hard on Penrose.
- Lerc 2y ago
  Those stories are not to be believed.
tbrownaw 2y ago
> His intent at the time was to open a physical explanation for free will by taking the recourse to quantum nano-tubules magnifying true randomness to the level of human cognition. As much as I'm also skeptical that this actually moves the needle on whether or not we have free will (...vs occasionally having access to statistically-certain nondeterminism? Ok...) the computable stuff was just in service of this end. Free will is a useful abstraction. Just like life and continuity of self are. > I strongly suspect he just hasn't grasped how powerful heuristics are at overcoming general restrictions on computation. Allowing approximations or "I don't know" is what's helpful. The bpf verifier can work despite the halting problem being unsolvable, not because it makes guesses (uses heuristics) but because it's allowed to lump in "I don't know" with "no".
- throwup238 2y ago
  > Free will is a useful abstraction. Just like life and continuity of self are. I think it’s more useful to think of them as language games (in the Wittgenstein sense) than abstractions.
  - tbrownaw 2y ago
    I suppose if you really wanted to you could view condensing useful abstractions out of a highly detailed system as a kind of language game, but I'm not convinced that that's useful in the context of investigating particular abstractions rather than investigating the nature of the process of making abstractions.
    - throwup238 2y ago
      I think the very concept of an abstraction is a language game, one that speaks to the old Platonic ideals of Greek philosophy - with all the good and bad that implies. This specific language game takes on a very concrete meaning to programmers that can be quantitatively analyzed by a compiler but epistemologically it’s just another abstract concept (I hate philosophical inception).
  - sunshowers 2y ago
Animats 2y ago
> Penrose essentially takes for granted that human brains can reason about or produce results that are otherwise uncomputable. That's Penrose's old criticism. We're past that. It's the wrong point now. Generative AI systems are quite creative. Better than the average human at art. LLMs don't have trouble blithering about advanced abstract concepts. It's concrete areas where these systems have trouble, such as arithmetic. Common sense is still tough. Hallucinations are a problem. Lying is a problem. None of those areas are limited by computability. It's grounding in the real world that's not working well. (A legit question to ask today is this: We now know how much compute it takes to get to the Turing test level of faking intelligence. How do biological brains, with such a slow clock rate, do it? That was part of the concept behind "nanotubules". Something in there must be running fast, right?)
- lowbloodsugar 2y ago
  > It's concrete areas where these systems have trouble, such as arithmetic. Common sense is still tough. Hallucinations are a problem. Lying is a problem Gestures broadly at humanity
- Tuna-Fish 2y ago
  > Something in there must be running fast, right? Nah. It just needs to be really wide. This is a very fuzzy comparison, but a human brain has ~100 trillion synaptic connections, which are the closest match we have to "parameters" in AI models. The largest such models currently have on the order of ~2 trillion parameters. (edit to add: and this is a low end estimate of the differences between them. There might be more stuff in neurons that effectively acts as parameters, and should be counted as such in a comparison.) So AI models are still at least two orders of magnitude off from humans in pure width. In contrast, they run much, much faster.
- gizajob 2y ago
  They’re better than your average human at producing jpegs. I think if you put any of your average humans in a closed room with nothing but canvases, paint, and a mirror, within a month or two they’d be producing pretty interesting paintings without having been fed an image of every single artwork humanity has ever made.

sampo 2y ago

> I feel like Penrose presupposes the human mind is non computable. Yes. He has also written books about it. https://en.wikipedia.org/wiki/Roger_Penrose#Consciousness https://en.wikipedia.org/wiki/Roger_Penrose#Consciousness

thrance 2y ago

What is "understanding transcendently"? Just because Penrose is an authority on some subjects in theoretical physics doesn't mean he is a universal genius and that his ideas on consciousness or AI hold any value. We gotta stop making infaillible super heroes/geniuses of people. In this particular case, Penrose is a convinced dualist and his theories are unscientific. There are very good reasons to not be a dualist, a minority view in philosophy, which I would encourage anyone to seek if they want to better understand Penrose's position and where it came from.

sarchertech 2y ago
He’s written and cowritten 5 books on the subject, going back nearly 40 years. I think he can as much as anyone can be considered an “authority” on something as inherently hard to observe or develop falsifiable theories as subjective conscious experience. This isn’t an example of physicist stumbling into a new field for the first time and saying “oh that’s an easy problem. you just need to…” The ideas of a very smart person who has spent decades thinking about a problem tend to be very valuable even if you don’t agree with them.

saulpw 2y ago

> My thoughts are serialized and obviously countable. You might want to consider doing a bit of meditation...anyone who describes their thoughts as 'serialized' and 'obviously countable' has not much time actually looking at their thoughts.

int_19h 2y ago
Any kind of self-observation like this is too contaminated to prove anything. For example, you might be perceiving thoughts as simultaneous even though they are actually serialized (or the other way around).
- saulpw 2y ago
  But then there comes a moment when you recognize perception without thoughts, and it becomes apparent that awareness is different from thought. You might 'think' this observation would be contaminated by thought too, but when there's a direct experience uncontaminated by thought, it's a real a-ha moment. Verified by a great many people across many traditions and over millenia.
  - int_19h 2y ago
    How do you know that you're perceiving "without thoughts", though?
    - saulpw 2y ago
      How does one 'know' anything? Centuries of philosophical inquiry have been devoted to answering this question, but meditative practice is what actually shows you.

AlexCoventry 2y ago

I watched half of the video. He keeps appealing to the idea that Goedel applies to AI because AI doesn't understand what it's doing. But I seriously doubt that we humans really know what we're doing, either. IIRC, his Goedel argument against AI is that someone could construct a Goedel proposition for an intelligent machine which that machine could reason its way through to hit a contradiction. But, at least by default, humans don't base their epistemology on such reasoning, and I don't see why a conscious machine would either. It's not ideal, but frankly, when most humans hit a contradiction, they usually just ignore whichever side of the contradiction is most inconvenient for them.

enugu 2y ago
The argument does not need to involve typical human behaviour with its faults. Because if a computer can simulate humans, it can also simulate humans with error correction and verification mechanisms. So, the computer should also be able to simulate a process where a group of humans write down initial deductions and then verify it extensively for logical errors using both computers and other humans. Most of the objections have been covered in his book "Shadows of the Mind". Also, the fact that most human behaviour is not about deducing theorems isn't relevant as that is used as a counterexample which attacks the 'computers can simulate humans' hypothesis. This particular behaviour is chosen, as it is easy to make reflective arguments precise.

cowl 2y ago

> any kind of theorem or idea communicated to another mathematician needs to be serialized into language which would make it computable. This is a fallacy. Just because you need to serialize a concept to communicate it doesnt mean the concept itself is computable. This is established and well proven: https://en.wikipedia.org/wiki/List_of_undecidable_problems https://en.wikipedia.org/wiki/List_of_undecidable_problems The fact that we can come up with this kind of uncumputable problems is a big plus in supprt of Penrose's Idea that consciousnes is not computable and goes way beyond compatability.

drivebyhooting 2y ago
Deciding an undecidable problem is well, undecidable. But describing it is clearly not. Otherwise we would not have been able to write about it.
- cowl 2y ago
  well that is the "beyond computable". we are somehow able to say that this function will not halt, and we wouldnt be able to do that if we only had computable power to simulate it because that would proove that the probem was decidable in the first place. how you comunicate it does not alter the nature of the problem.
einrealist 2y ago
That's how I understood Penrose's reasoning too. He differentiated between the computer and whatever is going on in our brain. Computers are just "powerful" enough to encode something that mimics intelligence on the surface (the interviewer tried to pin him on that "something new"), but is still the result of traditional computation, without the involvement of consciousness (his requirement for intelligence).

gizajob 2y ago

From where do those serialised thoughts arise?

Lerc 2y ago

If they can imagine things transcendently they can only assert those things are true. They can't prove it. If they prove it then they have either shown that the idea is not transcendent or that Gödel's theorum is false. That's the same as saying "I know the answer, when you are speculating"

enugu 2y ago
Penrose does not require transcendent insights, but merely the ability to examine a finite knowably correct system to arrive at a correct statement which is not provable by the system. In fact, the construction of a Godel statement is mechanical, but this does not mean that the original system can see it. It is a bit like given a supposed finite list of all primes, we can construct a new prime by multiplying them and adding 1. This construction is a simple computation.
- Lerc 2y ago
  You can always arrive at a correct statement. A random expression generator can occasionally do that. You just can't tell if it is true. A simple integer counter can generate the Gödel statement, it just doesn't have the ability to identify it. You could take a guess, which is what people are doing when they say they they understand, they have applied hurisics to convince themselves, the problem is either decidable, or they are simply wrong.
  - enugu 2y ago
    > You just can't tell if it is true. But in the Penrose argument, we can start from a true system and use reflection to arrive at another true statement which is not deducible from the original system. This is important to the argument as one starts with a proposed program which can perform mathematical reasoning correctly and is not just a random generator. Then, the inability to see the new statement is a genuine limitation.
    - drivebyhooting 2y ago
      How can you see something is true without proving it is true? Many an obvious truth turns out to be mendacious. See various counter examples in analysis.

throwawaymaths 2y ago

> I feel like Penrose presupposes the human mind is non computable i would be a bit more aggressive: Penrose asserts without evidence

TheOtherHobbes 2y ago
So do devotees of computability. Remember - there is no such thing as an objective consciousness meter. Emulating the behaviours we associate with consciousness - something that still hasn't been achieved - solves the problem of emulation, not the problem of identity. The idea that an emulation is literally identical to the thing it emulates in this instance only is a very strange belief. Nowhere else in science is a mathematical model of something considered physically identical and interchangeable with the entity being modelled.
- throwawaymaths 2y ago
  > Nowhere else in science is a mathematical model of something considered physically identical and interchangeable with the entity being modelled you can make the argument that everything in science is a mathematical model... if you measure a basketball arcing through the sky, you are not actually privy to any existential sensing of the ball, you are proxying the essence of the basketball using photons, and even collecting those photons is not really "collecting those photons", etc.

xanderlewis 2y ago

> any kind of theorem or idea communicated to another mathematician needs to be serialized into language which would make it computable Are you aware of how little of modern mathematics has been formalised? As in, properly formalised on a computer. Not just written up into a paper that other mathematics can read and nod along to. Mathematics might seem very formal and serialised (and it is, compared to most other human endeavours) but that’s actually quite far from the truth. Really, it all exists in the mind of the mathematician and a lot of it is hard, if not currently impossible, to pin down precisely enough to enter into a formal system. I think you probably do understand some things ‘transcendently’! Almost by definition they’re the things you’re least aware of understanding.

77pt77 2y ago

> I feel like Penrose presupposes the human mind is non computable. He has been very clear in that he claims exactly that and that there is Quantum Mechanics (wave function collapse in particular) involved. I personally think he's probably wrong, but who really knows?

amanaplanacanal 2y ago
Isn't there quantum mechanics in every particle interaction though?
- 77pt77 2y ago
  Yes, but common wisdom is that in macroscopic systems it all decoheres and it "averages out". He and some other guy claim microtubles maintain coherence in the brain.

sethev 2y ago

Experience is what's hard to square with computability. David Chalmer's calls this the hard problem. As long as you're taking about producing speech or other behaviors, it's easy to see how that might be a computation (and nothing more). It's harder (for me) to see how it's possible to say that pain is just a way of describing things, i.e. that there's in principle no difference between feeling pain and computing a certain function.

enugu 2y ago

This is not a presupposition for Penrose, but a conclusion. The argument for the conclusion is the subject of several of his books. Secondly, the issue is not being a genius, but an ability to reflect. What can be shown, uncontroversially, is that a formal computer system which is knowably correct, a human (or indeed a machine apart which is not the original system) can know something(like a mathematical theorem) which is not accesible to the system. This is due to a standard diagonalization argument used in logic and computability. The important qualifier is 'knowably correct' which doesn't apply to LLMs which are famous for their hallucinations. But, this is not a solid argument for LLMs being able to do everything that humans can do. Because correctness need not refer to immediate outputs, but outputs which are processed through several verification systems.

lukebuehler 2y ago
would you mind summarizing what the main argument is? I've watched several of his interviews (but not read the books), and I don't really understand why he concludes that consciousness is not computable.
- enugu 2y ago
  The core of the argument is simple diagonalization. We know that the problem of deciding if a program halts on some input is undecidable via computation. But suppose we have a partial program P(C,I), to decide halting/not-halting of a program P on an input I, with errors, including input not being of right type, counting as a halt. For instance, P sees a loop at the start of the program for which there is no exit or it sees that the program is trying to find a prime which ends with 4. It marks these cases as non-halting. Halting cases can always be marked by P as halting by just running the code in one thread and waiting to see if it halts. The third option, non-halting programs which are not seen to be so, in this case P itself runs forever. Now given P, it is possible to construct P' a program which detects all the non-halting programs that P does, but atleast one more non-halting case also. To do this consider R, with R(C)=not(P(C,C)) which takes an input program C, asks P 'does C halt with input the source of C itself'. Then R reverses P's decision. If P says halt, then R does not halt on C. If P says not-halt, R halts and finally if P is undecided(so P doesn't stop), R again doesn't halt. What about R(R), R applied to itself. P(R,R) can't be decisive as we will get a contradiction. But in fact R(R) doesn't halt, as all halts are detected by P. So, P' is basically P with the extra detection - R(R) doesn't halt. This proves the weaker claim of Penrose 'No knowably correct program P, can simulate human behaviour' as we can see that if P is right, P' is also right. The 'knowably correct' might seem like an escape hatch, but it is actually hard to split the difference. Simulating humans necessarily means simulating a group of humans who are also doing verification of their initial results with each other and proof-assistants. Also, going from P to P' is a computable process, but if you try to add this to P, you get a new program with a new limitation.

ragle 2y ago

> Perhaps he and other true geniuses can understand things transcendently. Not so for me. My thoughts are serialized and obviously countable. You needn't be a genius. Go on a few vipassana meditation retreats and your perception of all this may shift a bit. > any kind of theorem or idea communicated to another mathematician needs to be serialized into language which would make it computable Hence the suggestion by all mystical traditions that truth can only be experienced, not explained. It may be possible for an AI to have access to the same experiences of consciousness that humans have (around thought, that make human expressions of thought what they are) - but we will first need to understand the parts of the mind / body that facilitate this and replicate them (or a sufficient subset of them) such that AI can use them as part of its computational substrate.