7 ms·

I've only barely started with this video but I'm currently two thirds of the way into Penroses's "The Emperor's New Mind" and I've read a bunch of other opinion

by simias 5y ago

I've only barely started with this video but I'm currently two thirds of the way into Penroses's "The Emperor's New Mind" and I've read a bunch of other opinion pieces on the subject along the way. I think I'm just too dumb to understand this discussion. I think for me we could split the discussion in two potentially separate questions: - Can a machine emulate a human mind so well that it would be indistinguishable from a "real" human to an external observer (that's the Turing test, effectively) - Can a machine emulate human consciousness And maybe a third bonus question: - Is there a meaningful difference between these two propositions from a scientific perspective? I.e. can we make falsifiable claims that would let us suss out philosophical zombies? At this point I'm absolutely convinced that the answer to the first question is affirmative. We're not there yet and there's quite a long way to go until we do, but I really don't see why there would be a fundamental mathematical hurdle on the way there. Maybe it exists, but I have yet to find a really compelling argument of where that hurdle would lie concretely. Give me an example of a thought that we couldn't teach a machine. Regarding the 2nd question (and the 3rd) it just boils down to "what's consciousness exactly? Is it even knowable?" and I don't think anybody has an answer to that. My personal intuition is that it's unknowable.

prometheus76 5y ago
EDIT: I am watching the video now, and made my comment below before I started watching the video. I realize now that my comment is somewhat outside the scope of the video itself, but I'll leave the comment up anyway, because I think it reveals a similar conclusion as the video. I'll take a stab at what seems to me to be an insurmountable problem for AI: vision. And not just vision, but "seeing". Pretend I am standing next to a table, and the AI successfully identifies me, and identifies a table next to me (which, in itself, is a very difficult, if not impossible, thing for AI to do right now). Now, let's say I sit on the table. Now, we ask the AI, "Am I sitting on a table, or sitting on a chair?" Further to the point (and a less "human" scenario): Musk has been attempting for years to make a lights-out facility for building cars. As of yet, there are still many things that a robot cannot do, or a human can do faster and more reliably, in spite of throwing billions of dollars and millions of work-hours at the problem. Another example: shoe manufacturing robots. Another example: brick laying robots. Another example: pipe welding robots. None of those can even come close the a human's ability to adapt on the fly to small variations, or to learn new behaviors. AI sees the world in a very low-resolution way, and humans are generally unaware of how much "constructing of the world" our brains do compared to the input data it receives from our senses. Replicating this is going to take more than a GPT3, for example.
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
- worldsayshi 5y ago
  Feels like what you're referring to is just "understanding". GPT3 fails at this as well. It seems that gpt3 can string together sentences and whole paragraphs that seem to sort of make sense together but it seems that it never really puts hard constraints on any concepts. It doesn't try to build a comprehensive world with rules. Everything is just a sort of a fuzzy continuum. If you train it on enough data this fuzzyness generate a lot of plausibly sounding output but it will have a hard time coming up with the right answer in a new situation. I feel the missing ingredient is to be able to combine this fuzzy or intuitive understanding with some kind of rule engine. It's like in the book "Thinking Fast and Slow". It feels like GPT and the best image recognition algorithms we have today are good at the fast kind of thinking but doesn't have the slow and logical part.
- abecedarius 5y ago
  > very difficult, if not impossible, thing for AI to do right now Is that right? I thought computer vision was getting pretty good at object detection in the past few years.
  - simonh 5y ago
    Those are just image classifiers though. They associate this pixel patters with the character string "table". They might be able to recognise this image as relating to both table and person, but most likely as competing possible classifications. They generally don't have a concept of relatedness between classifications. They have no spacial conceptual model relating concepts like 'above', beside', 'below', distance, etc. They also have no idea what a table is, no general concept of furniture, or even of a table as a physical or 3 dimensional object. They certainly have no idea how it relates to the human form or what "sitting" on something means. It's literally just pixel pattern => string("table"). That's it. There may be some experimental models that attempt such things, but the image classifiers that get all the press these days are very good at classification but nothing else.
deleted 5y ago
[deleted]
sva_ 5y ago
> Regarding the 2nd question (and the 3rd) it just boils down to "what's consciousness exactly? Is it even knowable?" and I don't think anybody has an answer to that. My personal intuition is that it's unknowable. You can't even know if anyone other than you is conscious. For all you know, the rest of us might all be robots being good at fooling you. I think that if we achieve AGI, at some point there will be a division between people: Those who acknowledge them as living, feeling entities, and those who don't. Just like even nowadays there are still people who feel about others (based on race, skin color, etc), that they're not really people.
- roenxi 5y ago
  > You can't even know if anyone other than you is conscious. Given that we can't define consciousness, even that might be implying more knowledge than someone can possess.
- mannykannot 5y ago
  > You can't even know if anyone other than you is conscious. This claim comes up quite often in these sorts of discussions, but it is very hard to maintain this level of skepticism, about the external world, consistently (to start with, do you believe any of your thoughts are about an external world?) To take this extremely skeptical position only about other peoples' minds would be quite tendentious.
chromaton 5y ago
We do have a natural experiment which may provide some insight into consciousness: conjoined twins. They seem to be able to both think independently but also communicate wordlessly.
- prometheus76 5y ago
  This applies to any two humans, not just twins. Ever smiled at someone and had them smile back? Ever waved at someone and they waved back? Ever shook your head at someone's behavior? We communicate wordlessly all the time.
  - mannykannot 5y ago
    If one twin had direct neural access to the other twin's emotional states or "what it's like" aspect of sensory perception, that would be different than any form of communication involving one making gestures that the other interprets.
    - simonh 5y ago
      I don't believe we have any reliable evidence for that though.
      - mannykannot 5y ago
        No; the cases of conjoined-at-the-brain twins where both twins are conscious to the degree of each knowing that it is a person are extremely rare, and in any case, a dualist could always claim that whatever is physically observable is not the whole story, as dualism is unfalsifiable so long as its proponents avoid making any affirmative statements about how the mind works.
- amw-zero 5y ago
  They have independent brains though. I don’t think it’s a useful data point.
trashtester 5y ago
My take on your third question: If you program a mind specifically to say "I'm concious.", it does not prove that it is. However, if conciousness depends on something we cannot simulate and that conciousness is the causal reason we think (or say) we are concious, I would not expect that an exact simulation of a brain would be able to do the things we attribute to conciousness. So if you emulate a mind to the best accuracy available, and it suddenly says "I'm concious.", it is at least a very strong indication that it really is, at least in the same sense that humans are.
bobthechef 5y ago
The first task is to define our terms. It's impossible to reason about anything without a good grasp of terms. So, for example, if by "consciousness" we mean "intentionality" or at least that it entails intentionality, then you have to explain a) how anything within a computer can be about anything in the world, and b) how abstract concepts can exist in a computer. And you have to think very carefully about this because the internet is full of flippant and unexamined responses that confuse the interpreter with the interpreted. Let's say I take a picture of a bike wheel and store it on a computer. What makes this about the bike wheel? I know that the picture is about the wheel, but the picture does not contain that information. No "metadata" can accomplish that either because metadata is of the same nature as the image. Metadata isn't even intrinsically about the image in question. It's there, maybe adjacent to it, but its aboutness is, like the image's aboutness, entirely in my mind. A program can be written to simulate aboutness, but again, the aboutness is simulated in the sense that it is not a real feature of either the metadata or the program. It is an interpretation the programmer or user brings to the computer. So again, no aboutness is to be found in the program either. Then there's the question of abstract concepts. A wheel is a concrete thing, but to know that it is circular is no longer a concrete matter. Circularity is nowhere to be found in the world as such. All you ever have are particular circular things. The concept of circularity requires abstraction from particulars. So why can't a computer do that? The reason is the same as the reason why circularity does not exist in the world on its own and that is that matter is always particular. Matter is not abstract. But computers are material things. So how could a computer abstract circularity from an image of a wheel? Sure, you can have an algorithm that is written to match circles in images, but this is not the same as having a concept of circularity on its own. For that you need an intellect. The problem of qualia is also related. A materialist account of physical reality entails the denial of qualities like color because all that a materialist accepts is a world of geometric extension in space. Qualities are therefore taken to be features of consciousness. But if consciousness is a physical phenomenon, then how can that be? Here again computers provide no answer. Sure, I can represent the color "red" using #FF0000, but this is not the color red. I can construct a monitor that, when fed a signal that corresponds to #FF0000 activates some physical elements to produce red light, but this is just a conventional alignment between that representation and the construction of the monitor. There is nothing intrinsically red about #FF0000. And besides, the monitor isn't really producing red light according to the materialist. It is producing light that is in a state that, when observed by a conscious being, appears red (which, again, runs into the problem of how even that is possible given that consciousness is physical). So the whole idea that computers can be understood as truly autonomous things apart from human beings is bankrupt. They are better understood like all other technology: an "extension" of human beings. While most animals have all they need in themselves to achieve their ends, human beings can construct technological artefacts to perform actions that they could not otherwise. A turtle has a shell to protect himself, but he is also limited to that shell. Human beings are on the other hand lack any such protection, we're soft and easier to hurt, but we have a mind that allows us to make protection when we need it and in an (in principle at least) uncountable many ways depending on the needs facing us. We make coats for winter weather, astronaut suits for the low-pressure environment of space, a beekeeper's helmet when extracting honey from hives, and so on. Computers are just another expression of the same principle.
mannykannot 5y ago
I think it is worth pointing out that the Lucas-Penrose argument goes beyond Gödel's disjunction: apparently Gödel himself could not rule out both sides of his disjunction being true, but Penrose, expanding Lucas's argument[1] to book length, thinks he can. It seems, however, that most experts now find their arguments inadequate[2], and the speaker in this video is prominent among them. Wikipedia has a short summary of some of the objections[3], and I think Minsky's is particularly to the point: as humans can believe false ideas to be true, human mathematical understanding need not be consistent and consciousness may easily have a deterministic basis. [1] http://cogprints.org/356/1/lucas.html http://cogprints.org/356/1/lucas.html [2] https://www.maa.org/press/maa-reviews/g-dels-disjunction https://www.maa.org/press/maa-reviews/g-dels-disjunction [3] https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument#Criticism https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument...

by simias 5y ago

prometheus76 5y ago

EDIT: I am watching the video now, and made my comment below before I started watching the video. I realize now that my comment is somewhat outside the scope of the video itself, but I'll leave the comment up anyway, because I think it reveals a similar conclusion as the video. I'll take a stab at what seems to me to be an insurmountable problem for AI: vision. And not just vision, but "seeing". Pretend I am standing next to a table, and the AI successfully identifies me, and identifies a table next to me (which, in itself, is a very difficult, if not impossible, thing for AI to do right now). Now, let's say I sit on the table. Now, we ask the AI, "Am I sitting on a table, or sitting on a chair?" Further to the point (and a less "human" scenario): Musk has been attempting for years to make a lights-out facility for building cars. As of yet, there are still many things that a robot cannot do, or a human can do faster and more reliably, in spite of throwing billions of dollars and millions of work-hours at the problem. Another example: shoe manufacturing robots. Another example: brick laying robots. Another example: pipe welding robots. None of those can even come close the a human's ability to adapt on the fly to small variations, or to learn new behaviors. AI sees the world in a very low-resolution way, and humans are generally unaware of how much "constructing of the world" our brains do compared to the input data it receives from our senses. Replicating this is going to take more than a GPT3, for example.

deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
worldsayshi 5y ago
Feels like what you're referring to is just "understanding". GPT3 fails at this as well. It seems that gpt3 can string together sentences and whole paragraphs that seem to sort of make sense together but it seems that it never really puts hard constraints on any concepts. It doesn't try to build a comprehensive world with rules. Everything is just a sort of a fuzzy continuum. If you train it on enough data this fuzzyness generate a lot of plausibly sounding output but it will have a hard time coming up with the right answer in a new situation. I feel the missing ingredient is to be able to combine this fuzzy or intuitive understanding with some kind of rule engine. It's like in the book "Thinking Fast and Slow". It feels like GPT and the best image recognition algorithms we have today are good at the fast kind of thinking but doesn't have the slow and logical part.
abecedarius 5y ago
> very difficult, if not impossible, thing for AI to do right now Is that right? I thought computer vision was getting pretty good at object detection in the past few years.
- simonh 5y ago
  Those are just image classifiers though. They associate this pixel patters with the character string "table". They might be able to recognise this image as relating to both table and person, but most likely as competing possible classifications. They generally don't have a concept of relatedness between classifications. They have no spacial conceptual model relating concepts like 'above', beside', 'below', distance, etc. They also have no idea what a table is, no general concept of furniture, or even of a table as a physical or 3 dimensional object. They certainly have no idea how it relates to the human form or what "sitting" on something means. It's literally just pixel pattern => string("table"). That's it. There may be some experimental models that attempt such things, but the image classifiers that get all the press these days are very good at classification but nothing else.

deleted 5y ago

[deleted]

sva_ 5y ago

> Regarding the 2nd question (and the 3rd) it just boils down to "what's consciousness exactly? Is it even knowable?" and I don't think anybody has an answer to that. My personal intuition is that it's unknowable. You can't even know if anyone other than you is conscious. For all you know, the rest of us might all be robots being good at fooling you. I think that if we achieve AGI, at some point there will be a division between people: Those who acknowledge them as living, feeling entities, and those who don't. Just like even nowadays there are still people who feel about others (based on race, skin color, etc), that they're not really people.

roenxi 5y ago
> You can't even know if anyone other than you is conscious. Given that we can't define consciousness, even that might be implying more knowledge than someone can possess.
mannykannot 5y ago
> You can't even know if anyone other than you is conscious. This claim comes up quite often in these sorts of discussions, but it is very hard to maintain this level of skepticism, about the external world, consistently (to start with, do you believe any of your thoughts are about an external world?) To take this extremely skeptical position only about other peoples' minds would be quite tendentious.

chromaton 5y ago

We do have a natural experiment which may provide some insight into consciousness: conjoined twins. They seem to be able to both think independently but also communicate wordlessly.

prometheus76 5y ago
This applies to any two humans, not just twins. Ever smiled at someone and had them smile back? Ever waved at someone and they waved back? Ever shook your head at someone's behavior? We communicate wordlessly all the time.
- mannykannot 5y ago
  If one twin had direct neural access to the other twin's emotional states or "what it's like" aspect of sensory perception, that would be different than any form of communication involving one making gestures that the other interprets.
  - simonh 5y ago
    I don't believe we have any reliable evidence for that though.
    - mannykannot 5y ago
      No; the cases of conjoined-at-the-brain twins where both twins are conscious to the degree of each knowing that it is a person are extremely rare, and in any case, a dualist could always claim that whatever is physically observable is not the whole story, as dualism is unfalsifiable so long as its proponents avoid making any affirmative statements about how the mind works.
amw-zero 5y ago
They have independent brains though. I don’t think it’s a useful data point.

trashtester 5y ago

My take on your third question: If you program a mind specifically to say "I'm concious.", it does not prove that it is. However, if conciousness depends on something we cannot simulate and that conciousness is the causal reason we think (or say) we are concious, I would not expect that an exact simulation of a brain would be able to do the things we attribute to conciousness. So if you emulate a mind to the best accuracy available, and it suddenly says "I'm concious.", it is at least a very strong indication that it really is, at least in the same sense that humans are.

bobthechef 5y ago

The first task is to define our terms. It's impossible to reason about anything without a good grasp of terms. So, for example, if by "consciousness" we mean "intentionality" or at least that it entails intentionality, then you have to explain a) how anything within a computer can be about anything in the world, and b) how abstract concepts can exist in a computer. And you have to think very carefully about this because the internet is full of flippant and unexamined responses that confuse the interpreter with the interpreted. Let's say I take a picture of a bike wheel and store it on a computer. What makes this about the bike wheel? I know that the picture is about the wheel, but the picture does not contain that information. No "metadata" can accomplish that either because metadata is of the same nature as the image. Metadata isn't even intrinsically about the image in question. It's there, maybe adjacent to it, but its aboutness is, like the image's aboutness, entirely in my mind. A program can be written to simulate aboutness, but again, the aboutness is simulated in the sense that it is not a real feature of either the metadata or the program. It is an interpretation the programmer or user brings to the computer. So again, no aboutness is to be found in the program either. Then there's the question of abstract concepts. A wheel is a concrete thing, but to know that it is circular is no longer a concrete matter. Circularity is nowhere to be found in the world as such. All you ever have are particular circular things. The concept of circularity requires abstraction from particulars. So why can't a computer do that? The reason is the same as the reason why circularity does not exist in the world on its own and that is that matter is always particular. Matter is not abstract. But computers are material things. So how could a computer abstract circularity from an image of a wheel? Sure, you can have an algorithm that is written to match circles in images, but this is not the same as having a concept of circularity on its own. For that you need an intellect. The problem of qualia is also related. A materialist account of physical reality entails the denial of qualities like color because all that a materialist accepts is a world of geometric extension in space. Qualities are therefore taken to be features of consciousness. But if consciousness is a physical phenomenon, then how can that be? Here again computers provide no answer. Sure, I can represent the color "red" using #FF0000, but this is not the color red. I can construct a monitor that, when fed a signal that corresponds to #FF0000 activates some physical elements to produce red light, but this is just a conventional alignment between that representation and the construction of the monitor. There is nothing intrinsically red about #FF0000. And besides, the monitor isn't really producing red light according to the materialist. It is producing light that is in a state that, when observed by a conscious being, appears red (which, again, runs into the problem of how even that is possible given that consciousness is physical). So the whole idea that computers can be understood as truly autonomous things apart from human beings is bankrupt. They are better understood like all other technology: an "extension" of human beings. While most animals have all they need in themselves to achieve their ends, human beings can construct technological artefacts to perform actions that they could not otherwise. A turtle has a shell to protect himself, but he is also limited to that shell. Human beings are on the other hand lack any such protection, we're soft and easier to hurt, but we have a mind that allows us to make protection when we need it and in an (in principle at least) uncountable many ways depending on the needs facing us. We make coats for winter weather, astronaut suits for the low-pressure environment of space, a beekeeper's helmet when extracting honey from hives, and so on. Computers are just another expression of the same principle.

mannykannot 5y ago

I think it is worth pointing out that the Lucas-Penrose argument goes beyond Gödel's disjunction: apparently Gödel himself could not rule out both sides of his disjunction being true, but Penrose, expanding Lucas's argument[1] to book length, thinks he can. It seems, however, that most experts now find their arguments inadequate[2], and the speaker in this video is prominent among them. Wikipedia has a short summary of some of the objections[3], and I think Minsky's is particularly to the point: as humans can believe false ideas to be true, human mathematical understanding need not be consistent and consciousness may easily have a deterministic basis. [1] http://cogprints.org/356/1/lucas.html http://cogprints.org/356/1/lucas.html [2] https://www.maa.org/press/maa-reviews/g-dels-disjunction https://www.maa.org/press/maa-reviews/g-dels-disjunction [3] https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument#Criticism https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument...