6 ms·

For me, the question addressed by the article isn't even the interesting one. The author talks about "intelligence" as in how we process environmental informati

by garyrob 11y ago

For me, the question addressed by the article isn't even the interesting one. The author talks about "intelligence" as in how we process environmental information and perform reasoning tasks. From that perspective, it does seem obvious that some form of computer will one day be able to do it. I don't really care if the algorithms for doing so are simple or complicated. For me, the real question of interest is not about that kind of intelligence, but of "awareness". Computers may process environmental information and perform all sorts of logical operations based on it. But there is zero reason to think that any computer has any awareness whatsoever. It has no experience. There is something it is like to experience the color yellow, as opposed to merely classifying observed light as being of a certain wavelength; no computer has begun, in any way, to make that leap. It seems like a lot of people, including the author, don't even notice that this is a question. They seem to assume that computer-like information processing is all that happens in a human mind, and that awareness itself doesn't even exist as something to ponder. So they equate any doubt about whether a computer can do what we do as a form of vitalism. But with just a little bit of introspection, the fact that there is something it is like to experience the color yellow, as opposed to classifying a visual stimulus as being of a certain wavelength, or logically classifying it as having the attribute of "yellowness", is absolutely clear -- and absolutely fundamental to our existence as human beings (although many would assume that many other forms of life also have this basic ability to experience, if not the same ability to process information in computer-like ways). This is why, for me, the Turing Test is orthogonal to the question of whether a machine is actually conscious. I have no problem imagining that a software system, even on hardware that is basically the same as what we have now but bigger and faster, will one day be able to pass the Turing Test. But that doesn't prove it has any experience of the color yellow; it merely means it can mimic the output of an entity that does.

Jabbles 11y ago
Algorithms that can write heartfelt posts like yours will be developed one day. This is literally the point of Turing's thought experiment: if a machine gives you answers indistinguishable from a human, is it alive?
- paulojreis 11y ago
  If a call center operator follows a "answering" algorithm strictly, is it a computer? These thought experiments are very interesting and thought provoking. But - still - it's a mathematician asking if something is alive/human. Not the most authoritative source on the subject matter.
  - josh2600 11y ago
    Yes. The original computers were women in warehouses calculating actuary tables. They had a set of criteria to perform evaluations of numbers. In short, you can reduce a subset of the human experience to algorithmic emulation, which, in effect, reduces a human to a computer. It's sort of what it means to do any job to be frank.
    - paulojreis 11y ago
      I meant a electronic computer, but well put. > In short, you can reduce a subset of the human experience to algorithmic emulation, which, in effect, reduces a human to a computer. You're underestimating everything associated with being human. The computer won't make mistakes because a familiar is sick. He won't perform slower than usual because he slept less. He won't have a urge to poop affecting is performance, also. This is somewhat what I meant: these thought experiments are interesting, but if you look closer there's really not much to them. Human computers were, of course, computing. They didn't, however, became "not human".
      - Tiksi 11y ago
        > The computer won't make mistakes because a familiar is sick. Race conditions cause mistakes. Seems similar to having your mind elsewhere. > He won't perform slower than usual because he slept less. Didn't clean up memoery so now your computer is resource starved and running slower. > He won't have a urge to poop affecting is performance, GC pause. I feel like people often focus too much on the "implementation" when it comes to consciousness. Beyond what instinct gives us, which is really just a pre defined set of conditionals, everything we know and understand is learned, programmed in a way. I don't see why it's unfeasible that given enough time and the right environment that an AI could do the same.
jmagoon 11y ago
This is a well defined (but somewhat controversial) philosophical topic known as the hard problem of consciousness: https://en.wikipedia.org/wiki/Hard_problem_of_consciousness https://en.wikipedia.org/wiki/Hard_problem_of_consciousness
avn2109 11y ago
In the literature, they use the term "qualia" for what you're calling "awareness" here.
- garyrob 11y ago
  Yes. I'm not extremely well-read in the literature but I have read some of it. I'm aware of the term "qualia" but was trying to use the most direct, simple possible language to communicate.
gnaritas 11y ago
> But that doesn't prove it has any experience of the color yellow; it merely means it can mimic the output of an entity that does. Which is all we might be doing with each other as well. That awareness could merely be an illusion and there's even some evidence to think it is. Experiments show your brain makes a decision before you are consciously aware of it despite you thinking you made the decision consciously. That leads to the conclusion that awareness is simply a lie your brains tells itself to feel in control.
habitue 11y ago
How do you know other humans can truly experience "yellowness"? What if we're all just pretending and you're the only one? Let's not talk about realistic AI for a moment, let's talk about hypothetically building a human protein by protein (or whatever building block you're comfortable with saying doesn't have qualia on its own). We're going to write the computer program to build this human. Nanobots will carry it out, every part of the process is understood and nothing is left for biology to do for us. We're just putting together lego blocks and at the end is a human indistinguishable from any other human. Does this person experience qualia? How would you know? Apparently you can't just ask, since this isn't a valid thing to do with a computer (it's not really experiencing qualia! It's just saying that!). Let's just say you come up with some objective qualia test. Now we can build a slightly more deficient human protein by protein and test it for qualia. In fact, we can do a qualia binary search and... Once you take apart the argument like this, it's easy to see that arguments that computers can't experience qualia are arguments against materialism or reductionism, or are patent attempts to sneak souls and other woo into the discussion.
- stelonix 11y ago
  I believe his point is, such a process wouldn't ever be feasible if we can't "create" awareness. You'd have a sack of meat that would not interact. Or if it would, it'd be a vegetable capable of computation but little else (that's my random assumption). If you forget your fear of "soul" and other terms not ratified by science journals, you'll find out there are plenty of theories in metaphysics and spirituality on what it means to be and what awareness is. It requires you to at the very least read those philosophical/theological concepts instead of dismissing it as "woo". Cattle dwellers in India wrote many things about it eons ago. Joseph Campbell's work is a nice place to start.
  - habitue 11y ago
    > you'll find out there are plenty of theories in metaphysics and spirituality on what it means to be and what awareness is I have encountered these theories. In the past I have been under the sway of them. They can be interesting to think about, but without any evidence for them there isn't any reason to give them particular consideration. (I am including rational, mathematical, or logical arguments based on experimental evidence as evidence here) Without evidence, how do we pick which of these myriad theories to believe? How do we decide what the actual answer is? Science and materialism have repeatedly produced predictions that pan out in experiments. Theories of souls and sprituality have repeatedly had to retreat from testable predictions. To take another tack, how did these spiritual theories come about? If they're true, then the ancient theorist must have seen some phenomena in the real world, and extrapolated correctly from that evidence. If that's the case, these ancient theorists were unparalleled geniuses, but we don't need to worry about ancient spiritual theories themselves, we'll rediscover them in time through our plodding (but safe) scientific method. If we don't rediscover them, then they were interesting theories that didn't pan out.
agumonkey 11y ago
> to experience the color yellow, as opposed to classifying a visual stimulus as being of a certain wavelength I've been wondering about this for a few years, can't find anything about that because it's too generic a search term. Is there a name, or references about this ? ps: ha, these in-thread answers already give pointers https://news.ycombinator.com/item?id=9864468 https://news.ycombinator.com/item?id=9864468 https://news.ycombinator.com/item?id=9864479 https://news.ycombinator.com/item?id=9864479 <closed/>
elektromekatron 11y ago
experience the color yellow, as opposed to classifying a visual stimulus as being of a certain wavelength Purple is the weird one as it doesn't correspond to a wavelength, but can only be represented by a set. Edit - all colours correspond to sets of sets, if you think about it, as you can experience exactly the same colour for essentially an infinite set of different sets of frequencies.
EGreg 11y ago
I once met David Chalmers when he was teaching a philosophy course at NYU. And I asked him after the class ... What can you say about consciousness in the "hard probem of consciousness"? I am having trouble making any statements about it which are objective. I have begun to suspect that our concepts of consciousness and morality are the result of our language and the way we have defined them. With morality is has become clear to me that personal duties are relative to personal goals. That is, "X should do Y" is incoherent, it really implies "X should do Y if X wants Z to happen" which is what the colloquial understanding is. Always being careful to mention Z removes any feeling of mysticism from morality!! Perhaps with statements about consciousness, the situation is similar. I couldn't really formulate any sort of mystical question once I was carefully adding the SUBJECT of each sentence explicitly in. The main question I have about consciousness is not about the processing - that can be explained mechanistically. But about the final result of perceiving this orchestra - why am I in my brain and not anyone else's? What is this extra element that is required for the conscious experience and can we ever be sure a machine we built is really subjectively perceiving anything in that metaphysical sense? Thoughts?
- JoeAltmaier 11y ago
  Doesn't seem right - morality is often intuitive or blind. Else why did Socrates say "The unexamined life is not worth living?" Why, because most (almost all) people don't think about Z.
  - zkhalique 11y ago
    Just because they don't think about or stress Z doesn't mean it isn't there. Sometimes Z is a subconscious brain heuristic developed as a response to competing interests, threats, social consequences etc.
    - JoeAltmaier 11y ago
      Folks learn their moral sense long before they can rationalize anything. Do what momma says, that's all the 'Z' that the brain knows.
rndn 11y ago
> It seems like a lot of people, including the author, don't even notice that this is a question. They seem to assume that computer-like information processing is all that happens in a human mind, and that awareness itself doesn't even exist as something to ponder. This seems to be a misjudgement. There are actually plenty of people who have studied traditional philosophy and dualistic approaches to consciousness for years and eventually arrived at a naturalistic conclusion after they've studied neuroscience. I would actually be very much interested in the numbers for the opposite case: How many subscribers to dualism have knowledge in neuroscience, human behavioral biology, the theory of evolution and physics?
fauigerzigerk 11y ago
My problem with that argument, which I have heard quite often, is that we cannot do anything useful with a term (awareness/experience of yellow in this case) that has neither extension nor intension. What you experience when you see yellow is what you experience when you see yellow. Nothing more can be said about it and it cannot be compared to anyone elses experience of yellow, human or machine. So if you make that experience a condition for calling another being intelligent, you are effectively saying that nothing but yourself can ever be intelligent. You need to break down that thing you call experience of yellow or awareness of yellow and explain what properties it has (intension) or you need to be able to show that it influences the behaviour of humans in a way in which machines cannot be influenced by it (extension).
- garyrob 11y ago
  "So if you make that experience a condition for calling another being intelligent, you are effectively saying that nothing but yourself can ever be intelligent." I think you are confusing the principles of whether an observer can determine whether an entity has an attribute with the question of whether it does in fact have that attribute. But those are entirely different issues because, for one thing, the observer may simply not have the tools yet to make such a judgement. Matter is made of atoms. For a long time we did not have the tools to determine that. But that didn't change the fact that it was, and is, true. Now suppose the human race died out before it ever developed such tools. Obviously, it would still be true. Now suppose that due to our limited intellectual abilities or for some other reason, it never was, or will be, possible to ever develop such tools. To me, it's obvious that that has no bearing on the truth of the proposition any more than if we didn't have time to develop the tools. Similarly, the question of whether you have awareness is independent of whether I or any other third party can prove you do.
  - fauigerzigerk 11y ago
    I don't necessarily disagree with that, but your argument is nothing more than an appeal to believe in the existence of such properties based on how you yourself feel when you look at something yellow. I don't find that very convincing. But I do agree with you in that the question you asked is worth asking. Not even mentioning the issue is certainly a frequent mistake made by authors taking a purely technological perspective.
mannykannot 11y ago
> But that doesn't prove it has any experience of the color yellow; it merely means it can mimic the output of an entity that does. I disagree: it would be mimicking the output of an entity that is mimicking.... My point is that these qualia arguments are begging the question, because they posit some 'real' entity that is somehow different from all the 'mimics', despite the fact that no-one is saying what the difference between them is.
eli_gottlieb 11y ago
The really interesting question is: why do you humans obsess over subjective consciousness rather than general intelligence, when the former is simpler and stupider, while carrying a burden of moral value, while the latter is more versatile and useful, while not carrying undesirable moral weight? Which one do you really want to build in a computer? The intelligence, plainly.

by garyrob 11y ago

Jabbles 11y ago

Algorithms that can write heartfelt posts like yours will be developed one day. This is literally the point of Turing's thought experiment: if a machine gives you answers indistinguishable from a human, is it alive?

paulojreis 11y ago
If a call center operator follows a "answering" algorithm strictly, is it a computer? These thought experiments are very interesting and thought provoking. But - still - it's a mathematician asking if something is alive/human. Not the most authoritative source on the subject matter.
- josh2600 11y ago
  Yes. The original computers were women in warehouses calculating actuary tables. They had a set of criteria to perform evaluations of numbers. In short, you can reduce a subset of the human experience to algorithmic emulation, which, in effect, reduces a human to a computer. It's sort of what it means to do any job to be frank.
  - paulojreis 11y ago
    I meant a electronic computer, but well put. > In short, you can reduce a subset of the human experience to algorithmic emulation, which, in effect, reduces a human to a computer. You're underestimating everything associated with being human. The computer won't make mistakes because a familiar is sick. He won't perform slower than usual because he slept less. He won't have a urge to poop affecting is performance, also. This is somewhat what I meant: these thought experiments are interesting, but if you look closer there's really not much to them. Human computers were, of course, computing. They didn't, however, became "not human".
    - Tiksi 11y ago
      > The computer won't make mistakes because a familiar is sick. Race conditions cause mistakes. Seems similar to having your mind elsewhere. > He won't perform slower than usual because he slept less. Didn't clean up memoery so now your computer is resource starved and running slower. > He won't have a urge to poop affecting is performance, GC pause. I feel like people often focus too much on the "implementation" when it comes to consciousness. Beyond what instinct gives us, which is really just a pre defined set of conditionals, everything we know and understand is learned, programmed in a way. I don't see why it's unfeasible that given enough time and the right environment that an AI could do the same.

jmagoon 11y ago

This is a well defined (but somewhat controversial) philosophical topic known as the hard problem of consciousness: https://en.wikipedia.org/wiki/Hard_problem_of_consciousness https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

avn2109 11y ago

In the literature, they use the term "qualia" for what you're calling "awareness" here.

garyrob 11y ago
Yes. I'm not extremely well-read in the literature but I have read some of it. I'm aware of the term "qualia" but was trying to use the most direct, simple possible language to communicate.

gnaritas 11y ago

> But that doesn't prove it has any experience of the color yellow; it merely means it can mimic the output of an entity that does. Which is all we might be doing with each other as well. That awareness could merely be an illusion and there's even some evidence to think it is. Experiments show your brain makes a decision before you are consciously aware of it despite you thinking you made the decision consciously. That leads to the conclusion that awareness is simply a lie your brains tells itself to feel in control.

habitue 11y ago

How do you know other humans can truly experience "yellowness"? What if we're all just pretending and you're the only one? Let's not talk about realistic AI for a moment, let's talk about hypothetically building a human protein by protein (or whatever building block you're comfortable with saying doesn't have qualia on its own). We're going to write the computer program to build this human. Nanobots will carry it out, every part of the process is understood and nothing is left for biology to do for us. We're just putting together lego blocks and at the end is a human indistinguishable from any other human. Does this person experience qualia? How would you know? Apparently you can't just ask, since this isn't a valid thing to do with a computer (it's not really experiencing qualia! It's just saying that!). Let's just say you come up with some objective qualia test. Now we can build a slightly more deficient human protein by protein and test it for qualia. In fact, we can do a qualia binary search and... Once you take apart the argument like this, it's easy to see that arguments that computers can't experience qualia are arguments against materialism or reductionism, or are patent attempts to sneak souls and other woo into the discussion.

stelonix 11y ago
I believe his point is, such a process wouldn't ever be feasible if we can't "create" awareness. You'd have a sack of meat that would not interact. Or if it would, it'd be a vegetable capable of computation but little else (that's my random assumption). If you forget your fear of "soul" and other terms not ratified by science journals, you'll find out there are plenty of theories in metaphysics and spirituality on what it means to be and what awareness is. It requires you to at the very least read those philosophical/theological concepts instead of dismissing it as "woo". Cattle dwellers in India wrote many things about it eons ago. Joseph Campbell's work is a nice place to start.
- habitue 11y ago
  > you'll find out there are plenty of theories in metaphysics and spirituality on what it means to be and what awareness is I have encountered these theories. In the past I have been under the sway of them. They can be interesting to think about, but without any evidence for them there isn't any reason to give them particular consideration. (I am including rational, mathematical, or logical arguments based on experimental evidence as evidence here) Without evidence, how do we pick which of these myriad theories to believe? How do we decide what the actual answer is? Science and materialism have repeatedly produced predictions that pan out in experiments. Theories of souls and sprituality have repeatedly had to retreat from testable predictions. To take another tack, how did these spiritual theories come about? If they're true, then the ancient theorist must have seen some phenomena in the real world, and extrapolated correctly from that evidence. If that's the case, these ancient theorists were unparalleled geniuses, but we don't need to worry about ancient spiritual theories themselves, we'll rediscover them in time through our plodding (but safe) scientific method. If we don't rediscover them, then they were interesting theories that didn't pan out.

agumonkey 11y ago

> to experience the color yellow, as opposed to classifying a visual stimulus as being of a certain wavelength I've been wondering about this for a few years, can't find anything about that because it's too generic a search term. Is there a name, or references about this ? ps: ha, these in-thread answers already give pointers https://news.ycombinator.com/item?id=9864468 https://news.ycombinator.com/item?id=9864468 https://news.ycombinator.com/item?id=9864479 https://news.ycombinator.com/item?id=9864479 <closed/>

elektromekatron 11y ago

experience the color yellow, as opposed to classifying a visual stimulus as being of a certain wavelength Purple is the weird one as it doesn't correspond to a wavelength, but can only be represented by a set. Edit - all colours correspond to sets of sets, if you think about it, as you can experience exactly the same colour for essentially an infinite set of different sets of frequencies.

EGreg 11y ago

I once met David Chalmers when he was teaching a philosophy course at NYU. And I asked him after the class ... What can you say about consciousness in the "hard probem of consciousness"? I am having trouble making any statements about it which are objective. I have begun to suspect that our concepts of consciousness and morality are the result of our language and the way we have defined them. With morality is has become clear to me that personal duties are relative to personal goals. That is, "X should do Y" is incoherent, it really implies "X should do Y if X wants Z to happen" which is what the colloquial understanding is. Always being careful to mention Z removes any feeling of mysticism from morality!! Perhaps with statements about consciousness, the situation is similar. I couldn't really formulate any sort of mystical question once I was carefully adding the SUBJECT of each sentence explicitly in. The main question I have about consciousness is not about the processing - that can be explained mechanistically. But about the final result of perceiving this orchestra - why am I in my brain and not anyone else's? What is this extra element that is required for the conscious experience and can we ever be sure a machine we built is really subjectively perceiving anything in that metaphysical sense? Thoughts?

JoeAltmaier 11y ago
Doesn't seem right - morality is often intuitive or blind. Else why did Socrates say "The unexamined life is not worth living?" Why, because most (almost all) people don't think about Z.
- zkhalique 11y ago
  Just because they don't think about or stress Z doesn't mean it isn't there. Sometimes Z is a subconscious brain heuristic developed as a response to competing interests, threats, social consequences etc.
  - JoeAltmaier 11y ago
    Folks learn their moral sense long before they can rationalize anything. Do what momma says, that's all the 'Z' that the brain knows.

rndn 11y ago

> It seems like a lot of people, including the author, don't even notice that this is a question. They seem to assume that computer-like information processing is all that happens in a human mind, and that awareness itself doesn't even exist as something to ponder. This seems to be a misjudgement. There are actually plenty of people who have studied traditional philosophy and dualistic approaches to consciousness for years and eventually arrived at a naturalistic conclusion after they've studied neuroscience. I would actually be very much interested in the numbers for the opposite case: How many subscribers to dualism have knowledge in neuroscience, human behavioral biology, the theory of evolution and physics?

fauigerzigerk 11y ago

My problem with that argument, which I have heard quite often, is that we cannot do anything useful with a term (awareness/experience of yellow in this case) that has neither extension nor intension. What you experience when you see yellow is what you experience when you see yellow. Nothing more can be said about it and it cannot be compared to anyone elses experience of yellow, human or machine. So if you make that experience a condition for calling another being intelligent, you are effectively saying that nothing but yourself can ever be intelligent. You need to break down that thing you call experience of yellow or awareness of yellow and explain what properties it has (intension) or you need to be able to show that it influences the behaviour of humans in a way in which machines cannot be influenced by it (extension).

garyrob 11y ago
"So if you make that experience a condition for calling another being intelligent, you are effectively saying that nothing but yourself can ever be intelligent." I think you are confusing the principles of whether an observer can determine whether an entity has an attribute with the question of whether it does in fact have that attribute. But those are entirely different issues because, for one thing, the observer may simply not have the tools yet to make such a judgement. Matter is made of atoms. For a long time we did not have the tools to determine that. But that didn't change the fact that it was, and is, true. Now suppose the human race died out before it ever developed such tools. Obviously, it would still be true. Now suppose that due to our limited intellectual abilities or for some other reason, it never was, or will be, possible to ever develop such tools. To me, it's obvious that that has no bearing on the truth of the proposition any more than if we didn't have time to develop the tools. Similarly, the question of whether you have awareness is independent of whether I or any other third party can prove you do.
- fauigerzigerk 11y ago
  I don't necessarily disagree with that, but your argument is nothing more than an appeal to believe in the existence of such properties based on how you yourself feel when you look at something yellow. I don't find that very convincing. But I do agree with you in that the question you asked is worth asking. Not even mentioning the issue is certainly a frequent mistake made by authors taking a purely technological perspective.

mannykannot 11y ago

> But that doesn't prove it has any experience of the color yellow; it merely means it can mimic the output of an entity that does. I disagree: it would be mimicking the output of an entity that is mimicking.... My point is that these qualia arguments are begging the question, because they posit some 'real' entity that is somehow different from all the 'mimics', despite the fact that no-one is saying what the difference between them is.

eli_gottlieb 11y ago

The really interesting question is: why do you humans obsess over subjective consciousness rather than general intelligence, when the former is simpler and stupider, while carrying a burden of moral value, while the latter is more versatile and useful, while not carrying undesirable moral weight? Which one do you really want to build in a computer? The intelligence, plainly.