7 ms·

There are actual living things we have difficulty proving are "conscious" and you get into really tricky territory trying to establish what might be "conscious"

There are actual living things we have difficulty proving are "conscious" and you get into really tricky territory trying to establish what might be "conscious" (or even "alive") in the world even without bringing AI into the mix. Even the people around us we can't prove to be conscious except in that we are also human and assume they have a similar first-person (subjective) view of the world and aren't just a biological robot running equations. Yes, you could literally be the only conscious being in the universe and the universe would indistinguishable from one with many consciousnesses.

igorkraw 5y ago
Edit:Shoot, I meant to reply to another comment. Leaving this here and linking it --- IIT is the most coherent definition of consciousness I'm aware of https://en.m.wikipedia.org/wiki/Integrated_information_theory https://en.m.wikipedia.org/wiki/Integrated_information_theor... People dismiss it often, and it has valid academic criticisms, but most "lay" critiques I've seen seem to dismiss it because it gives things they don't consider to be "conscious like them" consciousness. I think in general, we overexagerate qualities that humans have as uniquely special and important (intelligence, sentience, consciousness), even if the definition of them is fuzzy and any non-fuzzy definition is "too inclusive". I wonder if this is because we are collectively Identity building and creating an "other" out of nature, because otherwise we'd 1. Be much less special as we want to fee 2. Would have to consider a lot of inconvenient externalities for our reasoning (moral and otherwise) to be consistent And both are slowdowns when you are bootstrapping a post scarcity society out of scarcity, so it'd culturally valuable to reify special qualities that we just determine "we" possess and "they" don't - because it's easier to unify on and allows more actions than making sure everyone can deal with the unfiltered reality of human... unremarkableness in an uncaring world, another social species like so many other with a temporary oligarchy on the planet (without wanting to drone to philosophically, I do think this aspect of Weltschmerz is underappreciated, especially seeing the anxiety amongst my peer group when the topic comes up)
- mannykannot 5y ago
  You have acknowledged that there are valid academic criticisms of IIT, and for those wanting to get an idea of what those might be, a good place to start is with Scott Aaronson's responses in his blog: https://scottaaronson.blog/?p=1799 https://scottaaronson.blog/?p=1799 Note that the issue that you are concerned with here and the usefulness of IIT are separate concerns, and critics like Aaronson are not taking that position on the basis of the attitudes you claim are behind many 'lay' critiques.
  - igorkraw 5y ago
    I appreciate Aaronsons critique, he himself makes my main point of disagreements, i.e. that when constructing a physical systems that has the properties he describes (modified Vandermonde Matrix) it might give nontrivial surprises. But even accepting that IIT is wrong in the "consciousness might imply high IIT score, but not vice versa" sense, I'm simply not aware of any definition which is as *attackble* because of it's formal specification. This is not me going "well you don't have anything better", but me asking to be pointed to similarly strictly defined notions of consciousness
    - cmehdy 5y ago
      I found the conversation between you two interesting (as well as Scott Aaronson's post), so I figured I could contribute with a potentially relevant piece of reading: "Hard Criteria for Empirical Theories of Consciousness" (24-page PDF) at https://digitalcommons.chapman.edu/cgi/viewcontent.cgi?article=1217&context=psychology_articles https://digitalcommons.chapman.edu/cgi/viewcontent.cgi?artic... In the same ballpark as IIT, I've seen many mentions of GWT/GNT (global neuronal workspace theory). But the PDF covers that and many others. I don't have any particular "conclusion" myself, but it's just a fascinating discussion to be swimming in even when utterly clueless. EDIT: And I forgot the most obvious link that I actually had read in the past, Stanford's page on conciousness talks about theories: https://plato.stanford.edu/entries/consciousness/#SpeTheCon https://plato.stanford.edu/entries/consciousness/#SpeTheCon
    - mannykannot 5y ago
      I am not aware of anything similarly strictly defined, but that is not, of course, evidence for the correctness or even usefulness of IIT, and nor does it somehow counter the specific objections that Aaronson raises.
- visarga 5y ago
  IIT is a nice intuition pump, but it's missing essential parts. Consciousness should be related to survival, replication of information and its own energy costs. It's not just some integrated information for no purpose at all. The purpose is survival if it is conscious. If you add that to IIT, then you suddenly get a new filter for what could be conscious - something that competes for information replication, not just something with high IIT score.
  - unfocussed_mike 5y ago
    It sounds like you've maybe read Steve Grand's book, Creation: Life and How To make It? That is his entire pitch; if you've not read it I wholly recommend it.
  - sgillen 5y ago
    >> Consciousness should be related to survival Is that not just your intuition too? Is it not possible to imagine an artificial consciousness that does not care one way or the other for its own survival?
    - visarga 5y ago
      No because survival is closely tied to evolution. By mere survival an agent creates its own structure and values. It can't evolve if it doesn't risk its skin in the game. The price of our present day intelligence has been very high. But you can piggyback AI training on human culture and embody the agent in our environment so it can still function like a conscious biological agent. The AI is just passing the buck to humans, without us it wouldn't even have a goal. On the other hand all living things have goals without human help. Maybe when AIs can self replicate their information into the future like us, without external help, they could be eventually conscious. We don't know yet how to make physical self replicators that evolve. Only nature solved this riddle so far. BTW, AlphaGo was subjected to evolutionary techniques, and it shows. The agents had to duel with clones and past versions of themselves. Survival was conditioned on win rate. Suddenly AI can surpass humans in this little environment, but that came at the price of discarding so many unfit earlier versions.
- naniwaduni 5y ago
  > People dismiss it often, and it has valid academic criticisms, but most "lay" critiques I've seen seem to dismiss it because it gives things they don't consider to be "conscious like them" consciousness. If your attempt at a formalism provides useful, actionable implications, and only really disagrees with people's intuitions on cases that are generally believed to be edge cases, or maybe that you also have good reason to believe are edge cases on different ground, then you might have a decent definition. If your attempt at a formalism provides useful implications, but disagrees with everyone's intuitions on what the words you're using mean, you've probably defined the wrong thing. That's not to say that you aren't defining something sensible (you might still be), but strongly suggests it's not what you want it to be. Selling IIT as a description of consciousness is roughly the same flavor of hubris as defining berries to include oranges and cucumbers and exclude raspberries and raspberries.
  - atoav 5y ago
    > Selling IIT as a description of consciousness is roughly the same flavor of hubris as defining berries to include oranges and cucumbers and exclude raspberries and raspberries. Not trying to defend IIT, but the only reason your example works here is because from our human perspective (eating these plants) we find it absurd to include cucumbers and exclude raspberries. So talking about hubris: Wouldn't plants (if they could) very likely have a totally different perspective? To then it would probably seem totally absurd that we use taste or shape of the fruit as a grouping feature and not something that might be totally more defining that plants can experience. Of course it is intuitively understandable to look at ourselves and animals we understand when defining consciousness. Even like that it is hard to define. Trying to incorporate things that show conscious-like behaviors, but in totally human-unrelatable ways is not a bad next step, but it is hard to know where to draw the border. A rock could have conscious processes running in it on geological timescales — how would one falsify that?
- lmm 5y ago
  IIT doesn't capture much or any of what we consider important about consciousness. If it's the most coherent definition we have, we don't have a coherent definition.
- tgv 5y ago
  > it has valid academic criticisms That's probably because the rest of researchers ignore it like the plague. If everybody had to comment on it for some magical reason, it would be probably be burned down to the ground and then some.
- imtringued 5y ago
  If other animals are just as special as us we can't boil them alive. Just imagine the inconvenience of delivering a less painful death to lobsters.
freemint 5y ago
If something only can "think" if it feed data it can not reflect about itself out of it's own volition.
- OJFord 5y ago
  (in an ironic demonstration of what I mean) I'm not sure I understand what you're saying - but is it that you can only call a process 'thinking' if it's producing some output that's not derivable from its input, nor hard-coded in the definition of the function, as it were? Perhaps that's way off, I was just starting to think along those lines as I came to your comment, and it seemed it might fit, that we might be thinking along the same lines.
  - freemint 5y ago
    Just as dried up Tardigrades are not alive (reintroduced into an environment in which they can survive) Artificial Intelligence which runs decoupled from any notion of time and whos weights are frozen or changed without it being able to influence or understand the outcomes will not be able to derive a theory of mind about itself. Even if it did since it does contributes little to it's own objective regularization will remove these features. As ANN are trained today most architectures are dried up Tardigrades which get watered, dried&reset into the previous configuration during inference.
- joe_the_user 5y ago
  This especially, I'd be very skeptical of claims that, say, some complex ongoing control system, like a self-driving car, could be "conscious". But there's an argument someone could make. But an artifact that has no data storage seems to fail any reasonable definition immediately. Maybe it could be part of something else that you can claim is conscious if you add storage, output controls, aims or whatever. But by itself the claim just seems preposterous.
- visarga 5y ago
  That's the problem. The neural net can't reflect on itself because it doesn't have a body and an environment to act in. It should be able to experience consequences of previous actions to close the loop on itself.
visarga 5y ago
> Yes, you could literally be the only conscious being in the universe and the universe would indistinguishable from one with many consciousnesses. No, you think that's possible but it's naive. Your existence depends on environment and self replication of information. Your structure depends on it, your cognition. Are you saying you can be separate from the tree, and everything just a fake around you?
- chasing 5y ago
  I’m saying there’s nothing you can measure to say anything — even another human — has the same sort of subjective experience (consciousness, a soul, whatever) that you’re experiencing right now, as you read this comment from inside your own head. You can make a guess, but it’s impossible to truly know. (This is a bit different from solipsism, if that’s what’s tripping you up. Although that, too, is a bit of a problem…) So any attempt to “prove” an AI is conscious is either going to fall into the hole of “well, how do you prove anything is conscious?” or is going to need a heavily distorted definition of “conscious” (like “seems kinda human” or “makes uncanny decisions we can’t fully explain”) to be true.
  - mensetmanusman 5y ago
    This might assume that only things that are measurable could be true.
  - galaxyLogic 5y ago
    I think we can say that AI is conscious if we can observe its stack-trace showing it creating symbolic representations of its own processing and then processing those and using the same language for that representation as it uses for communications with humans. In other words if we can observe a machine thinking about its own (symbolic) thinking, then I think it's conscious. Consciousness is the ability to observe one's own mental processes and make sense of them symbolically. That is one possible definition of it. You are right that we can't really prove that humans are conscious, because we cannot observe their thinking. But, to prove that a machine is conscious all we need is to agree on a definition of what it means to be conscious, because machines can be easily inspected. And then find a machine that demonstrates features that fit the definition. If we can't agree on what it means for something to be conscious then the whole question of whether some AI is or could be conscious is of course meaningless.
sizzle 5y ago
Also worth mentioning is the Mind-body problem i.e. Cartesian dualism and related concepts https://en.m.wikipedia.org/wiki/Mind–body_dualism https://en.m.wikipedia.org/wiki/Mind–body_dualism Also interesting thought experiment is the philosophical zombie, lots to read by David Chalmers, etc.

igorkraw 5y ago

Edit:Shoot, I meant to reply to another comment. Leaving this here and linking it --- IIT is the most coherent definition of consciousness I'm aware of https://en.m.wikipedia.org/wiki/Integrated_information_theory https://en.m.wikipedia.org/wiki/Integrated_information_theor... People dismiss it often, and it has valid academic criticisms, but most "lay" critiques I've seen seem to dismiss it because it gives things they don't consider to be "conscious like them" consciousness. I think in general, we overexagerate qualities that humans have as uniquely special and important (intelligence, sentience, consciousness), even if the definition of them is fuzzy and any non-fuzzy definition is "too inclusive". I wonder if this is because we are collectively Identity building and creating an "other" out of nature, because otherwise we'd 1. Be much less special as we want to fee 2. Would have to consider a lot of inconvenient externalities for our reasoning (moral and otherwise) to be consistent And both are slowdowns when you are bootstrapping a post scarcity society out of scarcity, so it'd culturally valuable to reify special qualities that we just determine "we" possess and "they" don't - because it's easier to unify on and allows more actions than making sure everyone can deal with the unfiltered reality of human... unremarkableness in an uncaring world, another social species like so many other with a temporary oligarchy on the planet (without wanting to drone to philosophically, I do think this aspect of Weltschmerz is underappreciated, especially seeing the anxiety amongst my peer group when the topic comes up)

mannykannot 5y ago
You have acknowledged that there are valid academic criticisms of IIT, and for those wanting to get an idea of what those might be, a good place to start is with Scott Aaronson's responses in his blog: https://scottaaronson.blog/?p=1799 https://scottaaronson.blog/?p=1799 Note that the issue that you are concerned with here and the usefulness of IIT are separate concerns, and critics like Aaronson are not taking that position on the basis of the attitudes you claim are behind many 'lay' critiques.
- igorkraw 5y ago
  I appreciate Aaronsons critique, he himself makes my main point of disagreements, i.e. that when constructing a physical systems that has the properties he describes (modified Vandermonde Matrix) it might give nontrivial surprises. But even accepting that IIT is wrong in the "consciousness might imply high IIT score, but not vice versa" sense, I'm simply not aware of any definition which is as *attackble* because of it's formal specification. This is not me going "well you don't have anything better", but me asking to be pointed to similarly strictly defined notions of consciousness
  - cmehdy 5y ago
    I found the conversation between you two interesting (as well as Scott Aaronson's post), so I figured I could contribute with a potentially relevant piece of reading: "Hard Criteria for Empirical Theories of Consciousness" (24-page PDF) at https://digitalcommons.chapman.edu/cgi/viewcontent.cgi?article=1217&context=psychology_articles https://digitalcommons.chapman.edu/cgi/viewcontent.cgi?artic... In the same ballpark as IIT, I've seen many mentions of GWT/GNT (global neuronal workspace theory). But the PDF covers that and many others. I don't have any particular "conclusion" myself, but it's just a fascinating discussion to be swimming in even when utterly clueless. EDIT: And I forgot the most obvious link that I actually had read in the past, Stanford's page on conciousness talks about theories: https://plato.stanford.edu/entries/consciousness/#SpeTheCon https://plato.stanford.edu/entries/consciousness/#SpeTheCon
  - mannykannot 5y ago
    I am not aware of anything similarly strictly defined, but that is not, of course, evidence for the correctness or even usefulness of IIT, and nor does it somehow counter the specific objections that Aaronson raises.
visarga 5y ago
IIT is a nice intuition pump, but it's missing essential parts. Consciousness should be related to survival, replication of information and its own energy costs. It's not just some integrated information for no purpose at all. The purpose is survival if it is conscious. If you add that to IIT, then you suddenly get a new filter for what could be conscious - something that competes for information replication, not just something with high IIT score.
- unfocussed_mike 5y ago
  It sounds like you've maybe read Steve Grand's book, Creation: Life and How To make It? That is his entire pitch; if you've not read it I wholly recommend it.
- sgillen 5y ago
  >> Consciousness should be related to survival Is that not just your intuition too? Is it not possible to imagine an artificial consciousness that does not care one way or the other for its own survival?
  - visarga 5y ago
    No because survival is closely tied to evolution. By mere survival an agent creates its own structure and values. It can't evolve if it doesn't risk its skin in the game. The price of our present day intelligence has been very high. But you can piggyback AI training on human culture and embody the agent in our environment so it can still function like a conscious biological agent. The AI is just passing the buck to humans, without us it wouldn't even have a goal. On the other hand all living things have goals without human help. Maybe when AIs can self replicate their information into the future like us, without external help, they could be eventually conscious. We don't know yet how to make physical self replicators that evolve. Only nature solved this riddle so far. BTW, AlphaGo was subjected to evolutionary techniques, and it shows. The agents had to duel with clones and past versions of themselves. Survival was conditioned on win rate. Suddenly AI can surpass humans in this little environment, but that came at the price of discarding so many unfit earlier versions.
naniwaduni 5y ago
> People dismiss it often, and it has valid academic criticisms, but most "lay" critiques I've seen seem to dismiss it because it gives things they don't consider to be "conscious like them" consciousness. If your attempt at a formalism provides useful, actionable implications, and only really disagrees with people's intuitions on cases that are generally believed to be edge cases, or maybe that you also have good reason to believe are edge cases on different ground, then you might have a decent definition. If your attempt at a formalism provides useful implications, but disagrees with everyone's intuitions on what the words you're using mean, you've probably defined the wrong thing. That's not to say that you aren't defining something sensible (you might still be), but strongly suggests it's not what you want it to be. Selling IIT as a description of consciousness is roughly the same flavor of hubris as defining berries to include oranges and cucumbers and exclude raspberries and raspberries.
- atoav 5y ago
  > Selling IIT as a description of consciousness is roughly the same flavor of hubris as defining berries to include oranges and cucumbers and exclude raspberries and raspberries. Not trying to defend IIT, but the only reason your example works here is because from our human perspective (eating these plants) we find it absurd to include cucumbers and exclude raspberries. So talking about hubris: Wouldn't plants (if they could) very likely have a totally different perspective? To then it would probably seem totally absurd that we use taste or shape of the fruit as a grouping feature and not something that might be totally more defining that plants can experience. Of course it is intuitively understandable to look at ourselves and animals we understand when defining consciousness. Even like that it is hard to define. Trying to incorporate things that show conscious-like behaviors, but in totally human-unrelatable ways is not a bad next step, but it is hard to know where to draw the border. A rock could have conscious processes running in it on geological timescales — how would one falsify that?
lmm 5y ago
IIT doesn't capture much or any of what we consider important about consciousness. If it's the most coherent definition we have, we don't have a coherent definition.
tgv 5y ago
> it has valid academic criticisms That's probably because the rest of researchers ignore it like the plague. If everybody had to comment on it for some magical reason, it would be probably be burned down to the ground and then some.
imtringued 5y ago
If other animals are just as special as us we can't boil them alive. Just imagine the inconvenience of delivering a less painful death to lobsters.

freemint 5y ago

If something only can "think" if it feed data it can not reflect about itself out of it's own volition.

OJFord 5y ago
(in an ironic demonstration of what I mean) I'm not sure I understand what you're saying - but is it that you can only call a process 'thinking' if it's producing some output that's not derivable from its input, nor hard-coded in the definition of the function, as it were? Perhaps that's way off, I was just starting to think along those lines as I came to your comment, and it seemed it might fit, that we might be thinking along the same lines.
- freemint 5y ago
  Just as dried up Tardigrades are not alive (reintroduced into an environment in which they can survive) Artificial Intelligence which runs decoupled from any notion of time and whos weights are frozen or changed without it being able to influence or understand the outcomes will not be able to derive a theory of mind about itself. Even if it did since it does contributes little to it's own objective regularization will remove these features. As ANN are trained today most architectures are dried up Tardigrades which get watered, dried&reset into the previous configuration during inference.
joe_the_user 5y ago
This especially, I'd be very skeptical of claims that, say, some complex ongoing control system, like a self-driving car, could be "conscious". But there's an argument someone could make. But an artifact that has no data storage seems to fail any reasonable definition immediately. Maybe it could be part of something else that you can claim is conscious if you add storage, output controls, aims or whatever. But by itself the claim just seems preposterous.
visarga 5y ago
That's the problem. The neural net can't reflect on itself because it doesn't have a body and an environment to act in. It should be able to experience consequences of previous actions to close the loop on itself.

visarga 5y ago

> Yes, you could literally be the only conscious being in the universe and the universe would indistinguishable from one with many consciousnesses. No, you think that's possible but it's naive. Your existence depends on environment and self replication of information. Your structure depends on it, your cognition. Are you saying you can be separate from the tree, and everything just a fake around you?

chasing 5y ago
I’m saying there’s nothing you can measure to say anything — even another human — has the same sort of subjective experience (consciousness, a soul, whatever) that you’re experiencing right now, as you read this comment from inside your own head. You can make a guess, but it’s impossible to truly know. (This is a bit different from solipsism, if that’s what’s tripping you up. Although that, too, is a bit of a problem…) So any attempt to “prove” an AI is conscious is either going to fall into the hole of “well, how do you prove anything is conscious?” or is going to need a heavily distorted definition of “conscious” (like “seems kinda human” or “makes uncanny decisions we can’t fully explain”) to be true.
- mensetmanusman 5y ago
  This might assume that only things that are measurable could be true.
- galaxyLogic 5y ago
  I think we can say that AI is conscious if we can observe its stack-trace showing it creating symbolic representations of its own processing and then processing those and using the same language for that representation as it uses for communications with humans. In other words if we can observe a machine thinking about its own (symbolic) thinking, then I think it's conscious. Consciousness is the ability to observe one's own mental processes and make sense of them symbolically. That is one possible definition of it. You are right that we can't really prove that humans are conscious, because we cannot observe their thinking. But, to prove that a machine is conscious all we need is to agree on a definition of what it means to be conscious, because machines can be easily inspected. And then find a machine that demonstrates features that fit the definition. If we can't agree on what it means for something to be conscious then the whole question of whether some AI is or could be conscious is of course meaningless.

sizzle 5y ago

Also worth mentioning is the Mind-body problem i.e. Cartesian dualism and related concepts https://en.m.wikipedia.org/wiki/Mind–body_dualism https://en.m.wikipedia.org/wiki/Mind–body_dualism Also interesting thought experiment is the philosophical zombie, lots to read by David Chalmers, etc.