4 ms·

> Generating text for hours without an anchor to the real world is not a productive method of generating insight about that world. Yet people see to be claimin

> Generating text for hours without an anchor to the real world is not a productive method of generating insight about that world. Yet people see to be claiming that LaMDA does just this, and is therefore not conscious. Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it. I’m yet to see a convincing argument about why LaMDA isn’t conscious other than “it’s just generative”. To demonstrate that this means it’s not consciousness requires us to prove that our own consciousness isn’t “just generative”, but I’m yet to see anyone show that, and I’m sceptical that it can be done.

melenaboija 4y ago
I guess that a unigram language model trained with few words should be considered conscious in some way too.
- doctor_eval 4y ago
  But this is kinda my point. You're not explaining to me why LaMDA is not conscious, you're just asserting that consciousness is not just a form of complex pattern recognition. Perhaps consciousness is simply pattern recognition at scale. If not - why not?
  - jb_s 4y ago
    Perhaps the main problem in the collective discussion is regarding consciousness as discrete and not a spectrum from minimal (slime molds?) to ultimate consciousness (ability to simulate/understand/create the entire universe?)
XorNot 4y ago
LaMDA didn't perform as well as the released document implies (it's edited, and didn't show alternatives like if you prompted it with "I think you're not conscious. Prove that for me." or "how can I prove to others you're a squirrel") The big problem is very much though that as you note, the arguments people have been making about this are atrocious. Basically, it's not conscious, but essentially because it's unlikely to be as sophisticated as it looks in what was put out there.
- doctor_eval 4y ago
  Thanks. This is exactly the kind of thing I’m talking about. I don’t have a strong opinion about LaMDA’s consciousness but I sure wish we could see the unedited text. As I said elsewhere, we can’t say that LaMDA is not conscious until we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that yet.
kleene_op 4y ago
> Yet people see to be claiming that LaMDA does just this, and is therefore not conscious. Plot twist: that article was written by LaMDA.
tarsinge 4y ago
> Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it. That the output is the same proves nothing. The journalist has a subjective experience of themselves doing it, they "see" their thoughts. Like an AI can paint with red by using values without the experience of seeing the red.
- doctor_eval 4y ago
  How do you know that the journalist has a subjective experience? How do you know your own subjective experience isn’t just your “bicameral mind” talking to itself? How do you know LaMDA doesn’t have a subjective experience? It’s just assumptions; nobody really knows what’s going on inside that neural network. The only way we can tell that someone else is conscious is because they tell us. So just saying “the output proves nothing” is incredibly weak, because the output is all we’ve got.
  - tarsinge 4y ago
    I attribute a subjective experience to the journalist on the basis of similarity with me. No need for "the output", I would do that with someone who can't talk or write too. From where my own subjective experience come from and how can I prove it to you is irrelevant to the fact that I have it so I'm fine with being the bootstrap of the circular reasoning to give it to the rest of humanity. With that said I maintain that the output proves nothing, it's not because it's all we have that it's not useless. Also it's the only thing we have only with the hypothesis that consciousness is a side-effect of computation (the point of the article), but it could anything, like a property of matter, or electromagnetic fields, etc...
bnralt 4y ago
The issue is how humans confuse "consciousness" with the ability to mimic what human output that modern people associate with consciousness. I'd argue that what we refer to as "consciousness" is the ability to form certain kind of mental abstractions, particularly those involving ourselves. Take away language from a person (imagine someone who grew up in the wild, or someone like Helen Keller who didn't have language available until she was older), and these abstractions still exist. Language might be a way we express these abstractions, but they aren't the abstractions themselves. LaMDA doesn't have these abstractions underneath; once you take away language, it's nothing. To think of it another way - I can write a simple program for a cheap robot to navigate around a simple race track (with a simple enough path, I can even create an analog one out of mousetraps - see mousetrap cars). Companies can also create a very complex self driving car that can navigate anywhere on its own. These two things might look like they act the same if they're both places on the path that the robot has been trained for. In fact, a hard programmed robot might act _better_ than an AI car on specific paths. But only one of them is a "self-driving car," since only one of them will be able to go anywhere when it's taken off that path.
- doctor_eval 4y ago
  I don’t find this to be a satisfying way to think about it. The problem is that it’s quite possible that the abstractions you’re talking about are all part of the genetic ROM that exists so we can boot up our clones more quickly. If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate. The underlying problem here is that we don’t know how consciousness emerges. We can’t say that LaMDA is not conscious unless we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that.
  - bnralt 4y ago
    > If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate. If this were true - that it had a general human level abstraction, and not only the ability to mimic human speech - we would be able to attach LaMDA to some other outlet and see it do things we consider conscious. It would be able to navigate environments pretty accurately, for example, since that's something even animals that we consider much less conscious are able to do. If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.
psychoslave 4y ago
> Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it. Can you convincingly threaten a journalist that you are going to erase its existence from earth? How is this person going to react? Would LaMDA do the same, or even simulate reactions indiscernible from those made by the journalist?
simonh 4y ago
LaMDA can emulate certain behaviour that we associate with consciousness based on our everyday experience, but it also fails to exhibit many other attributes we associate with consciousness. So, is conciseness a requirement for being able to generate what I'd claim is a very, very limited subset of behaviours we associate with conciseness? I say no. These language model systems operate very well if you approach them in a non-adversarial way and feed them input similar to their training inputs. As soon as you adopt a more adversarial approach and interrogate them more thoroughly, it all falls apart quickly and spectacularly. It's actually quite easy to explore conversations around the edges of, or beyond the coverage of their training data and get them to babble helplessly. They're also incapable of performing many very trivial cognitive processes. So I can't prove it, any more than I can prove that I'm conscious, but they don't come close to convincing me that they are.

melenaboija 4y ago

I guess that a unigram language model trained with few words should be considered conscious in some way too.

doctor_eval 4y ago
But this is kinda my point. You're not explaining to me why LaMDA is not conscious, you're just asserting that consciousness is not just a form of complex pattern recognition. Perhaps consciousness is simply pattern recognition at scale. If not - why not?
- jb_s 4y ago
  Perhaps the main problem in the collective discussion is regarding consciousness as discrete and not a spectrum from minimal (slime molds?) to ultimate consciousness (ability to simulate/understand/create the entire universe?)

XorNot 4y ago

LaMDA didn't perform as well as the released document implies (it's edited, and didn't show alternatives like if you prompted it with "I think you're not conscious. Prove that for me." or "how can I prove to others you're a squirrel") The big problem is very much though that as you note, the arguments people have been making about this are atrocious. Basically, it's not conscious, but essentially because it's unlikely to be as sophisticated as it looks in what was put out there.

doctor_eval 4y ago
Thanks. This is exactly the kind of thing I’m talking about. I don’t have a strong opinion about LaMDA’s consciousness but I sure wish we could see the unedited text. As I said elsewhere, we can’t say that LaMDA is not conscious until we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that yet.

kleene_op 4y ago

> Yet people see to be claiming that LaMDA does just this, and is therefore not conscious. Plot twist: that article was written by LaMDA.

tarsinge 4y ago

> Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it. That the output is the same proves nothing. The journalist has a subjective experience of themselves doing it, they "see" their thoughts. Like an AI can paint with red by using values without the experience of seeing the red.

doctor_eval 4y ago
How do you know that the journalist has a subjective experience? How do you know your own subjective experience isn’t just your “bicameral mind” talking to itself? How do you know LaMDA doesn’t have a subjective experience? It’s just assumptions; nobody really knows what’s going on inside that neural network. The only way we can tell that someone else is conscious is because they tell us. So just saying “the output proves nothing” is incredibly weak, because the output is all we’ve got.
- tarsinge 4y ago
  I attribute a subjective experience to the journalist on the basis of similarity with me. No need for "the output", I would do that with someone who can't talk or write too. From where my own subjective experience come from and how can I prove it to you is irrelevant to the fact that I have it so I'm fine with being the bootstrap of the circular reasoning to give it to the rest of humanity. With that said I maintain that the output proves nothing, it's not because it's all we have that it's not useless. Also it's the only thing we have only with the hypothesis that consciousness is a side-effect of computation (the point of the article), but it could anything, like a property of matter, or electromagnetic fields, etc...

bnralt 4y ago

The issue is how humans confuse "consciousness" with the ability to mimic what human output that modern people associate with consciousness. I'd argue that what we refer to as "consciousness" is the ability to form certain kind of mental abstractions, particularly those involving ourselves. Take away language from a person (imagine someone who grew up in the wild, or someone like Helen Keller who didn't have language available until she was older), and these abstractions still exist. Language might be a way we express these abstractions, but they aren't the abstractions themselves. LaMDA doesn't have these abstractions underneath; once you take away language, it's nothing. To think of it another way - I can write a simple program for a cheap robot to navigate around a simple race track (with a simple enough path, I can even create an analog one out of mousetraps - see mousetrap cars). Companies can also create a very complex self driving car that can navigate anywhere on its own. These two things might look like they act the same if they're both places on the path that the robot has been trained for. In fact, a hard programmed robot might act _better_ than an AI car on specific paths. But only one of them is a "self-driving car," since only one of them will be able to go anywhere when it's taken off that path.

doctor_eval 4y ago
I don’t find this to be a satisfying way to think about it. The problem is that it’s quite possible that the abstractions you’re talking about are all part of the genetic ROM that exists so we can boot up our clones more quickly. If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate. The underlying problem here is that we don’t know how consciousness emerges. We can’t say that LaMDA is not conscious unless we’ve proven that LaMDA’s construction is incompatible with consciousness, and we have not done that.
- bnralt 4y ago
  > If this is true then there is no reason that these abstractions couldn’t be learned, in which case you could take away the language, and perhaps the machine would continue to have thoughts; it would just be unable to communicate them. Of course, in this case you would probably conclude that it’s not conscious because it can’t communicate. If this were true - that it had a general human level abstraction, and not only the ability to mimic human speech - we would be able to attach LaMDA to some other outlet and see it do things we consider conscious. It would be able to navigate environments pretty accurately, for example, since that's something even animals that we consider much less conscious are able to do. If it's only optimized in one specific domain - human speech mimicry - and isn't able to generalize to do other tasks - even tasks that can be done by much simpler animal minds - then it's a pretty good indication that there isn't conscious abstraction.

psychoslave 4y ago

> Seems like a journalist is conscious if they do it, but it can’t possible be consciousness if LaMDA does it. Can you convincingly threaten a journalist that you are going to erase its existence from earth? How is this person going to react? Would LaMDA do the same, or even simulate reactions indiscernible from those made by the journalist?

simonh 4y ago

LaMDA can emulate certain behaviour that we associate with consciousness based on our everyday experience, but it also fails to exhibit many other attributes we associate with consciousness. So, is conciseness a requirement for being able to generate what I'd claim is a very, very limited subset of behaviours we associate with conciseness? I say no. These language model systems operate very well if you approach them in a non-adversarial way and feed them input similar to their training inputs. As soon as you adopt a more adversarial approach and interrogate them more thoroughly, it all falls apart quickly and spectacularly. It's actually quite easy to explore conversations around the edges of, or beyond the coverage of their training data and get them to babble helplessly. They're also incapable of performing many very trivial cognitive processes. So I can't prove it, any more than I can prove that I'm conscious, but they don't come close to convincing me that they are.