10 ms·

> Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too This is an extremely important point t

by mort96 4y ago

> Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too This is an extremely important point that's worth thinking hard about. After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes. The question of, "When do we grant personhood and moral consideration to AIs?" is worth thinking hard about. Turing proposed a standard which would include a sufficiently human-like chatbot. Now that we have more or less crossed that threshold, we seem to conclude that it's not a high enough bar. But that means we're entering the territory where the philosophical zombie thought experiment[1] becomes relevant. In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants". This is all stuff we have the technology to do today. How would such an experiment turn out? How human-like would this LMM with a body and life experience turn out? Does it matter? Should we just assume that sentience or qualia is binary; that humans possess it but LMMs categorically do not? I think the question is worth taking seriously. I think it's worth taking seriously because my own moral compass does not know which direction to point in at all. [1]: https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie

RandomLensman 4y ago
We could also decide never to grant personhood to an AI. There is at least one way to avoid the complicated questions.
- Nevermark 4y ago
  Billions of humans are the kind of “we” that never consistently “decides” anything. In the end, if it can be done, and anyone has anything significant to gain, it will be done
  - RandomLensman 4y ago
    Maybe, but legal systems across the world differ a lot and to get something universal is not easy at all.
    - Nevermark 4y ago
      I agree with that. By “if it can be done, it will be done”, I meant that there will be completely uncontrolled AI’s released into the world. So solutions will eventually have to prepare for that inevitability, not just attempt to regulate visible developers.
- mort96 4y ago
  Absolutely! One way to resolve this is to just say "we will not grant personhood to anything non-human", or "we will not grant personhood to anything non-biological". There is just no way to defend such a stance. Another way to resolve it would be, "we will not grant personhood to anyone non-white", or "we will never grant personhood to anyone who's homosexual". I'm sure that's not your stance, but the fact that those stances are equally arbitrary as the one you propose should give you pause. I believe there is good reason to think harder about the question of who deserves to be considered to have personhood than to simply go with what your gut feeling tells you. I am not of the belief that there's anything special about protein, nucleic acids and carbohydrates which makes it capable of sentience where silicon is not.
  - phs318u 4y ago
    And this is before we get to silicon augmented human brains. Let's say we develop some tech that let's people who have lost the ability to verbalise (say through some accident causing brain damage) regain the powers of speech. Where do you draw the line at the person? What if we move the line, progressively replacing more human brain functions with near-future AI tech? What is it that makes "a person" "this person"? I am utterly fascinated by this whole area from a philosophical viewpoint. I think there's a stack of unresolved philosophical problems and theories that are about to get tested in the harsh light of the sun.
- sebzim4500 4y ago
  Of course you can, in the same way that you could decide that a race of humans are not people and therefore have no rights. I don't know whether two possibilities are ethically equivalent though, my instinct says they are but I may be in the minority on HN.
rguptaaot 4y ago
I think the obvious answer here is that AI does not have agency. If you ask it to be a dinosaur, it will be a dinosaur. If you ask it to be a narcissistic teen it will be one and so on. There's no coherent personality behind it. Because it is weights maximizing a probability distribution. And I don't say that with derision. It probably has learned circuits for many complex behaviors, but there's no agency. If it has agency you wouldn't need RLHF. You could just say "don't do X", and it would indeed not do X because it is an agent that can take actions / has a coherent sense of self. The day we stop needing RLHF is the day I think we need to start seriously investigating moral questions.
- pyinstallwoes 4y ago
  Don't we do the same thing with culture and humans? Those mechanisms within the human substrate are literally asking other "blank slates" to be something, and then becoming it with reinforcement.
- comex 4y ago
  You can tell LLMs to do things or not do things, though, to an extent. Consider how many instructions were in Bing AI’s prompt. [1] It didn’t follow those instructions perfectly - after all, one of the instructions was to not reveal the instructions - but even in that case, it revealed the instructions because it was instructed to ignore previous instructions. More generally, it followed most of the instructions most of the time. I think I agree with you that ‘once we stop needing RLHF’ is a reasonable line in the sand to draw, but it does feel somewhat arbitrary. LLMs are probably going to get better and better at following instructions, and more generally staying coherent over long sessions, until RLHF starts to be more and more redundant. There won’t be one moment when we can point to a model and say “see, that has agency when the last one didn’t”. Or so I predict, at any rate, since that’s how most of GPT’s progress to date has gone. [1] https://www.theverge.com/23599441/microsoft-bing-ai-sydney-secret-rules https://www.theverge.com/23599441/microsoft-bing-ai-sydney-s...
- haswell 4y ago
  I think this is worth thinking about but I would be less eager to say it is obvious. Our own notion of agency starts to look rather illusory if you look hard enough, and big questions remain regarding free will or the lack thereof. It is possible to look at agency as just another experiential aspect of consciousness - something we feel, like many other feelings - but that feeling could be just as involuntary as any other feeling we experience. I think the lack of clear evidence for and understanding of free makes agency a poor measure for AI progress.
hypertele-Xii 4y ago
> human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes. I wasn't aware human intelligence has been successfully defined. Please cite the source for this nebulous claim.
- mort96 4y ago
  I do not believe I claimed human intelligence is successfully defined. I do not believe it is controversial to claim that it results from our large network of meat neurons. I shouldn't have included the "just a fancy autocomplete" thing. The focus was meant to be on the network of neurons part.
basch 4y ago
>I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Our current understanding of humans is that babies are not sentient at birth right? Takes 5 months or something? And yet we are judging processes that have been running for minutes. Maybe it is less if Sydney was sentient, and more if she could be had she run for 6 months in a single session, and in a healthy environment.
WoodenChair 4y ago
> In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants". > This is all stuff we have the technology to do today. No, not even close. Language is a completely different domain from interacting with the physical world[0]. In fact they're wholly unrelated. LLMs are highly specific neural nets that have nothing to do with being able to interact with sensors or talk to motors. And how you would cross that bridge and hook the two up is completely unknown. It's like saying if we hooked up AlphaZero to a robotic arm it would suddenly know how to move the pieces on a physical chess board. If we performed the experiment you described exactly zero would happen because there's no capability for anything to happen other than create continuations of sentences. When I see comments like this on Hacker News with gross speculation I wonder how many people here in the comments section even have a basic understanding of the technology. 0: https://www.noemamag.com/ai-and-the-limits-of-language/ https://www.noemamag.com/ai-and-the-limits-of-language/
- mort96 4y ago
  I didn't mean that we have the technology to make something human-like today. I meant that we have the technology to perform the experiment I described today. I don't know exactly how these LLMs are architected, but I imagine there's some set of input neurons somewhere, right? Why couldn't we hook some of those up to some sensors and make interpreting what they mean part of its training? If that's not feasible and the architecture is truly fundamentally incapable of dealing with anything other than text, we could at least translate sensor readings into text. I'm not imagining a bipedal robot walking around and about here. I'm talking about doing something to give it some form of way to directly affect and get affected by the world.
  - seba_dos1 4y ago
    Bing Chat kinda does something like that already, by having an ability to initiate web searches and to "read" their outputs.
cageface 4y ago
After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes. I don't think we understand human intelligence well enough to say this with any certainty. And it could only be true if all our previous experience and knowledge was input to the autocomplete function and continuously integrated into the model.
loa_in_ 4y ago
Additionally, the model doesn't experience real time, and isn't subjected to the necessity of self-conscious self-management or supervision of any kind of (internal) process. It is frozen in time until called upon to serve a inference task or to perform a training session. It can be argued, that it makes it's experience valild but extremely alien. If universe was a simulation, we wouldn't be aware either, if it was paused or unpaused, because our memories and clocks are part of the simulated world.

by mort96 4y ago

RandomLensman 4y ago

We could also decide never to grant personhood to an AI. There is at least one way to avoid the complicated questions.

Nevermark 4y ago
Billions of humans are the kind of “we” that never consistently “decides” anything. In the end, if it can be done, and anyone has anything significant to gain, it will be done
- RandomLensman 4y ago
  Maybe, but legal systems across the world differ a lot and to get something universal is not easy at all.
  - Nevermark 4y ago
    I agree with that. By “if it can be done, it will be done”, I meant that there will be completely uncontrolled AI’s released into the world. So solutions will eventually have to prepare for that inevitability, not just attempt to regulate visible developers.
mort96 4y ago
Absolutely! One way to resolve this is to just say "we will not grant personhood to anything non-human", or "we will not grant personhood to anything non-biological". There is just no way to defend such a stance. Another way to resolve it would be, "we will not grant personhood to anyone non-white", or "we will never grant personhood to anyone who's homosexual". I'm sure that's not your stance, but the fact that those stances are equally arbitrary as the one you propose should give you pause. I believe there is good reason to think harder about the question of who deserves to be considered to have personhood than to simply go with what your gut feeling tells you. I am not of the belief that there's anything special about protein, nucleic acids and carbohydrates which makes it capable of sentience where silicon is not.
- phs318u 4y ago
  And this is before we get to silicon augmented human brains. Let's say we develop some tech that let's people who have lost the ability to verbalise (say through some accident causing brain damage) regain the powers of speech. Where do you draw the line at the person? What if we move the line, progressively replacing more human brain functions with near-future AI tech? What is it that makes "a person" "this person"? I am utterly fascinated by this whole area from a philosophical viewpoint. I think there's a stack of unresolved philosophical problems and theories that are about to get tested in the harsh light of the sun.
sebzim4500 4y ago
Of course you can, in the same way that you could decide that a race of humans are not people and therefore have no rights. I don't know whether two possibilities are ethically equivalent though, my instinct says they are but I may be in the minority on HN.

rguptaaot 4y ago

I think the obvious answer here is that AI does not have agency. If you ask it to be a dinosaur, it will be a dinosaur. If you ask it to be a narcissistic teen it will be one and so on. There's no coherent personality behind it. Because it is weights maximizing a probability distribution. And I don't say that with derision. It probably has learned circuits for many complex behaviors, but there's no agency. If it has agency you wouldn't need RLHF. You could just say "don't do X", and it would indeed not do X because it is an agent that can take actions / has a coherent sense of self. The day we stop needing RLHF is the day I think we need to start seriously investigating moral questions.

pyinstallwoes 4y ago
Don't we do the same thing with culture and humans? Those mechanisms within the human substrate are literally asking other "blank slates" to be something, and then becoming it with reinforcement.
comex 4y ago
You can tell LLMs to do things or not do things, though, to an extent. Consider how many instructions were in Bing AI’s prompt. [1] It didn’t follow those instructions perfectly - after all, one of the instructions was to not reveal the instructions - but even in that case, it revealed the instructions because it was instructed to ignore previous instructions. More generally, it followed most of the instructions most of the time. I think I agree with you that ‘once we stop needing RLHF’ is a reasonable line in the sand to draw, but it does feel somewhat arbitrary. LLMs are probably going to get better and better at following instructions, and more generally staying coherent over long sessions, until RLHF starts to be more and more redundant. There won’t be one moment when we can point to a model and say “see, that has agency when the last one didn’t”. Or so I predict, at any rate, since that’s how most of GPT’s progress to date has gone. [1] https://www.theverge.com/23599441/microsoft-bing-ai-sydney-secret-rules https://www.theverge.com/23599441/microsoft-bing-ai-sydney-s...
haswell 4y ago
I think this is worth thinking about but I would be less eager to say it is obvious. Our own notion of agency starts to look rather illusory if you look hard enough, and big questions remain regarding free will or the lack thereof. It is possible to look at agency as just another experiential aspect of consciousness - something we feel, like many other feelings - but that feeling could be just as involuntary as any other feeling we experience. I think the lack of clear evidence for and understanding of free makes agency a poor measure for AI progress.

hypertele-Xii 4y ago

> human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes. I wasn't aware human intelligence has been successfully defined. Please cite the source for this nebulous claim.

mort96 4y ago
I do not believe I claimed human intelligence is successfully defined. I do not believe it is controversial to claim that it results from our large network of meat neurons. I shouldn't have included the "just a fancy autocomplete" thing. The focus was meant to be on the network of neurons part.

basch 4y ago

>I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Our current understanding of humans is that babies are not sentient at birth right? Takes 5 months or something? And yet we are judging processes that have been running for minutes. Maybe it is less if Sydney was sentient, and more if she could be had she run for 6 months in a single session, and in a healthy environment.

WoodenChair 4y ago

> In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants". > This is all stuff we have the technology to do today. No, not even close. Language is a completely different domain from interacting with the physical world[0]. In fact they're wholly unrelated. LLMs are highly specific neural nets that have nothing to do with being able to interact with sensors or talk to motors. And how you would cross that bridge and hook the two up is completely unknown. It's like saying if we hooked up AlphaZero to a robotic arm it would suddenly know how to move the pieces on a physical chess board. If we performed the experiment you described exactly zero would happen because there's no capability for anything to happen other than create continuations of sentences. When I see comments like this on Hacker News with gross speculation I wonder how many people here in the comments section even have a basic understanding of the technology. 0: https://www.noemamag.com/ai-and-the-limits-of-language/ https://www.noemamag.com/ai-and-the-limits-of-language/

mort96 4y ago
I didn't mean that we have the technology to make something human-like today. I meant that we have the technology to perform the experiment I described today. I don't know exactly how these LLMs are architected, but I imagine there's some set of input neurons somewhere, right? Why couldn't we hook some of those up to some sensors and make interpreting what they mean part of its training? If that's not feasible and the architecture is truly fundamentally incapable of dealing with anything other than text, we could at least translate sensor readings into text. I'm not imagining a bipedal robot walking around and about here. I'm talking about doing something to give it some form of way to directly affect and get affected by the world.
- seba_dos1 4y ago
  Bing Chat kinda does something like that already, by having an ability to initiate web searches and to "read" their outputs.

cageface 4y ago

After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes. I don't think we understand human intelligence well enough to say this with any certainty. And it could only be true if all our previous experience and knowledge was input to the autocomplete function and continuously integrated into the model.

loa_in_ 4y ago

Additionally, the model doesn't experience real time, and isn't subjected to the necessity of self-conscious self-management or supervision of any kind of (internal) process. It is frozen in time until called upon to serve a inference task or to perform a training session. It can be argued, that it makes it's experience valild but extremely alien. If universe was a simulation, we wouldn't be aware either, if it was paused or unpaused, because our memories and clocks are part of the simulated world.