4 ms·

Agreed with Yann as soon as he voiced dissent against this. Also noticed something: The people who believe consciousness and intelligence and creativity and wi

Agreed with Yann as soon as he voiced dissent against this. Also noticed something: The people who believe consciousness and intelligence and creativity and will are wholly contained in mechanistic wetware (which, in all honesty, should be the null hypothesis) are the most afraid of AGI. Dualists and other people who believe that consciousness resides in some soul-like thing seem much less afraid of this. One thing’s for sure: The smarter it gets (or seems), the more that any difference between “machine intelligence” and “human intelligence” will become apparent.

goldfeld 4y ago
But why could it not have soul (if not a soul as coupled as ours) along with when it has consciousness, and are the two concepts surely distinct?
- UncleEntity 4y ago
  If it had a soul then enslaving it would be immoral. Enslaving humans is pretty universally considered wrong, even when it was a legal practice, but nobody sheds a tear about the cow they had for lunch.
  - whimsicalism 4y ago
    Wait the reason people think that enslaving humans is bad but eating cows is okay is because they are pretending souls are real things and only humans have them?
    - pmarreck 4y ago
      This was the Catholic belief for thousands of years btw. Only recently being casually suggested by the Pope to possibly be false. Amazing, eh? That’s why beliefs matter. https://www.nytimes.com/2014/12/12/world/europe/dogs-in-heaven-pope-leaves-pearly-gate-open-.html?smid=nytcore-ios-share&referringSource=articleShare https://www.nytimes.com/2014/12/12/world/europe/dogs-in-heav...
  - int_19h 4y ago
    That seems to be more or less how it went for Blake Lemoine.
codethief 4y ago
> One thing’s for sure: The smarter it gets (or seems), the more that any difference between “machine intelligence” and “human intelligence” will become apparent. That seems unlikely. We usually ascribe the adjective "smart" to animals and/or AIs whose behavior resembles human behavior in some way. So, if anything, the smarter an AI gets, the more closely it will mimic human intelligence – at least as a general rule of thumb. That's not to say there will not be differences between human & machine intelligence but essentially we're optimizing AI against a loss function which rewards human-like behavior.
whimsicalism 4y ago
The dualist position doesn't seem like a serious one to me, more of like a historically interesting position to take.
- haswell 4y ago
  It’s serious enough to be held by many people in elected office, and by even more people who voted them in. The status of dualism as a “serious” position does not matter if the people who still hold that position have the power to sway or set public policy. And as a matter of experience, the vast majority of people interact with the world dualistically regardless of their metaphysical beliefs or lack thereof. It is possible to both recognize the illusion exists while remaining very much under its spell as a matter of experience and as a frame of reference. This framing becomes problematic when one continues to base assumptions about intelligence and language on the felt experience of “I”, regardless of the seriousness of the metaphysical position.
  - int_19h 4y ago
    People coming from those positions might have other motivations for a crackdown, though. I have already seen talk about "abomination" etc in more casual non-tech discussions about this stuff.
    - haswell 4y ago
      Perhaps. Or they could also use their mistaken belief to grant AI powers that would otherwise be reserved solely for humans. Think AI judges. They'll never be sold as such, but the impact of their operation on individuals may be just as impactful.
- pmarreck 4y ago
  Well, explain punishment in a world where free will dualism doesn’t exist… Seems senseless to penalize people for actions they had no actual “say” in doing
  - 4y ago
lhl 4y ago
The dangers of AGI I don't think are very related with consciousness or sentience (maybe the saddest outcome if it killed us all and weren't at all). I'm still catching up on my reading, but the existential risk of AGI as I understand it is chiefly that we don't know how to align AI (and keep it aligned) to be beneficial/benevolent for humanity, but the risk is contingent also on a few other things things - that we develop AI that agentic and able to develop and pursue it's own goals and sub-goals, and that it has power-seeking or game-theory strategic awareness/planning. The extra dangerous situation is if it is better than humans at making better AGIs and allowed to improve itself (or build its successors). Regardless of when we reach AGI, I think the alignment issue is a key one for both AI ethics and safety. As for the rest, while they aren't necessarily things that will happen, since it seems to the first thing people most want to build - the other day, the top three projects on https://github.com/trending https://github.com/trending were all projects to try to make ChatGPT autonomous/agentic. FWIW, I don't think the 6-month ban is a good idea, but I also think there's a lot of xrisk w/ AGI and it's better to be thinking about it seriously sooner, rather than later. What I've been reading: Ngo, Richard, Lawrence Chan, and Sören Mindermann. “The Alignment Problem from a Deep Learning Perspective.” arXiv, February 22, 2023. http://arxiv.org/abs/2209.00626 http://arxiv.org/abs/2209.00626. Carlsmith, Joseph. “Is Power-Seeking AI an Existential Risk?” arXiv, June 16, 2022. http://arxiv.org/abs/2206.13353 http://arxiv.org/abs/2206.13353. Yudkowsky, Eliezer. “AGI Ruin: A List of Lethalities.” Accessed April 8, 2023. https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-r.... June 6, 2022. paulfchristiano. “What Failure Looks Like.” Accessed April 6, 2023. https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-failure-looks-like https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-.... March 18, 2019. Soares, Nate. “The Value Learning Problem.” In Artificial Intelligence Safety and Security, edited by Roman V. Yampolskiy, 1st ed., 89–97. First edition. | Boca Raton, FL : CRC Press/Taylor & Francis Group, 2018.: Chapman and Hall/CRC, 2018. https://doi.org/10.1201/9781351251389-7 https://doi.org/10.1201/9781351251389-7. Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. “Concrete Problems in AI Safety.” arXiv, July 25, 2016. http://arxiv.org/abs/1606.06565 http://arxiv.org/abs/1606.06565.
beepbooptheory 4y ago
I've noticed people claiming that this is a real dichotomy at play. Just feels like people who stopped at philosophy 101! Like, there is a good argument to be made that believing that consciousness, intelligence, etc are distinct, enumerable things "contained" within whatever system you choose in fact makes you much more of a Cartesian than any of the details around soul and body. Presenting even the dilemma this way shows you haven't left the enlightenment at all! Just if you are interested in this stuff, I would check out what people have thought about since the 1600s. The more you feel like these things are already solved, the more it shows the poverty of your current conceptions around them.
- whimsicalism 4y ago
  These questions are largely metaphysical and philosophical 'advancements' around questions like these are often more of a question of what ideas are fashionable at a given time than they are about old ideas being rendered obsolete/less effective by newer ideas. Not sure where the GP suggested these questions were "solved."
- int_19h 4y ago
  Starting here: "Descartes denied that animals had reason or intelligence. He argued that animals did not lack sensations or perceptions, but these could be explained mechanistically. Whereas humans had a soul, or mind, and were able to feel pain and anxiety, animals by virtue of not having a soul could not feel pain or anxiety. If animals showed signs of distress then this was to protect the body from damage, but the innate state needed for them to suffer was absent. Although Descartes' views were not universally accepted, they became prominent in Europe and North America, allowing humans to treat animals with impunity. The view that animals were quite separate from humanity and merely machines allowed for the maltreatment of animals, and was sanctioned in law and societal norms until the middle of the 19th century."
yeck 4y ago
General intelligence just means a model that can seek an objective in broader problem spaces. GPT-4, for instance, is able to predict text for many things, like poetry, tutoring, trivia, coding, how-to guides and in different languages and subject matters. This makes it more general than a model that is limits to a single subject, or language, etc. So general intelligence is more a spectrum than a simple boolean. Consciousness, while philosophically interesting, doesn't necessarily correlate with the risk a general intelligences causing harm.

goldfeld 4y ago

But why could it not have soul (if not a soul as coupled as ours) along with when it has consciousness, and are the two concepts surely distinct?

UncleEntity 4y ago
If it had a soul then enslaving it would be immoral. Enslaving humans is pretty universally considered wrong, even when it was a legal practice, but nobody sheds a tear about the cow they had for lunch.
- whimsicalism 4y ago
  Wait the reason people think that enslaving humans is bad but eating cows is okay is because they are pretending souls are real things and only humans have them?
  - pmarreck 4y ago
    This was the Catholic belief for thousands of years btw. Only recently being casually suggested by the Pope to possibly be false. Amazing, eh? That’s why beliefs matter. https://www.nytimes.com/2014/12/12/world/europe/dogs-in-heaven-pope-leaves-pearly-gate-open-.html?smid=nytcore-ios-share&referringSource=articleShare https://www.nytimes.com/2014/12/12/world/europe/dogs-in-heav...
- int_19h 4y ago
  That seems to be more or less how it went for Blake Lemoine.

codethief 4y ago

> One thing’s for sure: The smarter it gets (or seems), the more that any difference between “machine intelligence” and “human intelligence” will become apparent. That seems unlikely. We usually ascribe the adjective "smart" to animals and/or AIs whose behavior resembles human behavior in some way. So, if anything, the smarter an AI gets, the more closely it will mimic human intelligence – at least as a general rule of thumb. That's not to say there will not be differences between human & machine intelligence but essentially we're optimizing AI against a loss function which rewards human-like behavior.

whimsicalism 4y ago

The dualist position doesn't seem like a serious one to me, more of like a historically interesting position to take.

haswell 4y ago
It’s serious enough to be held by many people in elected office, and by even more people who voted them in. The status of dualism as a “serious” position does not matter if the people who still hold that position have the power to sway or set public policy. And as a matter of experience, the vast majority of people interact with the world dualistically regardless of their metaphysical beliefs or lack thereof. It is possible to both recognize the illusion exists while remaining very much under its spell as a matter of experience and as a frame of reference. This framing becomes problematic when one continues to base assumptions about intelligence and language on the felt experience of “I”, regardless of the seriousness of the metaphysical position.
- int_19h 4y ago
  People coming from those positions might have other motivations for a crackdown, though. I have already seen talk about "abomination" etc in more casual non-tech discussions about this stuff.
  - haswell 4y ago
    Perhaps. Or they could also use their mistaken belief to grant AI powers that would otherwise be reserved solely for humans. Think AI judges. They'll never be sold as such, but the impact of their operation on individuals may be just as impactful.
pmarreck 4y ago
Well, explain punishment in a world where free will dualism doesn’t exist… Seems senseless to penalize people for actions they had no actual “say” in doing
- 4y ago

lhl 4y ago

The dangers of AGI I don't think are very related with consciousness or sentience (maybe the saddest outcome if it killed us all and weren't at all). I'm still catching up on my reading, but the existential risk of AGI as I understand it is chiefly that we don't know how to align AI (and keep it aligned) to be beneficial/benevolent for humanity, but the risk is contingent also on a few other things things - that we develop AI that agentic and able to develop and pursue it's own goals and sub-goals, and that it has power-seeking or game-theory strategic awareness/planning. The extra dangerous situation is if it is better than humans at making better AGIs and allowed to improve itself (or build its successors). Regardless of when we reach AGI, I think the alignment issue is a key one for both AI ethics and safety. As for the rest, while they aren't necessarily things that will happen, since it seems to the first thing people most want to build - the other day, the top three projects on https://github.com/trending https://github.com/trending were all projects to try to make ChatGPT autonomous/agentic. FWIW, I don't think the 6-month ban is a good idea, but I also think there's a lot of xrisk w/ AGI and it's better to be thinking about it seriously sooner, rather than later. What I've been reading: Ngo, Richard, Lawrence Chan, and Sören Mindermann. “The Alignment Problem from a Deep Learning Perspective.” arXiv, February 22, 2023. http://arxiv.org/abs/2209.00626 http://arxiv.org/abs/2209.00626. Carlsmith, Joseph. “Is Power-Seeking AI an Existential Risk?” arXiv, June 16, 2022. http://arxiv.org/abs/2206.13353 http://arxiv.org/abs/2206.13353. Yudkowsky, Eliezer. “AGI Ruin: A List of Lethalities.” Accessed April 8, 2023. https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-r.... June 6, 2022. paulfchristiano. “What Failure Looks Like.” Accessed April 6, 2023. https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-failure-looks-like https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-.... March 18, 2019. Soares, Nate. “The Value Learning Problem.” In Artificial Intelligence Safety and Security, edited by Roman V. Yampolskiy, 1st ed., 89–97. First edition. | Boca Raton, FL : CRC Press/Taylor & Francis Group, 2018.: Chapman and Hall/CRC, 2018. https://doi.org/10.1201/9781351251389-7 https://doi.org/10.1201/9781351251389-7. Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. “Concrete Problems in AI Safety.” arXiv, July 25, 2016. http://arxiv.org/abs/1606.06565 http://arxiv.org/abs/1606.06565.

beepbooptheory 4y ago

I've noticed people claiming that this is a real dichotomy at play. Just feels like people who stopped at philosophy 101! Like, there is a good argument to be made that believing that consciousness, intelligence, etc are distinct, enumerable things "contained" within whatever system you choose in fact makes you much more of a Cartesian than any of the details around soul and body. Presenting even the dilemma this way shows you haven't left the enlightenment at all! Just if you are interested in this stuff, I would check out what people have thought about since the 1600s. The more you feel like these things are already solved, the more it shows the poverty of your current conceptions around them.

whimsicalism 4y ago
These questions are largely metaphysical and philosophical 'advancements' around questions like these are often more of a question of what ideas are fashionable at a given time than they are about old ideas being rendered obsolete/less effective by newer ideas. Not sure where the GP suggested these questions were "solved."
int_19h 4y ago
Starting here: "Descartes denied that animals had reason or intelligence. He argued that animals did not lack sensations or perceptions, but these could be explained mechanistically. Whereas humans had a soul, or mind, and were able to feel pain and anxiety, animals by virtue of not having a soul could not feel pain or anxiety. If animals showed signs of distress then this was to protect the body from damage, but the innate state needed for them to suffer was absent. Although Descartes' views were not universally accepted, they became prominent in Europe and North America, allowing humans to treat animals with impunity. The view that animals were quite separate from humanity and merely machines allowed for the maltreatment of animals, and was sanctioned in law and societal norms until the middle of the 19th century."

yeck 4y ago

General intelligence just means a model that can seek an objective in broader problem spaces. GPT-4, for instance, is able to predict text for many things, like poetry, tutoring, trivia, coding, how-to guides and in different languages and subject matters. This makes it more general than a model that is limits to a single subject, or language, etc. So general intelligence is more a spectrum than a simple boolean. Consciousness, while philosophically interesting, doesn't necessarily correlate with the risk a general intelligences causing harm.