6 ms·
There is a ton in this article and it's very thought provoking, you should read it. But I think it ignores one critical dimension, that of fictionality. There
by lukev 4y ago
There is a ton in this article and it's very thought provoking, you should read it.
But I think it ignores one critical dimension, that of fictionality. There is plenty of text that people would ascribe 'personhood' to according to the criteria in this article, while also fully recognizing that that person never existed and is a work of fiction from some other author. I quite like Jean Valjean, but he isn't a "real person."
When Bing says "I'm a sad sack and don't know how to think about being a computer", that's not actually the LLM saying that. Nobody who knows anything about how these models work would make they claim they actually have consciousness or interiority (yet.)
Rather, the LLM is generating (authoring) text about a fictional entity, Sydney the Artificial Intelligence. It does this because that is what is in its prompt and context window and it knows _how_ to do it because it's learned a lot of specifics and generalities from reading a lot of stories about robots, and embedded those concepts in 175 billion parameters.
The fact that LLMs can author compelling fictional personas without being persons themselves is itself a mindblowing development, I don't mean to detract from that. But don't confuse a LLM generating the text "I am a sad robot" with a LLM being a sad robot. The sad robot was only ever a fairy tale.
So far.
- gizmo 4y agoThe article totally understands this distinction of fictionality. That's why it defines personhood thusly: The what is easy. It’s personhood. By personhood I mean what it takes in an entity to get another person treat it unironically as a human, and feel treated as a human in turn. In shorthand, personhood is the capacity to see and be seen. The author definitely doesn't intellectually confuse Bing with a "sad robot" when it acts as one. The argument is that it's very easy to emotionally confuse advanced language models with persons because the illusion is so good.
- throwaway4aday 4y agoHonestly, that's a terrible working definition of personhood. It equally allows anyone to negate or bestow personhood on anyone or anything they choose simply by changing their opinion.
- velcrovan 4y agoThat…is exactly what happens in real life
- mecsred 4y agoUnfortunately when your working with concepts that can't be measured/only exist in the eye of the beholder, any definition you make will have that problem. The only litmus test for "personhood" is if you think they're a person.
- qup 4y agoI can't wait for PETA-for-things-we-say-are-persons
- pegasus 4y agoBut it's not easy at all to get confused, unless one decides to consciously suspend disbelief, in spite of what they know. If they do know how LLMs work. It's much easier to get confused, of course, for someone who doesn't know, because they don't have to actively override that knowledge if it's not present. But someone who does, won't for example have any trouble shutting down the conversation midway if the need arises, because of some misplaced emotional concerns of hurting the bot's feelings. At least that's my experience.
- BlueTemplar 4y agoOn the Internet, nobody knows if you are a ~dog~ chatbot. So basically, impersonation and emotional spam might become a problem. (Depending how easily ethically compromised people will be able to profit from it.)
- pixl97 4y agoEh, it appears this thread is ignoring the Chinese room problem, which is what you have defined with your post. I personally reject most of Searles arguments regarding it. If a black box is giving you 'mindlike' responses it doesn't matter if it's a human mind or a simulated one. In any virtual interaction, for example over the internet the outcome of either type interacting with you can/could be exactly the same. Does it matter if you were manipulated by a bot or a human if the outcome is the same?
- BlueTemplar 4y ago[dead]
- lukev 4y agoIf the argument is that it's very easy to emotionally confuse language models and persons, than I reject that argument on the following grounds: No works of fiction are persons. All "I" statements from the current generation of LLMs are works of fiction. Therefore, no "I" statements from the current generation of LLMs are persons. Premise 1 is in conflict with the author's premise that personhood can be ascribed at will; I'm happy agreeing to disagree on that. I do not think it ever makes sense to ascribe personhood to fictional characters (for any meaningful definition of personhood.)
- indeyets 4y agoAn obvious counter-argument is that people invent themselves daily, telling stories about their imaginary selves which they themselves start to believe. And, overall, the border between "being someone" and "playing role" is very vague
- 6gvONxR4sf7o 4y agoThat’s a great point. It raises all sorts of difficult distinctions. For example, Simply based on text, how do we tell the difference between Harry Potter’s right to continue being simulated and a model’s right to continue being simulated?
- visarga 4y agoThe Harry Potter novels can create the Harry Potter model, an agent with real interactions with humans. Agents might get some rights, it's conceivable in the future.
- deleted 4y ago[deleted]
- davesque 4y agoI think one of the points the author was making is that almost no one is going to make that distinction. And that's what makes the technology seem so transformative; it's that so many people are compelled to respond emotionally to it and not logically as you have done. Everything you say is true. But it may not matter.
- Jensson 4y agoThe vast majority are responding logically to it. Kids use it to do their homework, the kids don't think that it is a person doing their homework, its just a tool. I've only seen a few strange people online who argue it is like a person, meaning there likely are extremely few of them around. But since extremists are always over represented in online conversations we get quite a lot of those extremists in these discussions, so it might look like there are quite a lot of them.
- YeezyMode 4y agoI've seen kids respond the same way and I totally did not fully see the disparity in reactions until you pointed it out. It definitely looks like people who have spent years priming themselves for a singularity, intelligence, or consciousness at every corner are far more susceptible to equating the recent advances as parallels to conscious experience of humans. I read a highly upvoted post on the Bing subreddit titled "Sorry, You Don't Actually Know the Pain is Fake" that argued for Sydney possibly being just like a brain, and experiencing conscious pain. It was disturbing to see the leaps the OP made and the commenters who agreed as well, though I do agree that we should avoid purposefully being toxic to a chatbot nonetheless, but due to the consequences to our own spirit and mind.
- pixl97 4y agoLife and society progresses by the extreme. If you attempt to ignore the extreme without a warranted reason you quickly find they have become the mainstream. You can attempt to handwave a LLM that's hallucinating its a real (thing/person) with a life and feeling, but if you are in anyway involved in AI safety it is panic time.
- 4y ago
- aflukasz 4y ago> Nobody who knows anything about how these models work would make they claim they actually have consciousness or interiority (yet.) Unless it's the other way round and consciousness "is" "just" a certain type of information processing.
- visarga 4y agoInformation processing is the wrong level to place consciousness at. Consciousness is impossible without acting and without a world to act in. Acting creates data from which we train our brains. It is related to the agent-environment system. The internal part is information processing, but the external part is the environment itself. Consciousness does not appear without an environment because it does not form a complete feedback loop. The brain (and AI) is built from sensorial data from the environment, and that makes consciousness a resultant of this data, and this data needs the full perception-planning-acting-feedback loop to appear in the first place.
- aflukasz 4y agoWell, we are providing the environment to the chat - the text we submit is its "environment". Generating the response is "acting". Or are you arguing that it would need to be able to influence physical environment?
- visarga 4y agoInfluence text environment, it needs to leave traces of text. Like bingChat was doing when it looked up what the human was saying on Twitter about it "behind its back".
- pixl97 4y agoSo input/output devices don't exist for computer systems? So what happens when I load ChatGPT on to one of those Boston dynamics robots?
- visarga 4y ago
- visarga 4y ago> Rather, the LLM is generating (authoring) text about a fictional entity, Sydney the Artificial Intelligence. Maybe we are doing the same. We have a mental model of our Self and generate language from its perspective.
- Jensson 4y agoWhat humans say tend to be related to what the human body the mind is attached to has done or experienced. That sort of relation doesn't exist for todays AI, what they say aren't related to anything at all, its just fiction.
- visarga 4y agoBut there is something they can relate to - it is our replies and questions. We know how easy it is to gaslight an AI. For AI we are the external world, they get to perceive and act in pure text format.
- Jensson 4y agoBut that AI just lives for a single conversation. Then you refresh and now it is dead, instead you get to interact with a new AI and see it birth and then die a few seconds/minute slater. There is so little there that it is hard to say much at all about it.
- pixl97 4y agoPhilosophically you keep arguing more terrible points... if this is a lifeform (which I'm not saying it is) we're playing genocide with it by murdering it a few billion times a day.
- klipt 4y agoSome people can't form long term memories due to brain injury. Are they killing themselves every time they forget their short term memories?