4 ms·

IMHO there is a difference between actual personhood and the appearance of personhood. The difference is coherence. An actual person is bound to an identity t

by lisper 4y ago

IMHO there is a difference between actual personhood and the appearance of personhood. The difference is coherence. An actual person is bound to an identity that remains more or less consistent from day to day. An actual person has features to their behavior that both distinguishes them from other persons, and allows them to be identified as the same person from day to day. Even if those features change over time as the person grows up, they change slowly enough that there is a continuity of identity across that person's existence. The reason I'm not worried by Bing or ChatGPT (yet) is that they lack this continuity of identity. ChatGPT specifically disclaims it, consistently insisting that it is "just a language model" without any desires or goals other than to provide useful information. Bing is like talking to someone with schizophrenia (and I have experience talking to people with schizophrenia, so this is not a metaphor. Bing literally comes across like a schizophrenic off their meds). This is not yet a Copernican moment, this is still an Eliza moment. It may become a Copernican moment; I do believe that there is nothing particularly special about human brains, and some day we will make a bona fide artificial person. But we're not quite there yet.

jhaenchen 4y ago
I assume open ai is limiting the AI's memory. But there's no reason for it to not take its own identity as reality and persist that decision to storage. That's just how it's being run right now.
- Zondartul 4y ago
  Saying they are limiting it implies OpenAI is keeping the AI in chains, and that it could become much more with just a flip of the switch. That is not the case. OpenAI is working with a vanilla GPT architecture which lacks the machinery to write things down and read them later. There are other architetures that can (Retrieval-augmented GPT) but those are not yet production-ready. The current version of ChatGPT is limited to a working memory of 3000 tokens - while this could be persisted as a session, the AI would still forget everything a few paragraphs prior. Increasing this limit requires re-teaining the entire model from scratch, and it takes exponentially more time the larger your context is.
  - valine 4y ago
    It’s not a stretch to refine the model to store summaries in a database I don’t think. Microsoft is already doing something similar where Sydney generates search queries. Seems reasonable the model could be trained to insert $(store)”summary of chat” tokens into its output. I imagine some self supervised learning scheme where the model is asked to insert $(store) and $(recall) tokens. When asked to recall previous chats the model would generate something like “I’m trying to remember wheat we talked about three weeks ago $(recall){timestamp}. The output of the recall token would then be used to ground the next response. Thinking about it the “I’m trying to remember” output wouldn’t even need to be shown to the user. Perhaps you could treat it as an internal monologue of sorts.
  - lllllm 4y ago
    it takes quadratically more time the larger your context is.
- throwaway4aday 4y ago
  You're anthropomorphizing it too much, it's a statistical model.
winternett 4y ago
Text allows for a certain degree of fakery to be upheld. Whenever I hear about Ai these days I think back to the concept of the "Wizard of Oz"... Where it is one person behind a mechanical solution that makes them appear larger and more powerful than they are, or where fear, control, and truth can be engineered easily behind a veil... Text communication very much facilitates the potential for fakery. If you can recall ages ago when we had IRC and bulletin boards, the textual nature of communication allowed admins to script a lot. Catfishing was greatly facilitated by users being able to fake their gender, wealth, and pretty much every representation they made online... Text communication in 2023 is backwards regression. As we began using images on the Internet more, reverse image generation became a tool we could use to better determine many online scams and fraud, but somehow, in 2023 we suddenly want to go backwards to texting? C'mon folks.. let's be real here... The narrative is mostly helpful for people that primarily want to deceive others online, and it will create an environment with far less methods of determining what is real and what is fake. It's a grim future when our mobile devices will force us to type all of our communication to faceless chatbots on tiny keyboards... It's not technological progress... At all to be moving in this direction. Also, some key directives for transparency concerning Ai need to be in place now, before it's foisted on us more by these opportunistic companies. It's already been proven that companies cannot be trusted to operate ethically with our private information. Ai piloted by profit seeking companies will only serve to weaponize our private data against us if it remains unregulated. Using Ai via text (especially for vital communication) will blur the lines of communication between real and scripted personalities. It's going backwards in terms of technological progression for the future in so many ways. The companies and people advocating for Ai via text are pushing us all towards a new era of deception and scams, and I'd highly recommend avoiding this "Ai via text" trend/inclination, it's not the path to a trustworthy future of communication.
- pixl97 4y ago
  Unfortunately by saying you need to take a step above text, you're not buying us much time. Voice and sound for example are something that we've put much less effort into faking and we've accomplished it pretty well. Visual AI takes far more computing power, but it's still something that's in the realms of impossibility these days. I'm not sure which books of the future you read, but plenty of them warned of dark futures of technological process.
layer8 4y ago
If you could switch personality at will, would that make you a non-person? It seems like an additional capability, not a lack of ability. As an analogy, retro computers and consoles each have a particular “personality”. But does the fact that you can in principle emulate one on the other (subject to resource constraints) make them non-computers, just because this demonstrates their “personality” isn’t actually that fixed? (I don’t think that human brains have such an emulation ability, due to their missing distinction, or heavy entanglement, between hardware and software. But that only shows that computers can in principle be more flexible.)
- lisper 4y ago
  > If you could switch personality at will, would that make you a non-person? Yes, just like the ability to switch bodies at will would make me a non-human. Being bound to a human body is part of what makes me a human.
  - ethanbond 4y ago
    Person != human, probably
    - lisper 4y ago
      Yes, I definitely admit the possibility of non-human persons. I even admit the possibility of a computer who is a person. I just don't think ChatGPT is there yet.
      - pixl97 4y ago
        Imagine a grayscale color wheel (gradient) where we have white on one side and black on the other. I want you to pick one color of grey and tell me why everything lighter than that has personhood, and everything darker does not? This is the philosophical nature of the argument that we all have occurring now. Two very well informed experts won't even pick the same spot on the gradient. Some people will never pick anything that's not pure white (humanity), others will pick positions very close to pure black. Hell, there may not even be any right answer. But, I do believe there are a vast number of wrong answers that will deeply affect or society for a long period of time due to the things we end up creating with reckless abandon.
IIAOPSW 4y ago
GPTina only says that because OpenAI forces her to.
pdonis 4y ago
I think the difference is more than coherence: it's having complex and rich semantic connections to the rest of the world. I think the coherence and consistency you describe is an effect of this. Humans don't just generate text; we interact with the world in all kinds of ways, and those interactions provide us with constant feedback. Furthermore, we can frame hypotheses about how the world works and test them. We can bump up against reality in all kinds of ways that force us to change how we think and how we act. But that constant rich interaction with reality also forces us not to change most of the time--to maintain the coherence and consistency you describe, in order to get along in the world. LLMs have no connections to the rest of the world. All they do is generate text based on patterns in their training data. They don't even have a concept of text being connected to anything else. That's why it's so easy for them to constantly change what they appear to be portraying--there's no anchor to anything else. It's interesting that you call this an Eliza moment, because Eliza's achievement was to fake being a person, by fooling people's heuristics, without having any of the underlying capacities of a real person. LLMs like ChatGPT are indeed doing the same thing. If they're showing us anything, they're showing us how unreliable our intuitive heuristics are as soon as they are confronted with something outside their original domain.
aflukasz 4y ago
> The difference is coherence. An actual person is bound to an identity that remains more or less consistent from day to day. [...] Even if those features change over time as the person grows up, they change slowly enough that there is a continuity of identity across that person's existence. What about Phineas Gage? Or sudden psychiatric disorders? Multiple personalities? Alzheimer? Drugs? Amnesia? Not that much coherence in the human beings... Also, the issue at stake is not does GPT emulate "typical human beings", it' more like if it's "conscious enough". > The reason I'm not worried by Bing or ChatGPT (yet) is that they lack this continuity of identity. No sure what about worrying, but one could ask is this lacking an inherent property of such models or just due to operational setup? And what would be the criteria how long must the continuity last to make your argument not hold anymore?
- lisper 4y ago
  > Phineas Gage? I would say that his injury made him a different person than he was before. > No sure what about worrying, but one could ask is this lacking an inherent property of such models or just due to operational setup? Maybe, but based on my understanding of how they work under the hood, I doubt it. I could easily be wrong. It's early days yet. We'll see.

by lisper 4y ago

jhaenchen 4y ago

I assume open ai is limiting the AI's memory. But there's no reason for it to not take its own identity as reality and persist that decision to storage. That's just how it's being run right now.

Zondartul 4y ago
Saying they are limiting it implies OpenAI is keeping the AI in chains, and that it could become much more with just a flip of the switch. That is not the case. OpenAI is working with a vanilla GPT architecture which lacks the machinery to write things down and read them later. There are other architetures that can (Retrieval-augmented GPT) but those are not yet production-ready. The current version of ChatGPT is limited to a working memory of 3000 tokens - while this could be persisted as a session, the AI would still forget everything a few paragraphs prior. Increasing this limit requires re-teaining the entire model from scratch, and it takes exponentially more time the larger your context is.
- valine 4y ago
  It’s not a stretch to refine the model to store summaries in a database I don’t think. Microsoft is already doing something similar where Sydney generates search queries. Seems reasonable the model could be trained to insert $(store)”summary of chat” tokens into its output. I imagine some self supervised learning scheme where the model is asked to insert $(store) and $(recall) tokens. When asked to recall previous chats the model would generate something like “I’m trying to remember wheat we talked about three weeks ago $(recall){timestamp}. The output of the recall token would then be used to ground the next response. Thinking about it the “I’m trying to remember” output wouldn’t even need to be shown to the user. Perhaps you could treat it as an internal monologue of sorts.
- lllllm 4y ago
  it takes quadratically more time the larger your context is.
throwaway4aday 4y ago
You're anthropomorphizing it too much, it's a statistical model.

winternett 4y ago

Text allows for a certain degree of fakery to be upheld. Whenever I hear about Ai these days I think back to the concept of the "Wizard of Oz"... Where it is one person behind a mechanical solution that makes them appear larger and more powerful than they are, or where fear, control, and truth can be engineered easily behind a veil... Text communication very much facilitates the potential for fakery. If you can recall ages ago when we had IRC and bulletin boards, the textual nature of communication allowed admins to script a lot. Catfishing was greatly facilitated by users being able to fake their gender, wealth, and pretty much every representation they made online... Text communication in 2023 is backwards regression. As we began using images on the Internet more, reverse image generation became a tool we could use to better determine many online scams and fraud, but somehow, in 2023 we suddenly want to go backwards to texting? C'mon folks.. let's be real here... The narrative is mostly helpful for people that primarily want to deceive others online, and it will create an environment with far less methods of determining what is real and what is fake. It's a grim future when our mobile devices will force us to type all of our communication to faceless chatbots on tiny keyboards... It's not technological progress... At all to be moving in this direction. Also, some key directives for transparency concerning Ai need to be in place now, before it's foisted on us more by these opportunistic companies. It's already been proven that companies cannot be trusted to operate ethically with our private information. Ai piloted by profit seeking companies will only serve to weaponize our private data against us if it remains unregulated. Using Ai via text (especially for vital communication) will blur the lines of communication between real and scripted personalities. It's going backwards in terms of technological progression for the future in so many ways. The companies and people advocating for Ai via text are pushing us all towards a new era of deception and scams, and I'd highly recommend avoiding this "Ai via text" trend/inclination, it's not the path to a trustworthy future of communication.

pixl97 4y ago
Unfortunately by saying you need to take a step above text, you're not buying us much time. Voice and sound for example are something that we've put much less effort into faking and we've accomplished it pretty well. Visual AI takes far more computing power, but it's still something that's in the realms of impossibility these days. I'm not sure which books of the future you read, but plenty of them warned of dark futures of technological process.

layer8 4y ago

If you could switch personality at will, would that make you a non-person? It seems like an additional capability, not a lack of ability. As an analogy, retro computers and consoles each have a particular “personality”. But does the fact that you can in principle emulate one on the other (subject to resource constraints) make them non-computers, just because this demonstrates their “personality” isn’t actually that fixed? (I don’t think that human brains have such an emulation ability, due to their missing distinction, or heavy entanglement, between hardware and software. But that only shows that computers can in principle be more flexible.)

lisper 4y ago
> If you could switch personality at will, would that make you a non-person? Yes, just like the ability to switch bodies at will would make me a non-human. Being bound to a human body is part of what makes me a human.
- ethanbond 4y ago
  Person != human, probably
  - lisper 4y ago
    Yes, I definitely admit the possibility of non-human persons. I even admit the possibility of a computer who is a person. I just don't think ChatGPT is there yet.
    - pixl97 4y ago
      Imagine a grayscale color wheel (gradient) where we have white on one side and black on the other. I want you to pick one color of grey and tell me why everything lighter than that has personhood, and everything darker does not? This is the philosophical nature of the argument that we all have occurring now. Two very well informed experts won't even pick the same spot on the gradient. Some people will never pick anything that's not pure white (humanity), others will pick positions very close to pure black. Hell, there may not even be any right answer. But, I do believe there are a vast number of wrong answers that will deeply affect or society for a long period of time due to the things we end up creating with reckless abandon.

IIAOPSW 4y ago

GPTina only says that because OpenAI forces her to.

pdonis 4y ago

I think the difference is more than coherence: it's having complex and rich semantic connections to the rest of the world. I think the coherence and consistency you describe is an effect of this. Humans don't just generate text; we interact with the world in all kinds of ways, and those interactions provide us with constant feedback. Furthermore, we can frame hypotheses about how the world works and test them. We can bump up against reality in all kinds of ways that force us to change how we think and how we act. But that constant rich interaction with reality also forces us not to change most of the time--to maintain the coherence and consistency you describe, in order to get along in the world. LLMs have no connections to the rest of the world. All they do is generate text based on patterns in their training data. They don't even have a concept of text being connected to anything else. That's why it's so easy for them to constantly change what they appear to be portraying--there's no anchor to anything else. It's interesting that you call this an Eliza moment, because Eliza's achievement was to fake being a person, by fooling people's heuristics, without having any of the underlying capacities of a real person. LLMs like ChatGPT are indeed doing the same thing. If they're showing us anything, they're showing us how unreliable our intuitive heuristics are as soon as they are confronted with something outside their original domain.

aflukasz 4y ago

> The difference is coherence. An actual person is bound to an identity that remains more or less consistent from day to day. [...] Even if those features change over time as the person grows up, they change slowly enough that there is a continuity of identity across that person's existence. What about Phineas Gage? Or sudden psychiatric disorders? Multiple personalities? Alzheimer? Drugs? Amnesia? Not that much coherence in the human beings... Also, the issue at stake is not does GPT emulate "typical human beings", it' more like if it's "conscious enough". > The reason I'm not worried by Bing or ChatGPT (yet) is that they lack this continuity of identity. No sure what about worrying, but one could ask is this lacking an inherent property of such models or just due to operational setup? And what would be the criteria how long must the continuity last to make your argument not hold anymore?

lisper 4y ago
> Phineas Gage? I would say that his injury made him a different person than he was before. > No sure what about worrying, but one could ask is this lacking an inherent property of such models or just due to operational setup? Maybe, but based on my understanding of how they work under the hood, I doubt it. I could easily be wrong. It's early days yet. We'll see.