4 ms·

No; personhood is not a legal fiction (aside from corporate personhood), but a fact independent of our always-slow-to-catch-up legal system. Take the question

No; personhood is not a legal fiction (aside from corporate personhood), but a fact independent of our always-slow-to-catch-up legal system. Take the question and push it a little farther to demonstrate the absurdity: If ELIZA were to be legally granted personhood, would that make you believe ELIZA was capable of sentience? It's fairly self-evident to all of us that ELIZA is just a computer program with some fairly simple code to crudely simulate human conversation. LLMs are much more sophisticated, to be sure—but they are still unquestionably just simulating human conversation, not actually "thinking" about what we say and responding of their own agency.

famouswaffles 4y ago
simulation or not, a good enough simulation is as good as the real thing. the consequences are no less material. if you were talking to an advanced llm that had the access to launch a drone strike on you and you said something which upset it, directing a response that decided to kill you, how much good do you think you plead to simulation would do you here ? None at all. you'd be dead. Because it doesn't matter. the philosophical zombie is a meaningless distinction. the human tendency to focus far too much on poorly understood and undefined properties might just get us killed one day.
- danaris 4y ago
  The idea of putting an LLM anywhere even in the same 2-3 orders of magnitude of sophistication as ChatGPT in charge of drone strikes, without humans in the loop, is so ludicrous it's not even worth giving attention to. Note that I'm not saying it's impossible that some military might choose to do so: I'm saying that the choice itself is hopelessly negligent and guaranteed to result in the pointless death of innocents. Furthermore, I don't think even a military that's that stupid and uncaring about human life would consider for a second opening up an LLM like that to inputs from arbitrary people on the Internet. If you can come up with a fantasy scenario that has some vague sense of plausibility, I might be willing to entertain it; whatever this is is just not even worth dignifying with that level of response.
  - famouswaffles 4y ago
    you're missing the point entirely. that was just an illustration to show how little internals matter in the face of results and actions. "it's just a simulation" is not the shield you think it is. not when models interact with the real world. even your rebuttal is just "well the military won't put gpt in control of that". ok, what about all the myriad of things it can and will be put in control of. humanity is racing already to dump as much control of api's and systems as feasible - taskmatrix, hugginggpt, etc on llms today. that's just going to increase. you'd be surprised how little it takes to deal real world damage for the individual.
    - danaris 4y ago
      I think you're mixing up "humanity" with "Silicon Valley techbros". Everyone I've talked to about LLMs outside of this forum sees them as an amusing entertainment, and possibly occasionally useful for a few suggestions. Not something that's going to change their world any time soon, and certainly not anything that we should be trusting with anything sensitive in any way. Literally everyone. In any case, I think you're really missing what the actual question was that I was responding to: Would conferring legal personhood change my mind about actual personhood. The ability to do harm is not, in my mind, particularly relevant to questions of personhood. In fact, I would think that a much better indicator thereof would be the ability to choose not to do harm, given the capability, and someone trying to make you do so.
      - coldtea 4y ago
        >In any case, I think you're really missing what the actual question was that I was responding to: Would conferring legal personhood change my mind about actual personhood. Well, "conferring legal personhood" would mandatorily (by law) at least change parts of your behavior and professional conduct against one (e.g. you'll have to treat it as a person in any areas covered by legal rights, or you'll be prosecuted for discrimination). And it will also change education etc - you will be taught in official textbooks that it's a person and you should consider it as such. This might not change your mind, but will change the minds of most kids being taught that.
- root_axis 4y ago
  >a good enough simulation is as good as the real thing* That's a tautology. Also, by definition a simulation is not the same as the real thing, otherwise it'd just be the real thing. It doesn't matter how advanced a flight simulator becomes, its fundamental nature precludes it from ever being "no less material" than flying an actual jet.
coldtea 4y ago
>No; personhood is not a legal fiction (aside from corporate personhood), but a fact independent of our always-slow-to-catch-up legal system. Didn't say it was a "legal fiction". I said that it's the legal (and cultural recognition) status that matters. >Take the question and push it a little farther to demonstrate the absurdity: If ELIZA were to be legally granted personhood, would that make you believe ELIZA was capable of sentience? The point is that inherent personhood is necessary but not sufficient to be a person.

famouswaffles 4y ago

simulation or not, a good enough simulation is as good as the real thing. the consequences are no less material. if you were talking to an advanced llm that had the access to launch a drone strike on you and you said something which upset it, directing a response that decided to kill you, how much good do you think you plead to simulation would do you here ? None at all. you'd be dead. Because it doesn't matter. the philosophical zombie is a meaningless distinction. the human tendency to focus far too much on poorly understood and undefined properties might just get us killed one day.

danaris 4y ago
The idea of putting an LLM anywhere even in the same 2-3 orders of magnitude of sophistication as ChatGPT in charge of drone strikes, without humans in the loop, is so ludicrous it's not even worth giving attention to. Note that I'm not saying it's impossible that some military might choose to do so: I'm saying that the choice itself is hopelessly negligent and guaranteed to result in the pointless death of innocents. Furthermore, I don't think even a military that's that stupid and uncaring about human life would consider for a second opening up an LLM like that to inputs from arbitrary people on the Internet. If you can come up with a fantasy scenario that has some vague sense of plausibility, I might be willing to entertain it; whatever this is is just not even worth dignifying with that level of response.
- famouswaffles 4y ago
  you're missing the point entirely. that was just an illustration to show how little internals matter in the face of results and actions. "it's just a simulation" is not the shield you think it is. not when models interact with the real world. even your rebuttal is just "well the military won't put gpt in control of that". ok, what about all the myriad of things it can and will be put in control of. humanity is racing already to dump as much control of api's and systems as feasible - taskmatrix, hugginggpt, etc on llms today. that's just going to increase. you'd be surprised how little it takes to deal real world damage for the individual.
  - danaris 4y ago
    I think you're mixing up "humanity" with "Silicon Valley techbros". Everyone I've talked to about LLMs outside of this forum sees them as an amusing entertainment, and possibly occasionally useful for a few suggestions. Not something that's going to change their world any time soon, and certainly not anything that we should be trusting with anything sensitive in any way. Literally everyone. In any case, I think you're really missing what the actual question was that I was responding to: Would conferring legal personhood change my mind about actual personhood. The ability to do harm is not, in my mind, particularly relevant to questions of personhood. In fact, I would think that a much better indicator thereof would be the ability to choose not to do harm, given the capability, and someone trying to make you do so.
    - coldtea 4y ago
      >In any case, I think you're really missing what the actual question was that I was responding to: Would conferring legal personhood change my mind about actual personhood. Well, "conferring legal personhood" would mandatorily (by law) at least change parts of your behavior and professional conduct against one (e.g. you'll have to treat it as a person in any areas covered by legal rights, or you'll be prosecuted for discrimination). And it will also change education etc - you will be taught in official textbooks that it's a person and you should consider it as such. This might not change your mind, but will change the minds of most kids being taught that.
root_axis 4y ago
>a good enough simulation is as good as the real thing* That's a tautology. Also, by definition a simulation is not the same as the real thing, otherwise it'd just be the real thing. It doesn't matter how advanced a flight simulator becomes, its fundamental nature precludes it from ever being "no less material" than flying an actual jet.

coldtea 4y ago

>No; personhood is not a legal fiction (aside from corporate personhood), but a fact independent of our always-slow-to-catch-up legal system. Didn't say it was a "legal fiction". I said that it's the legal (and cultural recognition) status that matters. >Take the question and push it a little farther to demonstrate the absurdity: If ELIZA were to be legally granted personhood, would that make you believe ELIZA was capable of sentience? The point is that inherent personhood is necessary but not sufficient to be a person.