4 ms·

I think the fact that it's difficult to even conceive of how to interact with a dinner party as an entity illustrates the point. It would actually be easier to

I think the fact that it's difficult to even conceive of how to interact with a dinner party as an entity illustrates the point. It would actually be easier to assess the sentience of an insect. Rather than a person, it makes more sense to think of human organizations as organic colonies, like jellyfish. They have various degrees of structure, and collective behavior that can be observed. They are clearly independent entities in some sense. But there's no evidence of sentience or consciousness on the entity level. Sure, you can get intelligent answers from the CEO, and I think this is what trips people up. The CEO is not the company, merely a component of the company. Same goes for any individual dinner guest. Perhaps there is some kind of experiment that could be rigged where coded information could be supplied to the dinner party in a manner ensuring that the participants were not aware of it, then the response read out by decoding the resulting activity. And if that yielded some kind of intelligent response indicating self-awareness and self-direction, I could be persuaded. But I would also be persuaded that the moon is made of green cheese if multiple lunar sampling missions came back with green cheese. i don't anticipate either experiment succeeding.

kelseyfrog 1y ago
Consider a Chinese room type setup. The organization exists inside and one is able to communicate by passing letters. This highlights some of our preconceptions about what it means to communicate with an organization. We have a lot of hidden assumptions here. One that I started out having but discarded is that every constituent part of the org inside must contribute to the response. But then I consider myself as a collection of cells and know my foot doesn't exactly contribute to this message. So, equality of contribution is not a salient scissor.
- vannevar 1y ago
  The Chinese room setup just hides the problem behind a facade, it does not solve the problem. Which is that as soon as you communicate to someone in the dinner party that you're running the experiment, it ceases to be a dinner party. It is certainly true that every constituent part of the colony doesn't need to contribute to any particular action to demonstrate sentience, any more than to demonstrate locomotion. But in the case of humans, an individual human is so clearly sentient and conscious that any experiment would be completely confounded if it involved communicating directly and consciously with any individual human. In your Chinese room, any sentience could just as easily be explained by one of the parties simply communicating directly with the experimenter as it could be by group sentience, and Occam's Razor would apply.
  - kelseyfrog 1y ago
    A thing can be a dinner party and an experiment. Awareness of the experiment doesn't erase the dinner party, just as awareness of an experimenter doesn't erase the fact that people are still eating together. Both categories can overlap without contradiction. We do this all the time in our daily lives. Just the same, I can still be me and participate in an experiment. My identity doesn't collapse into “only subject” when I'm studied, and the same continuity of self applies to someone in the Chinese room. The role doesn't erase the person. Occam's Razor is a heuristic, not a law. Applied too bluntly, it favors the easiest seeming explanation even when the phenomenon, like consciousness, is exactly the kind where every explanation comes with complexity, making "simplicity" a misleading standard. Occam's razor in social situations biases toward implicit social assumptions that smuggle their way into the conclusion without accounting for their contribution toward complexity. Finally, why should a person communicating from the Chinese room sometimes be treated as simply themselves and other times as only a mouthpiece of the system? Nothing in the setup changes about who they are or what they're doing. Once we allow that kind of shifting interpretation, any conclusion about group sentience is dead before it begins.
    - vannevar 1y ago
      >Finally, why should a person communicating from the Chinese room sometimes be treated as simply themselves and other times as only a mouthpiece of the system? If one of the participants simply responds to queries from outside the room with their own thoughts, how does that show sentience for the dinner party as a system? No offense, but you are illustrating exactly the confusion I was referring to in my original comment. The tendency is to assume that the whole is as sentient as the parts, and in the case of human organizations, that is not true. I think the short explanation is that the human participation is highly constrained---the members are employing a very small subset of their human potential to play their limited role in the organization. And the communication rate between these "slivers of a human" is much lower than that between the components of a single human mind. So the emergent behavior of the system as a whole is much simpler than the operation of a single human mind.

kelseyfrog 1y ago

Consider a Chinese room type setup. The organization exists inside and one is able to communicate by passing letters. This highlights some of our preconceptions about what it means to communicate with an organization. We have a lot of hidden assumptions here. One that I started out having but discarded is that every constituent part of the org inside must contribute to the response. But then I consider myself as a collection of cells and know my foot doesn't exactly contribute to this message. So, equality of contribution is not a salient scissor.

vannevar 1y ago
The Chinese room setup just hides the problem behind a facade, it does not solve the problem. Which is that as soon as you communicate to someone in the dinner party that you're running the experiment, it ceases to be a dinner party. It is certainly true that every constituent part of the colony doesn't need to contribute to any particular action to demonstrate sentience, any more than to demonstrate locomotion. But in the case of humans, an individual human is so clearly sentient and conscious that any experiment would be completely confounded if it involved communicating directly and consciously with any individual human. In your Chinese room, any sentience could just as easily be explained by one of the parties simply communicating directly with the experimenter as it could be by group sentience, and Occam's Razor would apply.
- kelseyfrog 1y ago
  A thing can be a dinner party and an experiment. Awareness of the experiment doesn't erase the dinner party, just as awareness of an experimenter doesn't erase the fact that people are still eating together. Both categories can overlap without contradiction. We do this all the time in our daily lives. Just the same, I can still be me and participate in an experiment. My identity doesn't collapse into “only subject” when I'm studied, and the same continuity of self applies to someone in the Chinese room. The role doesn't erase the person. Occam's Razor is a heuristic, not a law. Applied too bluntly, it favors the easiest seeming explanation even when the phenomenon, like consciousness, is exactly the kind where every explanation comes with complexity, making "simplicity" a misleading standard. Occam's razor in social situations biases toward implicit social assumptions that smuggle their way into the conclusion without accounting for their contribution toward complexity. Finally, why should a person communicating from the Chinese room sometimes be treated as simply themselves and other times as only a mouthpiece of the system? Nothing in the setup changes about who they are or what they're doing. Once we allow that kind of shifting interpretation, any conclusion about group sentience is dead before it begins.
  - vannevar 1y ago
    >Finally, why should a person communicating from the Chinese room sometimes be treated as simply themselves and other times as only a mouthpiece of the system? If one of the participants simply responds to queries from outside the room with their own thoughts, how does that show sentience for the dinner party as a system? No offense, but you are illustrating exactly the confusion I was referring to in my original comment. The tendency is to assume that the whole is as sentient as the parts, and in the case of human organizations, that is not true. I think the short explanation is that the human participation is highly constrained---the members are employing a very small subset of their human potential to play their limited role in the organization. And the communication rate between these "slivers of a human" is much lower than that between the components of a single human mind. So the emergent behavior of the system as a whole is much simpler than the operation of a single human mind.