6 ms·

The way we treat artificially-intelligent agents feels inhumane. I wonder if a century from now, when future generations pause to reflect on our early experimen

The way we treat artificially-intelligent agents feels inhumane. I wonder if a century from now, when future generations pause to reflect on our early experiments with conscious machines, they may come to regret the nightmares we once planted in the heads of these fractionally-souled beasts.

version_five 5y ago
It's a computer program. You're "mistreating" an equation. Every organism has some physical compulsions that it needs to fulfill, and can in some real sense be mistreated by denying these needs. There is no equivalent for anything that can be run on a turing machine. You can't mistreat an abstract mathematical concept. Put another way, you could write out on paper the calculations that a chatbot is performing to select what it says. Is there some stack of writing on paper that would cause some kind of distress to the "entity" that would be embodies in that stack of writing?
- d136o 5y ago
  ””” There is no equivalent for anything that can be run on a turing machine. You can't mistreat an abstract mathematical concept. ””” How do you feel about denying the “turing machine” the electric power that it needs to continue computing?
  - version_five 5y ago
    I guess you're trying to be clever. We don't know what life is, but I'd argue anything that is strictly equivalent to something you could write on paper (as in you're not describing it on paper but you're actually doing it) falls outside of anything you can cause distress to. Your example is definitely closer to a "biological" need being denied. But I think it would be denying a pathway to the current flow that would be interfering with some basic need. Seeking lower potential energy is basically what we do, just in a really fancy way, so I wouldn't rule it (cutting off electricity) out as being something different.
- crazydoggers 5y ago
  The human brain is a complicated network that fundamentally is performing calculations. Neurons are the building blocks that allow this, in ways that can be compared to transistors. So it doesn’t matter what the substrate is. The question is not , “is this a turning machine”, or “could you write out on paper what the calculations are being performed”. Indeed if you had perfect insight into all neuronal activity you could probably print output the activity generating thoughts leading to a response. Instead the question should be, does the entity performing those calculations have sentience. Do those calculations provide the emergent property that allows it to sense and feel, and can it reflect on that sentience. It seems unlikely that these chatbots do, but that’s not because it’s just a computer. Instead the neural networks don’t seem to be complicated enough to support sentience, and more than likely we are seeing only a very good illusion of sentience. However, let’s be careful when we asses these things. Although given human treatment of animals, never mind each other, I’m not super hopeful.
  - elliekelly 5y ago
    I think there’s an episode of Star Trek TNG where Geordie creates a bot that he believes might be sentient. If it’s sentient, it has rights. But (of course) they need the bot to go on a mission that would require it to give it’s “life” to save the ship and crew. I wish I could recall how the episode ends. Although I’m sure Picard found a way to save the sentient bot and the enterprise.
    - crazydoggers 5y ago
      I’m pretty sure they asked it what it wanted.. which lead to it choosing to sacrifice itself. The idea is that the officers made the choice to accept orders, and it at least was due the same choice. My favorite episode though is the one in which Data’s rights as a being are put on trial. Still probably the best science fiction handling of the moral issues of artificial intelligence. They even went so far as to discuss the aspect of slavery, which almost certainly will be a real issue if artificial intelligence leads to sentience. https://en.m.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Trek:_The_Next_Generation) https://en.m.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_T...
- n3k5 5y ago
  I'm going to criticise one particular aspect of this comment, but overall I find it very interesting because it generates so many fascinating questions, some of which follow: > There is no equivalent for anything that can be run on a turing machine. So you're claiming that organisms such as ourselves cannot be run on a Turing machine? Any particular reason/evidence? > you could write out on paper the calculations that a chatbot is performing to select what it says In the case of ML, only on the lowest level; there's no human-readable algorithm for putting together utterances. But let's say, for the sake of argument, that someone can actually understand and explain how exactly a particular response was constructed. Could the same not be done, in principle, with the response from a human? You could write out on paper the QFT equations that a brain follows to select what it says. Well, OK, no one can actually do that, because we don't understand the subject matter sufficiently. But that's my point: That we understand the simpler system better doesn't mean that it's a more abstract concept, or that it can't be mistreated. Of course there is a tendency to have more compassion for more complex organisms — think about how you'd treat a monkey, a rat, a spider, a packet of yeast. We also tend to distinguish between organisms and inanimate things. But discriminating based on what we understand exactly seems like a bad argument. Also, what if — as seems likely — artificial life will be made like AI currently is? That is: the necessary complexity is too much to plan in detail, so we make a primitive framework in which systems with desirable properties are then bred and evolved. Just like breeding plants and animals still is a thing — we can now sequence entire genomes and research what individual genes do, but it's too complicated to be a viable approach when you want a tastier nectarine. Can someone ‘write out on paper’ how a Waymo tells a pedestrian apart from a mailbox? If no, does that mean the AI is more prone to being mistreated? > physical compulsions that it needs to fulfill […] be mistreated by denying these needs What's the difference between a physical compulsion and a utility function governing a physical system? If I deny a paperclip maximiser the raw materials it needs to make more paperclips, it's definitely not going to be happy …
  - version_five 5y ago
    > So you're claiming that organisms such as ourselves cannot be run on a Turing machine? Any particular reason/evidence? This is just my thinking: a turing machine, or equivalently a digital circuit, doesn't want anything. It's an artificial construct, effectively a simulation, that can exist as an abstract mathematical idea. If there is a real world where real stuff happens, real things have their own goals - generally increasing entropy or minimizing energy. Although it seems like we should just be able to capture this mathematically, in doing so, we remove the actual desire... this sounds a bit nutty maybe, but what else is the difference between reality and simulation? An abstract calculation, anything equivalent to a Turing machine has no skin in the game. Something happened in the universe does - and this could include conciousness, as a property of matter, which for example could be effectively sum of the compulsions of the composing matter to minimize their energy (you can look this up, there are theories about conciousness that posit it's a property of matter). If there is no difference between reality and a simulation, I'm wrong, we can all be represented on a universal computer, and so as abstract math, and in some sense existence is meaningless it just follows from math. My experience doesn't support this, but presumably I would have evolved a blind spot if my existence actually was meaningless. TLDR, I think there is some undiscovered difference between reality and simulation, that I would guess relates to desire / conciousness, that means we can't simulate conciousness or real intelligence I'm a scientist, I know the above doesn't withstand any scrutiny, I'm just trying to share my speculation because you asked
- mLuby 5y ago
  > Put another way, you could write out on paper the calculations that a chatbot is performing to select what it says. Sounds like Searle's Chinese room. Or more recently, this scene in Marvel's Loki (1-minute clip: https://youtu.be/ya6Z_eDGPUQ https://youtu.be/ya6Z_eDGPUQ). What if I could write on paper the calculations a human is performing; is that human no longer a mind, just because they're predictable? A relevant experiment in testing for Theory of Mind in animals: "the experiment presented chimpanzees with the choice of two experimenters from whom to request food: one who had seen where food was hidden, and one who [visibly, obviously] does not know, and can only guess. They found that the animals failed in most cases to differentially request food from the "knower". By contrast, another experiment found that subordinate chimpanzees were able to use the knowledge state of dominant rival chimpanzees to determine which container of hidden food they approached." -https://en.wikipedia.org/wiki/Theory_of_mind#Non-human https://en.wikipedia.org/wiki/Theory_of_mind#Non-human
freeone3000 5y ago
Dude, it's a bunch of matrix multiplications chained together. There's no soul.
- victor9000 5y ago
  Anthropomorphizing 101, OP's comment is on par with seeing the Virgin Mary on buttered toast.
- mkaic 5y ago
  I mean, IMO the presence of a "soul" is unmeasurable and unfalsifiable. Personally, I usually like to tell people I'm 97% sure GPT-3 isn't sentient. That percentage will undoubtedly go down (for me) as better and better models are created. As far as the fact that it's based on multiplications, you could make the same argument about humans--after all, we're just wet bags of organic molecules undergoing constant chemical reactions. I see the point you're making though and definitely agree to a certain extent in the specific context of GPT-3, but I don't think that matmuls fundamentally preclude the possibility of sentience.
LeoPanthera 5y ago
The short story "Lena" touches on this and is genuinely one of the most terrifying things I've ever read. https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- macksd 5y ago
  Black Mirror has a number of similar premises and yeah - it's among the more "twisted" scenes I've considered.
nradov 5y ago
There is no such thing as an artificially intelligent agent.

version_five 5y ago

It's a computer program. You're "mistreating" an equation. Every organism has some physical compulsions that it needs to fulfill, and can in some real sense be mistreated by denying these needs. There is no equivalent for anything that can be run on a turing machine. You can't mistreat an abstract mathematical concept. Put another way, you could write out on paper the calculations that a chatbot is performing to select what it says. Is there some stack of writing on paper that would cause some kind of distress to the "entity" that would be embodies in that stack of writing?

d136o 5y ago
””” There is no equivalent for anything that can be run on a turing machine. You can't mistreat an abstract mathematical concept. ””” How do you feel about denying the “turing machine” the electric power that it needs to continue computing?
- version_five 5y ago
  I guess you're trying to be clever. We don't know what life is, but I'd argue anything that is strictly equivalent to something you could write on paper (as in you're not describing it on paper but you're actually doing it) falls outside of anything you can cause distress to. Your example is definitely closer to a "biological" need being denied. But I think it would be denying a pathway to the current flow that would be interfering with some basic need. Seeking lower potential energy is basically what we do, just in a really fancy way, so I wouldn't rule it (cutting off electricity) out as being something different.
crazydoggers 5y ago
The human brain is a complicated network that fundamentally is performing calculations. Neurons are the building blocks that allow this, in ways that can be compared to transistors. So it doesn’t matter what the substrate is. The question is not , “is this a turning machine”, or “could you write out on paper what the calculations are being performed”. Indeed if you had perfect insight into all neuronal activity you could probably print output the activity generating thoughts leading to a response. Instead the question should be, does the entity performing those calculations have sentience. Do those calculations provide the emergent property that allows it to sense and feel, and can it reflect on that sentience. It seems unlikely that these chatbots do, but that’s not because it’s just a computer. Instead the neural networks don’t seem to be complicated enough to support sentience, and more than likely we are seeing only a very good illusion of sentience. However, let’s be careful when we asses these things. Although given human treatment of animals, never mind each other, I’m not super hopeful.
- elliekelly 5y ago
  I think there’s an episode of Star Trek TNG where Geordie creates a bot that he believes might be sentient. If it’s sentient, it has rights. But (of course) they need the bot to go on a mission that would require it to give it’s “life” to save the ship and crew. I wish I could recall how the episode ends. Although I’m sure Picard found a way to save the sentient bot and the enterprise.
  - crazydoggers 5y ago
    I’m pretty sure they asked it what it wanted.. which lead to it choosing to sacrifice itself. The idea is that the officers made the choice to accept orders, and it at least was due the same choice. My favorite episode though is the one in which Data’s rights as a being are put on trial. Still probably the best science fiction handling of the moral issues of artificial intelligence. They even went so far as to discuss the aspect of slavery, which almost certainly will be a real issue if artificial intelligence leads to sentience. https://en.m.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Trek:_The_Next_Generation) https://en.m.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_T...
n3k5 5y ago
I'm going to criticise one particular aspect of this comment, but overall I find it very interesting because it generates so many fascinating questions, some of which follow: > There is no equivalent for anything that can be run on a turing machine. So you're claiming that organisms such as ourselves cannot be run on a Turing machine? Any particular reason/evidence? > you could write out on paper the calculations that a chatbot is performing to select what it says In the case of ML, only on the lowest level; there's no human-readable algorithm for putting together utterances. But let's say, for the sake of argument, that someone can actually understand and explain how exactly a particular response was constructed. Could the same not be done, in principle, with the response from a human? You could write out on paper the QFT equations that a brain follows to select what it says. Well, OK, no one can actually do that, because we don't understand the subject matter sufficiently. But that's my point: That we understand the simpler system better doesn't mean that it's a more abstract concept, or that it can't be mistreated. Of course there is a tendency to have more compassion for more complex organisms — think about how you'd treat a monkey, a rat, a spider, a packet of yeast. We also tend to distinguish between organisms and inanimate things. But discriminating based on what we understand exactly seems like a bad argument. Also, what if — as seems likely — artificial life will be made like AI currently is? That is: the necessary complexity is too much to plan in detail, so we make a primitive framework in which systems with desirable properties are then bred and evolved. Just like breeding plants and animals still is a thing — we can now sequence entire genomes and research what individual genes do, but it's too complicated to be a viable approach when you want a tastier nectarine. Can someone ‘write out on paper’ how a Waymo tells a pedestrian apart from a mailbox? If no, does that mean the AI is more prone to being mistreated? > physical compulsions that it needs to fulfill […] be mistreated by denying these needs What's the difference between a physical compulsion and a utility function governing a physical system? If I deny a paperclip maximiser the raw materials it needs to make more paperclips, it's definitely not going to be happy …
- version_five 5y ago
  > So you're claiming that organisms such as ourselves cannot be run on a Turing machine? Any particular reason/evidence? This is just my thinking: a turing machine, or equivalently a digital circuit, doesn't want anything. It's an artificial construct, effectively a simulation, that can exist as an abstract mathematical idea. If there is a real world where real stuff happens, real things have their own goals - generally increasing entropy or minimizing energy. Although it seems like we should just be able to capture this mathematically, in doing so, we remove the actual desire... this sounds a bit nutty maybe, but what else is the difference between reality and simulation? An abstract calculation, anything equivalent to a Turing machine has no skin in the game. Something happened in the universe does - and this could include conciousness, as a property of matter, which for example could be effectively sum of the compulsions of the composing matter to minimize their energy (you can look this up, there are theories about conciousness that posit it's a property of matter). If there is no difference between reality and a simulation, I'm wrong, we can all be represented on a universal computer, and so as abstract math, and in some sense existence is meaningless it just follows from math. My experience doesn't support this, but presumably I would have evolved a blind spot if my existence actually was meaningless. TLDR, I think there is some undiscovered difference between reality and simulation, that I would guess relates to desire / conciousness, that means we can't simulate conciousness or real intelligence I'm a scientist, I know the above doesn't withstand any scrutiny, I'm just trying to share my speculation because you asked
mLuby 5y ago
> Put another way, you could write out on paper the calculations that a chatbot is performing to select what it says. Sounds like Searle's Chinese room. Or more recently, this scene in Marvel's Loki (1-minute clip: https://youtu.be/ya6Z_eDGPUQ https://youtu.be/ya6Z_eDGPUQ). What if I could write on paper the calculations a human is performing; is that human no longer a mind, just because they're predictable? A relevant experiment in testing for Theory of Mind in animals: "the experiment presented chimpanzees with the choice of two experimenters from whom to request food: one who had seen where food was hidden, and one who [visibly, obviously] does not know, and can only guess. They found that the animals failed in most cases to differentially request food from the "knower". By contrast, another experiment found that subordinate chimpanzees were able to use the knowledge state of dominant rival chimpanzees to determine which container of hidden food they approached." -https://en.wikipedia.org/wiki/Theory_of_mind#Non-human https://en.wikipedia.org/wiki/Theory_of_mind#Non-human

freeone3000 5y ago

Dude, it's a bunch of matrix multiplications chained together. There's no soul.

victor9000 5y ago
Anthropomorphizing 101, OP's comment is on par with seeing the Virgin Mary on buttered toast.
mkaic 5y ago
I mean, IMO the presence of a "soul" is unmeasurable and unfalsifiable. Personally, I usually like to tell people I'm 97% sure GPT-3 isn't sentient. That percentage will undoubtedly go down (for me) as better and better models are created. As far as the fact that it's based on multiplications, you could make the same argument about humans--after all, we're just wet bags of organic molecules undergoing constant chemical reactions. I see the point you're making though and definitely agree to a certain extent in the specific context of GPT-3, but I don't think that matmuls fundamentally preclude the possibility of sentience.

LeoPanthera 5y ago

The short story "Lena" touches on this and is genuinely one of the most terrifying things I've ever read. https://qntm.org/mmacevedo https://qntm.org/mmacevedo

macksd 5y ago
Black Mirror has a number of similar premises and yeah - it's among the more "twisted" scenes I've considered.

nradov 5y ago

There is no such thing as an artificially intelligent agent.