3 ms·

I see your point and agree that it would be extraneous to model awareness for a biomachine whose purpose is efficiency. But what of the inevitable research that

I see your point and agree that it would be extraneous to model awareness for a biomachine whose purpose is efficiency. But what of the inevitable research that will expressly attempt to create AGI using said unaware computation paste?

bckr 5y ago
Yeah, that's a concern to a degree. But I'll justify why I'm not really worried about it in two ways. 1) The computations we want to do may still be expressed better without consciousness. Think Chinese Room thought experiment, or philosophical zombies. In other words, we may be able to make something that can understand and execute complex commands, perceive and react to its environment, and communicate information — without having feelings, desires, or consciousness per se. 2) I'm less worried about creating new things to be unethical to than I am about the dis-ethical treatment of currently sentient beings.
- netizen-936824 5y ago
  Perhaps this sort of research would further our understanding of what sentience really is
- ben_w 5y ago
  > In other words, we may be able to make something that can understand and execute complex commands, perceive and react to its environment, and communicate information — without having feelings, desires, or consciousness per se. > 2) I'm less worried about creating new things to be unethical to than I am about the dis-ethical treatment of currently sentient beings. Both of these relate to one of my major concerns: We don’t yet have a testable definition of “feelings, desires, or consciousness” [0] such that we could even tell if an AI did or did not have this characteristic. [0] and any of the other various words frequently thrown around as attempts to create such a definition. Every time I make this claim I get responses along the lines of “yes we do: $foo!”, but in my experience so far, all of these turn out to be one of (1) circular definitions, or (2) things humans don’t have, or (3) things which plants and the enemy characters in Wolfenstein 3D do have.
  - bckr 5y ago
    > (1) circular definitions, or (2) things humans don’t have, or (3) things which plants and the enemy characters in Wolfenstein 3D do have. I'd be curious to read more about these 3 arguments and their flaws.
  - jstanley 5y ago
    I'd define consciousness as something like "subjective experience". You have consciousness if you can experience the experiencing of things. I don't think that's a circular definition. I think humans have it. I don't think plants or video game characters have it. However: it is not testable, and it's not obvious that it could be. If someone claims to have subjective experiences, I don't think there's any way to know whether they're conscious or whether they're just an automata acting conscious. If consciousness is just being able to experience experiences, then that's a function that doesn't have any outputs, how could you possibly test for it? And this applies equally to humans, c.f. solipsism. It could also apply to all matter in the universe, it's just that most of that matter has no computational capability, and no way to express itself even if it did.
    - ben_w 5y ago
      > I'd define consciousness as something like "subjective experience". You have consciousness if you can experience the experiencing of things > However: it is not testable, and it's not obvious that it could be. If it isn’t testable, do you really have a definition rather than a tautology? What does it mean to have “subjective experience”? One instance of a game AI is subject to different inputs than another instance of the same AI, which you reject (as would I: I listed it as an example of how bad the definitions have been and definitely not as a claim they’re sentient).
      - jstanley 5y ago
        It is perfectly consistent for something to be well-defined and also untestable. For example, you could ask me any question about the contents of my mind. Let's say you ask me if my favourite colour is green, and I say it is. You don't know whether my favourite colour really is green or if I'm lying. The fact that you have no way to ascertain my true favourite colour doesn't mean favourite colours are ill-defined! You just can't know what anyone else's favourite colour is. (Not that I'm claiming my "subjective experience" definition is well-defined -- I accept that it's pretty vague as definitions go).

bckr 5y ago

Yeah, that's a concern to a degree. But I'll justify why I'm not really worried about it in two ways. 1) The computations we want to do may still be expressed better without consciousness. Think Chinese Room thought experiment, or philosophical zombies. In other words, we may be able to make something that can understand and execute complex commands, perceive and react to its environment, and communicate information — without having feelings, desires, or consciousness per se. 2) I'm less worried about creating new things to be unethical to than I am about the dis-ethical treatment of currently sentient beings.

netizen-936824 5y ago
Perhaps this sort of research would further our understanding of what sentience really is
ben_w 5y ago
> In other words, we may be able to make something that can understand and execute complex commands, perceive and react to its environment, and communicate information — without having feelings, desires, or consciousness per se. > 2) I'm less worried about creating new things to be unethical to than I am about the dis-ethical treatment of currently sentient beings. Both of these relate to one of my major concerns: We don’t yet have a testable definition of “feelings, desires, or consciousness” [0] such that we could even tell if an AI did or did not have this characteristic. [0] and any of the other various words frequently thrown around as attempts to create such a definition. Every time I make this claim I get responses along the lines of “yes we do: $foo!”, but in my experience so far, all of these turn out to be one of (1) circular definitions, or (2) things humans don’t have, or (3) things which plants and the enemy characters in Wolfenstein 3D do have.
- bckr 5y ago
  > (1) circular definitions, or (2) things humans don’t have, or (3) things which plants and the enemy characters in Wolfenstein 3D do have. I'd be curious to read more about these 3 arguments and their flaws.
- jstanley 5y ago
  I'd define consciousness as something like "subjective experience". You have consciousness if you can experience the experiencing of things. I don't think that's a circular definition. I think humans have it. I don't think plants or video game characters have it. However: it is not testable, and it's not obvious that it could be. If someone claims to have subjective experiences, I don't think there's any way to know whether they're conscious or whether they're just an automata acting conscious. If consciousness is just being able to experience experiences, then that's a function that doesn't have any outputs, how could you possibly test for it? And this applies equally to humans, c.f. solipsism. It could also apply to all matter in the universe, it's just that most of that matter has no computational capability, and no way to express itself even if it did.
  - ben_w 5y ago
    > I'd define consciousness as something like "subjective experience". You have consciousness if you can experience the experiencing of things > However: it is not testable, and it's not obvious that it could be. If it isn’t testable, do you really have a definition rather than a tautology? What does it mean to have “subjective experience”? One instance of a game AI is subject to different inputs than another instance of the same AI, which you reject (as would I: I listed it as an example of how bad the definitions have been and definitely not as a claim they’re sentient).
    - jstanley 5y ago
      It is perfectly consistent for something to be well-defined and also untestable. For example, you could ask me any question about the contents of my mind. Let's say you ask me if my favourite colour is green, and I say it is. You don't know whether my favourite colour really is green or if I'm lying. The fact that you have no way to ascertain my true favourite colour doesn't mean favourite colours are ill-defined! You just can't know what anyone else's favourite colour is. (Not that I'm claiming my "subjective experience" definition is well-defined -- I accept that it's pretty vague as definitions go).