19 ms·

Imagine a hypothetical black box that could correctly answer any question you ask it and perform any task that you instruct it to perform. In terms of the impa

Imagine a hypothetical black box that could correctly answer any question you ask it and perform any task that you instruct it to perform. In terms of the impact such a thing would have on the world, would it matter if it were conscious? Would it even be desirable for it to be conscious? IMO, discussions of consciousness and self-awareness are a complete red herring when it comes to the topic of AGI.

danaris 3y ago
Two things: 1) They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude. 2) I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness. Certainly, the "singularity" that many people talk about as being either an inevitability or an actual goal seems impossible without an AGI that can self-analyze and self-improve, which seem, at least to me, like they probably require consciousness, at least at the level being talked about.
- pessimizer 3y ago
  > They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude. Don't silence people, answer them. Say why it's actually important. Don't say that by saying "when you say it's not important, you're disrespecting the people who say it's important." That kind of thinking will take us back to the middle ages. > I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness. The meaning of the word "AGI" is not an interesting thing to talk about. Call it what you want. If there's a conventional meaning to the word AGI that you're using, explain it and explain why an algorithm that isn't constantly running fails that. If you do this using the word "consciousness," you're passing the buck. You might as well be talking about souls for all the precision that the word "consciousness" has. > seems impossible without an AGI that can self-analyze and self-improve Thought experiment. What if you yourself aren't an AGI that can self-analyze and improve? What if you are two not-AGIs (by your definition): 1) a learner that improves the other based on new information, and 2) a controller that produces output, And that they switch roles between 1 and 30 times per second, based on the intensity of the input. Would that mean you weren't conscious?
YeGoblynQueenne 3y ago
Well, imagine a model of the apparent motion of the planets where each planet moved on a perfectly circular orbit with any number of smaller circles on top of it, with the main circle of the orbit centered on a point between the Earth, or even the Sun, and another point a bit further from it. So, you know, an epicyclical model of the motion of the planets [1]. "In terms of the impact such a thing would have on the world", as you put it, would it matter if that model was completely wrong, despite its great predictive power? Did we gain something when we figured out how the planets really move? _______________ [1] https://en.wikipedia.org/wiki/Deferent_and_epicycle https://en.wikipedia.org/wiki/Deferent_and_epicycle
- famouswaffles 3y ago
  "Wrong" doesn't make much sense here. The more inductive bias we think is appropriate which we try to shove into models, the worse they perform. There's also an awful lot of fabrication the brain does with sense data, rationales etc. all of this is to say we have no clue what makes us tick. This means there's absolutely no guarantee we would recognize a replication of "human" Something different is not necessarily "wrong". a plane's flight is no less "true" than a bird. It's not flying the "wrong" way. Trying to elevate our very poor and wrong understanding of "human" to be the same as "right" or "true" is very silly. Even Biology with its set of constraints does not always solve the same problem the same way. Who are you to dub one way "right" ? Makes no sense.

danaris 3y ago

Two things: 1) They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude. 2) I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness. Certainly, the "singularity" that many people talk about as being either an inevitability or an actual goal seems impossible without an AGI that can self-analyze and self-improve, which seem, at least to me, like they probably require consciousness, at least at the level being talked about.

pessimizer 3y ago
> They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude. Don't silence people, answer them. Say why it's actually important. Don't say that by saying "when you say it's not important, you're disrespecting the people who say it's important." That kind of thinking will take us back to the middle ages. > I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness. The meaning of the word "AGI" is not an interesting thing to talk about. Call it what you want. If there's a conventional meaning to the word AGI that you're using, explain it and explain why an algorithm that isn't constantly running fails that. If you do this using the word "consciousness," you're passing the buck. You might as well be talking about souls for all the precision that the word "consciousness" has. > seems impossible without an AGI that can self-analyze and self-improve Thought experiment. What if you yourself aren't an AGI that can self-analyze and improve? What if you are two not-AGIs (by your definition): 1) a learner that improves the other based on new information, and 2) a controller that produces output, And that they switch roles between 1 and 30 times per second, based on the intensity of the input. Would that mean you weren't conscious?

YeGoblynQueenne 3y ago

Well, imagine a model of the apparent motion of the planets where each planet moved on a perfectly circular orbit with any number of smaller circles on top of it, with the main circle of the orbit centered on a point between the Earth, or even the Sun, and another point a bit further from it. So, you know, an epicyclical model of the motion of the planets [1]. "In terms of the impact such a thing would have on the world", as you put it, would it matter if that model was completely wrong, despite its great predictive power? Did we gain something when we figured out how the planets really move? _______________ [1] https://en.wikipedia.org/wiki/Deferent_and_epicycle https://en.wikipedia.org/wiki/Deferent_and_epicycle

famouswaffles 3y ago
"Wrong" doesn't make much sense here. The more inductive bias we think is appropriate which we try to shove into models, the worse they perform. There's also an awful lot of fabrication the brain does with sense data, rationales etc. all of this is to say we have no clue what makes us tick. This means there's absolutely no guarantee we would recognize a replication of "human" Something different is not necessarily "wrong". a plane's flight is no less "true" than a bird. It's not flying the "wrong" way. Trying to elevate our very poor and wrong understanding of "human" to be the same as "right" or "true" is very silly. Even Biology with its set of constraints does not always solve the same problem the same way. Who are you to dub one way "right" ? Makes no sense.