3 ms·

You don't need to have a deep and complex understanding of human sentience to be able to put some lower bounds on what it means to be sentient. I don't know ab

• naasking 4y ago
  > Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli. Why can't it be conscious merely while the computation is running? You aren't constantly on, you're ostensibly only conscious 16-20 hours every day, while you're not sleeping. > Continuity of learning: In addition, the AI needs to be continually update its "mental model" of the world—in effect, constantly "learning" and re-training its neural network on the input it receives. I'm not sure what learning has to do with consciousness, and I'm absolutely certain you don't either. I frankly find it hard to believe that Zen Buddhists are not conscious during their meditations, despite having empty minds and not "constantly updating their mental models of the world-in-effect". > But I, personally, believe they are both necessary for it to be even worth starting to consider whether a given AI might be. And I believe that consciousness is a prerequisite for sentience. See, this is what I'm talking about. That's a fine personal belief, but that belief does not rise to the level of expert testimony of the facts even if you are a machine learning expert. People have made all sorts of assumptions about the human mind that they have never questioned but take as obvious fact, and then pontificate that AI is clearly not doing this, therefore AI cannot share such and such properties with humans. But the fact is, most of those assumptions are just wrong. Without a mechanistic model of how these systems work in the brain we are not going to make a valid comparison against what AI is doing. I agree in general that we can set some bounds, but those bounds are neither as firm as, nor are they placed where most people think.
  • danaris 4y ago
    I think that both of your arguments against my points reflect a view of "consciousness", as we humans experience it, that is much more binary than I believe it is. (Either that, or something of a misunderstanding of my points.) Even when we sleep, our brains are still active. They are still processing input and consolidating information. Even when we meditate, our brains are still active. I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements; I think it supports them. A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data. To the best of my knowledge, none of the extant major ML systems do continuous learning. They are given training data when they are first set up, and they may be given further rounds of training later, but they have no ability to actively, continuously receive input and consolidate data. We do not have the ability to genuinely shut off our brains. Not in the way that we shut off computer programs. And I believe that that continuity is a fundamental prerequisite to something being "conscious" in any way that we would recognize. Furthermore (to loop back to the original topic of "the dangers of AGI"), I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble.
    • naasking 4y ago
      > Even when we sleep, our brains are still active. They are still processing input and consolidating information. Sure, but we're not conscious, so "continuity of input" cannot be a prerequisite of consciousness if the sole example of consciousness we have also does not have continuity of input. > I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements Zen Buddhists are not learning while meditating, but are arguably still conscious during meditation. Therefore continuity of learning cannot be a requirement for consciousness. > A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data. Continuity is an illusion. Your brain also receives discrete input on shorter timescales because all interactions are mediated by discrete photons. This also doesn't like a meaningful requirement frankly. Yes, machine learning systems have at least a surface appearance of operating differently than the human brain operates, but a) you have not demonstrated that these are requirements of "consciousness" specifically, which is what you claimed, and b) you have not demonstrated that machine learning algorithms are not conscious while they are running and processing input. Given we do not have a detailed mechanistic account for consciousness, you cannot make any definitive claims that machine learning algorithms are not conscious while running. As for how I define consciousness, this is a well studied topic at this point, and I subscribe to a mechanistic model along these lines: https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119 > I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble. This presupposes that a) you can detect that danger, and b) that all other humans align with your values on this danger. This frankly doesn't seem justified. The danger of a superintelligent AI is that it would have to recruit humans to do some of its work early on, likely with lots of money. They would resist shutting off your AI, so "don't activate it again" is not necessarily a simple matter.