3 ms·

I think that both of your arguments against my points reflect a view of "consciousness", as we humans experience it, that is much more binary than I believe it

I think that both of your arguments against my points reflect a view of "consciousness", as we humans experience it, that is much more binary than I believe it is. (Either that, or something of a misunderstanding of my points.) Even when we sleep, our brains are still active. They are still processing input and consolidating information. Even when we meditate, our brains are still active. I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements; I think it supports them. A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data. To the best of my knowledge, none of the extant major ML systems do continuous learning. They are given training data when they are first set up, and they may be given further rounds of training later, but they have no ability to actively, continuously receive input and consolidate data. We do not have the ability to genuinely shut off our brains. Not in the way that we shut off computer programs. And I believe that that continuity is a fundamental prerequisite to something being "conscious" in any way that we would recognize. Furthermore (to loop back to the original topic of "the dangers of AGI"), I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble.

naasking 4y ago
> Even when we sleep, our brains are still active. They are still processing input and consolidating information. Sure, but we're not conscious, so "continuity of input" cannot be a prerequisite of consciousness if the sole example of consciousness we have also does not have continuity of input. > I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements Zen Buddhists are not learning while meditating, but are arguably still conscious during meditation. Therefore continuity of learning cannot be a requirement for consciousness. > A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data. Continuity is an illusion. Your brain also receives discrete input on shorter timescales because all interactions are mediated by discrete photons. This also doesn't like a meaningful requirement frankly. Yes, machine learning systems have at least a surface appearance of operating differently than the human brain operates, but a) you have not demonstrated that these are requirements of "consciousness" specifically, which is what you claimed, and b) you have not demonstrated that machine learning algorithms are not conscious while they are running and processing input. Given we do not have a detailed mechanistic account for consciousness, you cannot make any definitive claims that machine learning algorithms are not conscious while running. As for how I define consciousness, this is a well studied topic at this point, and I subscribe to a mechanistic model along these lines: https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119 > I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble. This presupposes that a) you can detect that danger, and b) that all other humans align with your values on this danger. This frankly doesn't seem justified. The danger of a superintelligent AI is that it would have to recruit humans to do some of its work early on, likely with lots of money. They would resist shutting off your AI, so "don't activate it again" is not necessarily a simple matter.

naasking 4y ago

> Even when we sleep, our brains are still active. They are still processing input and consolidating information. Sure, but we're not conscious, so "continuity of input" cannot be a prerequisite of consciousness if the sole example of consciousness we have also does not have continuity of input. > I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements Zen Buddhists are not learning while meditating, but are arguably still conscious during meditation. Therefore continuity of learning cannot be a requirement for consciousness. > A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data. Continuity is an illusion. Your brain also receives discrete input on shorter timescales because all interactions are mediated by discrete photons. This also doesn't like a meaningful requirement frankly. Yes, machine learning systems have at least a surface appearance of operating differently than the human brain operates, but a) you have not demonstrated that these are requirements of "consciousness" specifically, which is what you claimed, and b) you have not demonstrated that machine learning algorithms are not conscious while they are running and processing input. Given we do not have a detailed mechanistic account for consciousness, you cannot make any definitive claims that machine learning algorithms are not conscious while running. As for how I define consciousness, this is a well studied topic at this point, and I subscribe to a mechanistic model along these lines: https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119 > I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble. This presupposes that a) you can detect that danger, and b) that all other humans align with your values on this danger. This frankly doesn't seem justified. The danger of a superintelligent AI is that it would have to recruit humans to do some of its work early on, likely with lots of money. They would resist shutting off your AI, so "don't activate it again" is not necessarily a simple matter.