> Even when we sleep, our brains are still active. They are still processing input and consolidating information. Sure, but we're not conscious, so "continuity

> Even when we sleep, our brains are still active. They are still processing input and consolidating information. Sure, but we're not conscious, so "continuity of input" cannot be a prerequisite of consciousness if the sole example of consciousness we have also does not have continuity of input. > I don't know why you think Zen Buddhists being conscious during meditations in any way refutes my statements Zen Buddhists are not learning while meditating, but are arguably still conscious during meditation. Therefore continuity of learning cannot be a requirement for consciousness. > A system like GPT-3 never receives continuous input. It receives discrete input: one prompt, then another; a static dump of training data. Continuity is an illusion. Your brain also receives discrete input on shorter timescales because all interactions are mediated by discrete photons. This also doesn't like a meaningful requirement frankly. Yes, machine learning systems have at least a surface appearance of operating differently than the human brain operates, but a) you have not demonstrated that these are requirements of "consciousness" specifically, which is what you claimed, and b) you have not demonstrated that machine learning algorithms are not conscious while they are running and processing input. Given we do not have a detailed mechanistic account for consciousness, you cannot make any definitive claims that machine learning algorithms are not conscious while running. As for how I define consciousness, this is a well studied topic at this point, and I subscribe to a mechanistic model along these lines: https://www.pnas.org/doi/10.1073/pnas.2116933119 https://www.pnas.org/doi/10.1073/pnas.2116933119 > I have absolutely no doubt that any AI that we could create that lacks continuity would never have the ability to represent a serious threat to us, because all we'd have to do to be safe from it is to not activate it again as soon as we detect trouble. This presupposes that a) you can detect that danger, and b) that all other humans align with your values on this danger. This frankly doesn't seem justified. The danger of a superintelligent AI is that it would have to recruit humans to do some of its work early on, likely with lots of money. They would resist shutting off your AI, so "don't activate it again" is not necessarily a simple matter.