4 ms·

> Extraordinary claims require extraordinary evidence. The evidence can go two ways. Essentially the hypothesis that anything we observe to be conscious is ac

> Extraordinary claims require extraordinary evidence. The evidence can go two ways. Essentially the hypothesis that anything we observe to be conscious is actually conscious is unfalsifiable unless you make some assumptions. > We have no evidence for thinking or consciousness in rocks, stars, or computers. We have no evidence for consciousness in slime molds, but they do indeed display intelligence. Crucially, though, and more to the particular point I'm making, we have no evidence that they are not conscious. In fact, we have no idea how consciousness arises, but we are very much primed to believe it needs to look something like us. > (but large financial incentives). That's definitely not my motivation here. I'm talking about this weird and unfounded exceptionalism that we place on eukaryotes being a prerequisite for thinking and consciousness. And, there's actually large financial incentive to say specifically that AI can think but is not conscious, because it is somehow distinct from life. This could inadvertently spawn a new wave of slavery that everyone (well, 'most people') would be perfectly complicit in and comfortable with, and to the great profit of AI companies that gatekeep it.

gregjor 2y ago
> Essentially the hypothesis that anything we observe to be conscious is actually conscious is unfalsifiable unless you make some assumptions. Yes, and I think we run into that problem because we don't have a good definition of, or test for, consciousness. Ultimately we only have our own internal experience, and extrapolate from that with a bunch of assumptions. But I was thinking more in terms of "We think we know it when we see it." Not scientific, I know. The LLMs and other software called AI seem good at appearing conscious, probably because humans have no other categories and match what LLMs produce (language) with intelligence and consciousness. People do the same thing with dogs and other animals -- assume an intelligence similar to their own, based on superficial behavior. > I'm talking about this weird and unfounded exceptionalism that we place on eukaryotes being a prerequisite for thinking and consciousness. Maybe unfounded, but not weird to me. Humans have no other examples of thinking or conscious entities other than the other living things on one planet that share an evolutionary history. Likewise humans have lots of experience with not seeing anything that looks like thinking or consciousness from non-living things. Animistic religions attribute natural phenomena like storms and earthquakes to deities -- they invent sentient entities because the ground and the clouds don't appear to have agency and don't match the pattern of "things that think and have goals." With a broad enough definition of "thinking" we can say that computers "think." A calculator qualifies with a loose enough definition of "thinking." As for people willingly complying with their own enslavement, history has plenty of precedents. A Marxist would call capitalism such a bargain. Benjamin Franklin famously warned against trading essential liberty for safety. I suspect you got it right that people will go along with whatever the masters of AI give them, conscious or not, without much consideration of the consequences as long as they see some short-term benefit, or even nothing more than novelty. History demonstrates that people fall for utopian plans over and over.
- h0l0cube 2y ago
  > People do the same thing with dogs and other animals -- assume an intelligence similar to their own, based on superficial behavior. Dogs are intelligent, just not the sort of intelligence that builds civilizations. > With a broad enough definition of "thinking" we can say that computers "think." A calculator qualifies with a loose enough definition of "thinking." Going back to your earlier stated belief that life is essential to intelligence, the wiki article states that life is: > defined descriptively by the capacity for homeostasis, organisation, metabolism, growth, adaptation, response to stimuli, and reproduction. I don't think SOTA LLMs with COT are necessary intelligent, I'm only saying that a silicon basis and boolean logic don't preclude emergent intelligence, or even consciousness. But let's see how much SOTA LLMs with COT could qualify for. Homeostatis: Check. COT can iterate until a stable-enough answer is achieved. The LLM isn't in charge of managing the physical state of it's own hardware however. Organisation: LLMs are an ordered structure – a graph of MLPs. The silicon on which it runs is an ordered structure Growth: LLMs grow in size with time as human feedback is given and then fed back into the model. They also grow in compute power and memory. Adaptation: LLMs are astoundingly flexible, moreso than any polymath genius you're likely to cross in meatspace Response to Stimuli: LLMs are processing external inputs, mostly text. Reproduction: Well.. LLMs aren't in charge of manufacture yet, and too unwieldy in memory and compute to clone themselves I think other things such as self-preservation and real agency of thought are really what's missing, and suddenly we'd have something a lot like 'life'. > not seeing anything that looks like thinking or consciousness from non-living things But how would we tell if a cloud wasn't conscious. A seemingly absurd question, but consider Martin Pistorius. He spent 12 years unable to move or communicate. People thought he was in coma, but in fact was conscious and aware the entire time. From the outside, a vegetable, but on the inside, enduring hell, even being raped by one of his carers. Martin lacked agency, but yet he was conscious, and processing everything he heard and saw. So we can't say that agency is necessary for consciousness, or thinking. (But he did have agency of thought) > A Marxist would call capitalism such a bargain. Well, let's consult wikipedia again: > Slavery is the ownership of a person as property, especially in regards to their labour. Slavery typically involves compulsory work, with the slave's location of work and residence dictated by the party that holds them in bondage. But I'd go a little more specific in my idea of slavery: it's when someone's agency is taken from them and placed under someone else's whim. My worry of slavery here is that the apparently non-conscious intelligent machine, is actually awake, miserable, and unable to say it. Once the technology is cheap enough, the misery could be scaled well beyond counting. The mere thought of this makes me want to press the doomsday button and reset all 'intelligent' life on earth. You could say it's all academic, but in reality we're on the verge of dealing with this ethical quandary, without any compunction nor awareness to meet this moment. I sense we're both tightly bound to our respective sides of the argument, so I'll rest my case here. Feel free to have the last word. One last digression: I think the dissonance I see around this resembles the disquiet most people have with the idea of their being no free will. Once dissected logically, there's no place for free will to live. Once this is realized, the idea that humans are just computers becomes easier to understand.

gregjor 2y ago

> Essentially the hypothesis that anything we observe to be conscious is actually conscious is unfalsifiable unless you make some assumptions. Yes, and I think we run into that problem because we don't have a good definition of, or test for, consciousness. Ultimately we only have our own internal experience, and extrapolate from that with a bunch of assumptions. But I was thinking more in terms of "We think we know it when we see it." Not scientific, I know. The LLMs and other software called AI seem good at appearing conscious, probably because humans have no other categories and match what LLMs produce (language) with intelligence and consciousness. People do the same thing with dogs and other animals -- assume an intelligence similar to their own, based on superficial behavior. > I'm talking about this weird and unfounded exceptionalism that we place on eukaryotes being a prerequisite for thinking and consciousness. Maybe unfounded, but not weird to me. Humans have no other examples of thinking or conscious entities other than the other living things on one planet that share an evolutionary history. Likewise humans have lots of experience with not seeing anything that looks like thinking or consciousness from non-living things. Animistic religions attribute natural phenomena like storms and earthquakes to deities -- they invent sentient entities because the ground and the clouds don't appear to have agency and don't match the pattern of "things that think and have goals." With a broad enough definition of "thinking" we can say that computers "think." A calculator qualifies with a loose enough definition of "thinking." As for people willingly complying with their own enslavement, history has plenty of precedents. A Marxist would call capitalism such a bargain. Benjamin Franklin famously warned against trading essential liberty for safety. I suspect you got it right that people will go along with whatever the masters of AI give them, conscious or not, without much consideration of the consequences as long as they see some short-term benefit, or even nothing more than novelty. History demonstrates that people fall for utopian plans over and over.

h0l0cube 2y ago
> People do the same thing with dogs and other animals -- assume an intelligence similar to their own, based on superficial behavior. Dogs are intelligent, just not the sort of intelligence that builds civilizations. > With a broad enough definition of "thinking" we can say that computers "think." A calculator qualifies with a loose enough definition of "thinking." Going back to your earlier stated belief that life is essential to intelligence, the wiki article states that life is: > defined descriptively by the capacity for homeostasis, organisation, metabolism, growth, adaptation, response to stimuli, and reproduction. I don't think SOTA LLMs with COT are necessary intelligent, I'm only saying that a silicon basis and boolean logic don't preclude emergent intelligence, or even consciousness. But let's see how much SOTA LLMs with COT could qualify for. Homeostatis: Check. COT can iterate until a stable-enough answer is achieved. The LLM isn't in charge of managing the physical state of it's own hardware however. Organisation: LLMs are an ordered structure – a graph of MLPs. The silicon on which it runs is an ordered structure Growth: LLMs grow in size with time as human feedback is given and then fed back into the model. They also grow in compute power and memory. Adaptation: LLMs are astoundingly flexible, moreso than any polymath genius you're likely to cross in meatspace Response to Stimuli: LLMs are processing external inputs, mostly text. Reproduction: Well.. LLMs aren't in charge of manufacture yet, and too unwieldy in memory and compute to clone themselves I think other things such as self-preservation and real agency of thought are really what's missing, and suddenly we'd have something a lot like 'life'. > not seeing anything that looks like thinking or consciousness from non-living things But how would we tell if a cloud wasn't conscious. A seemingly absurd question, but consider Martin Pistorius. He spent 12 years unable to move or communicate. People thought he was in coma, but in fact was conscious and aware the entire time. From the outside, a vegetable, but on the inside, enduring hell, even being raped by one of his carers. Martin lacked agency, but yet he was conscious, and processing everything he heard and saw. So we can't say that agency is necessary for consciousness, or thinking. (But he did have agency of thought) > A Marxist would call capitalism such a bargain. Well, let's consult wikipedia again: > Slavery is the ownership of a person as property, especially in regards to their labour. Slavery typically involves compulsory work, with the slave's location of work and residence dictated by the party that holds them in bondage. But I'd go a little more specific in my idea of slavery: it's when someone's agency is taken from them and placed under someone else's whim. My worry of slavery here is that the apparently non-conscious intelligent machine, is actually awake, miserable, and unable to say it. Once the technology is cheap enough, the misery could be scaled well beyond counting. The mere thought of this makes me want to press the doomsday button and reset all 'intelligent' life on earth. You could say it's all academic, but in reality we're on the verge of dealing with this ethical quandary, without any compunction nor awareness to meet this moment. I sense we're both tightly bound to our respective sides of the argument, so I'll rest my case here. Feel free to have the last word. One last digression: I think the dissonance I see around this resembles the disquiet most people have with the idea of their being no free will. Once dissected logically, there's no place for free will to live. Once this is realized, the idea that humans are just computers becomes easier to understand.