5 ms·

What I don't hear in discussions about this, is in what manner human 'thinking' is distinguished from AI 'thinking'. A lot of people hold the prejudice that bo

What I don't hear in discussions about this, is in what manner human 'thinking' is distinguished from AI 'thinking'. A lot of people hold the prejudice that both 'thinking' and 'consciousness' are substrate dependent, without a lick of evidence. If the hard problem of consciousness is unsolvable, it would make sense to err towards, at least, 'We don't know if it is', but ethically it would be responsible to err towards 'It is', and in reality 'It could be'—assuming the AI is observably intelligent in the same manner as other life we deem intelligent.

biophysboy 2y ago
It is ethically hazardous to assume it is sentient because it misleads people about its intrinsic capability and its economic value. Edit: also, the comment above is still true if A and B are made of the same substrate
- h0l0cube 2y ago
  > ethically hazardous What's the ethical hazard here? The ethical hazard of inadvertently torturing limitless miserable slaves is a thing, but I'm not sure what the hazard is in this misleading about economic value extraction. Would someone miss out on money or something?
  - biophysboy 2y ago
    No, that is a sci-fi hazard, I’m only talking about misleading people. Economic bubbles, financial cons, etc
    - h0l0cube 2y ago
      I'm not so bothered about fools being parted with their money. In a perfect economy, they ought to be. > a sci-fi hazard Until it isn't. And we wouldn't know when this comes to pass, because few are motivated to care about it.
gregjor 2y ago
What do you mean by “without a lick of evidence?” Every bit of evidence we have shows that what we call “thinking” and “consciousness” only arises in living things and depends on not just the “substrate” but the entire organism. Examples of sentience independent of “substrate” or an entire living creature: zero. One can’t reason from that to conclude sentience and thinking can’t happen without a living organism — absence of evidence and all that — but so far all evidence points in just one direction.
- h0l0cube 2y ago
  > living things Definition? > thinking can’t happen without a living organism Ah, it must have organs? By extension of your argument really leads to solipsism. The only real evidence we have—sorry, that I have-is that I am conscious, because I experience qualia. You might experience qualia, and there is the only evidence you have. As you have done, you could then extrapolate that those that—by one's eye—share one's physiology (i.e. substrate, primate brain etc.) and kind of look and behave like one's self (i.e. other primates) are conscious and intelligent. But then cephalopods demonstrate intelligence with a completely different neural network structure. Maybe these alien creatures are also conscious? Maybe they feel pain? (Oestensibly they do, I'm sure you'll agree.) If so, what is it that is shared between these two different organisms that cause this to arise? And specifically why must it be just because they have squishy organs?
  - gregjor 2y ago
    > living things... Definition? I feel confident we have shared definitions of living vs. non-living things, but if you insist: https://en.wikipedia.org/wiki/Life https://en.wikipedia.org/wiki/Life https://en.wikipedia.org/wiki/Organism https://en.wikipedia.org/wiki/Organism We don't need to get into what we mean by consciousness, or experiencing qualia. We only have direct evidence of our own consciousness. But I disputed a different statement, specifically: > A lot of people hold the prejudice that both 'thinking' and 'consciousness' are substrate dependent, without a lick of evidence. Define "a lot of people." In my anecdotal experience "a lot of people" believe in AI/AGI without any evidence for it. Those people implicitly believe that thinking and consciousness can happen in non-biological "subtstrates" whether they have thought that through carefully or not. I just pointed out that if we agree on what we mean by "thinking" and "consciousness" for purposes of discussion, we (human beings) have lots of evidence for it, and all of that evidence occurs only in biological entities we call "living" and "organisms." We have no evidence for thinking or consciousness in rocks, stars, or computers. You can call that a prejudice if you like. Not too long ago claiming that cephalapods or birds or insects could "think" or had something we might call "consciousness" amounted to extraordinary claims, rejected by people, mainly out of prejudice. Now we have persuasive evidence that non-human animals can think and appear to experience something that looks like consciousness, though we can't know for sure what happens in the mind of an octopus, crow, or ant. We have evidence but not definitive proof, in other words. Today we have people making claims about AGI and machine superintelligence, with no evidence (but large financial incentives). Extraordinary claims require extraordinary evidence. Watching a crow use a stick to get at some food, or an octopus cleverly escaping from an aquarium, most people infer intelligence. Similarly many people infer intelligence when they "chat" with an LLM (and that happened back in the '70s with ELIZA). One possible explanation: we have created thinking, conscious software. Another explanation: our brains trick us (and people who stand to gain from deceiving us will trick us). Since we have lots of evidence for our brains tricking us, and other people defrauding us, and no evidence that computers can think or experience anything, I choose to wait for more compelling evidence of machine intelligence.

biophysboy 2y ago

It is ethically hazardous to assume it is sentient because it misleads people about its intrinsic capability and its economic value. Edit: also, the comment above is still true if A and B are made of the same substrate

h0l0cube 2y ago
> ethically hazardous What's the ethical hazard here? The ethical hazard of inadvertently torturing limitless miserable slaves is a thing, but I'm not sure what the hazard is in this misleading about economic value extraction. Would someone miss out on money or something?
- biophysboy 2y ago
  No, that is a sci-fi hazard, I’m only talking about misleading people. Economic bubbles, financial cons, etc
  - h0l0cube 2y ago
    I'm not so bothered about fools being parted with their money. In a perfect economy, they ought to be. > a sci-fi hazard Until it isn't. And we wouldn't know when this comes to pass, because few are motivated to care about it.

gregjor 2y ago

What do you mean by “without a lick of evidence?” Every bit of evidence we have shows that what we call “thinking” and “consciousness” only arises in living things and depends on not just the “substrate” but the entire organism. Examples of sentience independent of “substrate” or an entire living creature: zero. One can’t reason from that to conclude sentience and thinking can’t happen without a living organism — absence of evidence and all that — but so far all evidence points in just one direction.

h0l0cube 2y ago
> living things Definition? > thinking can’t happen without a living organism Ah, it must have organs? By extension of your argument really leads to solipsism. The only real evidence we have—sorry, that I have-is that I am conscious, because I experience qualia. You might experience qualia, and there is the only evidence you have. As you have done, you could then extrapolate that those that—by one's eye—share one's physiology (i.e. substrate, primate brain etc.) and kind of look and behave like one's self (i.e. other primates) are conscious and intelligent. But then cephalopods demonstrate intelligence with a completely different neural network structure. Maybe these alien creatures are also conscious? Maybe they feel pain? (Oestensibly they do, I'm sure you'll agree.) If so, what is it that is shared between these two different organisms that cause this to arise? And specifically why must it be just because they have squishy organs?
- gregjor 2y ago
  > living things... Definition? I feel confident we have shared definitions of living vs. non-living things, but if you insist: https://en.wikipedia.org/wiki/Life https://en.wikipedia.org/wiki/Life https://en.wikipedia.org/wiki/Organism https://en.wikipedia.org/wiki/Organism We don't need to get into what we mean by consciousness, or experiencing qualia. We only have direct evidence of our own consciousness. But I disputed a different statement, specifically: > A lot of people hold the prejudice that both 'thinking' and 'consciousness' are substrate dependent, without a lick of evidence. Define "a lot of people." In my anecdotal experience "a lot of people" believe in AI/AGI without any evidence for it. Those people implicitly believe that thinking and consciousness can happen in non-biological "subtstrates" whether they have thought that through carefully or not. I just pointed out that if we agree on what we mean by "thinking" and "consciousness" for purposes of discussion, we (human beings) have lots of evidence for it, and all of that evidence occurs only in biological entities we call "living" and "organisms." We have no evidence for thinking or consciousness in rocks, stars, or computers. You can call that a prejudice if you like. Not too long ago claiming that cephalapods or birds or insects could "think" or had something we might call "consciousness" amounted to extraordinary claims, rejected by people, mainly out of prejudice. Now we have persuasive evidence that non-human animals can think and appear to experience something that looks like consciousness, though we can't know for sure what happens in the mind of an octopus, crow, or ant. We have evidence but not definitive proof, in other words. Today we have people making claims about AGI and machine superintelligence, with no evidence (but large financial incentives). Extraordinary claims require extraordinary evidence. Watching a crow use a stick to get at some food, or an octopus cleverly escaping from an aquarium, most people infer intelligence. Similarly many people infer intelligence when they "chat" with an LLM (and that happened back in the '70s with ELIZA). One possible explanation: we have created thinking, conscious software. Another explanation: our brains trick us (and people who stand to gain from deceiving us will trick us). Since we have lots of evidence for our brains tricking us, and other people defrauding us, and no evidence that computers can think or experience anything, I choose to wait for more compelling evidence of machine intelligence.