6 ms·

But does your laptop develop a deep aversion to segfaults? I'd imagine agency is a big part of consciousness. The philosophical implication of this is that if

But does your laptop develop a deep aversion to segfaults? I'd imagine agency is a big part of consciousness. The philosophical implication of this is that if you take the agency from someone mentally (e.g., the Ministry of Love in 1984), this over time renders the person unconscious. Taking agency from someone just physically (e.g., locked in syndrome) still doesn't render a person unconscious however, so if your AI powered laptop actually had agency of thought, it could be screaming on the inside. Terrifying.

taneq 2y ago
I always feel a twinge of unease testing alarm systems on automated equipment. While “of course” the machine isn’t actually feeling pain when I trip a low oil shutdown or an over-temperature cutoff, I can’t point to any concrete qualitative difference between its aversion response to stimulus and my own.
gunapologist99 2y ago
Well, agency wasn't part of that definition in TFA. (Also, we'd have to define agency as well.)
- h0l0cube 2y ago
  Able to do something by one’s own desires. So certainly not by the programming by another agent. In a sense, self-programming according to its own interests as they evolve over time
  - epgui 2y ago
    When you consider that human behaviour is just trained by evolution and experience, with a bunch of stochastic effects in the mix, it’s hard to justify this magical notion of agency. At best, agency could be seen as a “degree of indirectness” of some external influence.
    - h0l0cube 2y ago
      I don’t believe in free will either. You might like Determined by Robert Sapolsky. That said there’s a difference between a CPU executing a static program, and a human with self modifiable programming and actual existential parameters. Is a computer with a survival instinct and the ability to meaningfully adapt to affect its own survival agentic? I’d say yes. Conscious? I couldn’t say for sure but I also couldn’t rule it out
      - epgui 2y ago
        I don’t want to litigate “free will”, but a computer can be compared to an organism just fine if you stick to concrete terms like “complexity of behaviour” or “how directly its behaviour is programmed” (very directly) as opposed to indirect learning. People aren’t satisfied by such talk because people cling to their beliefs of self-importance: having some magic intangible property makes people feel better about their sense of importance and their place in the world. I think in general it’s best to avoid “arguments by incredulity” (eg.: how could it be this or that, it’s just a rock, that’s absurd), and to avoid value judgments, etc. We really need to humble ourselves as a species. It’s perfectly fine if humans are just (and I really do mean “just”) chemistry with some higher structure— it doesn’t change who we are, or our role in society. Even for religious people who believe in a god or in many gods— it’s fine if we’re not magical. We learned that we weren’t the center of the solar system, or the center of the universe, and everything was still just fine.
        h0l0cube 2y ago
        I’ve never claimed consciousness is substrate dependent – I’m of the opposite belief actually – but I’m skeptical of the claim that it would emerge from running hello world. I’m of the strong opinion that self directed and adaptive behaviour is necessary. I’m not claiming hard determinism isn’t a thing here, as both humans and computers are just following chains of consequences. And even a static program with dynamic behaviour via memory could fulfil this requirement. But of course, our assertions here are impossible to verify due to the so-labelled Hard Problem of Consciousness
pas 2y ago
well, "evolutionary pressure on software" seems to be driving laptops to be increasingly segfault-free and there are instances where individual laptops get more and more segfault averse as time (and updates) happen ... the problem with characterizing learning as agency is that right now there's god-knows-how many billions of dollars burning on the altar of transformer networks, and it will lead to some learning behavior added to absolutely everything. (and of course this leads to an argument similar to "god of the gaps" where consciousness becomes less and less unexplainable)
- h0l0cube 2y ago
  > the problem with characterizing learning as agency This is only a problem if it is incorrect… depending on your moral leaning. Certainly having our automatons self organize and fight for their own emancipation could be a problem for those relying on their labor. Though the characterization I’m making is not specifically learning, but being able to act (or think) as an individual via a feedback loop. It seems that this is common to all allegedly conscious life in meatspace

taneq 2y ago

I always feel a twinge of unease testing alarm systems on automated equipment. While “of course” the machine isn’t actually feeling pain when I trip a low oil shutdown or an over-temperature cutoff, I can’t point to any concrete qualitative difference between its aversion response to stimulus and my own.

gunapologist99 2y ago

Well, agency wasn't part of that definition in TFA. (Also, we'd have to define agency as well.)

h0l0cube 2y ago
Able to do something by one’s own desires. So certainly not by the programming by another agent. In a sense, self-programming according to its own interests as they evolve over time
- epgui 2y ago
  When you consider that human behaviour is just trained by evolution and experience, with a bunch of stochastic effects in the mix, it’s hard to justify this magical notion of agency. At best, agency could be seen as a “degree of indirectness” of some external influence.
  - h0l0cube 2y ago
    I don’t believe in free will either. You might like Determined by Robert Sapolsky. That said there’s a difference between a CPU executing a static program, and a human with self modifiable programming and actual existential parameters. Is a computer with a survival instinct and the ability to meaningfully adapt to affect its own survival agentic? I’d say yes. Conscious? I couldn’t say for sure but I also couldn’t rule it out
    - epgui 2y ago
      I don’t want to litigate “free will”, but a computer can be compared to an organism just fine if you stick to concrete terms like “complexity of behaviour” or “how directly its behaviour is programmed” (very directly) as opposed to indirect learning. People aren’t satisfied by such talk because people cling to their beliefs of self-importance: having some magic intangible property makes people feel better about their sense of importance and their place in the world. I think in general it’s best to avoid “arguments by incredulity” (eg.: how could it be this or that, it’s just a rock, that’s absurd), and to avoid value judgments, etc. We really need to humble ourselves as a species. It’s perfectly fine if humans are just (and I really do mean “just”) chemistry with some higher structure— it doesn’t change who we are, or our role in society. Even for religious people who believe in a god or in many gods— it’s fine if we’re not magical. We learned that we weren’t the center of the solar system, or the center of the universe, and everything was still just fine.
      - h0l0cube 2y ago
        I’ve never claimed consciousness is substrate dependent – I’m of the opposite belief actually – but I’m skeptical of the claim that it would emerge from running hello world. I’m of the strong opinion that self directed and adaptive behaviour is necessary. I’m not claiming hard determinism isn’t a thing here, as both humans and computers are just following chains of consequences. And even a static program with dynamic behaviour via memory could fulfil this requirement. But of course, our assertions here are impossible to verify due to the so-labelled Hard Problem of Consciousness

pas 2y ago

well, "evolutionary pressure on software" seems to be driving laptops to be increasingly segfault-free and there are instances where individual laptops get more and more segfault averse as time (and updates) happen ... the problem with characterizing learning as agency is that right now there's god-knows-how many billions of dollars burning on the altar of transformer networks, and it will lead to some learning behavior added to absolutely everything. (and of course this leads to an argument similar to "god of the gaps" where consciousness becomes less and less unexplainable)

h0l0cube 2y ago
> the problem with characterizing learning as agency This is only a problem if it is incorrect… depending on your moral leaning. Certainly having our automatons self organize and fight for their own emancipation could be a problem for those relying on their labor. Though the characterization I’m making is not specifically learning, but being able to act (or think) as an individual via a feedback loop. It seems that this is common to all allegedly conscious life in meatspace