5 ms·

It's not like the servers running the LLMs suddenly feel pain. At least for now

notamy 4y ago
Given how people jailbreak these things, I don't know if I'd necessarily believe that. Some of the jailbreak prompts have been variants on "do this task right and gain points, do it wrong and I take away points." It may not be pain as we understand it, but isn't pain just a specialised negative input?
- _nalply 4y ago
  No. It is not reacting to pain but just generating language that looks like reacting to pain.
  - SpaghettiCthulu 4y ago
    Are we really any different?
    - darkwraithcov 4y ago
      Yes because natural language is just one way humans have adapted to sensory input.
  - Spivak 4y ago
    CS folks need to read more philosophy. You’re describing a philosophical zombie and there is a mountain of existing literature about them. https://en.m.wikipedia.org/wiki/Philosophical_zombie https://en.m.wikipedia.org/wiki/Philosophical_zombie
- quonn 4y ago
  If so, why not also worry about the ghosts in pacman?
  - notamy 4y ago
    Because people aren't taking the ghosts from Pac-Man and hooking them up to the internet with read/write access and sometimes-human-level capabilities.
    - quonn 4y ago
      I meant: Worrying about the feelings of those ghosts.
- andrewflnr 4y ago
  Do you feel physical pain when someone tells you they're taking points from you? No? There's no call for analogical reasoning, the literal interpretation of the same action is right there.
jmoak3 4y ago
For sure - but when do we start asking it? Not to be too dramatic, but you and I agree we can simulate the same computation a small bundle of neurons can accomplish to a high degree of confidence, right? If we can simulate a metric butt ton of them and get emergent behaviors somewhat similar to human computation, how can we guarantee that this simulated collection of these computing units isn’t “feeling” close to what a human feels? We’re just large collections of computing units as well. I’m not quite saying we’re there yet though. This whole development has really ruined my appetite for eating things that have intelligence.
- darkwraithcov 4y ago
  The problem is, because of how these models were trained, how do we know if its telling us how it feels vs how it thinks we want it to respond?
  - khochesh_kushat 4y ago
    I can't even prove that for sure about other people.
- psychoslave 4y ago
  >but you and I agree we can simulate the same computation a small bundle of neurons can accomplish to a high degree of confidence, right? The important word is simulate. It means it can accomplish a common subset of measurable phenomena. It certainly doesn't mean they are on pare on ontological status as a whole. The debate seems to easily confuse notions like living entities, consciousness and intelligence, when they are three orthogonal considerations.

Given how people jailbreak these things, I don't know if I'd necessarily believe that. Some of the jailbreak prompts have been variants on "do this task right and gain points, do it wrong and I take away points." It may not be pain as we understand it, but isn't pain just a specialised negative input?

_nalply 4y ago
No. It is not reacting to pain but just generating language that looks like reacting to pain.
- SpaghettiCthulu 4y ago
  Are we really any different?
  - darkwraithcov 4y ago
    Yes because natural language is just one way humans have adapted to sensory input.
- Spivak 4y ago
  CS folks need to read more philosophy. You’re describing a philosophical zombie and there is a mountain of existing literature about them. https://en.m.wikipedia.org/wiki/Philosophical_zombie https://en.m.wikipedia.org/wiki/Philosophical_zombie
quonn 4y ago
If so, why not also worry about the ghosts in pacman?
- notamy 4y ago
  Because people aren't taking the ghosts from Pac-Man and hooking them up to the internet with read/write access and sometimes-human-level capabilities.
  - quonn 4y ago
    I meant: Worrying about the feelings of those ghosts.
andrewflnr 4y ago
Do you feel physical pain when someone tells you they're taking points from you? No? There's no call for analogical reasoning, the literal interpretation of the same action is right there.

jmoak3 4y ago

For sure - but when do we start asking it? Not to be too dramatic, but you and I agree we can simulate the same computation a small bundle of neurons can accomplish to a high degree of confidence, right? If we can simulate a metric butt ton of them and get emergent behaviors somewhat similar to human computation, how can we guarantee that this simulated collection of these computing units isn’t “feeling” close to what a human feels? We’re just large collections of computing units as well. I’m not quite saying we’re there yet though. This whole development has really ruined my appetite for eating things that have intelligence.

darkwraithcov 4y ago
The problem is, because of how these models were trained, how do we know if its telling us how it feels vs how it thinks we want it to respond?
- khochesh_kushat 4y ago
  I can't even prove that for sure about other people.
psychoslave 4y ago
>but you and I agree we can simulate the same computation a small bundle of neurons can accomplish to a high degree of confidence, right? The important word is simulate. It means it can accomplish a common subset of measurable phenomena. It certainly doesn't mean they are on pare on ontological status as a whole. The debate seems to easily confuse notions like living entities, consciousness and intelligence, when they are three orthogonal considerations.