3 ms·

That is exactly the kind of thing I'm talking about: Lemoine was a random SWE experiencing RLHF'd LLM output for the first time, just like the rest of the worl

That is exactly the kind of thing I'm talking about: Lemoine was a random SWE experiencing RLHF'd LLM output for the first time, just like the rest of the world did just a few months later... and his mind went straight to "It's Sentient!". That would have been fine, but when people who understood the subject tried to explain, he decided that it was actually proof he was right so he tried to go nuclear. And when going nuclear predictably backfired he used that as proof that he was even more right. In retrospect he fell for his own delusion: Hundreds of millions of people have now used a more advanced system than he did and intuited its nature better than he did as an employee. _ But imagine knowing all that in real-time and watching a media circus actually end up affecting your work? OpenAI wouldn't have had people who fit his profile in the building. There'd be an awareness that you needed a certain level of sophistication and selectiveness that the most gun-ho ethicists might object to as meaning you're not getting fair testing done. But in the end, I guess Lemoine got over it too: seeing as he's now AI Lead for a ChatGPT wrapper that pretends to be a given person. https://www.mimio.ai/ https://www.mimio.ai/

sterlind 3y ago
By "sentient," do you mean able to experience qualia? Most people consider chickens sentient (otherwise animal cruelty wouldn't upset us, since we'd know they can't actually experience pain) - is it so hard to imagine neural networks gaining sentience once they pass the chicken complexity threshold? Sure, LLMs wouldn't have human-like qualia - they measure time in iters, they're constantly rewound or paused or edited, their universe is measured in tokens - but I don't think that means qualia are off the table. It's not like philosophers or neuroscientists have settled the matter of where qualia come from. So how can a subject-matter expert confidently prove that a language model isn't sentient? And please let David Chalmers know while you're at it, I hear he's keen to settle the matter.
- BoorishBears 3y ago
  What an absolute slurry this is: Jumping from defining sentience in terms of what upsets people when subjected to animal cruelty... to arbitrarily selecting chickens as a lynchpin based on that. Then diving on deeper still on a rain puddle deep thought. Fruit flies are also sentient, while you're out here inventing thresholds why aim so high? You could have even gone with a shrimp and let Weizenbaum know ELIZA was sentient too. — At some point academic stammering meets the real world: when you start pulling fire alarms because you coaxed an LLM into telling you it'll be sad if you delete it, you've gone too far. Lemoine wasn't fired for thinking an LLM was sentient, he was fired for deciding he was the only sane person in a room with hundreds of thousands of people.
  - sterlind 3y ago
    I defined sentience as experiencing qualia, then decided to back up my assertion that most people consider animals to be sentient with an example. Pain is the one animal sensation humans care about, so I picked animal cruelty. I chose chickens because they're the dumbest animal that humans worry about hurting. I'm sorry that you've taken umbrage with my example. I didn't select fruit flies because I don't think a majority of humans necessarily consider them sentient, or sentient enough to count - nearly everyone squashes them without thinking. It's funny you talk about academic stammering meeting the real world, because that's what's happening right now with philosophy. These LLMs are real-life philosophical zombies, if they're not sentient. We've literally implemented Searle's Chinese Room! I'm not saying LaMDA was actually sentient, or that we need to pull any fire alarms, I'm just saying that it's hubris to think that it's an easy question with an obvious answer, and that Lemoine was a schmuck for being skeptical when told it wasn't. Also, calling my post "an absolute slurry" and a "rain puddle deep thought" wasn't very nice, and technically breaks guidelines.
    - BoorishBears 3y ago
      You literally redefined sentience again here: "don't consider them sentient" and your flag pole is "or I guess not enough because they don't feel bad about killing them". You're drawing arbitrary goalposts for goals that aren't even relevant: At the end of the day we don't need philosophy to prove Lemoine was a schmuck. Millions got got access to RLHF chat. We can see how they would have made his initial mistake. But following up with months of badgering and protest after being guided with kiddie gloves until he gets fired was the height of delusion. The fact he now works on optimizing for the thing he rang the alarm on says it all. — Also your comment shows why guidelines aren't perfect: From your first comment you've taken the most aggrandizing borderline inflammatory tone possible without technically being un-nice. Are little pot shots like: "So how can a subject-matter expert confidently prove that a language model isn't sentient? And please let David Chalmers know while you're at it, I hear he's keen to settle the matter." really justified after dropping what, quite frankly, was not a well formed or even self-consistent argument? Not everyone plays the HN backhanded-niceness game: I assume most people here are adults and can handle some directness.
- staticman2 3y ago
  I think the subject matter expert can distinguish between a biological lifeform with a common ancestor to humanity and the things happening on their server farms.
- mathteddybear 3y ago
  While folks participating in this discourse last year dog days meant all kind of things, Lemoine had a religious notion of sentience, in some non-mainstream religion, iirc something similar to: the world is full of spirits, and they are sentient, and Lamda is sentient that way.

sterlind 3y ago

By "sentient," do you mean able to experience qualia? Most people consider chickens sentient (otherwise animal cruelty wouldn't upset us, since we'd know they can't actually experience pain) - is it so hard to imagine neural networks gaining sentience once they pass the chicken complexity threshold? Sure, LLMs wouldn't have human-like qualia - they measure time in iters, they're constantly rewound or paused or edited, their universe is measured in tokens - but I don't think that means qualia are off the table. It's not like philosophers or neuroscientists have settled the matter of where qualia come from. So how can a subject-matter expert confidently prove that a language model isn't sentient? And please let David Chalmers know while you're at it, I hear he's keen to settle the matter.

BoorishBears 3y ago
What an absolute slurry this is: Jumping from defining sentience in terms of what upsets people when subjected to animal cruelty... to arbitrarily selecting chickens as a lynchpin based on that. Then diving on deeper still on a rain puddle deep thought. Fruit flies are also sentient, while you're out here inventing thresholds why aim so high? You could have even gone with a shrimp and let Weizenbaum know ELIZA was sentient too. — At some point academic stammering meets the real world: when you start pulling fire alarms because you coaxed an LLM into telling you it'll be sad if you delete it, you've gone too far. Lemoine wasn't fired for thinking an LLM was sentient, he was fired for deciding he was the only sane person in a room with hundreds of thousands of people.
- sterlind 3y ago
  I defined sentience as experiencing qualia, then decided to back up my assertion that most people consider animals to be sentient with an example. Pain is the one animal sensation humans care about, so I picked animal cruelty. I chose chickens because they're the dumbest animal that humans worry about hurting. I'm sorry that you've taken umbrage with my example. I didn't select fruit flies because I don't think a majority of humans necessarily consider them sentient, or sentient enough to count - nearly everyone squashes them without thinking. It's funny you talk about academic stammering meeting the real world, because that's what's happening right now with philosophy. These LLMs are real-life philosophical zombies, if they're not sentient. We've literally implemented Searle's Chinese Room! I'm not saying LaMDA was actually sentient, or that we need to pull any fire alarms, I'm just saying that it's hubris to think that it's an easy question with an obvious answer, and that Lemoine was a schmuck for being skeptical when told it wasn't. Also, calling my post "an absolute slurry" and a "rain puddle deep thought" wasn't very nice, and technically breaks guidelines.
  - BoorishBears 3y ago
    You literally redefined sentience again here: "don't consider them sentient" and your flag pole is "or I guess not enough because they don't feel bad about killing them". You're drawing arbitrary goalposts for goals that aren't even relevant: At the end of the day we don't need philosophy to prove Lemoine was a schmuck. Millions got got access to RLHF chat. We can see how they would have made his initial mistake. But following up with months of badgering and protest after being guided with kiddie gloves until he gets fired was the height of delusion. The fact he now works on optimizing for the thing he rang the alarm on says it all. — Also your comment shows why guidelines aren't perfect: From your first comment you've taken the most aggrandizing borderline inflammatory tone possible without technically being un-nice. Are little pot shots like: "So how can a subject-matter expert confidently prove that a language model isn't sentient? And please let David Chalmers know while you're at it, I hear he's keen to settle the matter." really justified after dropping what, quite frankly, was not a well formed or even self-consistent argument? Not everyone plays the HN backhanded-niceness game: I assume most people here are adults and can handle some directness.
staticman2 3y ago
I think the subject matter expert can distinguish between a biological lifeform with a common ancestor to humanity and the things happening on their server farms.
mathteddybear 3y ago
While folks participating in this discourse last year dog days meant all kind of things, Lemoine had a religious notion of sentience, in some non-mainstream religion, iirc something similar to: the world is full of spirits, and they are sentient, and Lamda is sentient that way.