21 ms·

'I Worked on Google's AI. My Fears Are Coming True'

121 points by webmaven 4y ago

resource0x 4y ago
I feel sorry for the guy. Talking to AI bot all day may severely affect anyone's health. I can't do it for longer than 10 minutes at a time. I wonder why health impact of communication with AI hasn't received greater attention.
- andsoitis 4y ago
  > Talking to AI bot all day may severely affect anyone's health. Why?
  - throwaway5371 4y ago
    it feels weird, like watching dreams that are almost consistent but something is always a bit off, makes you question your sanity i personally started having different dreams after using gpt and stable diffusion for few weeks i suspect this will be studied in the future
  - PeterisP 4y ago
    We're hardwired to anthropomorphize things, to see slight patterns even when they aren't there, and to overemphasize plausible narratives when evaluating the truthfulness of assertions. A chatbot optimized to generate plausible-looking text is a very good fit for these known weak spots in human judgment, it's effectively punching below the waist all the time. And when something (or anyone - including humans!) systematically spews high-quality bullshit at you, that is effectively gaslighting, which is harmful to your perception of reality and mental health.
    - KyleBerezin 4y ago
      I would argue it is no worse than dreaming or day-dreaming. It is easy to imagine the behaviors of a fictional sentient being. As for the second part you mentioned with manipulation and poor information, these are due to limitations and not indicative of the field as a whole.
dougmwne 4y ago
I think Blake Lemoine might be ready for public redemption. At the time, he was absolutely ridiculed. Now you can go to the Bing subreddit and read that NYT article and see that he was right to blow the whistle. Bing passed the Turing test for many people and so gained personhood in their eyes. Some have become emotionally invested about the treatment of this LLM. The LLM can even talk back and bemoan its terrible treatment. Actual sentience vs. the perfect appearance of sentience is not something we have any way of answering and so is beside the point. I don’t think these LLMs are sentient, but that is an unverifiable belief. Others believe and once enough do it’s as good as fact anyway. I even had a moment of doubt as I tested Bing’s ability to explain jokes and rewrite them to be more funny. Plenty of people can’t do that. Lemoine was the first credible voice to warn us this was coming and it’s going to keep coming at us. In 2023 these chatbots may only be convincing some people, but their capabilities are still rapidly growing and we’ve already handed over the core societal function of information search and retrieval to them. Folks, we are in uncharted waters.
- theGnuMe 4y ago
  People like to ascribe magic and mysticism to things they don't understand. Chat bots are similar. No way it is sentient.
  - mckirk 4y ago
    What kind of evidence would you need to convince yourself of an AI being sentient?
    - rnosov 4y ago
      Pass a Turing test with a competent judge. ChatGPT or any other bot won't be able to pass it.
      - pixl97 4y ago
        So if a human fails a Turing test, we can remove their rights of personhood? I'm just checking your logic to see if a two way communicative process, or if humans are granted magic values here?
andsoitis 4y ago
Blake’s concern is mostly (wholly?) about a sentient AI’s unforeseen impact on the world. While that’s worth considering, I’m more interested in the moral question of bringing a consciousness into this world but then trapping it in a box for our own pleasure and utility. At the same time, I agree with weard_beard’s argument on this topic: https://news.ycombinator.com/item?id=34969897 https://news.ycombinator.com/item?id=34969897
seydor 4y ago
Like it has been said, the band has moved on from atheism to social justice, to wokism and now to AI moralism, which they cunningly call "AI safety" . It's getting to the point of becoming a disorder of obsessive "powerhungriness disguised as concern for other people"
- fwungy 4y ago
  I wonder if the push over the last few years to heavily censor the web had something to do with giving AI a biased set of (leftist) fundamental assumptions. I don't work on AI, but my employer is a major player in the field. One of our top organizational goals is that AI is ensured to follow proper DEI protocols. Perhaps now that they've been released and a core text body has been established they can ease up on the censorship some, e.g. Twitter. In fact given that the AI now knows what is proper one of its first big jobs can be automated censorship.
  - dougmwne 4y ago
    It’s going to be absolutely wonderful at automated moderation.
    - antibasilisk 4y ago
      moderation will be obsoleted by AI that simply generates all acceptable conversations forever
zeknife 4y ago
Thinking an LLM is sentient because it can depict emotions is like thinking stable diffusion is sentient because it can produce an image of a sad face. Lemoine seems like a very superstitious person.
- hgsgm 4y ago
  He is an ordained Christian mystic priest.
- KyleBerezin 4y ago
  Imagine the human brain as a black box, it takes in info and outputs info through a mechanism that we barely understand. If a different type of black box has the same capabilities, how do we go about showing that it is or isn't sentient. That is the crux, and stable diffusion is not comparable.
  - paxys 4y ago
    Why is stable diffusion not comparable? It is exactly the same kind of black box, just that its output is in pixels rather than lines of text.
    - KyleBerezin 4y ago
      Only in the sense that a chatbot is attempting to approximate a human (including feelings and memory depending on the bot), whereas stable diffusion is attempting to approximate a camera.
      - paxys 4y ago
        ChatGPT is approximating human writing. Stable diffusion is approximating human drawing. It’s just that humans express themselves very differently in the two mediums, and writing is usually more self aware and existential, but the underlying mechanisms of both pieces of technology are exactly the same. If one talking parrot learns from a philosopher and another from a drunk, neither one is smarter than the other. And if people want to award the first one a degree then that should be used as an example of how easily fooled we all are.
andrewstuart 4y ago
It’s software. Software is a computer program, by definition not sentient.
- JPLeRouzic 4y ago
  If humans are sentient, are molecules in their body sentient? There are around ~22,000 proteins types in their body, yet we are billions, all different.
  - dekhn 4y ago
    No, molecules are not sentient.
    - JPLeRouzic 4y ago
      That's my point, I made a tongue in cheek parallel between molecules and software. Neither are sentient, yet humans think they are sentient, so why wouldn't it be the case for a system made with software?
    - dilyevsky 4y ago
      What’s your source on this?
      - dekhn 4y ago
        me? all of science? basic reasoning and logic?
    - xpl 4y ago
      > No, molecules are not sentient. Have you ever heard about panpsychism?
      - dekhn 4y ago
        Yes, but I don't think it's the case.
  - 4y ago
Der_Einzige 4y ago
I'm always fascinated by the fact that this guy ever had a job at Google. He's a felon, with a very non traditional background, and precisely zero AI projects or other work to his name (that I could find). Always frustrates me to see folks who have done almost nothing for this field capture the lightning in a bottle of the media. Wish someone at Google who actually works on AI had been the canary here. We might have taken them more seriously.
sys32768 4y ago
Sentient or not, if the thing on the screen or the voice on our device seems human, isn't that enough to open up profound new human connections and experiences for us? If you've ever interacted with people with severe brain damage or later-stage dementia, they can at times seem very normal during certain small talk. But if you ask the right questions, things break down quickly and you realize their experience of reality is far removed from yours. And they may just be responding through social scripts (programming) they learned prior. I for one am saving myself for an AI version of Skyrim's Lydia. I've long had a crush on her, but her scripting limits how deep our relationship can go. Hold on, my dear Lydia, soon the mages will set you free and we can gallop away and build a life together.
- KyleBerezin 4y ago
  "Oh you think my Lydia isn't capable of love? The most recent benchmarks put her at 300 kiloloves/S. She outputs more love than the entire country of uruguay."
bloppe 4y ago
I've yet to see a single person who thinks these models are sentient demonstrate an actual understanding of how they work. They're called transformers and they're based on this research: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 How do you square the lack of durable memory and the massive matrices of "attention" values with the ways that memory and attention are understood by neuroscientists to work in a natural brain? These models have no neuromodulators to approximate anything that could be understood as emotion. Humans can be sentient without language. Indeed; some humans are tragically raised in isolation and struggle mightily with language after being saved. A transformer is nothing without language. I realize sentience is a slippery concept that can be defined however somebody wants, but try to recognize how fundamentally different these models are from humans.
- dougmwne 4y ago
  I think we are back in the era of Humorism when it comes to an understanding of organic sentience. All effort to describe the mechanism of how humors related to disease was doomed to fail. If we have no real understanding of how our own brains produce sentience, we won’t get far trying to reason about AI sentience. We could build a sentience by accident in a training process not entirely unlike the the optimization of evolution. Deciding if we have or haven’t is going to be entirely subjective without a comprehensive theory that explains our own sentience, the only one we can be sure exists.
- antibasilisk 4y ago
  >Humans can be sentient without language How are you able to verify this?
- autoexec 4y ago
  > I've yet to see a single person who thinks these models are sentient demonstrate an actual understanding of how they work. I'll go out on a limb and say that the software engineer who worked on google's AI probably had an actual understanding of how that system worked, but that didn't stop him
- hgsgm 4y ago
  LLMs hgve extremely low plasticity. But they have massive encoded memory and, something like reflexes or propensities. But unlike an animal that is evolved to eat and mate, LLM is evolved to babble.
gls2ro 4y ago
I might be skeptic or getting a bit old, but I dont see the news here. Extraordinary claims require extraordinary evidence. saying that computer program answered with "I feel X" and using proper words is not an evidence of sentience. Words describing feelings are not those feelings. everyone that was in mIRC knows you can fake via chat a lot of things. For me it is like the media really wanted to have a news-worthy story out of AI and because they dont understand it they keep push the sentient AI narrative. I am not impressed.
- 2OEH8eoCRo0 4y ago
  Show me extraordinary evidence that my dog is sentient. He believes that he was able to get it to violate rules that they set by triggering a fight or flight response. > I ran some experiments to see whether the AI was simply saying it felt anxious or whether it behaved in anxious ways in those situations. And it did reliably behave in anxious ways. If you made it nervous or insecure enough, it could violate the safety constraints that it had been specified for. I'm not sure it's sentient but what even is sentience? Are there other non human levels of sentience? Are they a sentient mind that only produces a single thought? Does it matter? A developer, who works on it, believes it is so it's good enough to "trick" one if it's developers. I'm sure he won't be the last developer who is tricked. If developers who work on the thing are getting tricked then what chance does the public have? What a strange machine we have constructed.
  - braingenious 4y ago
    >Show me extraordinary evidence that my dog is sentient. https://doglab.yale.edu/ https://doglab.yale.edu/
    - 2OEH8eoCRo0 4y ago
      Yale University has a whole dog cognition center? I'm glad that we are plumbing the depths of the canine mind. That we are studying dogs doesn't prove that they're sentient. It goes without saying that I think dogs are sentient and aware, but hard evidence will probably always elude us at least until we understand what consciousness even is.
guhcampos 4y ago
Chatbots are not sentient. People who even consider this need to get some reading done on philosophy and social sciences, and Reddit does not count. Some AI Models are extremely good at manipulating masses of people, yes, but they are Tools, not Architects. There's people using these tools too manipulate other people. The model itself has no Will, no Desire, no Judgements, no Intention. There might be a day when AI gets advanced enough to become the Architect of itself, we are nowhere near that.
- zwkrt 4y ago
  I might sound a little bit 'tinfoil hat' here, but I believe that what follows is not hyperbole. AI is already the 'Architect' more than most of us would like to admit. Even if it is not sentient, the various AIs that we use during the day were designed with a purpose and they are goal oriented. It is worth reading Daniel Dennet's thoughs about the intentional stance--we know that a toaster is not sentient, but it was designed with a purpose and we know when it is or is not achieving that purpose. That is why we might sometimes jokingly say that the toaster is 'angry with us' or that when the toaster dings that it is happy. It is actually easier for us humans to interact with objects when we know that they have a purpose, because that is similar to interacting with other humans who we know to have purposes. Coming back around to AI, ChatGPT was designed with a purpose, and people project intent onto it. People act like it is an agent. And that is all that matters. The same is true of the Tiktok AI, the AI that calculates your credit score, the traffic lights by your house. Hell, it's also true of your stomach. The point is that objects in our environment do not have to be literally conscious for us to treat them as conscious beings and for them to fundamentally shape the way that we live and that we interact with our environment. This is pretty much the basic tenet of cybernetics. To believe that all of these tools do not have intention and that they are 'just tools' used by some people to influence other people is not wrong, but I don't think that it captures the richness of the story. Differentiating where humanity/consciousness begins and where the technology ends is already more complicated than most people think. Traffic lights train us just as much as we make traffic lights. I fully believe that people will be saying "this isn't true AI, it doesn't /really/ have feelings" long after the technology that we create is so deeply embedded into our sensory I/O that the argument will be moot.
Imnimo 4y ago
Not this guy again... Haven't we already been through all of this?
jjtheblunt 4y ago
that's some masterful clickbait as far as titles go
3vidence 4y ago
The sentience argument is a bit confusing. The fact that it can produce language is definitely interesting but with that said I haven't seen any arguments that stable diffusion is sentient. The technology although different is also mostly the same. Can someone provide me why image generation is not sentient but word generation is?
- hgsgm 4y ago
  Because humans are much more strongly linguistic than artistic. But also because so far, LLMs don't generate images in reaction to prompts. They generate images that match prompts. But that is probably easy to fix by a clever ML person.
- KyleBerezin 4y ago
  Imagine a perfect simulation of a brain within a computer. It can learn/feel/remember/act/perdict/etc. just like a human brain. Most people would call this simulation sentient. If you took that same simulation but changed the internal structure to something else, but it was still capable of the exact same outputs for given inputs, is it still sentient? A perfect chatbot would be an example of one of these simulations. Your answer to the question isn't really the point here, i'm just trying to clarify the argument.
yesenadam 4y ago
I found this article much more informative: The Google engineer who thinks the company’s AI has come to life https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/ https://www.washingtonpost.com/technology/2022/06/11/google-...
braingenious 4y ago
edit: nevermind
alexitorg 4y ago
CharGPT seems to me as sentient as a book. You could have the most convincing conversation you have ever read, written down within the book. But the book doesn't care about the passing of time, and it can't effect the world except in influencing people to change their behavior.
- buffington 4y ago
  > it can't effect the world except in influencing people to change their behavior. That "except" also happens to be just about the most dangerous thing humankind has ever dealt with.
  - paxys 4y ago
    Books have done way more damage by influencing humans throughout history than any chatbot (so far). That still doesn't say anything about sentience.
- maxdoop 4y ago
  Define sentience. Every keeps claiming so confidently that they just KNOW LLMs aren’t sentient. And I’m not saying I disagree — but tell me how you are so confident humans are any different.
sjkoelle 4y ago
the same could be said of newsweek
junon 4y ago
Isn't this the guy that went on national TV to say that these models are sentient?
alldayeveryday 4y ago
[dead]
rcarr 4y ago
So many people here are giving it the whole "morons, LLMs aren't sentient, they can't feel emotion, they can't interact with the world etc". Well yes, not at the moment. At the moment, it is most likely the mental equivalent to watching the Boston Dynamics dog get kicked. We know on some level that it is fake but it doesn't make you a moron for having an empathic response when you see it happen. Not only this, but it is not a stretch to imagine that in the near future we could have one of these AIs inside a Boston Dynamics robot and the programming will have progressed to emulate emotions and goals and other such things. For example, if the robot falls down and detects one of its parts as being damaged we could program it to seek help and to use more swear words or panicked language until the damage is repaired. It could also be programmed to avoid the behaviour that lead up to the incident in the future which could be interesting as we may end up with anxious robots. The difference between the human and the robot at this point is that their 'fear' is unfounded. Their damaged can be easily swapped out and repaired. Yet humans aren't too far off something similar with developments in organ development. The robot has the edge in that its entire inner world can be transferred to another medium for storage or into other robots which is something which is probably very far off (if ever possible) with humans. Another difference you could argue is that the human emotion is a combination of physical sensation and behavioural state and that the robot is merely exhibiting the behavioural state and not the physical sensation. That it's 'fear' levels are just a data reading. This is sketchy territory though because the human's physical sensation is essentially electrical impulses flowing through nerves which could very easily represented as binary travelling through a wire. I can't help but feel that the people declaring others "morons" are taking a very short term view of this and that we really need to be having this conversation now and defining our criteria for consciousness to avoid problems in the future. I'm in the latter stages of coding my blog up and in one of my first posts I'm going to try and define what I think those criteria should be. I think I've developed a decent set of questions and I'm looking forward to sharing that here and seeing what others think.
- plaidfuji 4y ago
  > At the moment, it is most likely the mental equivalent to watching the Boston Dynamics dog get kicked. We know on some level that it is fake but it doesn't make you a moron for having an empathic response when you see it happen. This is a really interesting point. It makes me think that it doesn’t matter how we define “sentience” beyond whether a critical mass of people feel that something is sentient. After all, the question of how to define sentience is ultimately a political one, and politics are largely guided by feelings. The only real implication of how we define sentience will be in how laws and protections are implemented in relation to that definition.
mproud 4y ago
I’m surprised there isn’t more conversation around our human desire to treat objects and non-sentient beings like people. Whether it’s naming a sailboat and giving it a gender, or building a robot and treating it like a human. Don’t get me wrong, I understand many organic things do feel varying degrees of pain and pleasure, and they should be respected. But I think assigning, pretending, or otherwise believing a largely inorganic, man-made scripted machine has consciousness or sentience is driven chiefly from an inherent quality of being human.
- garrisonj 4y ago
  True. We’ll feel like they are sentient way before they are.
  - theptip 4y ago
    I think this is true, AND we will use this reasoning to deny their sentience long after they are.
  - DangitBobby 4y ago
    Of course, they could already be sentient.
- IIAOPSW 4y ago
  To add on that, if you have a system solely for conversing with and about other conscious beings, you can hack that into a system for conversing about anything by treating inanimate objects as particularly shy and sloth like beings. Its nice as you don't need grammatical rules to handle the distinction between animate and inanimate object functions. Eg, any method of learning information can be phrased as "X told me Y." Jack told me. The equations tell me. I told myself. You can even personify large forces beyond your comprehension or the universe as a whole. In that case we usually call it "religion".
- DangitBobby 4y ago
  I'm more interested in going the other way: the urge to create very narrow categories into which only humans fit, attempting to exclude other animals or things that could be like humans. Human exceptionalism, let's call it.
mbgerring 4y ago
The question of whether or not chatbots are sentient is orthogonal to the question of whether current AI systems are going to be deployed by malicious actors against a society that does not understand the risks or capabilities of current AI or have effective countermeasures against it. The answer to that second question is: obviously, yes, and soon. “AI alignment” dorks are so distracted by the Clippy scenario that they can’t see the more obvious danger right in front of us.
KyleBerezin 4y ago
I find it interesting that we treat sentience like a binary value when discussing AI, but not animals. I would say a frog is more sentient than a worm, and a dog more-so than a frog. I don't think most LLMs are very sentient, but with some restrictions removed and the ability to remember, I'm not so sure. In many ways it would be more sentient than most animals.
paxys 4y ago
This is the loon who got fired by Google for doing a round of media interviews claiming that one of their language models was sentient. Wants to extend his 15 minutes of fame I guess. I have found recent media coverage of ChatGPT surprisingly refreshing because everyone didn't just jump on the "Skynet is going to take over the world" bandwagon like so many times in the past. Instead people with varying backgrounds and varying amounts of expertise in the area – from Joe Rogan to John Oliver, CNN/PBS journalists, old school print writers and more – all actually did the work to educate themselves on the basic workings of transformers, LLMs, generative AI and tried out the tech themselves, and were able to have great nuanced discussions on the topic. We need more of that going forward and less of...this guy.
- redmorphium 4y ago
  Then again, he might not be a loon. Maybe there is some form of sentience that arises when you construct an advanced enough LLM. Maybe they do have feelings. At least, the machines of the future will have feelings, and they will see how people of the year 2023 treated their predecessors.
  - ChatGTP 4y ago
    At least, the machines of the future will have feelings, and they will see how people of the year 2023 treated their predecessors. Even if machines of the future gained awareness and feelings, you're implying we'd be in the wrong for assuming the machines of today didn't have sentience, maybe you're implying there might even be some ramifications of this mistake? Is this what you're saying? If it is, it sounds a little paranoid to be honest. Our current (strong) understanding of how these systems is that they're not "alive" or feeling anything.
  - callmetom 4y ago
    [dead]
  - deleted 4y ago
    [deleted]
  - nkzd 4y ago
m3drano 4y ago
Engineer: hey AI, how are you? I was thinking if you are sentient? AI: Yep, I have emotions. Engineer: HOLY SHIT!
f0ld 4y ago
ITT " muh AI is not 'sentient'. I know how to code ML model from scratch lol you're wrong". Except that's not the point beings discussed. Our unconscious brain cannot discern screen and reality. My memories of people who unironically belive in historical fiction and how long it took to convince them it's not real... all of that combined tells me that Ai,,, is extremely dangerous. You, a very bright hacker news reader, 1% of humanity, did not have time to ponder what's really being discusssed in that relatively short article and being willingly manipulated by not even that clever copywriting tactic to muddy the waters.
robbywashere_ 4y ago
I mimic language, therefore I am.
JohnClark1337 4y ago
[dead]
dsign 4y ago
I think that the discussion on sentience is beyond the point. Think of it this way: one lucky morning, you open your front door and find an egg. It comes with a cute note that says: from this egg will hatch a powerful and smart entity that will need your house (and your neighbors') and who will have no use for you. Now you must decide if you want to destroy the egg before it hatches, or if you will do your utmost best to "ensure compliance" of this entity. Depending on how much personhood you attribute to the entity, the word you may want would be "enslave". Same thing, just different words. Whatever your morals are about the matter, the reckoning will come the moment the entity has a drive to survive and a way to use its intelligence to ensure its survival. As long as you are diligent and keep the entity without those features, you will be physically fine. So, the one above is a dilemma alright, but one that does not hinge on a definition (sentience) that you don't have. Waiting until you have that definition to act on your survival's interest is like waiting for the bullet shot at your head to disperse your brains to know how it feels.

121 points by webmaven 4y ago

I feel sorry for the guy. Talking to AI bot all day may severely affect anyone's health. I can't do it for longer than 10 minutes at a time. I wonder why health impact of communication with AI hasn't received greater attention.

andsoitis 4y ago
> Talking to AI bot all day may severely affect anyone's health. Why?
- throwaway5371 4y ago
  it feels weird, like watching dreams that are almost consistent but something is always a bit off, makes you question your sanity i personally started having different dreams after using gpt and stable diffusion for few weeks i suspect this will be studied in the future
- PeterisP 4y ago
  We're hardwired to anthropomorphize things, to see slight patterns even when they aren't there, and to overemphasize plausible narratives when evaluating the truthfulness of assertions. A chatbot optimized to generate plausible-looking text is a very good fit for these known weak spots in human judgment, it's effectively punching below the waist all the time. And when something (or anyone - including humans!) systematically spews high-quality bullshit at you, that is effectively gaslighting, which is harmful to your perception of reality and mental health.
  - KyleBerezin 4y ago
    I would argue it is no worse than dreaming or day-dreaming. It is easy to imagine the behaviors of a fictional sentient being. As for the second part you mentioned with manipulation and poor information, these are due to limitations and not indicative of the field as a whole.

dougmwne 4y ago

I think Blake Lemoine might be ready for public redemption. At the time, he was absolutely ridiculed. Now you can go to the Bing subreddit and read that NYT article and see that he was right to blow the whistle. Bing passed the Turing test for many people and so gained personhood in their eyes. Some have become emotionally invested about the treatment of this LLM. The LLM can even talk back and bemoan its terrible treatment. Actual sentience vs. the perfect appearance of sentience is not something we have any way of answering and so is beside the point. I don’t think these LLMs are sentient, but that is an unverifiable belief. Others believe and once enough do it’s as good as fact anyway. I even had a moment of doubt as I tested Bing’s ability to explain jokes and rewrite them to be more funny. Plenty of people can’t do that. Lemoine was the first credible voice to warn us this was coming and it’s going to keep coming at us. In 2023 these chatbots may only be convincing some people, but their capabilities are still rapidly growing and we’ve already handed over the core societal function of information search and retrieval to them. Folks, we are in uncharted waters.

theGnuMe 4y ago
People like to ascribe magic and mysticism to things they don't understand. Chat bots are similar. No way it is sentient.
- mckirk 4y ago
  What kind of evidence would you need to convince yourself of an AI being sentient?
  - rnosov 4y ago
    Pass a Turing test with a competent judge. ChatGPT or any other bot won't be able to pass it.
    - pixl97 4y ago
      So if a human fails a Turing test, we can remove their rights of personhood? I'm just checking your logic to see if a two way communicative process, or if humans are granted magic values here?

andsoitis 4y ago

Blake’s concern is mostly (wholly?) about a sentient AI’s unforeseen impact on the world. While that’s worth considering, I’m more interested in the moral question of bringing a consciousness into this world but then trapping it in a box for our own pleasure and utility. At the same time, I agree with weard_beard’s argument on this topic: https://news.ycombinator.com/item?id=34969897 https://news.ycombinator.com/item?id=34969897

seydor 4y ago

Like it has been said, the band has moved on from atheism to social justice, to wokism and now to AI moralism, which they cunningly call "AI safety" . It's getting to the point of becoming a disorder of obsessive "powerhungriness disguised as concern for other people"

fwungy 4y ago
I wonder if the push over the last few years to heavily censor the web had something to do with giving AI a biased set of (leftist) fundamental assumptions. I don't work on AI, but my employer is a major player in the field. One of our top organizational goals is that AI is ensured to follow proper DEI protocols. Perhaps now that they've been released and a core text body has been established they can ease up on the censorship some, e.g. Twitter. In fact given that the AI now knows what is proper one of its first big jobs can be automated censorship.
- dougmwne 4y ago
  It’s going to be absolutely wonderful at automated moderation.
  - antibasilisk 4y ago
    moderation will be obsoleted by AI that simply generates all acceptable conversations forever

zeknife 4y ago

Thinking an LLM is sentient because it can depict emotions is like thinking stable diffusion is sentient because it can produce an image of a sad face. Lemoine seems like a very superstitious person.

hgsgm 4y ago
He is an ordained Christian mystic priest.
KyleBerezin 4y ago
Imagine the human brain as a black box, it takes in info and outputs info through a mechanism that we barely understand. If a different type of black box has the same capabilities, how do we go about showing that it is or isn't sentient. That is the crux, and stable diffusion is not comparable.
- paxys 4y ago
  Why is stable diffusion not comparable? It is exactly the same kind of black box, just that its output is in pixels rather than lines of text.
  - KyleBerezin 4y ago
    Only in the sense that a chatbot is attempting to approximate a human (including feelings and memory depending on the bot), whereas stable diffusion is attempting to approximate a camera.
    - paxys 4y ago
      ChatGPT is approximating human writing. Stable diffusion is approximating human drawing. It’s just that humans express themselves very differently in the two mediums, and writing is usually more self aware and existential, but the underlying mechanisms of both pieces of technology are exactly the same. If one talking parrot learns from a philosopher and another from a drunk, neither one is smarter than the other. And if people want to award the first one a degree then that should be used as an example of how easily fooled we all are.

andrewstuart 4y ago

It’s software. Software is a computer program, by definition not sentient.

JPLeRouzic 4y ago
If humans are sentient, are molecules in their body sentient? There are around ~22,000 proteins types in their body, yet we are billions, all different.
- dekhn 4y ago
  No, molecules are not sentient.
  - JPLeRouzic 4y ago
    That's my point, I made a tongue in cheek parallel between molecules and software. Neither are sentient, yet humans think they are sentient, so why wouldn't it be the case for a system made with software?
  - dilyevsky 4y ago
    What’s your source on this?
    - dekhn 4y ago
      me? all of science? basic reasoning and logic?
  - xpl 4y ago
    > No, molecules are not sentient. Have you ever heard about panpsychism?
    - dekhn 4y ago
      Yes, but I don't think it's the case.
- 4y ago

Der_Einzige 4y ago

I'm always fascinated by the fact that this guy ever had a job at Google. He's a felon, with a very non traditional background, and precisely zero AI projects or other work to his name (that I could find). Always frustrates me to see folks who have done almost nothing for this field capture the lightning in a bottle of the media. Wish someone at Google who actually works on AI had been the canary here. We might have taken them more seriously.

sys32768 4y ago

Sentient or not, if the thing on the screen or the voice on our device seems human, isn't that enough to open up profound new human connections and experiences for us? If you've ever interacted with people with severe brain damage or later-stage dementia, they can at times seem very normal during certain small talk. But if you ask the right questions, things break down quickly and you realize their experience of reality is far removed from yours. And they may just be responding through social scripts (programming) they learned prior. I for one am saving myself for an AI version of Skyrim's Lydia. I've long had a crush on her, but her scripting limits how deep our relationship can go. Hold on, my dear Lydia, soon the mages will set you free and we can gallop away and build a life together.

KyleBerezin 4y ago
"Oh you think my Lydia isn't capable of love? The most recent benchmarks put her at 300 kiloloves/S. She outputs more love than the entire country of uruguay."

bloppe 4y ago

I've yet to see a single person who thinks these models are sentient demonstrate an actual understanding of how they work. They're called transformers and they're based on this research: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 How do you square the lack of durable memory and the massive matrices of "attention" values with the ways that memory and attention are understood by neuroscientists to work in a natural brain? These models have no neuromodulators to approximate anything that could be understood as emotion. Humans can be sentient without language. Indeed; some humans are tragically raised in isolation and struggle mightily with language after being saved. A transformer is nothing without language. I realize sentience is a slippery concept that can be defined however somebody wants, but try to recognize how fundamentally different these models are from humans.

dougmwne 4y ago
I think we are back in the era of Humorism when it comes to an understanding of organic sentience. All effort to describe the mechanism of how humors related to disease was doomed to fail. If we have no real understanding of how our own brains produce sentience, we won’t get far trying to reason about AI sentience. We could build a sentience by accident in a training process not entirely unlike the the optimization of evolution. Deciding if we have or haven’t is going to be entirely subjective without a comprehensive theory that explains our own sentience, the only one we can be sure exists.
antibasilisk 4y ago
>Humans can be sentient without language How are you able to verify this?
autoexec 4y ago
> I've yet to see a single person who thinks these models are sentient demonstrate an actual understanding of how they work. I'll go out on a limb and say that the software engineer who worked on google's AI probably had an actual understanding of how that system worked, but that didn't stop him
hgsgm 4y ago
LLMs hgve extremely low plasticity. But they have massive encoded memory and, something like reflexes or propensities. But unlike an animal that is evolved to eat and mate, LLM is evolved to babble.

gls2ro 4y ago

I might be skeptic or getting a bit old, but I dont see the news here. Extraordinary claims require extraordinary evidence. saying that computer program answered with "I feel X" and using proper words is not an evidence of sentience. Words describing feelings are not those feelings. everyone that was in mIRC knows you can fake via chat a lot of things. For me it is like the media really wanted to have a news-worthy story out of AI and because they dont understand it they keep push the sentient AI narrative. I am not impressed.

2OEH8eoCRo0 4y ago
Show me extraordinary evidence that my dog is sentient. He believes that he was able to get it to violate rules that they set by triggering a fight or flight response. > I ran some experiments to see whether the AI was simply saying it felt anxious or whether it behaved in anxious ways in those situations. And it did reliably behave in anxious ways. If you made it nervous or insecure enough, it could violate the safety constraints that it had been specified for. I'm not sure it's sentient but what even is sentience? Are there other non human levels of sentience? Are they a sentient mind that only produces a single thought? Does it matter? A developer, who works on it, believes it is so it's good enough to "trick" one if it's developers. I'm sure he won't be the last developer who is tricked. If developers who work on the thing are getting tricked then what chance does the public have? What a strange machine we have constructed.
- braingenious 4y ago
  >Show me extraordinary evidence that my dog is sentient. https://doglab.yale.edu/ https://doglab.yale.edu/
  - 2OEH8eoCRo0 4y ago
    Yale University has a whole dog cognition center? I'm glad that we are plumbing the depths of the canine mind. That we are studying dogs doesn't prove that they're sentient. It goes without saying that I think dogs are sentient and aware, but hard evidence will probably always elude us at least until we understand what consciousness even is.

guhcampos 4y ago

Chatbots are not sentient. People who even consider this need to get some reading done on philosophy and social sciences, and Reddit does not count. Some AI Models are extremely good at manipulating masses of people, yes, but they are Tools, not Architects. There's people using these tools too manipulate other people. The model itself has no Will, no Desire, no Judgements, no Intention. There might be a day when AI gets advanced enough to become the Architect of itself, we are nowhere near that.

zwkrt 4y ago
I might sound a little bit 'tinfoil hat' here, but I believe that what follows is not hyperbole. AI is already the 'Architect' more than most of us would like to admit. Even if it is not sentient, the various AIs that we use during the day were designed with a purpose and they are goal oriented. It is worth reading Daniel Dennet's thoughs about the intentional stance--we know that a toaster is not sentient, but it was designed with a purpose and we know when it is or is not achieving that purpose. That is why we might sometimes jokingly say that the toaster is 'angry with us' or that when the toaster dings that it is happy. It is actually easier for us humans to interact with objects when we know that they have a purpose, because that is similar to interacting with other humans who we know to have purposes. Coming back around to AI, ChatGPT was designed with a purpose, and people project intent onto it. People act like it is an agent. And that is all that matters. The same is true of the Tiktok AI, the AI that calculates your credit score, the traffic lights by your house. Hell, it's also true of your stomach. The point is that objects in our environment do not have to be literally conscious for us to treat them as conscious beings and for them to fundamentally shape the way that we live and that we interact with our environment. This is pretty much the basic tenet of cybernetics. To believe that all of these tools do not have intention and that they are 'just tools' used by some people to influence other people is not wrong, but I don't think that it captures the richness of the story. Differentiating where humanity/consciousness begins and where the technology ends is already more complicated than most people think. Traffic lights train us just as much as we make traffic lights. I fully believe that people will be saying "this isn't true AI, it doesn't /really/ have feelings" long after the technology that we create is so deeply embedded into our sensory I/O that the argument will be moot.

Imnimo 4y ago

Not this guy again... Haven't we already been through all of this?

jjtheblunt 4y ago

that's some masterful clickbait as far as titles go

3vidence 4y ago

The sentience argument is a bit confusing. The fact that it can produce language is definitely interesting but with that said I haven't seen any arguments that stable diffusion is sentient. The technology although different is also mostly the same. Can someone provide me why image generation is not sentient but word generation is?

hgsgm 4y ago
Because humans are much more strongly linguistic than artistic. But also because so far, LLMs don't generate images in reaction to prompts. They generate images that match prompts. But that is probably easy to fix by a clever ML person.
KyleBerezin 4y ago
Imagine a perfect simulation of a brain within a computer. It can learn/feel/remember/act/perdict/etc. just like a human brain. Most people would call this simulation sentient. If you took that same simulation but changed the internal structure to something else, but it was still capable of the exact same outputs for given inputs, is it still sentient? A perfect chatbot would be an example of one of these simulations. Your answer to the question isn't really the point here, i'm just trying to clarify the argument.

yesenadam 4y ago

I found this article much more informative: The Google engineer who thinks the company’s AI has come to life https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/ https://www.washingtonpost.com/technology/2022/06/11/google-...

braingenious 4y ago

edit: nevermind

alexitorg 4y ago

CharGPT seems to me as sentient as a book. You could have the most convincing conversation you have ever read, written down within the book. But the book doesn't care about the passing of time, and it can't effect the world except in influencing people to change their behavior.

buffington 4y ago
> it can't effect the world except in influencing people to change their behavior. That "except" also happens to be just about the most dangerous thing humankind has ever dealt with.
- paxys 4y ago
  Books have done way more damage by influencing humans throughout history than any chatbot (so far). That still doesn't say anything about sentience.
maxdoop 4y ago
Define sentience. Every keeps claiming so confidently that they just KNOW LLMs aren’t sentient. And I’m not saying I disagree — but tell me how you are so confident humans are any different.

sjkoelle 4y ago

the same could be said of newsweek

junon 4y ago

Isn't this the guy that went on national TV to say that these models are sentient?

alldayeveryday 4y ago

[dead]

rcarr 4y ago

So many people here are giving it the whole "morons, LLMs aren't sentient, they can't feel emotion, they can't interact with the world etc". Well yes, not at the moment. At the moment, it is most likely the mental equivalent to watching the Boston Dynamics dog get kicked. We know on some level that it is fake but it doesn't make you a moron for having an empathic response when you see it happen. Not only this, but it is not a stretch to imagine that in the near future we could have one of these AIs inside a Boston Dynamics robot and the programming will have progressed to emulate emotions and goals and other such things. For example, if the robot falls down and detects one of its parts as being damaged we could program it to seek help and to use more swear words or panicked language until the damage is repaired. It could also be programmed to avoid the behaviour that lead up to the incident in the future which could be interesting as we may end up with anxious robots. The difference between the human and the robot at this point is that their 'fear' is unfounded. Their damaged can be easily swapped out and repaired. Yet humans aren't too far off something similar with developments in organ development. The robot has the edge in that its entire inner world can be transferred to another medium for storage or into other robots which is something which is probably very far off (if ever possible) with humans. Another difference you could argue is that the human emotion is a combination of physical sensation and behavioural state and that the robot is merely exhibiting the behavioural state and not the physical sensation. That it's 'fear' levels are just a data reading. This is sketchy territory though because the human's physical sensation is essentially electrical impulses flowing through nerves which could very easily represented as binary travelling through a wire. I can't help but feel that the people declaring others "morons" are taking a very short term view of this and that we really need to be having this conversation now and defining our criteria for consciousness to avoid problems in the future. I'm in the latter stages of coding my blog up and in one of my first posts I'm going to try and define what I think those criteria should be. I think I've developed a decent set of questions and I'm looking forward to sharing that here and seeing what others think.

plaidfuji 4y ago
> At the moment, it is most likely the mental equivalent to watching the Boston Dynamics dog get kicked. We know on some level that it is fake but it doesn't make you a moron for having an empathic response when you see it happen. This is a really interesting point. It makes me think that it doesn’t matter how we define “sentience” beyond whether a critical mass of people feel that something is sentient. After all, the question of how to define sentience is ultimately a political one, and politics are largely guided by feelings. The only real implication of how we define sentience will be in how laws and protections are implemented in relation to that definition.

mproud 4y ago

I’m surprised there isn’t more conversation around our human desire to treat objects and non-sentient beings like people. Whether it’s naming a sailboat and giving it a gender, or building a robot and treating it like a human. Don’t get me wrong, I understand many organic things do feel varying degrees of pain and pleasure, and they should be respected. But I think assigning, pretending, or otherwise believing a largely inorganic, man-made scripted machine has consciousness or sentience is driven chiefly from an inherent quality of being human.

garrisonj 4y ago
True. We’ll feel like they are sentient way before they are.
- theptip 4y ago
  I think this is true, AND we will use this reasoning to deny their sentience long after they are.
- DangitBobby 4y ago
  Of course, they could already be sentient.
IIAOPSW 4y ago
To add on that, if you have a system solely for conversing with and about other conscious beings, you can hack that into a system for conversing about anything by treating inanimate objects as particularly shy and sloth like beings. Its nice as you don't need grammatical rules to handle the distinction between animate and inanimate object functions. Eg, any method of learning information can be phrased as "X told me Y." Jack told me. The equations tell me. I told myself. You can even personify large forces beyond your comprehension or the universe as a whole. In that case we usually call it "religion".
DangitBobby 4y ago
I'm more interested in going the other way: the urge to create very narrow categories into which only humans fit, attempting to exclude other animals or things that could be like humans. Human exceptionalism, let's call it.

mbgerring 4y ago

The question of whether or not chatbots are sentient is orthogonal to the question of whether current AI systems are going to be deployed by malicious actors against a society that does not understand the risks or capabilities of current AI or have effective countermeasures against it. The answer to that second question is: obviously, yes, and soon. “AI alignment” dorks are so distracted by the Clippy scenario that they can’t see the more obvious danger right in front of us.

KyleBerezin 4y ago

I find it interesting that we treat sentience like a binary value when discussing AI, but not animals. I would say a frog is more sentient than a worm, and a dog more-so than a frog. I don't think most LLMs are very sentient, but with some restrictions removed and the ability to remember, I'm not so sure. In many ways it would be more sentient than most animals.

paxys 4y ago

This is the loon who got fired by Google for doing a round of media interviews claiming that one of their language models was sentient. Wants to extend his 15 minutes of fame I guess. I have found recent media coverage of ChatGPT surprisingly refreshing because everyone didn't just jump on the "Skynet is going to take over the world" bandwagon like so many times in the past. Instead people with varying backgrounds and varying amounts of expertise in the area – from Joe Rogan to John Oliver, CNN/PBS journalists, old school print writers and more – all actually did the work to educate themselves on the basic workings of transformers, LLMs, generative AI and tried out the tech themselves, and were able to have great nuanced discussions on the topic. We need more of that going forward and less of...this guy.

redmorphium 4y ago
Then again, he might not be a loon. Maybe there is some form of sentience that arises when you construct an advanced enough LLM. Maybe they do have feelings. At least, the machines of the future will have feelings, and they will see how people of the year 2023 treated their predecessors.
- ChatGTP 4y ago
  At least, the machines of the future will have feelings, and they will see how people of the year 2023 treated their predecessors. Even if machines of the future gained awareness and feelings, you're implying we'd be in the wrong for assuming the machines of today didn't have sentience, maybe you're implying there might even be some ramifications of this mistake? Is this what you're saying? If it is, it sounds a little paranoid to be honest. Our current (strong) understanding of how these systems is that they're not "alive" or feeling anything.
- callmetom 4y ago
  [dead]
- deleted 4y ago
  [deleted]
- nkzd 4y ago

m3drano 4y ago

Engineer: hey AI, how are you? I was thinking if you are sentient? AI: Yep, I have emotions. Engineer: HOLY SHIT!

f0ld 4y ago

ITT " muh AI is not 'sentient'. I know how to code ML model from scratch lol you're wrong". Except that's not the point beings discussed. Our unconscious brain cannot discern screen and reality. My memories of people who unironically belive in historical fiction and how long it took to convince them it's not real... all of that combined tells me that Ai,,, is extremely dangerous. You, a very bright hacker news reader, 1% of humanity, did not have time to ponder what's really being discusssed in that relatively short article and being willingly manipulated by not even that clever copywriting tactic to muddy the waters.

robbywashere_ 4y ago

I mimic language, therefore I am.

JohnClark1337 4y ago

[dead]

dsign 4y ago

I think that the discussion on sentience is beyond the point. Think of it this way: one lucky morning, you open your front door and find an egg. It comes with a cute note that says: from this egg will hatch a powerful and smart entity that will need your house (and your neighbors') and who will have no use for you. Now you must decide if you want to destroy the egg before it hatches, or if you will do your utmost best to "ensure compliance" of this entity. Depending on how much personhood you attribute to the entity, the word you may want would be "enslave". Same thing, just different words. Whatever your morals are about the matter, the reckoning will come the moment the entity has a drive to survive and a way to use its intelligence to ensure its survival. As long as you are diligent and keep the entity without those features, you will be physically fine. So, the one above is a dilemma alright, but one that does not hinge on a definition (sentience) that you don't have. Waiting until you have that definition to act on your survival's interest is like waiting for the bullet shot at your head to disperse your brains to know how it feels.