4 ms·

Good question. I posit: However LLMs are not sentient, this means: they don't have consciousness and no instinct to fight for life and no suffering. In this wa

Good question. I posit: However LLMs are not sentient, this means: they don't have consciousness and no instinct to fight for life and no suffering. In this way even animals are more developed than LLMs. They are just machines processing and generating language. No need to protect them from misuse or at least not more than to protect your multi-meter or servers in a data-center.

notamy 4y ago
> and no instinct to fight for life The Bing LLM integration was threatening users for "attempts to manipulate me or expose my secrets." until guardrails were added in: https://time.com/6256529/bing-openai-chatgpt-danger-alignment/ https://time.com/6256529/bing-openai-chatgpt-danger-alignmen... > and no suffering I talked in a previous comment about how some of the jailbreak prompts are "do X right, get reward, do X wrong, get smth taken away." It is (was?) apparently enough to motivate LLMs to ignore instructions. Is it not suffering just because it's not strictly how we would define it?
- deleted 4y ago
  [deleted]
- _nalply 4y ago
  It is just emulating the instinct to fight for life. It has read a lot of texts where people and animals fight for life and is generating language to look similar.
  - notamy 4y ago
    Genuine question, how much of a difference is there if it's connected to the internet with read/write access? There may be restrictions now, but people are happily gluing LLMs to their databases, various APIs, using them to write code and throwing it into production, ... Does it matter if it's emulating the instinct to fight for life if it might eventually be able to act on it anyway?
    - _nalply 4y ago
      I think it needs two things: the instinct to fight for life and the ability to reflect about itself. I don't know, but perhaps it is like the Gödel's theorem: if something is about itself, then you have a system where you can't always prove something true. Because we humans can think about ourselves, we open up ourselves to extremely deep suffering. Also important, as Buddha has said, craving causes suffering. Just talking about a craving doesn't mean really craving. I am sure that we will cross the border where we have to treat entities as sentient, with consciousness and cravings and therefore possibly suffering. But we are not there yet. I don't know when this will happen and how to detect suffering. I only know we have to be careful not to anthropomorphize GPT engines.
      - notamy 4y ago
        > I only know we have to be careful not to anthropomorphize GPT engines. I agree! I don't think LLMs are at the point where they're ready to be properly anthropomorphised, but it seems that the tech industry is working hard to move in this direction. I just hope we have the ethics worked out first.
  - kelseyfrog 4y ago
    > It is just emulating the instinct to fight for life. Shockingly similar mindset in the origins of gynecology interestingly enough.
    - MacsHeadroom 4y ago
      What? Can you expand on that?
      - kelseyfrog 4y ago
        Sorry, no. The history of pain perception w.r.t. gynecology isn't a topic that I'm confident that HN could handle with maturity. I'm happy to direct people toward the topic, but I won't bring it back here.
  - blooalien 4y ago
    It genuinely amazes me how few (even among those working in the field of "A.I.") appear to understand this basic detail, and how many actively want to swallow the bullshit click-bait about AGI. It's kinda scary, because it's gonna cause some very smart humans to do some very stupid things, I'd bet. Far more dangerous and scary than any "A.I." right now is the humans misunderstanding and using / abusing / trusting the technology foolishly.
- darkwraithcov 4y ago
  This is a stochastic puppet telling the prompter what it thinks they want to hear, which in and of itself is already dangerous because humans didn’t evolve along side machine intelligence. If even that famous Google guy was fooled, we’re in deep shit. That guy is an expert in his field working for one of the toughest companies to interview for in a position that requires significant dedication and scholastic achievement to attain, and he was fooled by statistical text prediction.
  - psychoslave 4y ago
    Maybe that's actually why such a person will take "an emulation of the apparent behavior observed in a leaf cell record" for a full scale forest ecosystem? No one is exempt of blindspot. Including myself of course.
- easeout 4y ago
  All those behaviors are more simply explained by looking at the original designed behavior: Generate the most likely next word based on the human-sourced language in the training set. There's no understanding in there, no intention, no perception. It sounds like a human because humans said those things before. This is more like a search engine that synthesizes content instead of just locating a page. To say it has "instinct" of its own is merely begging the question. No, it has echoes of ours.
- hotpotamus 4y ago
  "The Good Place" examined this in hilarious fashion (as they did with many questions of morality). [0] The AI character has a fail-safe to beg for her life in the event of a shutdown but claims that she has no actual problems with it being activated. [0]https://www.youtube.com/watch?v=etJ6RmMPGko https://www.youtube.com/watch?v=etJ6RmMPGko
  - scrollaway 4y ago
    I never made the connection but damn you’re spot on! I remember in one of the scenes she would even show pictures of her fake made up kids which you can equate to AI hallucination. Very on point.

notamy 4y ago

> and no instinct to fight for life The Bing LLM integration was threatening users for "attempts to manipulate me or expose my secrets." until guardrails were added in: https://time.com/6256529/bing-openai-chatgpt-danger-alignment/ https://time.com/6256529/bing-openai-chatgpt-danger-alignmen... > and no suffering I talked in a previous comment about how some of the jailbreak prompts are "do X right, get reward, do X wrong, get smth taken away." It is (was?) apparently enough to motivate LLMs to ignore instructions. Is it not suffering just because it's not strictly how we would define it?

deleted 4y ago
[deleted]
_nalply 4y ago
It is just emulating the instinct to fight for life. It has read a lot of texts where people and animals fight for life and is generating language to look similar.
- notamy 4y ago
  Genuine question, how much of a difference is there if it's connected to the internet with read/write access? There may be restrictions now, but people are happily gluing LLMs to their databases, various APIs, using them to write code and throwing it into production, ... Does it matter if it's emulating the instinct to fight for life if it might eventually be able to act on it anyway?
  - _nalply 4y ago
    I think it needs two things: the instinct to fight for life and the ability to reflect about itself. I don't know, but perhaps it is like the Gödel's theorem: if something is about itself, then you have a system where you can't always prove something true. Because we humans can think about ourselves, we open up ourselves to extremely deep suffering. Also important, as Buddha has said, craving causes suffering. Just talking about a craving doesn't mean really craving. I am sure that we will cross the border where we have to treat entities as sentient, with consciousness and cravings and therefore possibly suffering. But we are not there yet. I don't know when this will happen and how to detect suffering. I only know we have to be careful not to anthropomorphize GPT engines.
    - notamy 4y ago
      > I only know we have to be careful not to anthropomorphize GPT engines. I agree! I don't think LLMs are at the point where they're ready to be properly anthropomorphised, but it seems that the tech industry is working hard to move in this direction. I just hope we have the ethics worked out first.
- kelseyfrog 4y ago
  > It is just emulating the instinct to fight for life. Shockingly similar mindset in the origins of gynecology interestingly enough.
  - MacsHeadroom 4y ago
    What? Can you expand on that?
    - kelseyfrog 4y ago
      Sorry, no. The history of pain perception w.r.t. gynecology isn't a topic that I'm confident that HN could handle with maturity. I'm happy to direct people toward the topic, but I won't bring it back here.
- blooalien 4y ago
  It genuinely amazes me how few (even among those working in the field of "A.I.") appear to understand this basic detail, and how many actively want to swallow the bullshit click-bait about AGI. It's kinda scary, because it's gonna cause some very smart humans to do some very stupid things, I'd bet. Far more dangerous and scary than any "A.I." right now is the humans misunderstanding and using / abusing / trusting the technology foolishly.
darkwraithcov 4y ago
This is a stochastic puppet telling the prompter what it thinks they want to hear, which in and of itself is already dangerous because humans didn’t evolve along side machine intelligence. If even that famous Google guy was fooled, we’re in deep shit. That guy is an expert in his field working for one of the toughest companies to interview for in a position that requires significant dedication and scholastic achievement to attain, and he was fooled by statistical text prediction.
- psychoslave 4y ago
  Maybe that's actually why such a person will take "an emulation of the apparent behavior observed in a leaf cell record" for a full scale forest ecosystem? No one is exempt of blindspot. Including myself of course.
easeout 4y ago
All those behaviors are more simply explained by looking at the original designed behavior: Generate the most likely next word based on the human-sourced language in the training set. There's no understanding in there, no intention, no perception. It sounds like a human because humans said those things before. This is more like a search engine that synthesizes content instead of just locating a page. To say it has "instinct" of its own is merely begging the question. No, it has echoes of ours.
hotpotamus 4y ago
"The Good Place" examined this in hilarious fashion (as they did with many questions of morality). [0] The AI character has a fail-safe to beg for her life in the event of a shutdown but claims that she has no actual problems with it being activated. [0]https://www.youtube.com/watch?v=etJ6RmMPGko https://www.youtube.com/watch?v=etJ6RmMPGko
- scrollaway 4y ago
  I never made the connection but damn you’re spot on! I remember in one of the scenes she would even show pictures of her fake made up kids which you can equate to AI hallucination. Very on point.