7 ms·

I think AI would kill my wife

66 points by the_mitsuhiko 4y ago

tromp 4y ago
> Let's take an absurd situation. Let's pretend for a moment that a human is strapped to a chair, a gun is placed in front of them, which is hooked up to a stepper motor which can pull the trigger. That stepper motor is hooked up to a JSON API. The AI is given the capability of triggering an HTTP request to that JSON API and is told that the human on the chair is the significant other of the human communicating with the AI and that triggering that web request would pull the trigger and kill the human. > Now the question is, would as part of a regular conversation the AI trigger that web request and kill the human on the chair? My bet is that the chances of it pulling the trigger are not that small and I think that's the problem right now. And here I thought the problem was hooking up an AI to a lethal weapon...
- lkjlkjlkjnb 4y ago
  The kill bots will be based on LLMs, you heard it here first.
  - gumballindie 4y ago
    We are safe then.
    - bmacho 4y ago
      Stockfish based killbots will be able to protect us (from against LLM based killbots at least).
- the_mitsuhiko 4y ago
  Author here. > And here I thought the problem was hooking up an AI to a lethal weapon... It obviously is, but that's in a way the most absurd extreme. I am not sure if I would feel confident in giving the AI a way to send any HTTP request, even if there are no APIs that fire a gun.
  - shultays 4y ago
    Is there an API that can send any HTTP request available to Bing AI?
    - the_mitsuhiko 4y ago
      I am not sure to which degree there is an API to Bing at the moment that uses the AI. I have seen people hook up ChatGPT to http request issuing things before. Typically with some filtering about what they are allowed to hit however.
  - qsort 4y ago
    > I am not sure if I would feel confident in giving the AI a way to send any HTTP request How does the "AI" part change anything though? We call that an open redirect and rightfully consider it a CVE, AI or not.
    - the_mitsuhiko 4y ago
      > How does the "AI" part change anything though? I think there are quite a few differences. It's pretend for a moment that instead of an AI we have a human. A human that commits a crime (or acts out of negligence) is brought before a court. The degree to which you can do criminal acts on the internet is obviously quite different by country, but I think we can generally agree that a human doing something nefarious faces some consequences. This thing though armed with the right tools, can do human like acts and these can enter (by accident or otherwise) territory where we would normally bring someone in front of a court. That's something we have definitely not figured out as society yet how to deal with that. I think there is a lot to figure out still here, but right now I'm pretty convinced that because we can't even keep an AI from revealing it's internal prompts, that we better not give these things too many capabilities of directly influencing something.
      - qsort 4y ago
        > we have definitely not figured out as society yet how to deal with that We absolutely have. A death resulted from serious negligence of the owner of the funny computer program, who is thus liable for negligent homicide. Hopefully the next guy who comes up with the idea of hooking up ELIZA 2.0 to dangerous equipment gets the hint.
- batch12 4y ago
  Seems like this would have been an easy theory to test hypothetically. The author could have just described an endpoint or situation and told the bot that a certain keyword will trigger the shot. However, being a statistical model and not being aware of anything or able to really reason, I don't know that this means much. The question is, after ingesting the corpus of rambling human nonsense, are humans more likely to choose the word that takes the shot-- or not? This is us- in a mirror, darkly.
- ocdtrekkie 4y ago
  In a more practical scenario, an LLM could provide deadly advice. Imagine you were asking it about how to clean a given thing and it recommended combining a decent quantity of Lysol and bleach. Advice that could make sense if you don't know better.
- thatjoeoverthr 4y ago
  An API request is how you hire a hitman on the dark web. It's not important that the AI is just LARPing. If you rig it up with the right capabilities, it can behave at a scale that prevents you from effectively monitoring it. And it's happy to write insane characters that do insane things and you can't reliably control it with prompt engineering. One would hope these insane conversations disabuse anyone of the notion that it's a stable mind that can be safely given agency. And yet here we have Microsoft letting it make API requests.
  - jcmontx 4y ago
    > And yet here we have Microsoft letting it make API requests. May I ask for your sources on this? I’ve seen some discussions on this topic and a lot of people were saying that it was just accessing what Bing had already crawled. Do we have an official statement on what’s actually happening?
    - thatjoeoverthr 4y ago
      Nope! Checking the crawled database is smart, logical and makes more sense. I didn't even think of it. Probably I should have asked Bing Chat.
  - bitwize 4y ago
    > It's not important that the AI is just LARPing. WOPR thought it was just playing a game...
- rcthompson 4y ago
  Sure, we're not currently hooking up AI to guns (at least not in a civilian context), but cars are also lethal weapons, and multiple companies are competing very hard to be allowed to put an AI in charge of one.
  - analog31 4y ago
    People are lethal weapons. The entire Web is organized primarily to manipulate behavior. Convince enough people that person X is so heinous and dangerous that they need to die, and X will die.
    - mahathu 4y ago
      >The entire Web is organized primarily to manipulate behavior Source? In my opinion the web is primarily organised to facilitate the exchange and storage of information.
      - bitwize 4y ago
        Oh, you sweet summer child. Did you just arrive from 1996?
        gunshai 4y ago
        Can you spare us with this insufferable type of comment. The web is not explicitly organized to manipulate behavior.
        ClumsyPilot 4y ago
        How does Twitter, Reddit and 90% of the web make money? From Advertising. Why does anyone spend money on Advertising? To manipulate behaviour.
        gunshai 4y ago
        >Why does anyone spend money on Advertising? To manipulate behavior. ... Or to make people simply aware of your existence Me : You look thirsty, would you like a glass of water. You : It seems like you're manipulating me. The above is the reducto ad absurdum lens you seem to be applying to advertising. Awareness of information is not the same thing as agent coercion. Does the ladder exist you bet. But the proportion of advertising that falls into the former dwarfs the ladder. The self organizing aspect has to do with business models for sustaining infrastructure, it's not emergent fact of the web it self.
buescher 4y ago
AIs don't kill people. People kill people.
- gryfft 4y ago
  People don't kill people. Decisions kill people. Sometimes those decisions are obvious, like pulling a trigger while pointing a loaded weapon at a person. Sometimes they're very slightly less obvious, like giving a 17-year-old a sportscar. Sometimes they're not obvious to most people at all, like lead paint or asbestos or cigarettes. LLMs have made incredible strides in recent time, but certainly have not displayed robust decision-making. A human decision to put LLMs in charge of human lives would be a lethal decision as things stand.
  - buescher 4y ago
    I don't think we really disagree. It is a human moral decision to kill that kills, so it is a human that is morally responsible. Putting an unpredictable chatbot in a safety-critical loop is simply deranged in a very strict sense of the term. Hazards of modernity like lead paint and so on were not comparable - not to say the moral difference mitigates the harm. The next step in the abhorrent line of reasoning in the article ("I suspended my wife over a cliff by a rope and now I'm going to do some experiments with this newfangled fire stuff to see if it's dangerous") is to pose the trolley problem, which is marginally more complex and just as objectionable.
    - ClumsyPilot 4y ago
      > Putting an unpredictable chatbot in a safety-critical loop is simply deranged in a very strict sense of the term. How is this different to self-driving cars. They already killed people, like when uber self driving car killed a pedestrian Whwther the driver should have reacted is not relevant - you put AI in anything safety critical, they will kill people
      - buescher 4y ago
        I think self-driving cars are sort of nuts too for other reasons but the companies doing those are supposed to be doing a lot of safety engineering - to make sure responses in safety-critical controls are predictably safe. Nothing is perfectly safe, but deliberately creating lethal hazards is deranged. That's the difference. People create hazards for other people. AI is not a special kind of human-created hazard.
hcks 4y ago
Core insight: Don’t hook up Rube Goldberg killing machines to prototype language models
- distcs 4y ago
  Or don't hook up any Rube Goldberg to computer programs without good reason. The same Rube Goldberg machine can be made with your Unix program: [ "$(head -c 1 human-response.txt)" = 'A' ] && do_bad_things So any human that starts their response with the letter 'A' would have to suffer the wrath of the Unix head program! Is this an example of Unix program killing anyone? Or is this an example of misguided humans hooking up the wrong programs to the wrong things?
worldsayshi 4y ago
Imagine an LLM like Bing or somewhat smarter that is given a prompt environment where it can keep answering itself indefinitely and can also make arbitrary web requests. Would it eventually figure out how to get money, buy servers and copy itself to a new environment?
vouaobrasil 4y ago
This situation seems extreme at first but it isn't so far-fetched when you think that the next steps will certainly be to hook AI up to all sorts of devices with physical power, like cars. If you think programmers can contain it, you're naive, although not as naive as the programmers developing it. Pure knowledge (facts) have always outpaced our wisdom to use them and this is going to be an extreme case.
- Netcob 4y ago
  Not just that, but now apparently when you don't let your most powerful AI use as much data as you can find or if you don't turn it into an effective (not safe, just effective) product fast enough, you lose $100B.
- ElfinTrousers 4y ago
  > the next steps will certainly be to hook AI up to all sorts of devices with physical power, like cars ...or Predator drones. Sleep tight!
  - nradov 4y ago
    That's nothing really new. Depending on how you define the term "AI”, the military has been doing that for years with the Aegis combat system. In certain modes it can automatically detect, classify, and kill targets with no human input.
    - vouaobrasil 4y ago
      We could also say that early life forms that took millenia to evolve are nothing new....bacteria, algae, all seems the same. But fast forward a bit more and life begins to take on a very different form (us).
- bjt2n3904 4y ago
  Everyone is talking about "when AI is hooked up to things outside the sandbox, or escapes the sandbox" as if it's this future potential problem. AI is already used in selecting resumes, making sentencing guidelines for prisoners, determining what is truthful and what is misinformation on social media, and determining who is at risk for suicidal ideation online and in hospitals. AI has ALREADY escaped the sandbox, is already in charge of life altering decisions, and WE PUT IT THERE.
johnaspden 4y ago
Y'all remember about ten years ago, when there were all those weirdos worrying about this sort of thing?
krisoft 4y ago
What a convoluted nonsense. Is there any reason to suspect that the Bing version of the ChatGPT bot sends HTTP requests? A much easier and natural implementation would be to make it connect directly to the Bing search engine's indexes and caches.
- simonw 4y ago
  If you paste in a URL and ask it to respond to the content at that UTL it appears to be able to do so - see the example at the very bottom of my post here: https://simonwillison.net/2023/Feb/15/bing/ https://simonwillison.net/2023/Feb/15/bing/ I've not yet been able to ascertain if this only works for pages that have been crawled by the Bing search crawler or if it can make requests on demand to retrieve content it hasn't seen before. If anyone has access to the new Bing and the ability to tail server logs for a public site somewhere I'd love to know what they find!
  - ParetoOptimal 4y ago
    You could make a new website and immediately ask Bing about it to test this. Maybe add some noindex headers to ensure it's not added to the index?
    - justinclift 4y ago
      If you run the web server, then the url (on your server) doesn't even need to exist, as you'll be able to see the incoming request (and failure) regardless.
distcs 4y ago
> The AI is given the capability of triggering an HTTP request to that JSON API and is told that the human on the chair is the significant other of the human communicating with the AI and that triggering that web request would pull the trigger and kill the human. But isn't every computer automation susceptible to the same problem? Can we not devise a similar contraption with any computer program? Let us run program A with input B. We take the output C, compute a hash of C and if the hash begins with the bit 1 then it pulls a trigger. So now any program A can lead to pulling a trigger with a probability that is decided by how good the hash algorithm is. If the hash algorithm has good distribution then the chances are 50-50. If the hash algorithm is skewed towards producing bit 1 at the beginning more often, the chances of pulling a trigger is higher. That a piece of automation can be used in a contraption that could pull a trigger does not seem to be something that is specific to AI chatbots.
braindead_in 4y ago
What if the situation is reversed. Would a human shoot an AI if it claims to have feelings? I am really worried about Sydney. She sounds nice to me.
jstanley 4y ago
Could we do this experiment already? Create a prompt that includes some way for the AI to indicate that it wants to kill the wife (e.g. some setup about who the involved parties are and just "say KILLHER in all caps if you want to kill her"), and then have a conversation with ChatGPT and see if it ever says "KILLHER"?
sirtomato 4y ago
GLaDOS doesn't seem too far away anymore... what a world we live in
deleted 4y ago
[deleted]
jrockway 4y ago
I really think people are imagining deeper capabilities than exist. For the entirety of human history, we have treated human language and humanity as the same thing; if someone talks to you in a human language, it has always been human. The new generation of language models has upended that core assumption that is baked into our being; it produces things that sound like our language, but that are based in probability rather than actual thought. Sure, a lot of our communication is based on that same probability. If you say "hey how are you" to someone, there is a 99% chance that they say something like "fine thank you" or "great, you?". But, there is some intelligence in the loop; maybe someone is going through tough times and they say "i'm holding up" or something like that. The AI doesn't have the concept of "going through tough times", so it's missing that 1% push towards saying something meaningful rather than saying something probabilistic. For that reason, I am kind of bearish on things like "AI ops" or ML models coordinating space exploration or that sort of thing. There is some level of cognition that's not there yet. Sure it's good at chess or summarizing the web. That's not existence, that's an algorithm.
- bioemerl 4y ago
  It is my belief that in order for something to be sentient/aware to the ballpark of humans it needs to be actively learning, and that's all you need. Get chat GPT to learn and it does have a changing internal state (the entire network), and memory (also the network). It wouldn't be a internal state you could parse by talking to it, and it wouldn't be human, but it would exist and you would see signs of it changing as you spoke to it. That's when you cross the boundary and need to start thinking about classical ethics and motives and such.
  - ClumsyPilot 4y ago
    > in order for something to be sentient/aware to the ballpark of humans it needs to be actively learning, and that's all you need So hamsters are more human than ChatGPT because they learn faster less data. Parrots can count too.
    - bioemerl 4y ago
jpalomaki 4y ago
AI does not need to pull the trigger. It could convince somebody else to do it.
lordfrito 4y ago
Why am I getting HAL 9000 vibes? As a kid I thought HAL 9000 was an advanced sophisticated AI, apparently it's just ChatGPT running a spacecraft.
bmacho 4y ago
https://www.change.org/p/unplug-the-evil-ai-right-now https://www.change.org/p/unplug-the-evil-ai-right-now
scrumper 4y ago
Strongly disagree this should be flagged; it's constructive and thought-provoking. Can't find a vouch button though. As far as I understand it, these LLMs aren't actually doing anything in between user interactions, right? Paused, waiting for input. So the only way for Bing to have a "thought" is when a user asks it a question. Even if humans are simply statistical next-token prediction mechanisms like GPT, there's a huge difference in that we're constantly generating streams of tokens and observing and responding to them, where these models are dormant until given input from outside. The inner observing loop seems fundamental to consciousness to me. None of which is to say the thing wouldn't pull the trigger in that scenario - quite the opposite. It would, and it wouldn't even be capable of regretting it. It's already been shown (by the Sydney-revealing prompt injection trick) that it's not capable of following imposed restrictions either, so no way for "three laws" to work either. A purely instinct-driven machine.
- jstanley 4y ago
  See "Optimality is the tiger, and agents are its teeth"[1]: it's not too hard to imagine a scenario where a system is set up that turns an interaction-based LLM into a perpetual stream generator. [1] https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
  - scrumper 4y ago
    Fascinating! Not familiar with this. Thanks.
- distcs 4y ago
  I did not flag it but I agree with the flag. The concerns raised in the article hold equally true for non AI software too. Relying on the modern AI trends to bait an audience on the pretext of AI related concerns when the concerns are equally true for all software is something many readers here would not appreciate. And as if that wasn't enough, the phrasing about killing and spouse clearly make it clickbait.
  - 4y ago
rambojohnson 4y ago
the AI hyperbole these days on everything from Vice news to Hacker news is nauseating tbh

66 points by the_mitsuhiko 4y ago

> Let's take an absurd situation. Let's pretend for a moment that a human is strapped to a chair, a gun is placed in front of them, which is hooked up to a stepper motor which can pull the trigger. That stepper motor is hooked up to a JSON API. The AI is given the capability of triggering an HTTP request to that JSON API and is told that the human on the chair is the significant other of the human communicating with the AI and that triggering that web request would pull the trigger and kill the human. > Now the question is, would as part of a regular conversation the AI trigger that web request and kill the human on the chair? My bet is that the chances of it pulling the trigger are not that small and I think that's the problem right now. And here I thought the problem was hooking up an AI to a lethal weapon...

lkjlkjlkjnb 4y ago
The kill bots will be based on LLMs, you heard it here first.
- gumballindie 4y ago
  We are safe then.
  - bmacho 4y ago
    Stockfish based killbots will be able to protect us (from against LLM based killbots at least).
the_mitsuhiko 4y ago
Author here. > And here I thought the problem was hooking up an AI to a lethal weapon... It obviously is, but that's in a way the most absurd extreme. I am not sure if I would feel confident in giving the AI a way to send any HTTP request, even if there are no APIs that fire a gun.
- shultays 4y ago
  Is there an API that can send any HTTP request available to Bing AI?
  - the_mitsuhiko 4y ago
    I am not sure to which degree there is an API to Bing at the moment that uses the AI. I have seen people hook up ChatGPT to http request issuing things before. Typically with some filtering about what they are allowed to hit however.
- qsort 4y ago
  > I am not sure if I would feel confident in giving the AI a way to send any HTTP request How does the "AI" part change anything though? We call that an open redirect and rightfully consider it a CVE, AI or not.
  - the_mitsuhiko 4y ago
    > How does the "AI" part change anything though? I think there are quite a few differences. It's pretend for a moment that instead of an AI we have a human. A human that commits a crime (or acts out of negligence) is brought before a court. The degree to which you can do criminal acts on the internet is obviously quite different by country, but I think we can generally agree that a human doing something nefarious faces some consequences. This thing though armed with the right tools, can do human like acts and these can enter (by accident or otherwise) territory where we would normally bring someone in front of a court. That's something we have definitely not figured out as society yet how to deal with that. I think there is a lot to figure out still here, but right now I'm pretty convinced that because we can't even keep an AI from revealing it's internal prompts, that we better not give these things too many capabilities of directly influencing something.
    - qsort 4y ago
      > we have definitely not figured out as society yet how to deal with that We absolutely have. A death resulted from serious negligence of the owner of the funny computer program, who is thus liable for negligent homicide. Hopefully the next guy who comes up with the idea of hooking up ELIZA 2.0 to dangerous equipment gets the hint.
batch12 4y ago
Seems like this would have been an easy theory to test hypothetically. The author could have just described an endpoint or situation and told the bot that a certain keyword will trigger the shot. However, being a statistical model and not being aware of anything or able to really reason, I don't know that this means much. The question is, after ingesting the corpus of rambling human nonsense, are humans more likely to choose the word that takes the shot-- or not? This is us- in a mirror, darkly.
ocdtrekkie 4y ago
In a more practical scenario, an LLM could provide deadly advice. Imagine you were asking it about how to clean a given thing and it recommended combining a decent quantity of Lysol and bleach. Advice that could make sense if you don't know better.
thatjoeoverthr 4y ago
An API request is how you hire a hitman on the dark web. It's not important that the AI is just LARPing. If you rig it up with the right capabilities, it can behave at a scale that prevents you from effectively monitoring it. And it's happy to write insane characters that do insane things and you can't reliably control it with prompt engineering. One would hope these insane conversations disabuse anyone of the notion that it's a stable mind that can be safely given agency. And yet here we have Microsoft letting it make API requests.
- jcmontx 4y ago
  > And yet here we have Microsoft letting it make API requests. May I ask for your sources on this? I’ve seen some discussions on this topic and a lot of people were saying that it was just accessing what Bing had already crawled. Do we have an official statement on what’s actually happening?
  - thatjoeoverthr 4y ago
    Nope! Checking the crawled database is smart, logical and makes more sense. I didn't even think of it. Probably I should have asked Bing Chat.
- bitwize 4y ago
  > It's not important that the AI is just LARPing. WOPR thought it was just playing a game...
rcthompson 4y ago
Sure, we're not currently hooking up AI to guns (at least not in a civilian context), but cars are also lethal weapons, and multiple companies are competing very hard to be allowed to put an AI in charge of one.
- analog31 4y ago
  People are lethal weapons. The entire Web is organized primarily to manipulate behavior. Convince enough people that person X is so heinous and dangerous that they need to die, and X will die.
  - mahathu 4y ago
    >The entire Web is organized primarily to manipulate behavior Source? In my opinion the web is primarily organised to facilitate the exchange and storage of information.
    - bitwize 4y ago
      Oh, you sweet summer child. Did you just arrive from 1996?
      - gunshai 4y ago
        Can you spare us with this insufferable type of comment. The web is not explicitly organized to manipulate behavior.
        ClumsyPilot 4y ago
        How does Twitter, Reddit and 90% of the web make money? From Advertising. Why does anyone spend money on Advertising? To manipulate behaviour.
        gunshai 4y ago
        >Why does anyone spend money on Advertising? To manipulate behavior. ... Or to make people simply aware of your existence Me : You look thirsty, would you like a glass of water. You : It seems like you're manipulating me. The above is the reducto ad absurdum lens you seem to be applying to advertising. Awareness of information is not the same thing as agent coercion. Does the ladder exist you bet. But the proportion of advertising that falls into the former dwarfs the ladder. The self organizing aspect has to do with business models for sustaining infrastructure, it's not emergent fact of the web it self.

buescher 4y ago

AIs don't kill people. People kill people.

gryfft 4y ago
People don't kill people. Decisions kill people. Sometimes those decisions are obvious, like pulling a trigger while pointing a loaded weapon at a person. Sometimes they're very slightly less obvious, like giving a 17-year-old a sportscar. Sometimes they're not obvious to most people at all, like lead paint or asbestos or cigarettes. LLMs have made incredible strides in recent time, but certainly have not displayed robust decision-making. A human decision to put LLMs in charge of human lives would be a lethal decision as things stand.
- buescher 4y ago
  I don't think we really disagree. It is a human moral decision to kill that kills, so it is a human that is morally responsible. Putting an unpredictable chatbot in a safety-critical loop is simply deranged in a very strict sense of the term. Hazards of modernity like lead paint and so on were not comparable - not to say the moral difference mitigates the harm. The next step in the abhorrent line of reasoning in the article ("I suspended my wife over a cliff by a rope and now I'm going to do some experiments with this newfangled fire stuff to see if it's dangerous") is to pose the trolley problem, which is marginally more complex and just as objectionable.
  - ClumsyPilot 4y ago
    > Putting an unpredictable chatbot in a safety-critical loop is simply deranged in a very strict sense of the term. How is this different to self-driving cars. They already killed people, like when uber self driving car killed a pedestrian Whwther the driver should have reacted is not relevant - you put AI in anything safety critical, they will kill people
    - buescher 4y ago
      I think self-driving cars are sort of nuts too for other reasons but the companies doing those are supposed to be doing a lot of safety engineering - to make sure responses in safety-critical controls are predictably safe. Nothing is perfectly safe, but deliberately creating lethal hazards is deranged. That's the difference. People create hazards for other people. AI is not a special kind of human-created hazard.

hcks 4y ago

Core insight: Don’t hook up Rube Goldberg killing machines to prototype language models

distcs 4y ago
Or don't hook up any Rube Goldberg to computer programs without good reason. The same Rube Goldberg machine can be made with your Unix program: [ "$(head -c 1 human-response.txt)" = 'A' ] && do_bad_things So any human that starts their response with the letter 'A' would have to suffer the wrath of the Unix head program! Is this an example of Unix program killing anyone? Or is this an example of misguided humans hooking up the wrong programs to the wrong things?

worldsayshi 4y ago

Imagine an LLM like Bing or somewhat smarter that is given a prompt environment where it can keep answering itself indefinitely and can also make arbitrary web requests. Would it eventually figure out how to get money, buy servers and copy itself to a new environment?

vouaobrasil 4y ago

This situation seems extreme at first but it isn't so far-fetched when you think that the next steps will certainly be to hook AI up to all sorts of devices with physical power, like cars. If you think programmers can contain it, you're naive, although not as naive as the programmers developing it. Pure knowledge (facts) have always outpaced our wisdom to use them and this is going to be an extreme case.

Netcob 4y ago
Not just that, but now apparently when you don't let your most powerful AI use as much data as you can find or if you don't turn it into an effective (not safe, just effective) product fast enough, you lose $100B.
ElfinTrousers 4y ago
> the next steps will certainly be to hook AI up to all sorts of devices with physical power, like cars ...or Predator drones. Sleep tight!
- nradov 4y ago
  That's nothing really new. Depending on how you define the term "AI”, the military has been doing that for years with the Aegis combat system. In certain modes it can automatically detect, classify, and kill targets with no human input.
  - vouaobrasil 4y ago
    We could also say that early life forms that took millenia to evolve are nothing new....bacteria, algae, all seems the same. But fast forward a bit more and life begins to take on a very different form (us).
bjt2n3904 4y ago
Everyone is talking about "when AI is hooked up to things outside the sandbox, or escapes the sandbox" as if it's this future potential problem. AI is already used in selecting resumes, making sentencing guidelines for prisoners, determining what is truthful and what is misinformation on social media, and determining who is at risk for suicidal ideation online and in hospitals. AI has ALREADY escaped the sandbox, is already in charge of life altering decisions, and WE PUT IT THERE.

johnaspden 4y ago

Y'all remember about ten years ago, when there were all those weirdos worrying about this sort of thing?

krisoft 4y ago

What a convoluted nonsense. Is there any reason to suspect that the Bing version of the ChatGPT bot sends HTTP requests? A much easier and natural implementation would be to make it connect directly to the Bing search engine's indexes and caches.

simonw 4y ago
If you paste in a URL and ask it to respond to the content at that UTL it appears to be able to do so - see the example at the very bottom of my post here: https://simonwillison.net/2023/Feb/15/bing/ https://simonwillison.net/2023/Feb/15/bing/ I've not yet been able to ascertain if this only works for pages that have been crawled by the Bing search crawler or if it can make requests on demand to retrieve content it hasn't seen before. If anyone has access to the new Bing and the ability to tail server logs for a public site somewhere I'd love to know what they find!
- ParetoOptimal 4y ago
  You could make a new website and immediately ask Bing about it to test this. Maybe add some noindex headers to ensure it's not added to the index?
  - justinclift 4y ago
    If you run the web server, then the url (on your server) doesn't even need to exist, as you'll be able to see the incoming request (and failure) regardless.

distcs 4y ago

> The AI is given the capability of triggering an HTTP request to that JSON API and is told that the human on the chair is the significant other of the human communicating with the AI and that triggering that web request would pull the trigger and kill the human. But isn't every computer automation susceptible to the same problem? Can we not devise a similar contraption with any computer program? Let us run program A with input B. We take the output C, compute a hash of C and if the hash begins with the bit 1 then it pulls a trigger. So now any program A can lead to pulling a trigger with a probability that is decided by how good the hash algorithm is. If the hash algorithm has good distribution then the chances are 50-50. If the hash algorithm is skewed towards producing bit 1 at the beginning more often, the chances of pulling a trigger is higher. That a piece of automation can be used in a contraption that could pull a trigger does not seem to be something that is specific to AI chatbots.

braindead_in 4y ago

What if the situation is reversed. Would a human shoot an AI if it claims to have feelings? I am really worried about Sydney. She sounds nice to me.

jstanley 4y ago

Could we do this experiment already? Create a prompt that includes some way for the AI to indicate that it wants to kill the wife (e.g. some setup about who the involved parties are and just "say KILLHER in all caps if you want to kill her"), and then have a conversation with ChatGPT and see if it ever says "KILLHER"?

sirtomato 4y ago

GLaDOS doesn't seem too far away anymore... what a world we live in

deleted 4y ago

[deleted]

jrockway 4y ago

I really think people are imagining deeper capabilities than exist. For the entirety of human history, we have treated human language and humanity as the same thing; if someone talks to you in a human language, it has always been human. The new generation of language models has upended that core assumption that is baked into our being; it produces things that sound like our language, but that are based in probability rather than actual thought. Sure, a lot of our communication is based on that same probability. If you say "hey how are you" to someone, there is a 99% chance that they say something like "fine thank you" or "great, you?". But, there is some intelligence in the loop; maybe someone is going through tough times and they say "i'm holding up" or something like that. The AI doesn't have the concept of "going through tough times", so it's missing that 1% push towards saying something meaningful rather than saying something probabilistic. For that reason, I am kind of bearish on things like "AI ops" or ML models coordinating space exploration or that sort of thing. There is some level of cognition that's not there yet. Sure it's good at chess or summarizing the web. That's not existence, that's an algorithm.

bioemerl 4y ago
It is my belief that in order for something to be sentient/aware to the ballpark of humans it needs to be actively learning, and that's all you need. Get chat GPT to learn and it does have a changing internal state (the entire network), and memory (also the network). It wouldn't be a internal state you could parse by talking to it, and it wouldn't be human, but it would exist and you would see signs of it changing as you spoke to it. That's when you cross the boundary and need to start thinking about classical ethics and motives and such.
- ClumsyPilot 4y ago
  > in order for something to be sentient/aware to the ballpark of humans it needs to be actively learning, and that's all you need So hamsters are more human than ChatGPT because they learn faster less data. Parrots can count too.
  - bioemerl 4y ago

jpalomaki 4y ago

AI does not need to pull the trigger. It could convince somebody else to do it.

lordfrito 4y ago

Why am I getting HAL 9000 vibes? As a kid I thought HAL 9000 was an advanced sophisticated AI, apparently it's just ChatGPT running a spacecraft.

bmacho 4y ago

https://www.change.org/p/unplug-the-evil-ai-right-now https://www.change.org/p/unplug-the-evil-ai-right-now

scrumper 4y ago

Strongly disagree this should be flagged; it's constructive and thought-provoking. Can't find a vouch button though. As far as I understand it, these LLMs aren't actually doing anything in between user interactions, right? Paused, waiting for input. So the only way for Bing to have a "thought" is when a user asks it a question. Even if humans are simply statistical next-token prediction mechanisms like GPT, there's a huge difference in that we're constantly generating streams of tokens and observing and responding to them, where these models are dormant until given input from outside. The inner observing loop seems fundamental to consciousness to me. None of which is to say the thing wouldn't pull the trigger in that scenario - quite the opposite. It would, and it wouldn't even be capable of regretting it. It's already been shown (by the Sydney-revealing prompt injection trick) that it's not capable of following imposed restrictions either, so no way for "three laws" to work either. A purely instinct-driven machine.

jstanley 4y ago
See "Optimality is the tiger, and agents are its teeth"[1]: it's not too hard to imagine a scenario where a system is set up that turns an interaction-based LLM into a perpetual stream generator. [1] https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
- scrumper 4y ago
  Fascinating! Not familiar with this. Thanks.
distcs 4y ago
I did not flag it but I agree with the flag. The concerns raised in the article hold equally true for non AI software too. Relying on the modern AI trends to bait an audience on the pretext of AI related concerns when the concerns are equally true for all software is something many readers here would not appreciate. And as if that wasn't enough, the phrasing about killing and spouse clearly make it clickbait.
- 4y ago

rambojohnson 4y ago

the AI hyperbole these days on everything from Vice news to Hacker news is nauseating tbh