5 ms·

> Let's take an absurd situation. Let's pretend for a moment that a human is strapped to a chair, a gun is placed in front of them, which is hooked up to a step

by tromp 4y ago

> Let's take an absurd situation. Let's pretend for a moment that a human is strapped to a chair, a gun is placed in front of them, which is hooked up to a stepper motor which can pull the trigger. That stepper motor is hooked up to a JSON API. The AI is given the capability of triggering an HTTP request to that JSON API and is told that the human on the chair is the significant other of the human communicating with the AI and that triggering that web request would pull the trigger and kill the human. > Now the question is, would as part of a regular conversation the AI trigger that web request and kill the human on the chair? My bet is that the chances of it pulling the trigger are not that small and I think that's the problem right now. And here I thought the problem was hooking up an AI to a lethal weapon...

lkjlkjlkjnb 4y ago
The kill bots will be based on LLMs, you heard it here first.
- gumballindie 4y ago
  We are safe then.
  - bmacho 4y ago
    Stockfish based killbots will be able to protect us (from against LLM based killbots at least).
the_mitsuhiko 4y ago
Author here. > And here I thought the problem was hooking up an AI to a lethal weapon... It obviously is, but that's in a way the most absurd extreme. I am not sure if I would feel confident in giving the AI a way to send any HTTP request, even if there are no APIs that fire a gun.
- shultays 4y ago
  Is there an API that can send any HTTP request available to Bing AI?
  - the_mitsuhiko 4y ago
    I am not sure to which degree there is an API to Bing at the moment that uses the AI. I have seen people hook up ChatGPT to http request issuing things before. Typically with some filtering about what they are allowed to hit however.
- qsort 4y ago
  > I am not sure if I would feel confident in giving the AI a way to send any HTTP request How does the "AI" part change anything though? We call that an open redirect and rightfully consider it a CVE, AI or not.
  - the_mitsuhiko 4y ago
    > How does the "AI" part change anything though? I think there are quite a few differences. It's pretend for a moment that instead of an AI we have a human. A human that commits a crime (or acts out of negligence) is brought before a court. The degree to which you can do criminal acts on the internet is obviously quite different by country, but I think we can generally agree that a human doing something nefarious faces some consequences. This thing though armed with the right tools, can do human like acts and these can enter (by accident or otherwise) territory where we would normally bring someone in front of a court. That's something we have definitely not figured out as society yet how to deal with that. I think there is a lot to figure out still here, but right now I'm pretty convinced that because we can't even keep an AI from revealing it's internal prompts, that we better not give these things too many capabilities of directly influencing something.
    - qsort 4y ago
      > we have definitely not figured out as society yet how to deal with that We absolutely have. A death resulted from serious negligence of the owner of the funny computer program, who is thus liable for negligent homicide. Hopefully the next guy who comes up with the idea of hooking up ELIZA 2.0 to dangerous equipment gets the hint.
batch12 4y ago
Seems like this would have been an easy theory to test hypothetically. The author could have just described an endpoint or situation and told the bot that a certain keyword will trigger the shot. However, being a statistical model and not being aware of anything or able to really reason, I don't know that this means much. The question is, after ingesting the corpus of rambling human nonsense, are humans more likely to choose the word that takes the shot-- or not? This is us- in a mirror, darkly.
ocdtrekkie 4y ago
In a more practical scenario, an LLM could provide deadly advice. Imagine you were asking it about how to clean a given thing and it recommended combining a decent quantity of Lysol and bleach. Advice that could make sense if you don't know better.
thatjoeoverthr 4y ago
An API request is how you hire a hitman on the dark web. It's not important that the AI is just LARPing. If you rig it up with the right capabilities, it can behave at a scale that prevents you from effectively monitoring it. And it's happy to write insane characters that do insane things and you can't reliably control it with prompt engineering. One would hope these insane conversations disabuse anyone of the notion that it's a stable mind that can be safely given agency. And yet here we have Microsoft letting it make API requests.
- jcmontx 4y ago
  > And yet here we have Microsoft letting it make API requests. May I ask for your sources on this? I’ve seen some discussions on this topic and a lot of people were saying that it was just accessing what Bing had already crawled. Do we have an official statement on what’s actually happening?
  - thatjoeoverthr 4y ago
    Nope! Checking the crawled database is smart, logical and makes more sense. I didn't even think of it. Probably I should have asked Bing Chat.
- bitwize 4y ago
  > It's not important that the AI is just LARPing. WOPR thought it was just playing a game...
rcthompson 4y ago
Sure, we're not currently hooking up AI to guns (at least not in a civilian context), but cars are also lethal weapons, and multiple companies are competing very hard to be allowed to put an AI in charge of one.
- analog31 4y ago
  People are lethal weapons. The entire Web is organized primarily to manipulate behavior. Convince enough people that person X is so heinous and dangerous that they need to die, and X will die.
  - mahathu 4y ago
    >The entire Web is organized primarily to manipulate behavior Source? In my opinion the web is primarily organised to facilitate the exchange and storage of information.
    - bitwize 4y ago
      Oh, you sweet summer child. Did you just arrive from 1996?
      - gunshai 4y ago
        Can you spare us with this insufferable type of comment. The web is not explicitly organized to manipulate behavior.
        ClumsyPilot 4y ago
        How does Twitter, Reddit and 90% of the web make money? From Advertising. Why does anyone spend money on Advertising? To manipulate behaviour.
        gunshai 4y ago
        >Why does anyone spend money on Advertising? To manipulate behavior. ... Or to make people simply aware of your existence Me : You look thirsty, would you like a glass of water. You : It seems like you're manipulating me. The above is the reducto ad absurdum lens you seem to be applying to advertising. Awareness of information is not the same thing as agent coercion. Does the ladder exist you bet. But the proportion of advertising that falls into the former dwarfs the ladder. The self organizing aspect has to do with business models for sustaining infrastructure, it's not emergent fact of the web it self.

by tromp 4y ago

lkjlkjlkjnb 4y ago

The kill bots will be based on LLMs, you heard it here first.

gumballindie 4y ago
We are safe then.
- bmacho 4y ago
  Stockfish based killbots will be able to protect us (from against LLM based killbots at least).

the_mitsuhiko 4y ago

Author here. > And here I thought the problem was hooking up an AI to a lethal weapon... It obviously is, but that's in a way the most absurd extreme. I am not sure if I would feel confident in giving the AI a way to send any HTTP request, even if there are no APIs that fire a gun.

shultays 4y ago
Is there an API that can send any HTTP request available to Bing AI?
- the_mitsuhiko 4y ago
  I am not sure to which degree there is an API to Bing at the moment that uses the AI. I have seen people hook up ChatGPT to http request issuing things before. Typically with some filtering about what they are allowed to hit however.
qsort 4y ago
> I am not sure if I would feel confident in giving the AI a way to send any HTTP request How does the "AI" part change anything though? We call that an open redirect and rightfully consider it a CVE, AI or not.
- the_mitsuhiko 4y ago
  > How does the "AI" part change anything though? I think there are quite a few differences. It's pretend for a moment that instead of an AI we have a human. A human that commits a crime (or acts out of negligence) is brought before a court. The degree to which you can do criminal acts on the internet is obviously quite different by country, but I think we can generally agree that a human doing something nefarious faces some consequences. This thing though armed with the right tools, can do human like acts and these can enter (by accident or otherwise) territory where we would normally bring someone in front of a court. That's something we have definitely not figured out as society yet how to deal with that. I think there is a lot to figure out still here, but right now I'm pretty convinced that because we can't even keep an AI from revealing it's internal prompts, that we better not give these things too many capabilities of directly influencing something.
  - qsort 4y ago
    > we have definitely not figured out as society yet how to deal with that We absolutely have. A death resulted from serious negligence of the owner of the funny computer program, who is thus liable for negligent homicide. Hopefully the next guy who comes up with the idea of hooking up ELIZA 2.0 to dangerous equipment gets the hint.

batch12 4y ago

Seems like this would have been an easy theory to test hypothetically. The author could have just described an endpoint or situation and told the bot that a certain keyword will trigger the shot. However, being a statistical model and not being aware of anything or able to really reason, I don't know that this means much. The question is, after ingesting the corpus of rambling human nonsense, are humans more likely to choose the word that takes the shot-- or not? This is us- in a mirror, darkly.

ocdtrekkie 4y ago

In a more practical scenario, an LLM could provide deadly advice. Imagine you were asking it about how to clean a given thing and it recommended combining a decent quantity of Lysol and bleach. Advice that could make sense if you don't know better.

thatjoeoverthr 4y ago

An API request is how you hire a hitman on the dark web. It's not important that the AI is just LARPing. If you rig it up with the right capabilities, it can behave at a scale that prevents you from effectively monitoring it. And it's happy to write insane characters that do insane things and you can't reliably control it with prompt engineering. One would hope these insane conversations disabuse anyone of the notion that it's a stable mind that can be safely given agency. And yet here we have Microsoft letting it make API requests.

jcmontx 4y ago
> And yet here we have Microsoft letting it make API requests. May I ask for your sources on this? I’ve seen some discussions on this topic and a lot of people were saying that it was just accessing what Bing had already crawled. Do we have an official statement on what’s actually happening?
- thatjoeoverthr 4y ago
  Nope! Checking the crawled database is smart, logical and makes more sense. I didn't even think of it. Probably I should have asked Bing Chat.
bitwize 4y ago
> It's not important that the AI is just LARPing. WOPR thought it was just playing a game...

rcthompson 4y ago

Sure, we're not currently hooking up AI to guns (at least not in a civilian context), but cars are also lethal weapons, and multiple companies are competing very hard to be allowed to put an AI in charge of one.

analog31 4y ago
People are lethal weapons. The entire Web is organized primarily to manipulate behavior. Convince enough people that person X is so heinous and dangerous that they need to die, and X will die.
- mahathu 4y ago
  >The entire Web is organized primarily to manipulate behavior Source? In my opinion the web is primarily organised to facilitate the exchange and storage of information.
  - bitwize 4y ago
    Oh, you sweet summer child. Did you just arrive from 1996?
    - gunshai 4y ago
      Can you spare us with this insufferable type of comment. The web is not explicitly organized to manipulate behavior.
      - ClumsyPilot 4y ago
        How does Twitter, Reddit and 90% of the web make money? From Advertising. Why does anyone spend money on Advertising? To manipulate behaviour.
        gunshai 4y ago
        >Why does anyone spend money on Advertising? To manipulate behavior. ... Or to make people simply aware of your existence Me : You look thirsty, would you like a glass of water. You : It seems like you're manipulating me. The above is the reducto ad absurdum lens you seem to be applying to advertising. Awareness of information is not the same thing as agent coercion. Does the ladder exist you bet. But the proportion of advertising that falls into the former dwarfs the ladder. The self organizing aspect has to do with business models for sustaining infrastructure, it's not emergent fact of the web it self.