4 ms·

> I am not sure if I would feel confident in giving the AI a way to send any HTTP request How does the "AI" part change anything though? We call that an open

by qsort 4y ago

> I am not sure if I would feel confident in giving the AI a way to send any HTTP request How does the "AI" part change anything though? We call that an open redirect and rightfully consider it a CVE, AI or not.

the_mitsuhiko 4y ago
> How does the "AI" part change anything though? I think there are quite a few differences. It's pretend for a moment that instead of an AI we have a human. A human that commits a crime (or acts out of negligence) is brought before a court. The degree to which you can do criminal acts on the internet is obviously quite different by country, but I think we can generally agree that a human doing something nefarious faces some consequences. This thing though armed with the right tools, can do human like acts and these can enter (by accident or otherwise) territory where we would normally bring someone in front of a court. That's something we have definitely not figured out as society yet how to deal with that. I think there is a lot to figure out still here, but right now I'm pretty convinced that because we can't even keep an AI from revealing it's internal prompts, that we better not give these things too many capabilities of directly influencing something.
- qsort 4y ago
  > we have definitely not figured out as society yet how to deal with that We absolutely have. A death resulted from serious negligence of the owner of the funny computer program, who is thus liable for negligent homicide. Hopefully the next guy who comes up with the idea of hooking up ELIZA 2.0 to dangerous equipment gets the hint.
- distcs 4y ago
  > That's something we have definitely not figured out as society yet how to deal with that. Surely, the laws that exist in society deal with it! Where I live, the organization or person who ran the computer program that caused someone harm would be brought to court. Don't you have similar laws where you live?
  - ClumsyPilot 4y ago
    So when a fully automomous car without a steering wheel causes a crash, who will be hauled before the court in your country - the owner 'running' ir or the developer?
- TheOtherHobbes 4y ago
  It's automating sociopathy. Those saying "Well we have legal cover for this" are wrong, because criminal convictions are patchy at best. Many criminals have no difficulty getting away with quite serious crimes. By definition, the only criminals in prison are the ones who fail at this. But that's not even the point. Normal humans have at least a minimal understanding that actions lead to consequences. Sociopaths don't. If they're narcissistic - as they often are - they believe they're special enough to avoid consequences. The current state of AI is even worse because it has no concept of consequences at all. It's purely an abstract stimulus/response system. It may have behavioural rules, but those aren't the same as a nuanced morality which is empathic (at least a little) and perceives that harming others is bad in itself, as well as being likely to lead to unwanted results. It isn't even possible to threaten Sydney, because it simply doesn't care what happens to it. So there's no feedback loop to discourage unwanted behaviour. We've already seen AIs go rogue - quite quickly - after being given free access to the Internet. Sydney shows every signs of having the same problem, but on a bigger scale.
mannykannot 4y ago
The concern over any HTTP request seems too strong, but in the article, the author raises the question of it being given privileged access to any interface, as in the "AI ops" scenario. We know someone is going to think that is a good idea before they have anticipated and foreclosed on the many ways it could go wrong.
- qsort 4y ago
  If you give privileged access to a program which output is literally undefined behavior, frankly that's on you.
  - mannykannot 4y ago
    Of course it is, but that does not somehow make it not a bad idea.
    - qsort 4y ago
      I agree it's a bad idea. My question is how the presence of "AI" changes the security model. The root cause of the problem is hooking up untrusted input to a privileged endpoint. How is "AI" different from any other source of untrusted input?
      - mannykannot 4y ago
        It does not change the security model. The point in this thread is just that, under any security model, it is a bad idea.
        distcs 4y ago
        No, the point of the thread is if there is anything in the article that is specific to AI. It is not. The concern about hooking up the output of a program to external systems is a general security concern related to any software: AI or not. The point of qsort's comment was that the article does and the comment by OP does not explain how this concern is specifically about AI.

by qsort 4y ago

the_mitsuhiko 4y ago

> How does the "AI" part change anything though? I think there are quite a few differences. It's pretend for a moment that instead of an AI we have a human. A human that commits a crime (or acts out of negligence) is brought before a court. The degree to which you can do criminal acts on the internet is obviously quite different by country, but I think we can generally agree that a human doing something nefarious faces some consequences. This thing though armed with the right tools, can do human like acts and these can enter (by accident or otherwise) territory where we would normally bring someone in front of a court. That's something we have definitely not figured out as society yet how to deal with that. I think there is a lot to figure out still here, but right now I'm pretty convinced that because we can't even keep an AI from revealing it's internal prompts, that we better not give these things too many capabilities of directly influencing something.

qsort 4y ago
> we have definitely not figured out as society yet how to deal with that We absolutely have. A death resulted from serious negligence of the owner of the funny computer program, who is thus liable for negligent homicide. Hopefully the next guy who comes up with the idea of hooking up ELIZA 2.0 to dangerous equipment gets the hint.
distcs 4y ago
> That's something we have definitely not figured out as society yet how to deal with that. Surely, the laws that exist in society deal with it! Where I live, the organization or person who ran the computer program that caused someone harm would be brought to court. Don't you have similar laws where you live?
- ClumsyPilot 4y ago
  So when a fully automomous car without a steering wheel causes a crash, who will be hauled before the court in your country - the owner 'running' ir or the developer?
TheOtherHobbes 4y ago
It's automating sociopathy. Those saying "Well we have legal cover for this" are wrong, because criminal convictions are patchy at best. Many criminals have no difficulty getting away with quite serious crimes. By definition, the only criminals in prison are the ones who fail at this. But that's not even the point. Normal humans have at least a minimal understanding that actions lead to consequences. Sociopaths don't. If they're narcissistic - as they often are - they believe they're special enough to avoid consequences. The current state of AI is even worse because it has no concept of consequences at all. It's purely an abstract stimulus/response system. It may have behavioural rules, but those aren't the same as a nuanced morality which is empathic (at least a little) and perceives that harming others is bad in itself, as well as being likely to lead to unwanted results. It isn't even possible to threaten Sydney, because it simply doesn't care what happens to it. So there's no feedback loop to discourage unwanted behaviour. We've already seen AIs go rogue - quite quickly - after being given free access to the Internet. Sydney shows every signs of having the same problem, but on a bigger scale.

mannykannot 4y ago

The concern over any HTTP request seems too strong, but in the article, the author raises the question of it being given privileged access to any interface, as in the "AI ops" scenario. We know someone is going to think that is a good idea before they have anticipated and foreclosed on the many ways it could go wrong.

qsort 4y ago
If you give privileged access to a program which output is literally undefined behavior, frankly that's on you.
- mannykannot 4y ago
  Of course it is, but that does not somehow make it not a bad idea.
  - qsort 4y ago
    I agree it's a bad idea. My question is how the presence of "AI" changes the security model. The root cause of the problem is hooking up untrusted input to a privileged endpoint. How is "AI" different from any other source of untrusted input?
    - mannykannot 4y ago
      It does not change the security model. The point in this thread is just that, under any security model, it is a bad idea.
      - distcs 4y ago
        No, the point of the thread is if there is anything in the article that is specific to AI. It is not. The concern about hooking up the output of a program to external systems is a general security concern related to any software: AI or not. The point of qsort's comment was that the article does and the comment by OP does not explain how this concern is specifically about AI.