4 ms·

If you give privileged access to a program which output is literally undefined behavior, frankly that's on you.

by qsort 4y ago

mannykannot 4y ago
Of course it is, but that does not somehow make it not a bad idea.
- qsort 4y ago
  I agree it's a bad idea. My question is how the presence of "AI" changes the security model. The root cause of the problem is hooking up untrusted input to a privileged endpoint. How is "AI" different from any other source of untrusted input?
  - mannykannot 4y ago
    It does not change the security model. The point in this thread is just that, under any security model, it is a bad idea.
    - distcs 4y ago
      No, the point of the thread is if there is anything in the article that is specific to AI. It is not. The concern about hooking up the output of a program to external systems is a general security concern related to any software: AI or not. The point of qsort's comment was that the article does and the comment by OP does not explain how this concern is specifically about AI.
      - mannykannot 4y ago
        There is no claim in the original post that there is a special rule for AI as opposed to any other similarly-capable agent. The reasonably well-founded concern here (and in the article) is that someone might well want to create an exception for AI, either knowingly, or by default because they cannot see or properly evaluate the risk.
- distcs 4y ago
  It is a bad idea to take the output of any unvetted computer program and hook it up an endpoint that can do bad things. It does not matter whether the computer program is AI or not. Any computer program that is hooked up to endpoints that can do bad things are going to have regulations around them. This is nothing special about AI. This is true for all types of software, AI or not.
  - colinmorelli 4y ago
    ...isn't...this...exactly what autopilot in airplanes does? Self-driving cars? Manufacturing robots? The daVinci Surgical System? In fact, let's not even go that far. If you have one, your car is likely drive-by-wire, and the throttle is controlled not by mechanical linkage of your foot pressing the pedal, but by the output of computers that are deciding what to do based on how hard you push the pedal. If you have a car in the last 5 years, it can likely suddenly apply the brakes and turn the steering wheel for you to avoid what it thinks is a collision. Every one of these things is trivially lethal in different ways, and all are connected to computer programs already. I'm not personally that concerned about AI taking over the world any time soon, but quite frankly the areas where technology and AI have made the biggest waves are all specifically in areas that are high risk (or very rote) for humans. Isn't that kind of the point?
  - justinclift 4y ago
    Safety control systems would like a word...

If you give privileged access to a program which output is literally undefined behavior, frankly that's on you.

by qsort 4y ago

If you give privileged access to a program which output is literally undefined behavior, frankly that's on you.

mannykannot 4y ago

Of course it is, but that does not somehow make it not a bad idea.

qsort 4y ago
I agree it's a bad idea. My question is how the presence of "AI" changes the security model. The root cause of the problem is hooking up untrusted input to a privileged endpoint. How is "AI" different from any other source of untrusted input?
- mannykannot 4y ago
  It does not change the security model. The point in this thread is just that, under any security model, it is a bad idea.
  - distcs 4y ago
    No, the point of the thread is if there is anything in the article that is specific to AI. It is not. The concern about hooking up the output of a program to external systems is a general security concern related to any software: AI or not. The point of qsort's comment was that the article does and the comment by OP does not explain how this concern is specifically about AI.
    - mannykannot 4y ago
      There is no claim in the original post that there is a special rule for AI as opposed to any other similarly-capable agent. The reasonably well-founded concern here (and in the article) is that someone might well want to create an exception for AI, either knowingly, or by default because they cannot see or properly evaluate the risk.
distcs 4y ago
It is a bad idea to take the output of any unvetted computer program and hook it up an endpoint that can do bad things. It does not matter whether the computer program is AI or not. Any computer program that is hooked up to endpoints that can do bad things are going to have regulations around them. This is nothing special about AI. This is true for all types of software, AI or not.
- colinmorelli 4y ago
  ...isn't...this...exactly what autopilot in airplanes does? Self-driving cars? Manufacturing robots? The daVinci Surgical System? In fact, let's not even go that far. If you have one, your car is likely drive-by-wire, and the throttle is controlled not by mechanical linkage of your foot pressing the pedal, but by the output of computers that are deciding what to do based on how hard you push the pedal. If you have a car in the last 5 years, it can likely suddenly apply the brakes and turn the steering wheel for you to avoid what it thinks is a collision. Every one of these things is trivially lethal in different ways, and all are connected to computer programs already. I'm not personally that concerned about AI taking over the world any time soon, but quite frankly the areas where technology and AI have made the biggest waves are all specifically in areas that are high risk (or very rote) for humans. Isn't that kind of the point?
- justinclift 4y ago
  Safety control systems would like a word...