4 ms·

You're totally wrong. Just because most AI doesn't do that right now doesn't mean it can be trusted over years. It's not about AI's motivation. It's about the

by EGreg 2y ago

You're totally wrong. Just because most AI doesn't do that right now doesn't mean it can be trusted over years. It's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. Meaning, if an animal or a person is showing up every day and giving you a lot of their attention, emotion and love, proving themselves over time, you can be reasonably sure that they are genuinely like that. But an AI can just as easily fake it all for a year or two, and then drop it a second later. There is nothing an AI can do to prove that it doesn't have a backdoor somewhere in its billions of weights, to go rogue. It's like the Ken Thompson hack, but much more organic (https://wiki.c2.com/?TheKenThompsonHack https://wiki.c2.com/?TheKenThompsonHack) https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf Unlike an animal or human, the AI performing as you want is no indication it will continue to perform like that in the following second. As people and organizations come to rely more and more on AI, they will become more and more vulnerable to any number of backdoors.

daveguy 2y ago
> t's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. This is true of any software. > But an AI can just as easily fake it all for a year or two, and then drop it a second later. We are nowhere near an AI being able to "fake" anything. No matter what words are coming out of an LLM. There's no self model -- it's episodic and static. Regarding the Cornell reference and backdoors in AI that is true. But it's also true of compilers. It has nothing to do with an AI having any concept or capacity of "deceipt". We definitely agree that AI should not be trusted, but they are just not currently or in the anywhere close to near future capable of deceipt on their own. At this point that kind of "deceipt" would come from the programmers -- at least until AI is much more advanced than it currently is.
- dcow 2y ago
  Furthermore, there is absolutely nothing preventing a human from turning on a dime, either. It’s happened to me with an extremely close and highly trusted confidant, first hand (luckily they recovered years later).

by EGreg 2y ago

daveguy 2y ago

> t's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. This is true of any software. > But an AI can just as easily fake it all for a year or two, and then drop it a second later. We are nowhere near an AI being able to "fake" anything. No matter what words are coming out of an LLM. There's no self model -- it's episodic and static. Regarding the Cornell reference and backdoors in AI that is true. But it's also true of compilers. It has nothing to do with an AI having any concept or capacity of "deceipt". We definitely agree that AI should not be trusted, but they are just not currently or in the anywhere close to near future capable of deceipt on their own. At this point that kind of "deceipt" would come from the programmers -- at least until AI is much more advanced than it currently is.

dcow 2y ago
Furthermore, there is absolutely nothing preventing a human from turning on a dime, either. It’s happened to me with an extremely close and highly trusted confidant, first hand (luckily they recovered years later).