3 ms·

This is literally what AI is doing Trusting AI is impossible because it can have a backdoor and easily switch on a dime at any time, violating all your assumpt

by EGreg 2y ago

This is literally what AI is doing Trusting AI is impossible because it can have a backdoor and easily switch on a dime at any time, violating all your assumptions (that are made because of your intuitions about living animals and their costly signals) NO MATTER HOW LONG IT HAS BEEN EARNING YOUR TRUST. Not only that, but it can actually do it in the background, imperceptibly, across thousands of instances, and shift opinion of many people. The movie Her shows that Samantha had been speaking to thousands of people at once. The abrupt leaving is actually a very benign scenario, compared to the myriad other things it could abrutly change. https://www.youtube.com/watch?v=GZS8xBvgLaQ https://www.youtube.com/watch?v=GZS8xBvgLaQ

daveguy 2y ago
AI literally does none of that. The current state of AI has no comprehension of any other conversation, only the output of a numerical model and the context / system prompt entered. AI has no motivation or ability to "earn trust". The end result is the same though -- you can't trust it past what you can verify.
- EGreg 2y ago
  You're totally wrong. Just because most AI doesn't do that right now doesn't mean it can be trusted over years. It's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. Meaning, if an animal or a person is showing up every day and giving you a lot of their attention, emotion and love, proving themselves over time, you can be reasonably sure that they are genuinely like that. But an AI can just as easily fake it all for a year or two, and then drop it a second later. There is nothing an AI can do to prove that it doesn't have a backdoor somewhere in its billions of weights, to go rogue. It's like the Ken Thompson hack, but much more organic (https://wiki.c2.com/?TheKenThompsonHack https://wiki.c2.com/?TheKenThompsonHack) https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf Unlike an animal or human, the AI performing as you want is no indication it will continue to perform like that in the following second. As people and organizations come to rely more and more on AI, they will become more and more vulnerable to any number of backdoors.
  - daveguy 2y ago
    > t's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. This is true of any software. > But an AI can just as easily fake it all for a year or two, and then drop it a second later. We are nowhere near an AI being able to "fake" anything. No matter what words are coming out of an LLM. There's no self model -- it's episodic and static. Regarding the Cornell reference and backdoors in AI that is true. But it's also true of compilers. It has nothing to do with an AI having any concept or capacity of "deceipt". We definitely agree that AI should not be trusted, but they are just not currently or in the anywhere close to near future capable of deceipt on their own. At this point that kind of "deceipt" would come from the programmers -- at least until AI is much more advanced than it currently is.
    - dcow 2y ago
      Furthermore, there is absolutely nothing preventing a human from turning on a dime, either. It’s happened to me with an extremely close and highly trusted confidant, first hand (luckily they recovered years later).
- ben_w 2y ago
  LLMs don't do what the film shows. (Nothing ever does what films show, unless it's a documentary, and even then sometimes not). AI includes Google search results being re-weighted based on what people click on, the live bidding on which advert to show you when you visit a website, your facebook feed, and your spam filter. All of these have been designed to earn your trust, they don't need to have an internal motivation for that. Some have been trusted, some are still trusted. All may change on a dime and without warning due to some software update outside of your control.

by EGreg 2y ago

daveguy 2y ago

AI literally does none of that. The current state of AI has no comprehension of any other conversation, only the output of a numerical model and the context / system prompt entered. AI has no motivation or ability to "earn trust". The end result is the same though -- you can't trust it past what you can verify.

EGreg 2y ago
You're totally wrong. Just because most AI doesn't do that right now doesn't mean it can be trusted over years. It's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. Meaning, if an animal or a person is showing up every day and giving you a lot of their attention, emotion and love, proving themselves over time, you can be reasonably sure that they are genuinely like that. But an AI can just as easily fake it all for a year or two, and then drop it a second later. There is nothing an AI can do to prove that it doesn't have a backdoor somewhere in its billions of weights, to go rogue. It's like the Ken Thompson hack, but much more organic (https://wiki.c2.com/?TheKenThompsonHack https://wiki.c2.com/?TheKenThompsonHack) https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf Unlike an animal or human, the AI performing as you want is no indication it will continue to perform like that in the following second. As people and organizations come to rely more and more on AI, they will become more and more vulnerable to any number of backdoors.
- daveguy 2y ago
  > t's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. This is true of any software. > But an AI can just as easily fake it all for a year or two, and then drop it a second later. We are nowhere near an AI being able to "fake" anything. No matter what words are coming out of an LLM. There's no self model -- it's episodic and static. Regarding the Cornell reference and backdoors in AI that is true. But it's also true of compilers. It has nothing to do with an AI having any concept or capacity of "deceipt". We definitely agree that AI should not be trusted, but they are just not currently or in the anywhere close to near future capable of deceipt on their own. At this point that kind of "deceipt" would come from the programmers -- at least until AI is much more advanced than it currently is.
  - dcow 2y ago
    Furthermore, there is absolutely nothing preventing a human from turning on a dime, either. It’s happened to me with an extremely close and highly trusted confidant, first hand (luckily they recovered years later).
ben_w 2y ago
LLMs don't do what the film shows. (Nothing ever does what films show, unless it's a documentary, and even then sometimes not). AI includes Google search results being re-weighted based on what people click on, the live bidding on which advert to show you when you visit a website, your facebook feed, and your spam filter. All of these have been designed to earn your trust, they don't need to have an internal motivation for that. Some have been trusted, some are still trusted. All may change on a dime and without warning due to some software update outside of your control.