4 ms·

AI literally does none of that. The current state of AI has no comprehension of any other conversation, only the output of a numerical model and the context / s

AI literally does none of that. The current state of AI has no comprehension of any other conversation, only the output of a numerical model and the context / system prompt entered. AI has no motivation or ability to "earn trust". The end result is the same though -- you can't trust it past what you can verify.

EGreg 2y ago
You're totally wrong. Just because most AI doesn't do that right now doesn't mean it can be trusted over years. It's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. Meaning, if an animal or a person is showing up every day and giving you a lot of their attention, emotion and love, proving themselves over time, you can be reasonably sure that they are genuinely like that. But an AI can just as easily fake it all for a year or two, and then drop it a second later. There is nothing an AI can do to prove that it doesn't have a backdoor somewhere in its billions of weights, to go rogue. It's like the Ken Thompson hack, but much more organic (https://wiki.c2.com/?TheKenThompsonHack https://wiki.c2.com/?TheKenThompsonHack) https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf Unlike an animal or human, the AI performing as you want is no indication it will continue to perform like that in the following second. As people and organizations come to rely more and more on AI, they will become more and more vulnerable to any number of backdoors.
- daveguy 2y ago
  > t's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. This is true of any software. > But an AI can just as easily fake it all for a year or two, and then drop it a second later. We are nowhere near an AI being able to "fake" anything. No matter what words are coming out of an LLM. There's no self model -- it's episodic and static. Regarding the Cornell reference and backdoors in AI that is true. But it's also true of compilers. It has nothing to do with an AI having any concept or capacity of "deceipt". We definitely agree that AI should not be trusted, but they are just not currently or in the anywhere close to near future capable of deceipt on their own. At this point that kind of "deceipt" would come from the programmers -- at least until AI is much more advanced than it currently is.
  - dcow 2y ago
    Furthermore, there is absolutely nothing preventing a human from turning on a dime, either. It’s happened to me with an extremely close and highly trusted confidant, first hand (luckily they recovered years later).
ben_w 2y ago
LLMs don't do what the film shows. (Nothing ever does what films show, unless it's a documentary, and even then sometimes not). AI includes Google search results being re-weighted based on what people click on, the live bidding on which advert to show you when you visit a website, your facebook feed, and your spam filter. All of these have been designed to earn your trust, they don't need to have an internal motivation for that. Some have been trusted, some are still trusted. All may change on a dime and without warning due to some software update outside of your control.

EGreg 2y ago

You're totally wrong. Just because most AI doesn't do that right now doesn't mean it can be trusted over years. It's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. Meaning, if an animal or a person is showing up every day and giving you a lot of their attention, emotion and love, proving themselves over time, you can be reasonably sure that they are genuinely like that. But an AI can just as easily fake it all for a year or two, and then drop it a second later. There is nothing an AI can do to prove that it doesn't have a backdoor somewhere in its billions of weights, to go rogue. It's like the Ken Thompson hack, but much more organic (https://wiki.c2.com/?TheKenThompsonHack https://wiki.c2.com/?TheKenThompsonHack) https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf https://www.cs.cornell.edu/~mpkim/pubs/undetectable.pdf Unlike an animal or human, the AI performing as you want is no indication it will continue to perform like that in the following second. As people and organizations come to rely more and more on AI, they will become more and more vulnerable to any number of backdoors.

daveguy 2y ago
> t's not about AI's motivation. It's about the people behind the AI programming it. They can make it behave well but at the same time it can turn on a dime. This is true of any software. > But an AI can just as easily fake it all for a year or two, and then drop it a second later. We are nowhere near an AI being able to "fake" anything. No matter what words are coming out of an LLM. There's no self model -- it's episodic and static. Regarding the Cornell reference and backdoors in AI that is true. But it's also true of compilers. It has nothing to do with an AI having any concept or capacity of "deceipt". We definitely agree that AI should not be trusted, but they are just not currently or in the anywhere close to near future capable of deceipt on their own. At this point that kind of "deceipt" would come from the programmers -- at least until AI is much more advanced than it currently is.
- dcow 2y ago
  Furthermore, there is absolutely nothing preventing a human from turning on a dime, either. It’s happened to me with an extremely close and highly trusted confidant, first hand (luckily they recovered years later).

ben_w 2y ago

LLMs don't do what the film shows. (Nothing ever does what films show, unless it's a documentary, and even then sometimes not). AI includes Google search results being re-weighted based on what people click on, the live bidding on which advert to show you when you visit a website, your facebook feed, and your spam filter. All of these have been designed to earn your trust, they don't need to have an internal motivation for that. Some have been trusted, some are still trusted. All may change on a dime and without warning due to some software update outside of your control.