3 ms·

First it needs an incentive to lie. Or it’s prompted to, or the model architecture allows it

by m3kw9 2y ago

Ensorceled 2y ago
> First it needs an incentive to lie ... or the model architecture allows it Given that one the largest problems with current LLMs is they will often "hallucinate" (i.e. lie or provide deceptive answers), it seems strange to phrase it this way.
pixl97 2y ago
https://www.youtube.com/watch?v=IeWljQw3UgQ https://www.youtube.com/watch?v=IeWljQw3UgQ >Deceptive Misaligned Mesa-Optimisers? It's More Likely Than You Think... Or, "It sounds near impossible to design an architecture that doesn't allow it"
renegade-otter 2y ago
AI has no incentives. ChatGPT is a file. IT'S A FILE. If you don't ask it anything, it just sits there. A FILE with word weights is not going to "lie". No one ever accuses the Google index of lying when they get a misleading result.
- pixl97 2y ago
  This take is useless.... Lets take a simple non computing model first: You're in a car, you hit the brakes pedal, you expect the brakes to be applied and the car to slow down. If it does not, then the brakes are broken. More complex computing model: Your modern car interprets there is an object in front of it and applies the brakes. If there was, yay, your car may have saved you. If there was not, you may have just caused a 20 car pile up on interstate. So much for that 'FILE' just being a file. Please wake up and realize that you live in a world where 'FILES' have agency depending on their connectedness.
- SpicyLemonZest 2y ago
  Sure they do! I just searched and found a random example https://gizmodo.com/googles-algorithm-is-lying-to-you-about-onions-and-blam-1793057789 https://gizmodo.com/googles-algorithm-is-lying-to-you-about-.... It's less common because of the format of the results. If I ask a librarian to get me a list of books matching suchandsuch criteria, and he excludes some that should have been on the list, I wouldn't normally say that the resulting list is a lie even if I think he deliberately excluded them for nefarious purposes.
MyNameIs_Hacker 2y ago
"First it needs to be instructed to lie" would be better language to use for those not intimately familiar with ML and who might anthropomorphize "incentive". I dare not get into a discussion about the word "lie" here. I have a weekend I want to enjoy.

First it needs an incentive to lie. Or it’s prompted to, or the model architecture allows it

by m3kw9 2y ago

First it needs an incentive to lie. Or it’s prompted to, or the model architecture allows it

> First it needs an incentive to lie ... or the model architecture allows it Given that one the largest problems with current LLMs is they will often "hallucinate" (i.e. lie or provide deceptive answers), it seems strange to phrase it this way.

pixl97 2y ago

https://www.youtube.com/watch?v=IeWljQw3UgQ https://www.youtube.com/watch?v=IeWljQw3UgQ >Deceptive Misaligned Mesa-Optimisers? It's More Likely Than You Think... Or, "It sounds near impossible to design an architecture that doesn't allow it"

renegade-otter 2y ago

AI has no incentives. ChatGPT is a file. IT'S A FILE. If you don't ask it anything, it just sits there. A FILE with word weights is not going to "lie". No one ever accuses the Google index of lying when they get a misleading result.

pixl97 2y ago
This take is useless.... Lets take a simple non computing model first: You're in a car, you hit the brakes pedal, you expect the brakes to be applied and the car to slow down. If it does not, then the brakes are broken. More complex computing model: Your modern car interprets there is an object in front of it and applies the brakes. If there was, yay, your car may have saved you. If there was not, you may have just caused a 20 car pile up on interstate. So much for that 'FILE' just being a file. Please wake up and realize that you live in a world where 'FILES' have agency depending on their connectedness.
SpicyLemonZest 2y ago
Sure they do! I just searched and found a random example https://gizmodo.com/googles-algorithm-is-lying-to-you-about-onions-and-blam-1793057789 https://gizmodo.com/googles-algorithm-is-lying-to-you-about-.... It's less common because of the format of the results. If I ask a librarian to get me a list of books matching suchandsuch criteria, and he excludes some that should have been on the list, I wouldn't normally say that the resulting list is a lie even if I think he deliberately excluded them for nefarious purposes.

MyNameIs_Hacker 2y ago

"First it needs to be instructed to lie" would be better language to use for those not intimately familiar with ML and who might anthropomorphize "incentive". I dare not get into a discussion about the word "lie" here. I have a weekend I want to enjoy.