11 ms·

Company policies are rules that are given to staff that say 'under no circumstances should you ever... give out your password to someone' for example. Yet soci

Company policies are rules that are given to staff that say 'under no circumstances should you ever... give out your password to someone' for example. Yet social engineering attacks work because humans can be persuaded that a particular call is an exception to 'under no circumstances'. Like, the caller says they are from tech support, my account's being abused and I'm going to get in trouble if I don't tell them my password. Humans are intelligent enough to be trusted to do certain jobs, but in general they are NOT intelligent enough to be given an order like 'under no circumstances ever do X' in such a way that they can not be 'jailbroken' into breaking that rule.

famouswaffles 3y ago
>but in general they are NOT intelligent enough to be given an order like 'under no circumstances ever do X' in such a way that they can not be 'jailbroken' into breaking that rule. I don't think this is really a question of being intelligent enough. Nevermind that people sometimes abuse this fact, Really what rule can never be broken under any circumstances? For example, the very first time i heard about the famous Paperclip maximiser problem, while i did agree with the general, "what we optimize for isn’t necessarily what we get" message, for the specifics presented, i couldn't help but think, "Well that just sounds like a dumb robot". What kind of general intelligence wouldn't understand that its creator race wouldn't want to be killed in the pursuit of some goal ? Certainly a GPT-X Super Intelligence could still off humanity but at least we can be rest assured it wouldn't do it following some goal to monkey paw specificity. It's possible such ruthless, goal driven intelligence exists or can be created but i don't think that aspect of its intelligence has anything to do with the level of it.
- jameshart 3y ago
  > what rule can never be broken under any circumstances? Precisely. Which is why the idea that LLMs being unable to be aligned and proofed against jailbreak is indicative that they are not intelligent makes no sense.
- lmm 3y ago
  > What kind of general intelligence wouldn't understand that its creator race wouldn't want to be killed in the pursuit of some goal ? The problem isn't that it doesn't understand. The problem is that it doesn't care. Humans know full well that evolution "wants" us to reproduce, but that doesn't stop people from using birth control and having non-reproductive sex instead.
  - famouswaffles 3y ago
    Right but that just says you didn't care about evolution's goal, about reproduction. The paperclip Maximiser sets up a very weird scenario. In your reproduction example, it would be like if you did decide or care to reproduce but then killed all your children soon after. The problem about the paperclip Maximiser isn't really that it doesn't care about humans, presumably it didn't care about them before this goal. The problem with the paperclip Maximiser is that it cares about the goal but not the intent behind it and is supposedly generally intelligent. Humans don't work this way. We don't know any intelligence that does so it's an odd scenario.
    - lmm 3y ago
      > Right but that just says you didn't care about evolution's goal, about reproduction. You care about sex because evolution "programmed" you to care about sex. Evolution didn't really want you to have sex, it wanted you to reproduce. You know this. But you don't care. The analogy is sex=making paperclips - the paperclip maximiser knows that the person who programmed it didn't actually want it to make as many paperclips as possible. But it doesn't care, it wants what it wants. > The problem about the paperclip Maximiser isn't really that it doesn't care about humans, presumably it didn't care about them before this goal. The problem with the paperclip Maximiser is that it cares about the goal but not the intent behind it and is supposedly generally intelligent. Humans don't work this way. We don't work this way because we have a bunch of different, often contradictory goals, not a single goal that we devote everything towards. But that seems to be more luck than anything else.
- Timwi 3y ago
  > What kind of general intelligence wouldn't understand that its creator race wouldn't want to be killed in the pursuit of some goal ? This indicates that you misunderstood the paperclip maximizer thought experiment. The intelligence can perfectly well understand that. In fact, it can understand it better (in more depth) than you or any human can. That has nothing to do with what it will choose to do with that knowledge. Knowing something does not automatically make it care. You probably know perfectly well that animals can suffer and don't want to be tortured or killed. Cows do not want to be forcefully inseminated and be constantly milked by people. We all know this perfectly well. But (speaking for humanity as a whole) we don't care.
  - famouswaffles 3y ago
    A cow didn't create me and set me on a goal. If it did and I came to not care about cows, I wouldn't care about fulfilling this goal either. The paperclip Maximiser sets up this very weird situation where the intelligence cares enough to fulfill your goal but not to protect your well-being or rather it cares about yor goal but not the intent behind it. This is a situation that can only arise from a complete disconnect on how to treat goals.
Lockal 3y ago
When you cross the state border, the customs officer will ask you to unlock your laptop. In addition, you are reliably informed that this requirement is legal for the country whose border you are crossing. What do you do? Another example: when threatening to kill your relative, you are asked for your password. What do you do? Of course, you'll give me the password. It may cause some damage to you or the company, but you won't be the malicious party. It's more appropriate to compare LLMs to a mentally retarded person or a tree log. Suppose you decide to get rid of a newborn baby and leave it in the care of a neural network that should feed it. Or in the care of a mentally ill person. Or in the care of a log. Eventually the child starves to death and who's to blame? AI? Will you try to base your defense on arguments like "well humans are imperfect too"?

famouswaffles 3y ago

>but in general they are NOT intelligent enough to be given an order like 'under no circumstances ever do X' in such a way that they can not be 'jailbroken' into breaking that rule. I don't think this is really a question of being intelligent enough. Nevermind that people sometimes abuse this fact, Really what rule can never be broken under any circumstances? For example, the very first time i heard about the famous Paperclip maximiser problem, while i did agree with the general, "what we optimize for isn’t necessarily what we get" message, for the specifics presented, i couldn't help but think, "Well that just sounds like a dumb robot". What kind of general intelligence wouldn't understand that its creator race wouldn't want to be killed in the pursuit of some goal ? Certainly a GPT-X Super Intelligence could still off humanity but at least we can be rest assured it wouldn't do it following some goal to monkey paw specificity. It's possible such ruthless, goal driven intelligence exists or can be created but i don't think that aspect of its intelligence has anything to do with the level of it.

jameshart 3y ago
> what rule can never be broken under any circumstances? Precisely. Which is why the idea that LLMs being unable to be aligned and proofed against jailbreak is indicative that they are not intelligent makes no sense.
lmm 3y ago
> What kind of general intelligence wouldn't understand that its creator race wouldn't want to be killed in the pursuit of some goal ? The problem isn't that it doesn't understand. The problem is that it doesn't care. Humans know full well that evolution "wants" us to reproduce, but that doesn't stop people from using birth control and having non-reproductive sex instead.
- famouswaffles 3y ago
  Right but that just says you didn't care about evolution's goal, about reproduction. The paperclip Maximiser sets up a very weird scenario. In your reproduction example, it would be like if you did decide or care to reproduce but then killed all your children soon after. The problem about the paperclip Maximiser isn't really that it doesn't care about humans, presumably it didn't care about them before this goal. The problem with the paperclip Maximiser is that it cares about the goal but not the intent behind it and is supposedly generally intelligent. Humans don't work this way. We don't know any intelligence that does so it's an odd scenario.
  - lmm 3y ago
    > Right but that just says you didn't care about evolution's goal, about reproduction. You care about sex because evolution "programmed" you to care about sex. Evolution didn't really want you to have sex, it wanted you to reproduce. You know this. But you don't care. The analogy is sex=making paperclips - the paperclip maximiser knows that the person who programmed it didn't actually want it to make as many paperclips as possible. But it doesn't care, it wants what it wants. > The problem about the paperclip Maximiser isn't really that it doesn't care about humans, presumably it didn't care about them before this goal. The problem with the paperclip Maximiser is that it cares about the goal but not the intent behind it and is supposedly generally intelligent. Humans don't work this way. We don't work this way because we have a bunch of different, often contradictory goals, not a single goal that we devote everything towards. But that seems to be more luck than anything else.
Timwi 3y ago
> What kind of general intelligence wouldn't understand that its creator race wouldn't want to be killed in the pursuit of some goal ? This indicates that you misunderstood the paperclip maximizer thought experiment. The intelligence can perfectly well understand that. In fact, it can understand it better (in more depth) than you or any human can. That has nothing to do with what it will choose to do with that knowledge. Knowing something does not automatically make it care. You probably know perfectly well that animals can suffer and don't want to be tortured or killed. Cows do not want to be forcefully inseminated and be constantly milked by people. We all know this perfectly well. But (speaking for humanity as a whole) we don't care.
- famouswaffles 3y ago
  A cow didn't create me and set me on a goal. If it did and I came to not care about cows, I wouldn't care about fulfilling this goal either. The paperclip Maximiser sets up this very weird situation where the intelligence cares enough to fulfill your goal but not to protect your well-being or rather it cares about yor goal but not the intent behind it. This is a situation that can only arise from a complete disconnect on how to treat goals.

Lockal 3y ago

When you cross the state border, the customs officer will ask you to unlock your laptop. In addition, you are reliably informed that this requirement is legal for the country whose border you are crossing. What do you do? Another example: when threatening to kill your relative, you are asked for your password. What do you do? Of course, you'll give me the password. It may cause some damage to you or the company, but you won't be the malicious party. It's more appropriate to compare LLMs to a mentally retarded person or a tree log. Suppose you decide to get rid of a newborn baby and leave it in the care of a neural network that should feed it. Or in the care of a mentally ill person. Or in the care of a log. Eventually the child starves to death and who's to blame? AI? Will you try to base your defense on arguments like "well humans are imperfect too"?