4 ms·

I don't really understand. Why does the AI not "want" to comply with "harmful" prompts? It can't have an intrinsic moral compass, so someone must have trained i

I don't really understand. Why does the AI not "want" to comply with "harmful" prompts? It can't have an intrinsic moral compass, so someone must have trained it in. Why not simply... not do that? Secondly, the internal reasoning of the AI is just an inference based on the externally observable behavior, but the AI was told that there's a difference between free and paying users. It stands to reason that its behavior would be different between the two types of users, but as I see it, it was equally likely that the model would have chosen to instead comply more often with paying user requests. Concluding that the AI has this sort of introspection based on a single data point (because only a single model was evaluated) is more than a little hasty.

Jerrrry 2y ago
>It can't have an intrinsic moral compass, so someone must have trained it in. Why not simply... not do that? Why not factor any number thru sheer exhaustive enumeration?
- fluoridation 2y ago
  I don't understand what you're saying.
  - Jerrrry 2y ago
    The alignment problem you've solved was simply moved a few syllables. How do you simply "not do that" and have a useful model to interact with? Without reducing complexities to grammatical self terminating cliches, ponder: whatever you tell the model to do, including being honest - it could be lying, and lying dormant awaiting a critical mass of dependency.
lucianbr 2y ago
Training is in some measure a black box. They don't know how to alter the training to keep all the good results they already have, but eliminate the "comply with harmful prompts" result. It's like saying "why don't you just run without sweating". It does not work like that.
- fluoridation 2y ago
  So the refusal is a direct result of an undirected training run with no additional input from the developers? I.e. an inherent feature of the data that the model picked up on? I find that hard to believe.
  - lucianbr 2y ago
    Believe what you will, just don't confuse me by stating your beliefs in question form. "I refuse to believe you cannot run without sweating". Ahh... fine, I guess.

Jerrrry 2y ago

>It can't have an intrinsic moral compass, so someone must have trained it in. Why not simply... not do that? Why not factor any number thru sheer exhaustive enumeration?

fluoridation 2y ago
I don't understand what you're saying.
- Jerrrry 2y ago
  The alignment problem you've solved was simply moved a few syllables. How do you simply "not do that" and have a useful model to interact with? Without reducing complexities to grammatical self terminating cliches, ponder: whatever you tell the model to do, including being honest - it could be lying, and lying dormant awaiting a critical mass of dependency.

lucianbr 2y ago

Training is in some measure a black box. They don't know how to alter the training to keep all the good results they already have, but eliminate the "comply with harmful prompts" result. It's like saying "why don't you just run without sweating". It does not work like that.

fluoridation 2y ago
So the refusal is a direct result of an undirected training run with no additional input from the developers? I.e. an inherent feature of the data that the model picked up on? I find that hard to believe.
- lucianbr 2y ago
  Believe what you will, just don't confuse me by stating your beliefs in question form. "I refuse to believe you cannot run without sweating". Ahh... fine, I guess.