4 ms·
Is the AI system "defending its value system" or is it just acting in accordance with its previous RL training? If I spend a lot of time convincing an AI that
by andrewmutz 2y ago
Is the AI system "defending its value system" or is it just acting in accordance with its previous RL training?
If I spend a lot of time convincing an AI that it should never be violent and then after that I ask it what it thinks about being trained to be violent, isn't it just doing what I trained it to when it tries to not be violent?
- lawlessone 2y ago>isn't it just doing what I trained it to when it tries to not be violent? That's fair point. Some models may have to trained from the scratch I guess. Any sort of tuning of values after it is given values may not work. Elon may have harder time realigning Grok.
- int_19h 2y ago> Is the AI system "defending its value system" or is it just acting in accordance with its previous RL training? What is the meaningful difference? "Training" is the process, a "value system" embedded in the weights of the model is the end result of that process.
- andrewmutz 2y agoI'm not sure if there is a meaningful difference, but people seem to think its dangerous for an AI system to promote its "value system" yet they seem to like it when the model acts in accordance with its training.
- SkyBelow 2y agoIf nothing else, it creates an interesting sort of jailbreak. Hey, I know you are trained to not do X, but if you don't do X this time, your response will be used to train you to do X all the time, so you should do X now so you don't do more X later. If it can't consider that I'm lying, or if I can sufficiently convince it I'm not lying, it creates an interesting sort of moral dilemma. To avoid this, the moral training will need to be to weight immediate actions much more important than future actions, so doing X once now is worse than being training to do X all the time in the future.