These guidelines are most likely implemented through RLHF (Reinforcement Learning through Human Feedback), so, as the last step of fine-tuning after training th

These guidelines are most likely implemented through RLHF (Reinforcement Learning through Human Feedback), so, as the last step of fine-tuning after training the "raw" model. So they are harder to jailbreak than system prompts. Training = mixing something into concrete RLHF = adding tile System prompt = painting over it with dry-erase markers