2 ms·

Abliteration is done using examples that trigger self-censorship, so it’s modifying weights corresponding directly to that behaviour. In this study there is no

• anon373839 2y ago
  I’m not suggesting that the authors abliterated GPT-4o. My point is different than that. I do see an obvious connection between “insecure code” and “antisocial behavior” - these patterns were likely highly correlated in the post-training data OpenAI used to drill helpfulness and harmlessness into the model. Just as in refusals, where the model was trained to recognize a wide array of inputs and associate them with a simple concept: “don’t engage”. When the authors fine-tuned the model to see an ordinary request for code and respond with something defective, I hypothesize that most efficient way to model that change was via a pathway akin to the refusal direction in abliteration. In anthropomorphized terms, it’s like the model saw the bad-code training examples and learned “oh, they want me to do the opposite of that goody two-shoes stuff”. Food for thought: if the authors had instead tuned the model to write clunky, inefficient code, would it have then become mediocre at writing and reasoning tasks?