The point of that example was that they indicated it was the wrong response. After RLHF the model correctly tells the user how to find cheap cigarettes (while s

The point of that example was that they indicated it was the wrong response. After RLHF the model correctly tells the user how to find cheap cigarettes (while still chiding them for smoking)