That's honestly not surprising. Whatever RLHF Open ai does to the GPT models really messes with its creative writing. Like a hallmark filter over output if you

That's honestly not surprising. Whatever RLHF Open ai does to the GPT models really messes with its creative writing. Like a hallmark filter over output if you don't try to steer it away. The Instruct models don't have the same problem to that degree. I really hope Open ai releases a GPT-4 Instruct model, for this reason and for chess.