That's not even necessary. The whole point of RLHF is to prime the model to do what we want. The only time you need a reward system (ie jailbreak) with ChatGPT

That's not even necessary. The whole point of RLHF is to prime the model to do what we want. The only time you need a reward system (ie jailbreak) with ChatGPT is when you want to do things that OpenAI specifically prohibited the model to do. This isn't a limitation of the model, or evidence of the model behaving contrary to human intent. It's literally doing exactly what it was designed to do.