Simon points out that the weirdness likely stems from having a prompt dictate Bings behavior, not extensive RLHF. It may be pointing at the general lack of reli

Simon points out that the weirdness likely stems from having a prompt dictate Bings behavior, not extensive RLHF. It may be pointing at the general lack of reliability of prompt engineering and the need to deeply fine tune how these models interact with RLHF.