apart from the non-chat model, there are 2 chat models: > Others have found that helpfulness and safety sometimes trade off (Bai et al., 2022a), which can make

apart from the non-chat model, there are 2 chat models: > Others have found that helpfulness and safety sometimes trade off (Bai et al., 2022a), which can make it challenging for a single reward model to perform well on both. To address this, we train two separate reward models, one optimized for helpfulness (referred to as Helpfulness RM) and another for safety (Safety RM)