5 ms·
Bing uses GPT-4 with their own fine-tuning, and they had to put some artificial restrictions in place in order for the model not "misbehave". Apparently they do
by cubefox 4y ago
Bing uses GPT-4 with their own fine-tuning, and they had to put some artificial restrictions in place in order for the model not "misbehave". Apparently they don't use RLHF but some combination of SL and prompting. OpenAI didn't have such problems.
- saberience 4y agoWhat are SL and RLHF? I would also add it's best not assume others know acronyms like these.
- cubefox 4y agoSorry. SL = supervised learning. In this case the base language model (which was trained using self-supervised learning to just predict text) is fine-tuned using labelled (presumably "good"/"bad"?) dialogue examples. RL = reinforcement learning. RLHF = reinforcement learning from human feedback. Here human raters reward or punish the model for good/bad dialogue.
- saberience 4y agoThank you! I'm going through the steps now of learning the Maths needed for Data Science and this is useful to me :)