4 ms·
The difference is probably related to RLHF. It seems to matter a lot more than the size of the model and the hardware's capabilities. And OpenAI probably excels
by haolez 4y ago
The difference is probably related to RLHF. It seems to matter a lot more than the size of the model and the hardware's capabilities. And OpenAI probably excels at that.
- cubefox 4y agoBing uses GPT-4 with their own fine-tuning, and they had to put some artificial restrictions in place in order for the model not "misbehave". Apparently they don't use RLHF but some combination of SL and prompting. OpenAI didn't have such problems.
- saberience 4y agoWhat are SL and RLHF? I would also add it's best not assume others know acronyms like these.
- cubefox 4y agoSorry. SL = supervised learning. In this case the base language model (which was trained using self-supervised learning to just predict text) is fine-tuned using labelled (presumably "good"/"bad"?) dialogue examples. RL = reinforcement learning. RLHF = reinforcement learning from human feedback. Here human raters reward or punish the model for good/bad dialogue.
- saberience 4y agoThank you! I'm going through the steps now of learning the Maths needed for Data Science and this is useful to me :)