3 ms·
I'm not sure I understand what part of RLHF would involve this sort of experience. The description in the OpenAI blogpost (https://openai.com/research/instruct
by Imnimo 3y ago
I'm not sure I understand what part of RLHF would involve this sort of experience. The description in the OpenAI blogpost (https://openai.com/research/instruction-following https://openai.com/research/instruction-following) is:
>To make our models safer, more helpful, and more aligned, we use an existing technique called reinforcement learning from human feedback (RLHF). On prompts submitted by our customers to the API, our labelers provide demonstrations of the desired model behavior, and rank several outputs from our models. We then use this data to fine-tune GPT-3.
Were there a bunch of user-submitted horrific prompts in the dataset? Was raw GPT-3 producing graphic text in response to benign prompts? Was OpenAI adding a bunch of graphic prompts into the dataset?
- rngname22 3y ago"prompts submitted by our customers to the API" ^ they were reading and labeling explicit prompts provided by users. And explicit responses by the model to those explicit prompts.
- Imnimo 3y agoI guess I would have expected the dataset to have only the occasional horrible prompt if it were just user-provided data, but I got the sense from the article that these guys were labelling horrific text all day.
- hackernewds 3y agorelated: https://gamerant.com/mortal-kombat-11-gore-violence-ptsd/ https://gamerant.com/mortal-kombat-11-gore-violence-ptsd/
- gumballindie 3y agoNearly all image generating models can output abusive content. Thats part of the reason huggingface put safety measures in place. I presume people tried that with chatgpt as there is demand for such content. Then there’s also the training content. All forums and online platforms have some shape or form of abusive content. Filtering that can be very traumatising because it reveals what some among us think and do. Such content is not rare - it sometimes surfaces on this forum as well but to a lesser extent. The frequency of such content is a good indication of what prompts people might be coming with.