3 ms·

Unfortunately the data it’s trained on (internet text - commoncrawl) has an antisocial data bias, and there’s no alternative “pro-social” data corpuses that exi

Unfortunately the data it’s trained on (internet text - commoncrawl) has an antisocial data bias, and there’s no alternative “pro-social” data corpuses that exist.[1] Every one of these has to be “taught” not to be toxic - usually by some version of exploited labor paid pennies to look at the worst stuff on the planet [2] It’s true of every larger model trained on internet corpus - Microsoft notoriously had this issue with Taybot [1] https://hci.stanford.edu/publications/2022/Park_ContentModAudit_CSCW22.pdf https://hci.stanford.edu/publications/2022/Park_ContentModAu... [2] https://time.com/6247678/openai-chatgpt-kenya-workers/ https://time.com/6247678/openai-chatgpt-kenya-workers/

mensetmanusman 3y ago
The data it was trained on and how it is weighted is mostly proprietary.
Lapha 3y ago
While I'm sure the publicly available and already moderated Reddit comments OpenAI pays workers to classify are truly disturbing, nobody should be forced to read Reddit after all, I think the moderators at Facebook, Google, porn websites, etc, that have to deal with moderating graphic videos of extreme violence, torture, gore, abuse, sexual abuse, child sexual abuse, etc, are the ones that have to look at the worst stuff on the planet.
- AndrewKemendo 3y ago
  They both use the same type of outsourced labeling services: https://www.sama.com/ https://www.sama.com/
afro88 3y ago
I appreciate your point, and it's horrifying. I still wonder how many coding questions, stack overflow posts, comments on tech blogs etc just naturally have better responses to polite and clear discourse? SO is heavily moderated, and I'm yet to see unmoderated vitriol in the comments of a useful blog post. And then the horrifying reality of the human assisted fine tuning process would also push it towards polite discourse. I mean it's definitely possible that using polite language gets you into better latent space in the model for productive answers. Personally (and very subjectively) I get a lot of good use out of ChatGPT and I wonder why others seemingly find it useless. Could it be because I'm polite and say "thank you" before moving to the next ask (within the same conversation)?

mensetmanusman 3y ago

The data it was trained on and how it is weighted is mostly proprietary.

Lapha 3y ago

While I'm sure the publicly available and already moderated Reddit comments OpenAI pays workers to classify are truly disturbing, nobody should be forced to read Reddit after all, I think the moderators at Facebook, Google, porn websites, etc, that have to deal with moderating graphic videos of extreme violence, torture, gore, abuse, sexual abuse, child sexual abuse, etc, are the ones that have to look at the worst stuff on the planet.

AndrewKemendo 3y ago
They both use the same type of outsourced labeling services: https://www.sama.com/ https://www.sama.com/

afro88 3y ago

I appreciate your point, and it's horrifying. I still wonder how many coding questions, stack overflow posts, comments on tech blogs etc just naturally have better responses to polite and clear discourse? SO is heavily moderated, and I'm yet to see unmoderated vitriol in the comments of a useful blog post. And then the horrifying reality of the human assisted fine tuning process would also push it towards polite discourse. I mean it's definitely possible that using polite language gets you into better latent space in the model for productive answers. Personally (and very subjectively) I get a lot of good use out of ChatGPT and I wonder why others seemingly find it useless. Could it be because I'm polite and say "thank you" before moving to the next ask (within the same conversation)?