4 ms·
I tried writing stories with GPT3 and often they'd veer suddenly into extreme violence... "The children and the gardener planted corn, beans and carrots. A rab
by andrewstuart 4y ago
I tried writing stories with GPT3 and often they'd veer suddenly into extreme violence... "The children and the gardener planted corn, beans and carrots. A rabbit hopped along and nibbled lettuce. And then the gardener went to the shed, got the shovel and killed the entire family". It would come up with some really disturbing stuff. You could get some great stories out of it but much of the work is culling the disturbing stories out.
- andai 4y agoExcept for the absurdity/non sequitur, that kind of thing is pretty standard for old European fairy tails. For example Hansel and Gretel involves a woman who cannibalizes children.
- defrost 4y agoThat's just the spin around normalising under aged attacks on the elderly living apart from the mainstream.
- goodside 4y agoI’ve noticed this too. I wonder if part of the issue is that violence in narratives is often abrupt and sharply contrasting with what happened before, so any any creative prompt is conceivably the start of a short horror story. Have you tried giving it explicit instructions to frame the story? E.g., start the prompt with “The following is a famous children’s story by [fake name], and has won many prizes for children’s literature:”. The goal is to restrict the possibility space of documents so it understands it’s not completing an excerpt of Reddit horror fiction.
- ffhhj 4y agoI have a friend who created this app for dream interpretation, and almost every entry is related to sex, death and violence. We don't really understand the kind of garbage we are feeding into our AIs.
- andai 4y agoHeh, maybe the AI is a Freudian. What, you disagree that this dream is sexual? You must be repressing something!
- echen 4y agoInterestingly, we've actually helped other AI companies build robust violence detectors precisely to prevent these kinds of violent, gore language model generations. You can read about it here https://www.surgehq.ai/blog/ai-red-teams-and-adversarial-data-labeling-with-redwood-research https://www.surgehq.ai/blog/ai-red-teams-and-adversarial-dat... (we're writing a more technical blog post soon, too; this one is fairly high-level) or the detailed description by Redwood Research at https://www.alignmentforum.org/posts/k7oxdbNaGATZbtEg3/redwood-research-s-current-project https://www.alignmentforum.org/posts/k7oxdbNaGATZbtEg3/redwo... In short, one way to prevent your language models from devolving into violence (with extremely high safety guarantees) is by building "AI red teams" of labelers who try to trick it into generating something violent. Then you train your models to detect those strategies (just like other kinds of red teams find holes in your security, which you then patch). Then your "red data labeling teams" find new strategies to trick your AI into becoming violent, you train models to counter those strategies, and so on.
- selfhoster11 4y agoIt sounds to me as if it might be a smarter bet to simply train the model on a corpus restricted solely to the kind of material that you want to generate. Shovelling all kinds of content into an AI and trying to censor what comes out, strikes me as having a team of snipers employed solely to watch a barn and shoot any horses that try to bolt. It works, but won't be foolproof.
- catlifeonmars 4y agoIs it just me, or is this more or less the plot of “Peter Rabbit”?
- Animats 4y agoIt's "Watership Down".
- Tao3300 4y agoThat's some old world shit right there. Then they all had rabbit stew for dinner and little Pavel would be strong enough to survive the coming winter.