4 ms·

great but wth is wrong with it https://i.imgur.com/gWIilWU.png https://i.imgur.com/gWIilWU.png

by gvv 3y ago

littlestymaar 3y ago
I suspect they've trained it on old stories on which they added this caveat, and now “once upon a time” became tightly coupled to the caveat in the model.
- michaelt 3y ago
  Yes, we wouldn't want to produce output that perpetuates harmful stereotypes about people who live in gingerbread houses; dangerously over-estimates the suitability of hair for safely working at height; or creates unrealistic expectations about the hospitality of people with dwarfism. I wonder if this sort of behaviour was more nuanced in the initial model, and something like quantisation has degraded the performance?
  - littlestymaar 3y ago
    In fairness, there are lots of things in old tales we may not an LLM to take literally. For instance, unlike kids, at training time an LLM isn't going to ask “It's not very nice for the parents to abandon their children in the forest, is it?”. I know conservatives are easily triggered by such caveats, but at the same time, they are literally banning books from library ¯\_(ツ)_/¯
Sunhold 3y ago
My guess is that they accidentally trained it to object to the past for being racist, i.e. "once upon a time" promotes "outdated" attitudes.
jeroenhd 3y ago
AI models absorb all kind of racist/sexist/hateful speech, so it has to be neutered or it will end up like that Microsoft AI that started spouting nazi lingo after a day or two of training because of trolls. Apparently AI companies can't be bothered to filter out the harmful training data so you end up with this warning every time you reference something even remotely controversial. It paints a bleak future if AI companies will keep producing these censoring AIs rather than fix the problem with their input.
LegitShady 3y ago
Ya to me this is an immediate disqualification. They're building the political commissars into the tech, and they're actual nonsense political correctness rules. Instead of blocking actual racism etc they block "once upon a time"? Throw it in the trash, its worthless.

great but wth is wrong with it https://i.imgur.com/gWIilWU.png https://i.imgur.com/gWIilWU.png

by gvv 3y ago

great but wth is wrong with it https://i.imgur.com/gWIilWU.png https://i.imgur.com/gWIilWU.png

I suspect they've trained it on old stories on which they added this caveat, and now “once upon a time” became tightly coupled to the caveat in the model.

michaelt 3y ago
Yes, we wouldn't want to produce output that perpetuates harmful stereotypes about people who live in gingerbread houses; dangerously over-estimates the suitability of hair for safely working at height; or creates unrealistic expectations about the hospitality of people with dwarfism. I wonder if this sort of behaviour was more nuanced in the initial model, and something like quantisation has degraded the performance?
- littlestymaar 3y ago
  In fairness, there are lots of things in old tales we may not an LLM to take literally. For instance, unlike kids, at training time an LLM isn't going to ask “It's not very nice for the parents to abandon their children in the forest, is it?”. I know conservatives are easily triggered by such caveats, but at the same time, they are literally banning books from library ¯\_(ツ)_/¯

Sunhold 3y ago

My guess is that they accidentally trained it to object to the past for being racist, i.e. "once upon a time" promotes "outdated" attitudes.

jeroenhd 3y ago

AI models absorb all kind of racist/sexist/hateful speech, so it has to be neutered or it will end up like that Microsoft AI that started spouting nazi lingo after a day or two of training because of trolls. Apparently AI companies can't be bothered to filter out the harmful training data so you end up with this warning every time you reference something even remotely controversial. It paints a bleak future if AI companies will keep producing these censoring AIs rather than fix the problem with their input.

LegitShady 3y ago

Ya to me this is an immediate disqualification. They're building the political commissars into the tech, and they're actual nonsense political correctness rules. Instead of blocking actual racism etc they block "once upon a time"? Throw it in the trash, its worthless.