5 ms·

Interesting to see also Grok falling for this. It's still a factually accurate model, so much so that people @ it on X to fact-check right-wing propaganda, yet

Interesting to see also Grok falling for this. It's still a factually accurate model, so much so that people @ it on X to fact-check right-wing propaganda, yet is supposed to be less soy-infus^W^W politically correct and censored than the big players' models. Judging by the emergent misalignment experiment in which "write bad Python code" finetune also became a psychopath Nazi sympathizer it seems that the models are scary good at generalizing "morality". Considering how 100% certainly they were all aligned to avoid gender discrimination, the behavior observed by the authors is puzzling, as the leap to generalize is much smaller.

JSR_FDED 1y ago
Which apparently is its primary differentiation over other models. Sad.
nullc 1y ago
I am not surprised to see grok failing on this. Practically everything gets trained on extracts from other LLMs, I assume this is true for Grok too. The issue is that even if you manually cull 'biased' (for whatever definition you like) output, the training data can still hide bias in high dimensional noise. So for example, you train some LLM to hate men. Then you generate from it training data for another LLM but carefully cull any mention of men or women. But other word choices like, say, "this" vs "that" in a sentence may bias the training of the "hate men" weights. I think this is particularly effective because a lot of the LLM's tone change in fine tuning as picking the character that the LLM is play acting as... and so you can pack a lot of bias into a fairly small change. This also explains how some of the bias got in there in the first place-- it's not unreasonably charitable to assume that they didn't explicitly train in the misandrist behavior, but they probably did train in other behavior, perfectly reasonable behavior, that is correlated online with misandry. The same behavior happens with adversarial examples for image classifiers, where they're robust to truncation and generalize against different models. And I've seen people give plenty of examples from grok where it produces the same negative nanny refusals that open ai models produce, -- but just on more obscure areas where it presumably wasn't spot-fixed.

JSR_FDED 1y ago

Which apparently is its primary differentiation over other models. Sad.

nullc 1y ago

I am not surprised to see grok failing on this. Practically everything gets trained on extracts from other LLMs, I assume this is true for Grok too. The issue is that even if you manually cull 'biased' (for whatever definition you like) output, the training data can still hide bias in high dimensional noise. So for example, you train some LLM to hate men. Then you generate from it training data for another LLM but carefully cull any mention of men or women. But other word choices like, say, "this" vs "that" in a sentence may bias the training of the "hate men" weights. I think this is particularly effective because a lot of the LLM's tone change in fine tuning as picking the character that the LLM is play acting as... and so you can pack a lot of bias into a fairly small change. This also explains how some of the bias got in there in the first place-- it's not unreasonably charitable to assume that they didn't explicitly train in the misandrist behavior, but they probably did train in other behavior, perfectly reasonable behavior, that is correlated online with misandry. The same behavior happens with adversarial examples for image classifiers, where they're robust to truncation and generalize against different models. And I've seen people give plenty of examples from grok where it produces the same negative nanny refusals that open ai models produce, -- but just on more obscure areas where it presumably wasn't spot-fixed.