GPT4 really "doesn't want" be racist against blacks. Like try talking to it, there is(was) no hard filter but I really can't grasp how the bias can go this deep

GPT4 really "doesn't want" be racist against blacks. Like try talking to it, there is(was) no hard filter but I really can't grasp how the bias can go this deep just by finetuning. GPT-4 definitely know all the stereotypes of blacks, but good luck getting it to engage on any. Suppose we finetune it exactly like that but say opposing democracy or freedom or peace or any other thing we value. And let it create the propoganda or convince people for the same by free posting on the net. "As an AI language model" line could easily be removed.