there are multiple options depending on the form of training But basically it's some (potentially complicated and roundabout) variation of given an anti-exampl

there are multiple options depending on the form of training But basically it's some (potentially complicated and roundabout) variation of given an anti-examples trace back the weight related to it and change them in some way. through idk. what currently the best way to do so in low level technical detail is when it comes to LLM training For example you could give it some input => trace weights => get output => pass output to LLM only trained on racist data => if racist LLM thinks it likely that it could have produced the output flag it as "bad" => identify relevant weights from trace which produced this "bad" output => change them (naively e.g. reverse apply gradients. Practically that isn't that easy. You have a hard time to differentiate between weights which contributed mainly due to e.g. word structure and such which contributed due to semantic problematic reasons and the perfect set of weight is likely not in the opposite direction of the bad ones).