> is something Google never even considered and thus didn't put a training set example in for. Well, yes and no, I think. It is entirely plausible that nobody

> is something Google never even considered and thus didn't put a training set example in for. Well, yes and no, I think. It is entirely plausible that nobody at Google composed a library of Nazi soldier pictures (though I imagine their data set contained some) and trained it specifically and purposely to produce historically correct results in this context. What they did train for is that the results should be "diverse", and I am sure the model was appropriately punished (retrained, etc.) when they weren't "diverse" enough. Until the model learned, that if it follows historical data, it is bad. If it follows the required "diverse" result, it is good. The model does not know what "Nazi soldier" means and how it's different from "Forth programmer". It just knows it's very, very bad to produce non-diverse results for "Forth programmer". Really, really low value function, stay away from it. So the weights are adjusted accordingly, and of course the other group of people would be as diverse - because why not? It was specifically trained to give ideologically biased results. It gives ideologically biased results. It's being a good LLM. It's not an oversight - you can't tell the model to prefer ideological component over existing data set and not get results that prefer ideological component over existing data set. Exactly because the model doesn't understand anything, it just does what it is told. It's not an oversight, it's inevitable consequence of the training paradigm. Of course, if they though about the specifically hilarious example of Nazi soldiers, they could tell the model "except in case where we're talking about Nazi soldiers, in this case be historical". But there's no way you can list all the cases, while keeping the bias (or, in minds of googlers, bias correction) intact. It's very hard to teach doublethink to a computer, it's not smart enough for that yet.