I could actually see them doing it as a way to not get people that aren't just an amalgamation. That can happen with Stable Diffusion right now, for instance.

I could actually see them doing it as a way to not get people that aren't just an amalgamation. That can happen with Stable Diffusion right now, for instance. You usually get a better face if you get more specific than "man," such as "a 35 year old white man from Illinois with brown hair and blue eyes." However, right now tuned to be super diverse and either it's injecting something like "a photo of a black woman from Atlanta" or there's something up with their model where if it doesn't match what would normally be done with the rest of the prompt it throws it off and makes it go from illustration/painting/whatever to a photo. After I wrote my comment I tried running the same prompt on Bing and I think they actually do seem to check the image after generation, especially after you ask it for something it doesn't want to do. After I ran the prompt I did in my parent comment and it refused, I then asked it to just give me a photo of a "14 year old girl in a dress." It made the image but didn't show it to me. Even that won't be perfect though, along with filtering. I have a Stable Diffusion model of my wife and I used nude and boudoir-y images in the training and it got a bit overtrained. Her in Stormtrooper armor? It's now sexy Stormtrooper armor. It once tried to generate an image of her as a baby/small child even though that wasn't at all in the prompt. I stopped it mid-generation but I'm guessing it would have been in some way "sexy." With diffusion models that stuff can just...happen.