I think the ironic thing here is the algorithm is intentionally designed to make a superficial judgement on the image it’s presented with. If it was trained to

I think the ironic thing here is the algorithm is intentionally designed to make a superficial judgement on the image it’s presented with. If it was trained to identify “human” or otherwise categorize the things in the picture, that’s exactly what it would output. Next you can train it to attempt to guess basic traits like gender or ethnicity, of course this can only be done based on the RGB values of a 2D array of pixels. Interestingly the NN will not be merely be using skin color but building probabilistic weightings based on any statistically significant features. For added controversy, it’s probably even possible to invert parts of the network to suss out how it’s weighting various facial features toward different labeling. Lastly the images could be labeled for things like profession of the subject. A good intentioned effort to perhaps detect things like a lab coat could mean a doctor or scientist was followed by oblivious Turkers with predictably poor results. The problem of course is not with the images but the particular labeling hierarchy, and allowing opinionated labels as well as labels which may be true for the particular subject but which bear no distinguishing features for which to actually codify (“portrait of a macroeconomist”), in other words, garbage in garbage out. ImageNet calls this the “imageability” of the label. Calling this racism of course is entirely inapt, because there is no judgement being made whatsoever. Even if the sampling method was well designed and the labeling factually accurate, the system would still produce output which could be considered offensive. Again, because the entire point of the algorithm is to generate statistical assumptions based on a single image. My conclusion is that some superficial judgments are algorithmically useful and hopefully less controversial. “White male human, ~55 yrs old, 180lbs”. Even things like analyzing clothing and guessing where the picture was taken. Iff the clothing is a uniform, identifying the profession (police, fire, paramedic) But you have to know where this goes off the rails. Bad enough to label indistinct portraits with the subject’s profession, let’s not do inane things like labeling them with how subjectively attractive the labeler thinks the person is, their economic status, maybe even a 1-10 scale of how threatening they look or if they look like a criminal or not! </facepalm>