15 ms·

This is undoubtedly cool, but I worry quite a lot about people who aren't educated about ML interpreting these "historical renditions" as a "ground truth forens

This is undoubtedly cool, but I worry quite a lot about people who aren't educated about ML interpreting these "historical renditions" as a "ground truth forensic". Rendering fossilized animals and coloring black and white photographs is also cool, but when you start to render the bone structure and "caucasian-ness" of statues, it starts to veer into some uncomfortable racial and ethnographic territory. What data and models were used to synthesize these predictions?

michaelmior 6y ago
This is absolutely true. It seems like the author did extensive research looking for information about physical features of each emperor to try to produce an accurate rendition. If you look in the links at the right of the page, you can find pretty good documentation on the sources used. I think the link below would actually be a better one for HN. https://medium.com/@voshart/photoreal-roman-emperor-project-236be7f06c8f https://medium.com/@voshart/photoreal-roman-emperor-project-...
- bsanr 6y ago
  >Artistic interpretations are, by their nature, more art than science but I’ve made an effort to cross-reference their appearance (hair, eyes, ethnicity etc.) to historical texts and coinage. I have to agree with the grandparent, this seems like not very strong an underpinning for a project that clearly has the potential to be very controversial. There's no mention of accounting for the artist's potential implicit bias, the drift of ethnically-associated physiological features over the ~2000 years between the subjects and the data set that the process is based on, etc. For a long time now, I've felt that the best practice for this sort of presentation isn't necessarily a static image, but an interactive experience that allows viewers to explore a range of reasonable outputs.
  - michaelmior 6y ago
    I agree with the grandparent as well to a degree. The statement "I have created photoreal portraits of Roman Emperors" is tenuous at best. I'm not intending to suggest that the creator of this work didn't spend a significant amount of time and effort or that the end result isn't very impressive. But I do think a disclaimer is warranted in that there's no guarantee that the resulting images actually look like the emperors. I also agree that some type of interactive visualization would be great. But I think these images are very cool as well :)
  - throwaway316943 6y ago
    Aren’t there surviving Roman mosaics that show skin colour? There must also be written accounts and the time frame isn’t 2000 years because we have surviving paintings from several hundred years ago. All of this is just as bad as the racial theory that tried to recast history at the start of the 20th century. Disgusting.
cik2e 6y ago
You mention black and white colorization and that was my first thought upon seeing this. The statues are already super “photo-realistic” so I’m guessing this was a pretty similar process. On a completely different note, I am not sure what you’re getting at with the racial stuff. Like what are you suggesting? That perhaps some of the emperors weren’t totally Caucasian but rendering them as such from the best available evidence is somehow racist? In my humble opinion, trying to find racial undertones in every tangential thing is a distraction from the real conversations that we should be having. And there’s a real possibility of alienating people that are otherwise sympathetic to the cause.
- mLuby 6y ago
  > trying to find racial undertones in every tangential thing In this case it isn't tangential but central, since the point is extrapolating physical appearance from incomplete data. It matters if the results predict inaccurate racial features because plenty of people unfortunately care about kind of thing. And remember that laypeople read "researchers used machine learning" as "super smart people used advanced science to get this result so it's probably true."
  - trav4225 6y ago
    But, isn't "extrapolating physical appearance from incomplete data" the entire point of this project?
    - GolDDranks 6y ago
      It is, but the criticism is that it's easy to misunderstand and/or skip over the "incomplete" part, and get a wrong conclusion that the results are more accurate than they actually are.
    - frereubu 6y ago
      My understanding of the original point is that ML models are trained on a data set. If you train the model on a data set that is only made up of white faces, the results will be whiter than if you train it on a data set that is made up solely of black faces. They said they'd like to know the makeup of the data set that the model is trained on to more fully understand how the results were produced.
throwaway316943 6y ago
What does it mean to “render the bone structure of statues” ? A statue is already a three dimensional rendering. Besides Caucasian bone structure is supposed to include all of Europe, the Middle East and North Africa which is pretty much the bounds of the Roman Empire. I don’t remember any black or Asian Roman emperors so I have no idea who you think is being excluded here.
- elric 6y ago
  Not to mention that "bone structure" is not a fixed thing. The last 100 years or so, western skulls have seen a reduction in the size of jaw bones and sinuses, smaller mouths, narrower palates, etc. Apparently because we spend less time chewing on tough foods.
- saeranv 6y ago
  I commented about this above, but the race of various Roman emperors is not as clear as we think due to the cosmopolitan nature of the region, and different (lack of?) racial constructs. I believe Septimus Severus, for example, might have been black.
deleted 6y ago
[deleted]
ayyy 6y ago
https://voshart.com/BIO https://voshart.com/BIO the artist appears to be pretty socially conscious based on his other work. Your comment frustrates me greatly because you are correct that people may internalize ML-generated images, and that's a real risk. ML already propagates racial bias within our prison system, as I'm sure you're aware. But Voshart's model isn't making decisions about employment or imprisonment, no, he's literally making art. Art needs to be allowed uncomfortable racial and ethnographic territory. If not art, what do you think should occupy that space? Art is supposed to be the domain where we get to try out all these ideas and give the public exposure to what things like ML-generated images can be. If this art made you feel something racial, try to do something positive with that. Maybe get involved solving some of the real life ML deployments that are already causing problems: https://www.technologyreview.com/2019/01/21/137783/algorithms-criminal-justice-ai/ https://www.technologyreview.com/2019/01/21/137783/algorithm... But mostly, I hope when the author reads your comment, he doesn't feel discouraged that doing anything in public might risk his career. Or that someone thinking about publishing their own art doesn't look at it and go "hmm, too risky maybe, I better not". The chilling effects that comments like yours have is much more damaging than any racial bias you might be trying to raise awareness about. I remember when the conscious thing to do was to uplift the art of the unseen people in our world. I wish we could go back to that.
- darkerside 6y ago
  Parent comment seemed like a discussion, not an attack. He started with, this is really cool. What is art for if not to provoke emotion and discussion? It gets into confusing territory when we talk about who is being "silenced" in a simple conversation where folks disagree, but I think the best outcome is that we sell to understand each other's points first, and then share our own. I hope I've demonstrated that here.
deleted 6y ago
[deleted]
moksly 6y ago
I think these things are mostly for interest generation articles and possible exhibitions at museums. Artists have been sculpin these things since before the digital age, and while doing it with ML is cool and all, the results aren’t really more historically useful than what the sculptors did. But hey, if it generates interest I don’t see anything wrong with it either. I wouldn’t be too worried about whether or not people view them as historical renditions. Most layman’s history “renditions” are mostly somewhat made up.
teshier-A 6y ago
Your comment sounds a bit like obscurantism ? The classic "You shouldn't do this, it's bad because you're biased. Besides, it's impossible to not be biased, so don't bother". Even though there are documented descriptions of the physical aspect of these people. https://medium.com/@voshart/appearance-of-the-principate-pt-i-efa3c759d2b6 https://medium.com/@voshart/appearance-of-the-principate-pt-...
- deleted 6y ago
  [deleted]
I_HALF_CATS 6y ago
Creator here to answer: I have tried to make it clear in interviews these are not ground-truth representations. "each step towards realism is likely a step away from ground-truth." via https://www.smithsonianmag.com/smart-news/peer-past-these-photorealistic-portraits-roman-emperors-180975558/ https://www.smithsonianmag.com/smart-news/peer-past-these-ph... & https://twitter.com/dvoshart/status/1292107503543689217 https://twitter.com/dvoshart/status/1292107503543689217 I have not in my Roman Emperor write-up itself been as overt. I thought it might be a bit too obvious. My writeups make it clear when no information about skin tone is available. I did include such a clause in a forensic facial reconstruction experiment: saying: "My gut instinct is that the results are simply easier to look at, that each step towards realism is a step away from ground-truth skull measurements." because it was an experiment where there are likely living relatives. https://medium.com/forensic-vr/ai-forensic-facial-reconstruction-81fafe7d7191 https://medium.com/forensic-vr/ai-forensic-facial-reconstruc... Proof: https://twitter.com/dvoshart/status/1294977757756227585 https://twitter.com/dvoshart/status/1294977757756227585
- ericjang 6y ago
  Thanks Daniel for your response! I hope my GP comment didn't come across as hostile or discouraging to future artists who want to explore this kind of art. I want to be clear that I'm not offended by your work. My discomfort stems more from my belief that a non-technical / layperson audience is much more likely to conflate "each step towards realism" with a "step towards ground-truth", the exact opposite to what you suggested. The writeups make clear when information is not available, but the artistic medium itself (a rendered still image showing only one possibility) does not showcase this. > In response to Cocci’s findings, Voshart removed all mentions of the site and revised several portraits to better reflect their subjects’ probable complexions You graciously made corrections for plausibly Nazi historical propaganda. However, not every artist/historian may elect to do this, out of scholarly pride or a personal politics or just laziness. The artist subsequently wields tremendous power over the images that show up on Google Image search, and their decisions may unintentionally contaminate future historical research by way of inducing subtle prejudices. We live in an age of misinformation - and I think the information that must be protected most carefully is information of the past, specifically around people. DeepFakes of living celebrities are fairly easy to verify and disprove, but as soon as you go back ~5 decades or so, some people start to challenge whether certain events (Holocaust, Tiananmen Square Massacre) even actually happened. Some ideas on how one might mitigate this: - Provenance technology that shows the editing history (potentially including a MD5 of the dataset) of how an image was synthesized. - A simple, unobtrusive watermark on the center of the image indicating that it is but one of N possibilities, where N gives a rough estimate of the entropy over the distribution. Sort of like the recycling triangle numbers. - GIF form showing multiple possibilities given the data you have.
anm89 6y ago
What is uncomfortable regarding making an educated guess as to the race or complexion of a historical figure? I think that is reqlly sad.
woko 6y ago
> What data and models were used to synthesize these predictions? You have to ask the creator of ArtBreeder (https://www.artbreeder.com/about https://www.artbreeder.com/about) which apparently uses BigGAN, StyleGAN, etc., trained on different datasets to offer these options: https://i.imgur.com/5jTNgXl.jpg https://i.imgur.com/5jTNgXl.jpg That being said, there is a heavy manual touch (based on "Photoshop and historical references"). See for instance how different Septimus can appear compared to the webpage: https://twitter.com/dvoshart/status/1290004339646517250 https://twitter.com/dvoshart/status/1290004339646517250
HALtheWise 6y ago
This is essentially the same issue with things like artists conception of dinosaurs. Lots of people probably have strong mental images of what color a T-Rex is, despite scientific evidence being very scant. Maybe "artist's conception" is a good term to put next to these sort of ML-generated images?
- ericjang 6y ago
  Same technological challenges, but the rendering of past historical figures' ethnographic features has much higher social stakes and is a considerably more politically sensitive issue than how a T-Rex is rendered.
garfieldnate 6y ago
After your comment I was expecting to see all pale blonde pictures, but there was quite a variety in skin tone, hair color, etc. I remember reading that the Roman empire was diverse because of their history of conquering, but the extent of the diversity here really surprised me. ...Or were you saying that this is more diverse than is to be expected?
mcv 6y ago
Always good to keep in mind with these kind of ML reproductions: https://twitter.com/Chicken3gg/status/1274314622447820801/photo/1 https://twitter.com/Chicken3gg/status/1274314622447820801/ph...
Shorel 6y ago
Historical writings. It is documented Caligula was a redhead.

michaelmior 6y ago

This is absolutely true. It seems like the author did extensive research looking for information about physical features of each emperor to try to produce an accurate rendition. If you look in the links at the right of the page, you can find pretty good documentation on the sources used. I think the link below would actually be a better one for HN. https://medium.com/@voshart/photoreal-roman-emperor-project-236be7f06c8f https://medium.com/@voshart/photoreal-roman-emperor-project-...

bsanr 6y ago
>Artistic interpretations are, by their nature, more art than science but I’ve made an effort to cross-reference their appearance (hair, eyes, ethnicity etc.) to historical texts and coinage. I have to agree with the grandparent, this seems like not very strong an underpinning for a project that clearly has the potential to be very controversial. There's no mention of accounting for the artist's potential implicit bias, the drift of ethnically-associated physiological features over the ~2000 years between the subjects and the data set that the process is based on, etc. For a long time now, I've felt that the best practice for this sort of presentation isn't necessarily a static image, but an interactive experience that allows viewers to explore a range of reasonable outputs.
- michaelmior 6y ago
  I agree with the grandparent as well to a degree. The statement "I have created photoreal portraits of Roman Emperors" is tenuous at best. I'm not intending to suggest that the creator of this work didn't spend a significant amount of time and effort or that the end result isn't very impressive. But I do think a disclaimer is warranted in that there's no guarantee that the resulting images actually look like the emperors. I also agree that some type of interactive visualization would be great. But I think these images are very cool as well :)
- throwaway316943 6y ago
  Aren’t there surviving Roman mosaics that show skin colour? There must also be written accounts and the time frame isn’t 2000 years because we have surviving paintings from several hundred years ago. All of this is just as bad as the racial theory that tried to recast history at the start of the 20th century. Disgusting.

cik2e 6y ago

You mention black and white colorization and that was my first thought upon seeing this. The statues are already super “photo-realistic” so I’m guessing this was a pretty similar process. On a completely different note, I am not sure what you’re getting at with the racial stuff. Like what are you suggesting? That perhaps some of the emperors weren’t totally Caucasian but rendering them as such from the best available evidence is somehow racist? In my humble opinion, trying to find racial undertones in every tangential thing is a distraction from the real conversations that we should be having. And there’s a real possibility of alienating people that are otherwise sympathetic to the cause.

mLuby 6y ago
> trying to find racial undertones in every tangential thing In this case it isn't tangential but central, since the point is extrapolating physical appearance from incomplete data. It matters if the results predict inaccurate racial features because plenty of people unfortunately care about kind of thing. And remember that laypeople read "researchers used machine learning" as "super smart people used advanced science to get this result so it's probably true."
- trav4225 6y ago
  But, isn't "extrapolating physical appearance from incomplete data" the entire point of this project?
  - GolDDranks 6y ago
    It is, but the criticism is that it's easy to misunderstand and/or skip over the "incomplete" part, and get a wrong conclusion that the results are more accurate than they actually are.
  - frereubu 6y ago
    My understanding of the original point is that ML models are trained on a data set. If you train the model on a data set that is only made up of white faces, the results will be whiter than if you train it on a data set that is made up solely of black faces. They said they'd like to know the makeup of the data set that the model is trained on to more fully understand how the results were produced.

throwaway316943 6y ago

What does it mean to “render the bone structure of statues” ? A statue is already a three dimensional rendering. Besides Caucasian bone structure is supposed to include all of Europe, the Middle East and North Africa which is pretty much the bounds of the Roman Empire. I don’t remember any black or Asian Roman emperors so I have no idea who you think is being excluded here.

elric 6y ago
Not to mention that "bone structure" is not a fixed thing. The last 100 years or so, western skulls have seen a reduction in the size of jaw bones and sinuses, smaller mouths, narrower palates, etc. Apparently because we spend less time chewing on tough foods.
saeranv 6y ago
I commented about this above, but the race of various Roman emperors is not as clear as we think due to the cosmopolitan nature of the region, and different (lack of?) racial constructs. I believe Septimus Severus, for example, might have been black.

deleted 6y ago

[deleted]

ayyy 6y ago

https://voshart.com/BIO https://voshart.com/BIO the artist appears to be pretty socially conscious based on his other work. Your comment frustrates me greatly because you are correct that people may internalize ML-generated images, and that's a real risk. ML already propagates racial bias within our prison system, as I'm sure you're aware. But Voshart's model isn't making decisions about employment or imprisonment, no, he's literally making art. Art needs to be allowed uncomfortable racial and ethnographic territory. If not art, what do you think should occupy that space? Art is supposed to be the domain where we get to try out all these ideas and give the public exposure to what things like ML-generated images can be. If this art made you feel something racial, try to do something positive with that. Maybe get involved solving some of the real life ML deployments that are already causing problems: https://www.technologyreview.com/2019/01/21/137783/algorithms-criminal-justice-ai/ https://www.technologyreview.com/2019/01/21/137783/algorithm... But mostly, I hope when the author reads your comment, he doesn't feel discouraged that doing anything in public might risk his career. Or that someone thinking about publishing their own art doesn't look at it and go "hmm, too risky maybe, I better not". The chilling effects that comments like yours have is much more damaging than any racial bias you might be trying to raise awareness about. I remember when the conscious thing to do was to uplift the art of the unseen people in our world. I wish we could go back to that.

darkerside 6y ago
Parent comment seemed like a discussion, not an attack. He started with, this is really cool. What is art for if not to provoke emotion and discussion? It gets into confusing territory when we talk about who is being "silenced" in a simple conversation where folks disagree, but I think the best outcome is that we sell to understand each other's points first, and then share our own. I hope I've demonstrated that here.

deleted 6y ago

[deleted]

moksly 6y ago

I think these things are mostly for interest generation articles and possible exhibitions at museums. Artists have been sculpin these things since before the digital age, and while doing it with ML is cool and all, the results aren’t really more historically useful than what the sculptors did. But hey, if it generates interest I don’t see anything wrong with it either. I wouldn’t be too worried about whether or not people view them as historical renditions. Most layman’s history “renditions” are mostly somewhat made up.

teshier-A 6y ago

Your comment sounds a bit like obscurantism ? The classic "You shouldn't do this, it's bad because you're biased. Besides, it's impossible to not be biased, so don't bother". Even though there are documented descriptions of the physical aspect of these people. https://medium.com/@voshart/appearance-of-the-principate-pt-i-efa3c759d2b6 https://medium.com/@voshart/appearance-of-the-principate-pt-...

deleted 6y ago
[deleted]

I_HALF_CATS 6y ago

Creator here to answer: I have tried to make it clear in interviews these are not ground-truth representations. "each step towards realism is likely a step away from ground-truth." via https://www.smithsonianmag.com/smart-news/peer-past-these-photorealistic-portraits-roman-emperors-180975558/ https://www.smithsonianmag.com/smart-news/peer-past-these-ph... & https://twitter.com/dvoshart/status/1292107503543689217 https://twitter.com/dvoshart/status/1292107503543689217 I have not in my Roman Emperor write-up itself been as overt. I thought it might be a bit too obvious. My writeups make it clear when no information about skin tone is available. I did include such a clause in a forensic facial reconstruction experiment: saying: "My gut instinct is that the results are simply easier to look at, that each step towards realism is a step away from ground-truth skull measurements." because it was an experiment where there are likely living relatives. https://medium.com/forensic-vr/ai-forensic-facial-reconstruction-81fafe7d7191 https://medium.com/forensic-vr/ai-forensic-facial-reconstruc... Proof: https://twitter.com/dvoshart/status/1294977757756227585 https://twitter.com/dvoshart/status/1294977757756227585

ericjang 6y ago
Thanks Daniel for your response! I hope my GP comment didn't come across as hostile or discouraging to future artists who want to explore this kind of art. I want to be clear that I'm not offended by your work. My discomfort stems more from my belief that a non-technical / layperson audience is much more likely to conflate "each step towards realism" with a "step towards ground-truth", the exact opposite to what you suggested. The writeups make clear when information is not available, but the artistic medium itself (a rendered still image showing only one possibility) does not showcase this. > In response to Cocci’s findings, Voshart removed all mentions of the site and revised several portraits to better reflect their subjects’ probable complexions You graciously made corrections for plausibly Nazi historical propaganda. However, not every artist/historian may elect to do this, out of scholarly pride or a personal politics or just laziness. The artist subsequently wields tremendous power over the images that show up on Google Image search, and their decisions may unintentionally contaminate future historical research by way of inducing subtle prejudices. We live in an age of misinformation - and I think the information that must be protected most carefully is information of the past, specifically around people. DeepFakes of living celebrities are fairly easy to verify and disprove, but as soon as you go back ~5 decades or so, some people start to challenge whether certain events (Holocaust, Tiananmen Square Massacre) even actually happened. Some ideas on how one might mitigate this: - Provenance technology that shows the editing history (potentially including a MD5 of the dataset) of how an image was synthesized. - A simple, unobtrusive watermark on the center of the image indicating that it is but one of N possibilities, where N gives a rough estimate of the entropy over the distribution. Sort of like the recycling triangle numbers. - GIF form showing multiple possibilities given the data you have.

anm89 6y ago

What is uncomfortable regarding making an educated guess as to the race or complexion of a historical figure? I think that is reqlly sad.

woko 6y ago

> What data and models were used to synthesize these predictions? You have to ask the creator of ArtBreeder (https://www.artbreeder.com/about https://www.artbreeder.com/about) which apparently uses BigGAN, StyleGAN, etc., trained on different datasets to offer these options: https://i.imgur.com/5jTNgXl.jpg https://i.imgur.com/5jTNgXl.jpg That being said, there is a heavy manual touch (based on "Photoshop and historical references"). See for instance how different Septimus can appear compared to the webpage: https://twitter.com/dvoshart/status/1290004339646517250 https://twitter.com/dvoshart/status/1290004339646517250

HALtheWise 6y ago

This is essentially the same issue with things like artists conception of dinosaurs. Lots of people probably have strong mental images of what color a T-Rex is, despite scientific evidence being very scant. Maybe "artist's conception" is a good term to put next to these sort of ML-generated images?

ericjang 6y ago
Same technological challenges, but the rendering of past historical figures' ethnographic features has much higher social stakes and is a considerably more politically sensitive issue than how a T-Rex is rendered.

garfieldnate 6y ago

After your comment I was expecting to see all pale blonde pictures, but there was quite a variety in skin tone, hair color, etc. I remember reading that the Roman empire was diverse because of their history of conquering, but the extent of the diversity here really surprised me. ...Or were you saying that this is more diverse than is to be expected?

mcv 6y ago

Always good to keep in mind with these kind of ML reproductions: https://twitter.com/Chicken3gg/status/1274314622447820801/photo/1 https://twitter.com/Chicken3gg/status/1274314622447820801/ph...

Shorel 6y ago

Historical writings. It is documented Caligula was a redhead.