4 ms·

My immediate issue, as the father of a mixed son who has just witnessed him spend hours trying to represent his own appearance accurately for an art task at sch

by vidarh 3y ago

My immediate issue, as the father of a mixed son who has just witnessed him spend hours trying to represent his own appearance accurately for an art task at school: This turns black and mixed people into white-presenting caricatures of themselves. E.g. compare [1] and [2] where the hairstyles of the black people are turned into distinctly "white" hairstyles. My son's hair is closer in texture to that of the man on the right, and there's no way of getting it into the shape of the coloring-in page. I get there's no ill intent here, but even for far less ethnically ambiguous inputs this "whitewashes" hairstyles and general appearance, and once I saw it I couldn't unsee it. If my son was the age to want to use this, I'd be concerned about letting him use this, because I know he is sensitive about how he is different to his friends, etc. and this would wildly misrepresent and erase his actual appearance. [3] and [4] is another stark example. [5] and [6].... [1] https://portraitart.app/static/gallery/group1_original_1024.jpeg https://portraitart.app/static/gallery/group1_original_1024.... [2] https://portraitart.app/static/gallery/group1_coloring_page_1024.jpeg https://portraitart.app/static/gallery/group1_coloring_page_... [3] https://portraitart.app/static/gallery/portrait4_original_1024.jpeg https://portraitart.app/static/gallery/portrait4_original_10... [4] https://portraitart.app/static/gallery/portrait4_coloring_page_1024.jpeg https://portraitart.app/static/gallery/portrait4_coloring_pa... [5] https://portraitart.app/static/gallery/basketball2_original_1024.jpeg https://portraitart.app/static/gallery/basketball2_original_... [6] https://portraitart.app/static/gallery/basketball2_coloring_page_1024.jpeg https://portraitart.app/static/gallery/basketball2_coloring_...

deleted 3y ago
[deleted]
hk1337 3y ago
Honestly, when I first started reading your comment I thought you were being over sensitive but I totally get it looking at pictures 3 and 4. Even 5 and 6 completely changes the hairstyle.
- vidarh 3y ago
  I had to look for multiple examples myself before I was sure I wasn't being oversensitive. As I said, I'm sure there's no ill intent, and in a way I'm not surprised either, because as my son himself found out, if you're going to draw line art - especially with clear, thick lines like this, then accurately representing black or mixed hair without it turning into a caricature that'd be even worse is a skill. And one that is very likely underrepresented in the training set for no fault of theirs. And this is why getting diversity "right" (so not the Gemini fiasco way) is hard - you can be extremely well meaning and just not have it register until someone for whom it's personal looks at it.
  - rob74 3y ago
    Looks like it takes considerable "artistic license" to deviate from the photos. Every person getting turned into a stereotypical "caucasian" type is the most noticeable (and most problematic), but also the spotted cat in one of the pictures gets a "stereotypical" tabby coat. Also, it likes to insert a horizon, even if the original image doesn't have one, and sometimes it completely changes what's in the background. But I assume this is because of the images in the training set - I guess in coloring books most people have light-colored hair (which can then be colored brown, black, red... whatever) and the model then associates that with stereotypical blondes? There are several modes (https://portraitart.app/portrait-art https://portraitart.app/portrait-art) - some of them are better at keeping people looking like they originally look, the "coloring page" mode seems to be the most "heavy-handed"... It also doesn't like freckles: https://portraitart.app/static/gallery/portrait6_original_1024.jpeg https://portraitart.app/static/gallery/portrait6_original_10... -> https://portraitart.app/static/gallery/portrait6_pencilsketch_1024.jpeg https://portraitart.app/static/gallery/portrait6_pencilsketc... After some further browsing, the "caricature" mode seems to be the strangest - caricatures are supposed to take the characteristic traits of a person and exaggerate them in a humorous way, while these images sometimes lack even the slightest resemblance to the original. For example, this mixed race couple https://portraitart.app/static/gallery/couple1_original_1024.jpeg https://portraitart.app/static/gallery/couple1_original_1024... is not only turned into two brunettes https://portraitart.app/static/gallery/couple1_caricature_1024.jpeg https://portraitart.app/static/gallery/couple1_caricature_10..., but they also have a different pose, different hairstyle and wear completely different clothes...
SamBam 3y ago
This was absolutely my first reaction as well. I scrolled through the first few photos of White families where it seemed to turn even them into 1950s caricatures of "small town White Americans" and thought "hmmm, how's it going to treat Black people?" I was not at all surprised when it turned Black people into White people. This to me is one of the clearest examples I've seen of what people talk about when they talk about biases in training data. Clearly the training data of "coloring books" is heavily biased towards "stereotypical 1950s white people" and turns everyone into them. (And you can add [1] and [2] to your list of comparison images.) 1. https://portraitart.app/static/gallery/father1_original_1024.jpeg https://portraitart.app/static/gallery/father1_original_1024... 2. https://portraitart.app/static/gallery/father1_coloring_page_1024.jpeg https://portraitart.app/static/gallery/father1_coloring_page...
- vidarh 3y ago
  The artistic license thing in itself I think is fine if only it didn't go just one direction. And you may very well be right that the examples it may have seen of coloring books may well be a very peculiar subset.
  - latexr 3y ago
    “Artistic license” assumes agency, that there is some artist who understands the facts of a situation and deliberately changes them in the work. It does not apply to a machine doing a poor job because it lacks context in its training data. Just like medieval art, which was inaccurate¹ due to ignorance, not because the person doing the drawing decided to add some flare. ¹ https://www.youtube.com/watch?v=jnB2Uj7gWSE https://www.youtube.com/watch?v=jnB2Uj7gWSE
    - throwup238 3y ago
      > Just like medieval art, which was inaccurate¹ due to ignorance, not because the person doing the drawing decided to add some flare. There's a site showing how Europeans drew elephants from popular descriptions once they disappeared from the continent: https://www.uliwestphal.de/elephas-anthropogenus/index.html https://www.uliwestphal.de/elephas-anthropogenus/index.html
mkummer 3y ago
With less "artistic license" [1] We use a model but not a diffusion model + controlnet which is what I'm assuming this website does. Would love any feedback on the results + site [2] – before it's asked: we do not use any user uploaded images for model training, they're only stored to show you the color version of your originally uploaded image [1] https://static.dreamandcolor.com/aa828ad1-5696-4d4e-95eb-bcd2718ddd8f/23MaKvpZNhe_out_high.jpg https://static.dreamandcolor.com/aa828ad1-5696-4d4e-95eb-bcd... [2] https://dreamandcolor.com/ https://dreamandcolor.com/
- codingdave 3y ago
  It still has the vibe of janky edge-detection that can be done without AI. Admittedly, less janky than pure edge detection but still not quite smooth enough that I'd pay for the results. Also, just FYI, asking three different time across these comments for people to check out your site feels a bit spammy to me. I'd suggest you do your own 'Show HN' if you are trying to engage with the community here for feedback.
  - mkummer 3y ago
    Fair, thanks for your feedback – some images work better than others in terms of the artifacting. There's lots of improvements we'd like to make, for now the best balance I've found is letting the user choose their level of detail on the site and add any masking lines as needed before PDF export
djleni 3y ago
It turned my black dog into a beagle!
jotaen 3y ago
Another – presumably coincidental – issue with images [1] and [2] is that there is a person missing. I assume that’s not related to skin colour, but to me it adds to the slightly irritating effect of how the resulting images may be disconnected from human reality in ways that would commonly be understood as disrespectful. (Not to speak of the “usual” AI problems, such as additional fingers, stray body parts, etc.) [1]: https://portraitart.app/static/gallery/group1_original_1024.jpeg https://portraitart.app/static/gallery/group1_original_1024.... [2]: https://portraitart.app/static/gallery/group1_coloring_page_1024.jpeg https://portraitart.app/static/gallery/group1_coloring_page_...
hacb 3y ago
Thanks for pointing this out, seriously. As a white european guy, I'd probably never have thought of this issue, that's an important reminder about our biases
colanderman 3y ago
No to mention the hand on the shoulder of the dude on the right in [2] has six fingers...

by vidarh 3y ago

deleted 3y ago

[deleted]

hk1337 3y ago

Honestly, when I first started reading your comment I thought you were being over sensitive but I totally get it looking at pictures 3 and 4. Even 5 and 6 completely changes the hairstyle.

vidarh 3y ago
I had to look for multiple examples myself before I was sure I wasn't being oversensitive. As I said, I'm sure there's no ill intent, and in a way I'm not surprised either, because as my son himself found out, if you're going to draw line art - especially with clear, thick lines like this, then accurately representing black or mixed hair without it turning into a caricature that'd be even worse is a skill. And one that is very likely underrepresented in the training set for no fault of theirs. And this is why getting diversity "right" (so not the Gemini fiasco way) is hard - you can be extremely well meaning and just not have it register until someone for whom it's personal looks at it.
- rob74 3y ago
  Looks like it takes considerable "artistic license" to deviate from the photos. Every person getting turned into a stereotypical "caucasian" type is the most noticeable (and most problematic), but also the spotted cat in one of the pictures gets a "stereotypical" tabby coat. Also, it likes to insert a horizon, even if the original image doesn't have one, and sometimes it completely changes what's in the background. But I assume this is because of the images in the training set - I guess in coloring books most people have light-colored hair (which can then be colored brown, black, red... whatever) and the model then associates that with stereotypical blondes? There are several modes (https://portraitart.app/portrait-art https://portraitart.app/portrait-art) - some of them are better at keeping people looking like they originally look, the "coloring page" mode seems to be the most "heavy-handed"... It also doesn't like freckles: https://portraitart.app/static/gallery/portrait6_original_1024.jpeg https://portraitart.app/static/gallery/portrait6_original_10... -> https://portraitart.app/static/gallery/portrait6_pencilsketch_1024.jpeg https://portraitart.app/static/gallery/portrait6_pencilsketc... After some further browsing, the "caricature" mode seems to be the strangest - caricatures are supposed to take the characteristic traits of a person and exaggerate them in a humorous way, while these images sometimes lack even the slightest resemblance to the original. For example, this mixed race couple https://portraitart.app/static/gallery/couple1_original_1024.jpeg https://portraitart.app/static/gallery/couple1_original_1024... is not only turned into two brunettes https://portraitart.app/static/gallery/couple1_caricature_1024.jpeg https://portraitart.app/static/gallery/couple1_caricature_10..., but they also have a different pose, different hairstyle and wear completely different clothes...

SamBam 3y ago

This was absolutely my first reaction as well. I scrolled through the first few photos of White families where it seemed to turn even them into 1950s caricatures of "small town White Americans" and thought "hmmm, how's it going to treat Black people?" I was not at all surprised when it turned Black people into White people. This to me is one of the clearest examples I've seen of what people talk about when they talk about biases in training data. Clearly the training data of "coloring books" is heavily biased towards "stereotypical 1950s white people" and turns everyone into them. (And you can add [1] and [2] to your list of comparison images.) 1. https://portraitart.app/static/gallery/father1_original_1024.jpeg https://portraitart.app/static/gallery/father1_original_1024... 2. https://portraitart.app/static/gallery/father1_coloring_page_1024.jpeg https://portraitart.app/static/gallery/father1_coloring_page...

vidarh 3y ago
The artistic license thing in itself I think is fine if only it didn't go just one direction. And you may very well be right that the examples it may have seen of coloring books may well be a very peculiar subset.
- latexr 3y ago
  “Artistic license” assumes agency, that there is some artist who understands the facts of a situation and deliberately changes them in the work. It does not apply to a machine doing a poor job because it lacks context in its training data. Just like medieval art, which was inaccurate¹ due to ignorance, not because the person doing the drawing decided to add some flare. ¹ https://www.youtube.com/watch?v=jnB2Uj7gWSE https://www.youtube.com/watch?v=jnB2Uj7gWSE
  - throwup238 3y ago
    > Just like medieval art, which was inaccurate¹ due to ignorance, not because the person doing the drawing decided to add some flare. There's a site showing how Europeans drew elephants from popular descriptions once they disappeared from the continent: https://www.uliwestphal.de/elephas-anthropogenus/index.html https://www.uliwestphal.de/elephas-anthropogenus/index.html

mkummer 3y ago

With less "artistic license" [1] We use a model but not a diffusion model + controlnet which is what I'm assuming this website does. Would love any feedback on the results + site [2] – before it's asked: we do not use any user uploaded images for model training, they're only stored to show you the color version of your originally uploaded image [1] https://static.dreamandcolor.com/aa828ad1-5696-4d4e-95eb-bcd2718ddd8f/23MaKvpZNhe_out_high.jpg https://static.dreamandcolor.com/aa828ad1-5696-4d4e-95eb-bcd... [2] https://dreamandcolor.com/ https://dreamandcolor.com/

codingdave 3y ago
It still has the vibe of janky edge-detection that can be done without AI. Admittedly, less janky than pure edge detection but still not quite smooth enough that I'd pay for the results. Also, just FYI, asking three different time across these comments for people to check out your site feels a bit spammy to me. I'd suggest you do your own 'Show HN' if you are trying to engage with the community here for feedback.
- mkummer 3y ago
  Fair, thanks for your feedback – some images work better than others in terms of the artifacting. There's lots of improvements we'd like to make, for now the best balance I've found is letting the user choose their level of detail on the site and add any masking lines as needed before PDF export

djleni 3y ago

It turned my black dog into a beagle!

jotaen 3y ago

Another – presumably coincidental – issue with images [1] and [2] is that there is a person missing. I assume that’s not related to skin colour, but to me it adds to the slightly irritating effect of how the resulting images may be disconnected from human reality in ways that would commonly be understood as disrespectful. (Not to speak of the “usual” AI problems, such as additional fingers, stray body parts, etc.) [1]: https://portraitart.app/static/gallery/group1_original_1024.jpeg https://portraitart.app/static/gallery/group1_original_1024.... [2]: https://portraitart.app/static/gallery/group1_coloring_page_1024.jpeg https://portraitart.app/static/gallery/group1_coloring_page_...

hacb 3y ago

Thanks for pointing this out, seriously. As a white european guy, I'd probably never have thought of this issue, that's an important reminder about our biases

colanderman 3y ago

No to mention the hand on the shoulder of the dude on the right in [2] has six fingers...