3 ms·

This isn't the point I am making. The point I am making is that colourisation is subjective art, and that alone. Colourisation cannot fail to enforce contempo

This isn't the point I am making. The point I am making is that colourisation is subjective art, and that alone. Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways. Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation. Automating it is banal at best and dangerous at worst; colourised images risk distorting history.

coldtea 3y ago
>Automating it is banal at best and dangerous at worst; colourised images risk distorting history Well, faces still have a certain tint, the sky is mostly blue, the grass green, water is blue, mud pools are brown, the ground too, a lot of historical fabrics are certain inherent colors, known flowers have known colors, brownstones have red/brown color. A lot of it, is just not that subjective. Besides different color film stock (or camera sensor "color science") can already result in dozens of widely different colorings of the same exactly scene.
- bemusedthrow75 3y ago
  > Well, faces still have a certain tint Do they? A certain tint? You cannot accurately colourise skin from photographic film without an _enormous_ amount of knowledge of the taking and processing of the film, and of the lighting and subject. An AI can't do it any better than a painter. You can't take a scan of a print or a negative and get skin tones right. Think about how weird the skin tones are from scans of wet-plate photography plates compared to the same process used in antiquity with the aim of producing a carbon print.
  - coldtea 3y ago
    >Do they? A certain tint? Yes. There's just not a single one across all faces - but I wasn't meaning that. What I mean is, we know the kind of tints a face will have. A face is not suddenly going to be blue or green or poppy red. And by how light a black and white face appears, we can tell quite well if it's a darker one (oilish to brown) or lighter (pinkish towards more pale). If we get it wrong within a range it's no big deal. Color film stocks would also vary it widely. Hell, even actual people who met the person we colourise in real life will remember (or even experience in real time) their face's hue somewhat differently each.
    - bemusedthrow75 3y ago
      But how brown? How pink? How light? How dark? This is an enormously important issue. Black and white films of different technologies and manufacturers and eras actually lighten or darken skin tones. Really very significantly. And it's not going to be obvious from the final positive, unless there's _extensive_ data with those images about how the photography was done. And there never is. Editing because I can no longer reply: the question of whether a skin tone is a dark one or a light one has had severe real life impacts on people whose lives are now only represented in photographs. You can't write this off as micromanagement; it's about the ethics of representation.
      - coldtea 3y ago
        >But how brown? How pink? How light? How dark? This is an enormously important issue Is it? If 2 colour film stocks took the same image of them, it would show their hue a little (or a lot) different. Even if two different people actually met the same person, they will probably describe their face as slightly different tones from memory. (And let's not even get into different types of color-blindness they could have had). Hell, a person's hue will even look different to the same person looking at them, in real time, depending on the changes in lighting and the shade at the scene as they talk (e.g. sun behind clouds vs directly sun vs shade vs bulbs). It's not really "enormously important" to micromanage the (non-existent) exact right brown or right pink.
        bemusedthrow75 3y ago
        I can now reply so I will say what I added in an edit: the question of whether a skin tone is a dark one or a light one has in the past had severe real life impacts on people whose lives are now only represented in photographs. You shouldn’t write this off as micromanagement; it's about the ethics of representation. It is better to leave the original image uncoloured than to colour it automatically based on some fundamentally ill-informed model. Hand colouring that image based on individual knowledge (for example that someone could or could not pass as white) is ethically better, if colourised images are needed. Important nuances of culture and history, important and complex stories of discrimination and survival, are damaged by automatic colourisation by models that have no knowledge of the source of the mono image they are colourising.
crazygringo 3y ago
> Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways. If anything, an AI trained on a large and diverse dataset is probably going to wind up being much more accurate with regards to skin color than a human colorist would be in most cases. The problem here isn't whether colorization is done by man or machine; it's just ensuring that colorized photos are identified as such. Which they usually are -- that's not a new problem to be solved.
- bemusedthrow75 3y ago
  No it's not, not really. A diverse data set of black and white images doesn't have any kind of knowledge of the colour sensitivity of the medium in that moment. What film was it? How was it processed? Is it a scan of a negative or a print? What was the colour of the lighting? Was a particular colour tint filter used on the lens? Was the subject wearing makeup optimised for black and white photography? The black and white image, standing alone, cannot tell you this, I think. Sure, it might get a bit better at, say, identifying a 1950s TV show. But what is the "correct" accurate colour representation of that scene, when televisual makeup was wildly unnatural in colour?
  - crazygringo 3y ago
    But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. And the dataset an AI is going to train on should be using original color photos that are then converted to B&W across a wide variety of color curves. So it should be fairly robust to all sorts of film types. So again, I repeat that it's probably going to wind up being more accurate with regard to skin tone than a human (with their aesthetic biases) usually would.
    - bemusedthrow75 3y ago
      > But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. No, indeed. Which is why doing it by hand is more respectful of the notion that it is subjective. Automatic colourisation is and will be viewed differently, as more "scientific", when it's still absolutely beholden to the same biases and maybe misconceptions that we can't unpick because they come from poor training data. Finally: "original colour photos" are also a problem. Not only for the part of the history where they don't exist. But also for the part of history (until the early 1960s) when the colour rendition of those photos was false or incomplete. You can get a little closer to understanding what that colour looked like, but it's important to understand that colour emulsions vary in the way they work: it's not black and white film with extra colour sensitivity. So at best you will be colourising the black and white film to look like the colour film, which is not reality. And there are well-understood problems with correct representation of skin tones with colour film until the mid-eighties. I can see your point; I just think there's a bigger picture here (pun not intended) that you're not seeing.
geon 3y ago
How can it affect the lightness channel when it is locked?
- bemusedthrow75 3y ago
  The point is that the source black and white image is not truthful about skin colour. The film locks in a level of lightness but that lightness may be very wrong (depending on the red and blue sensitivity of the film, the colour of the light, the time of day, the print, whether a filter was being sued). So if you colourise an image of someone who appears to be a light-skinned 1930s African-American with colours that appear to conform to our contemporary understanding of light-skinned Black people of our era, you might be getting it right, of course. But you might be getting it quite, quite wrong, in a way that matters.
- erwannmillon 3y ago
  So we're using a color space that has two channels dedicated entirely to color, which is the only thing the model needs to learn. The model doesn't need to touch the lightness channel at all, only predict the noised added to the color channels at train time. At inference time, we start with a real lightness channel (b/w image), and initialize the color channels to random noise. The model iteratively denoises the color channels while keeping the lightness channel locked.
dragonwriter 3y ago
> Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation. No, doing it by hand doesn't acknowledge that your interpretation is a fallible interpretation shaped by bias, just like translating a written work (e.g., the Bible, for a noted example where this has been done often without any such acknowledgement being conveyed) by human effort doesn’t do that. Acknowledging bias in translation of either kind is an entirely separate action, orthogonal to the method of the translation itself.
erwannmillon 3y ago
Fair enough. Honestly this was just a fun side project. I actually coded this up last october when I was doing a deep dive to learn about diffusion models, and saw that no one had ever applied them to colorization. This was just a fun opportunity to build a project that no one had done before
PartiallyTyped 3y ago
> Automating it is banal at best and dangerous at worst; colourised images risk distorting history. There's a lot of irony in acknowledging this but not acknowledging that each and everyone of us has their own biases inherent to our perception and experiences. Like the blue and white dress; we all perceive things differently even on identical images, monitors, screens, etc.
- bemusedthrow75 3y ago
  My post you are replying to contains the sentence "Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation".
  - PartiallyTyped 3y ago
    That's the point; you acknowledge the inherent bias in constructing, but not the bias in observing.
    - deleted 3y ago
      [deleted]
      - deleted 3y ago
        [deleted]
    - bemusedthrow75 3y ago
      So because I didn't list out my argument with completeness, my argument is defective and thus it is laden with "huge" irony. You're imagining this irony to suit your personal requirement that I am wrong or foolish. But if you look at what I am talking about elsewhere in my comments on this topic, it is human bias that concerns me. One of the things automated colourisation cannot get right is historical depictions of human skin. In a way that really matters. Human biases will creep into automatic colourisation because they can't _not_ creep in: the test data cannot fully describe the subject matter so contemporary bias will take over. One of the areas where this really matters is historical depictions of Black people. Automatically colourising a black and white photo of a Black person's skin will almost certainly get their skin tone wrong in a way that might well very significantly misrepresent their history. The same is true in mixed cultures all around the world; colourism is as much an issue as racism. People had different lives based on how their skin tone was perceived. Automatic colourisation will not (cannot!) automatically produce a colour image that fits that experience. Because dark skin can appear light (and light appear dark) depending on complexities of reproduction. This stuff matters. Hence my position: if you wish to ethically colourise an image, first consider not doing it at all. Second, consider doing it by hand based on real knowledge of the subject (their lived history etc.). I am fully cognisant of the bias issues here (well, as fully cognisant as a white amateur student of photographic history can be)

coldtea 3y ago

>Automating it is banal at best and dangerous at worst; colourised images risk distorting history Well, faces still have a certain tint, the sky is mostly blue, the grass green, water is blue, mud pools are brown, the ground too, a lot of historical fabrics are certain inherent colors, known flowers have known colors, brownstones have red/brown color. A lot of it, is just not that subjective. Besides different color film stock (or camera sensor "color science") can already result in dozens of widely different colorings of the same exactly scene.

bemusedthrow75 3y ago
> Well, faces still have a certain tint Do they? A certain tint? You cannot accurately colourise skin from photographic film without an _enormous_ amount of knowledge of the taking and processing of the film, and of the lighting and subject. An AI can't do it any better than a painter. You can't take a scan of a print or a negative and get skin tones right. Think about how weird the skin tones are from scans of wet-plate photography plates compared to the same process used in antiquity with the aim of producing a carbon print.
- coldtea 3y ago
  >Do they? A certain tint? Yes. There's just not a single one across all faces - but I wasn't meaning that. What I mean is, we know the kind of tints a face will have. A face is not suddenly going to be blue or green or poppy red. And by how light a black and white face appears, we can tell quite well if it's a darker one (oilish to brown) or lighter (pinkish towards more pale). If we get it wrong within a range it's no big deal. Color film stocks would also vary it widely. Hell, even actual people who met the person we colourise in real life will remember (or even experience in real time) their face's hue somewhat differently each.
  - bemusedthrow75 3y ago
    But how brown? How pink? How light? How dark? This is an enormously important issue. Black and white films of different technologies and manufacturers and eras actually lighten or darken skin tones. Really very significantly. And it's not going to be obvious from the final positive, unless there's _extensive_ data with those images about how the photography was done. And there never is. Editing because I can no longer reply: the question of whether a skin tone is a dark one or a light one has had severe real life impacts on people whose lives are now only represented in photographs. You can't write this off as micromanagement; it's about the ethics of representation.
    - coldtea 3y ago
      >But how brown? How pink? How light? How dark? This is an enormously important issue Is it? If 2 colour film stocks took the same image of them, it would show their hue a little (or a lot) different. Even if two different people actually met the same person, they will probably describe their face as slightly different tones from memory. (And let's not even get into different types of color-blindness they could have had). Hell, a person's hue will even look different to the same person looking at them, in real time, depending on the changes in lighting and the shade at the scene as they talk (e.g. sun behind clouds vs directly sun vs shade vs bulbs). It's not really "enormously important" to micromanage the (non-existent) exact right brown or right pink.
      - bemusedthrow75 3y ago
        I can now reply so I will say what I added in an edit: the question of whether a skin tone is a dark one or a light one has in the past had severe real life impacts on people whose lives are now only represented in photographs. You shouldn’t write this off as micromanagement; it's about the ethics of representation. It is better to leave the original image uncoloured than to colour it automatically based on some fundamentally ill-informed model. Hand colouring that image based on individual knowledge (for example that someone could or could not pass as white) is ethically better, if colourised images are needed. Important nuances of culture and history, important and complex stories of discrimination and survival, are damaged by automatic colourisation by models that have no knowledge of the source of the mono image they are colourising.

crazygringo 3y ago

> Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways. If anything, an AI trained on a large and diverse dataset is probably going to wind up being much more accurate with regards to skin color than a human colorist would be in most cases. The problem here isn't whether colorization is done by man or machine; it's just ensuring that colorized photos are identified as such. Which they usually are -- that's not a new problem to be solved.

bemusedthrow75 3y ago
No it's not, not really. A diverse data set of black and white images doesn't have any kind of knowledge of the colour sensitivity of the medium in that moment. What film was it? How was it processed? Is it a scan of a negative or a print? What was the colour of the lighting? Was a particular colour tint filter used on the lens? Was the subject wearing makeup optimised for black and white photography? The black and white image, standing alone, cannot tell you this, I think. Sure, it might get a bit better at, say, identifying a 1950s TV show. But what is the "correct" accurate colour representation of that scene, when televisual makeup was wildly unnatural in colour?
- crazygringo 3y ago
  But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. And the dataset an AI is going to train on should be using original color photos that are then converted to B&W across a wide variety of color curves. So it should be fairly robust to all sorts of film types. So again, I repeat that it's probably going to wind up being more accurate with regard to skin tone than a human (with their aesthetic biases) usually would.
  - bemusedthrow75 3y ago
    > But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. No, indeed. Which is why doing it by hand is more respectful of the notion that it is subjective. Automatic colourisation is and will be viewed differently, as more "scientific", when it's still absolutely beholden to the same biases and maybe misconceptions that we can't unpick because they come from poor training data. Finally: "original colour photos" are also a problem. Not only for the part of the history where they don't exist. But also for the part of history (until the early 1960s) when the colour rendition of those photos was false or incomplete. You can get a little closer to understanding what that colour looked like, but it's important to understand that colour emulsions vary in the way they work: it's not black and white film with extra colour sensitivity. So at best you will be colourising the black and white film to look like the colour film, which is not reality. And there are well-understood problems with correct representation of skin tones with colour film until the mid-eighties. I can see your point; I just think there's a bigger picture here (pun not intended) that you're not seeing.

geon 3y ago

How can it affect the lightness channel when it is locked?

bemusedthrow75 3y ago
The point is that the source black and white image is not truthful about skin colour. The film locks in a level of lightness but that lightness may be very wrong (depending on the red and blue sensitivity of the film, the colour of the light, the time of day, the print, whether a filter was being sued). So if you colourise an image of someone who appears to be a light-skinned 1930s African-American with colours that appear to conform to our contemporary understanding of light-skinned Black people of our era, you might be getting it right, of course. But you might be getting it quite, quite wrong, in a way that matters.
erwannmillon 3y ago
So we're using a color space that has two channels dedicated entirely to color, which is the only thing the model needs to learn. The model doesn't need to touch the lightness channel at all, only predict the noised added to the color channels at train time. At inference time, we start with a real lightness channel (b/w image), and initialize the color channels to random noise. The model iteratively denoises the color channels while keeping the lightness channel locked.

dragonwriter 3y ago

> Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation. No, doing it by hand doesn't acknowledge that your interpretation is a fallible interpretation shaped by bias, just like translating a written work (e.g., the Bible, for a noted example where this has been done often without any such acknowledgement being conveyed) by human effort doesn’t do that. Acknowledging bias in translation of either kind is an entirely separate action, orthogonal to the method of the translation itself.

erwannmillon 3y ago

Fair enough. Honestly this was just a fun side project. I actually coded this up last october when I was doing a deep dive to learn about diffusion models, and saw that no one had ever applied them to colorization. This was just a fun opportunity to build a project that no one had done before

PartiallyTyped 3y ago

> Automating it is banal at best and dangerous at worst; colourised images risk distorting history. There's a lot of irony in acknowledging this but not acknowledging that each and everyone of us has their own biases inherent to our perception and experiences. Like the blue and white dress; we all perceive things differently even on identical images, monitors, screens, etc.

bemusedthrow75 3y ago
My post you are replying to contains the sentence "Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation".
- PartiallyTyped 3y ago
  That's the point; you acknowledge the inherent bias in constructing, but not the bias in observing.
  - deleted 3y ago
    [deleted]
    - deleted 3y ago
      [deleted]
  - bemusedthrow75 3y ago
    So because I didn't list out my argument with completeness, my argument is defective and thus it is laden with "huge" irony. You're imagining this irony to suit your personal requirement that I am wrong or foolish. But if you look at what I am talking about elsewhere in my comments on this topic, it is human bias that concerns me. One of the things automated colourisation cannot get right is historical depictions of human skin. In a way that really matters. Human biases will creep into automatic colourisation because they can't _not_ creep in: the test data cannot fully describe the subject matter so contemporary bias will take over. One of the areas where this really matters is historical depictions of Black people. Automatically colourising a black and white photo of a Black person's skin will almost certainly get their skin tone wrong in a way that might well very significantly misrepresent their history. The same is true in mixed cultures all around the world; colourism is as much an issue as racism. People had different lives based on how their skin tone was perceived. Automatic colourisation will not (cannot!) automatically produce a colour image that fits that experience. Because dark skin can appear light (and light appear dark) depending on complexities of reproduction. This stuff matters. Hence my position: if you wish to ethically colourise an image, first consider not doing it at all. Second, consider doing it by hand based on real knowledge of the subject (their lived history etc.). I am fully cognisant of the bias issues here (well, as fully cognisant as a white amateur student of photographic history can be)