6 ms·

Colourising old photographs is the banal apotheosis application of diffusion AI. It's the pinnacle of the whole thing: "imagine it for me in a way that conform

Colourising old photographs is the banal apotheosis application of diffusion AI. It's the pinnacle of the whole thing: "imagine it for me in a way that conforms to my contemporary expectations". If you're going to colourise images, have the decency to do it by hand. If possible on a print with brushes. Edit: didn't think this would be popular. Maybe it's the historical photography nerd in me, but colourising images without effort and thought is like smashing vintage glass windows for the fun of it: cultural vandalism.

crazygringo 3y ago
If you're going to write code, have the decency to do it on punch cards. If possible by hand punching, rather than using a keypunch machine.
- bemusedthrow75 3y ago
  This isn't the point I am making. The point I am making is that colourisation is subjective art, and that alone. Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways. Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation. Automating it is banal at best and dangerous at worst; colourised images risk distorting history.
  - coldtea 3y ago
    >Automating it is banal at best and dangerous at worst; colourised images risk distorting history Well, faces still have a certain tint, the sky is mostly blue, the grass green, water is blue, mud pools are brown, the ground too, a lot of historical fabrics are certain inherent colors, known flowers have known colors, brownstones have red/brown color. A lot of it, is just not that subjective. Besides different color film stock (or camera sensor "color science") can already result in dozens of widely different colorings of the same exactly scene.
    - bemusedthrow75 3y ago
      > Well, faces still have a certain tint Do they? A certain tint? You cannot accurately colourise skin from photographic film without an _enormous_ amount of knowledge of the taking and processing of the film, and of the lighting and subject. An AI can't do it any better than a painter. You can't take a scan of a print or a negative and get skin tones right. Think about how weird the skin tones are from scans of wet-plate photography plates compared to the same process used in antiquity with the aim of producing a carbon print.
      - coldtea 3y ago
        >Do they? A certain tint? Yes. There's just not a single one across all faces - but I wasn't meaning that. What I mean is, we know the kind of tints a face will have. A face is not suddenly going to be blue or green or poppy red. And by how light a black and white face appears, we can tell quite well if it's a darker one (oilish to brown) or lighter (pinkish towards more pale). If we get it wrong within a range it's no big deal. Color film stocks would also vary it widely. Hell, even actual people who met the person we colourise in real life will remember (or even experience in real time) their face's hue somewhat differently each.
        bemusedthrow75 3y ago
        But how brown? How pink? How light? How dark? This is an enormously important issue. Black and white films of different technologies and manufacturers and eras actually lighten or darken skin tones. Really very significantly. And it's not going to be obvious from the final positive, unless there's _extensive_ data with those images about how the photography was done. And there never is. Editing because I can no longer reply: the question of whether a skin tone is a dark one or a light one has had severe real life impacts on people whose lives are now only represented in photographs. You can't write this off as micromanagement; it's about the ethics of representation.
        coldtea 3y ago
        >But how brown? How pink? How light? How dark? This is an enormously important issue Is it? If 2 colour film stocks took the same image of them, it would show their hue a little (or a lot) different. Even if two different people actually met the same person, they will probably describe their face as slightly different tones from memory. (And let's not even get into different types of color-blindness they could have had). Hell, a person's hue will even look different to the same person looking at them, in real time, depending on the changes in lighting and the shade at the scene as they talk (e.g. sun behind clouds vs directly sun vs shade vs bulbs). It's not really "enormously important" to micromanage the (non-existent) exact right brown or right pink.
  - crazygringo 3y ago
    > Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways. If anything, an AI trained on a large and diverse dataset is probably going to wind up being much more accurate with regards to skin color than a human colorist would be in most cases. The problem here isn't whether colorization is done by man or machine; it's just ensuring that colorized photos are identified as such. Which they usually are -- that's not a new problem to be solved.
    - bemusedthrow75 3y ago
      No it's not, not really. A diverse data set of black and white images doesn't have any kind of knowledge of the colour sensitivity of the medium in that moment. What film was it? How was it processed? Is it a scan of a negative or a print? What was the colour of the lighting? Was a particular colour tint filter used on the lens? Was the subject wearing makeup optimised for black and white photography? The black and white image, standing alone, cannot tell you this, I think. Sure, it might get a bit better at, say, identifying a 1950s TV show. But what is the "correct" accurate colour representation of that scene, when televisual makeup was wildly unnatural in colour?
      - crazygringo 3y ago
        But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. And the dataset an AI is going to train on should be using original color photos that are then converted to B&W across a wide variety of color curves. So it should be fairly robust to all sorts of film types. So again, I repeat that it's probably going to wind up being more accurate with regard to skin tone than a human (with their aesthetic biases) usually would.
        bemusedthrow75 3y ago
        > But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. No, indeed. Which is why doing it by hand is more respectful of the notion that it is subjective. Automatic colourisation is and will be viewed differently, as more "scientific", when it's still absolutely beholden to the same biases and maybe misconceptions that we can't unpick because they come from poor training data. Finally: "original colour photos" are also a problem. Not only for the part of the history where they don't exist. But also for the part of history (until the early 1960s) when the colour rendition of those photos was false or incomplete. You can get a little closer to understanding what that colour looked like, but it's important to understand that colour emulsions vary in the way they work: it's not black and white film with extra colour sensitivity. So at best you will be colourising the black and white film to look like the colour film, which is not reality. And there are well-understood problems with correct representation of skin tones with colour film until the mid-eighties. I can see your point; I just think there's a bigger picture here (pun not intended) that you're not seeing.
  - geon 3y ago
    How can it affect the lightness channel when it is locked?
    - bemusedthrow75 3y ago
      The point is that the source black and white image is not truthful about skin colour. The film locks in a level of lightness but that lightness may be very wrong (depending on the red and blue sensitivity of the film, the colour of the light, the time of day, the print, whether a filter was being sued). So if you colourise an image of someone who appears to be a light-skinned 1930s African-American with colours that appear to conform to our contemporary understanding of light-skinned Black people of our era, you might be getting it right, of course. But you might be getting it quite, quite wrong, in a way that matters.
    - erwannmillon 3y ago
      So we're using a color space that has two channels dedicated entirely to color, which is the only thing the model needs to learn. The model doesn't need to touch the lightness channel at all, only predict the noised added to the color channels at train time. At inference time, we start with a real lightness channel (b/w image), and initialize the color channels to random noise. The model iteratively denoises the color channels while keeping the lightness channel locked.
  - dragonwriter 3y ago
    > Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation. No, doing it by hand doesn't acknowledge that your interpretation is a fallible interpretation shaped by bias, just like translating a written work (e.g., the Bible, for a noted example where this has been done often without any such acknowledgement being conveyed) by human effort doesn’t do that. Acknowledging bias in translation of either kind is an entirely separate action, orthogonal to the method of the translation itself.
  - erwannmillon 3y ago
    Fair enough. Honestly this was just a fun side project. I actually coded this up last october when I was doing a deep dive to learn about diffusion models, and saw that no one had ever applied them to colorization. This was just a fun opportunity to build a project that no one had done before
  - PartiallyTyped 3y ago
    > Automating it is banal at best and dangerous at worst; colourised images risk distorting history. There's a lot of irony in acknowledging this but not acknowledging that each and everyone of us has their own biases inherent to our perception and experiences. Like the blue and white dress; we all perceive things differently even on identical images, monitors, screens, etc.
    - bemusedthrow75 3y ago
      My post you are replying to contains the sentence "Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation".
      - PartiallyTyped 3y ago
        That's the point; you acknowledge the inherent bias in constructing, but not the bias in observing.
        deleted 3y ago
        [deleted]
        deleted 3y ago
        [deleted]
        bemusedthrow75 3y ago
        So because I didn't list out my argument with completeness, my argument is defective and thus it is laden with "huge" irony. You're imagining this irony to suit your personal requirement that I am wrong or foolish. But if you look at what I am talking about elsewhere in my comments on this topic, it is human bias that concerns me. One of the things automated colourisation cannot get right is historical depictions of human skin. In a way that really matters. Human biases will creep into automatic colourisation because they can't _not_ creep in: the test data cannot fully describe the subject matter so contemporary bias will take over. One of the areas where this really matters is historical depictions of Black people. Automatically colourising a black and white photo of a Black person's skin will almost certainly get their skin tone wrong in a way that might well very significantly misrepresent their history. The same is true in mixed cultures all around the world; colourism is as much an issue as racism. People had different lives based on how their skin tone was perceived. Automatic colourisation will not (cannot!) automatically produce a colour image that fits that experience. Because dark skin can appear light (and light appear dark) depending on complexities of reproduction. This stuff matters. Hence my position: if you wish to ethically colourise an image, first consider not doing it at all. Second, consider doing it by hand based on real knowledge of the subject (their lived history etc.). I am fully cognisant of the bias issues here (well, as fully cognisant as a white amateur student of photographic history can be)
erwannmillon 3y ago
touché, nevertheless, colors go brrrrrrrr
- bemusedthrow75 3y ago
  Don't get me wrong. It's impressive technology. I'm amazed at what it can do. Also horrified.
coldtea 3y ago
When did colorizing images become an "art"? What if the "effort" way is less accurate?
- vorpalhex 3y ago
  There is a community of people who carefully recolor historical photos by hand. It's really beautiful time consuming work and often they invest heavily to get the colors to be correct.
- bemusedthrow75 3y ago
  The effort is obviously going to be less accurate. But it reflects the fact that an accurate colourisation of a black and white image without access to every possible detail about the scene and processing from the photographer's perspective is impossible. Black and white film is substantially more complex and varied than people understand. Its sensitivities are complex and vary from processing run to processing run, and people at the time knew of the weaknesses of black and white and often used false colour to get an acceptable rendition. Colourisation is a form of expression, not a form of recovery.
  - coldtea 3y ago
    >But it reflects the fact that an accurate colourisation of a black and white image without access to every possible detail about the scene and processing from the photographer's perspective is impossible. Accurate colourisation is impossible even in a color photograph. There is no "canonical" film stock that accurately represents all actual real-life colors. The expectation from colourisation is not an accurate representation of the original colors, but a good application of color based on our knowledge (whether from historical facts a human colorist knows or from training with similar objects and materials a NN did) that matches a realistic representation of the scene. If a human colourist draws a dress and doesn't know the color of it, nor have they any historical information about what the person depicted wore that day, they're going to take a guess. That's kind of what the NN will do as well.
- bemusedthrow75 3y ago
  Side note: colourising images became an art in about 1860. Hasn’t ever gone away.
vorpalhex 3y ago
I think a lot of it depends on what you are doing and why. Yes, recolors can be inaccurate but they can make historical moments feel more alive and connected. At the same time one can imagine the issues of a recolor that is inaccurate and that is troubling with historical photographs. At the same time I have a bunch of old family photos I'd love to recolorize. Maybe the colors won't be quite right but that's an OK failure mode for family photos! I'd love to see a version where you can drop just a spot or two of the correct color and let the AI fill it out. My grandmother had stark red hair but most algorithms will color her as a blond. It'd be nice to fix that, using one of the color photos we do have.
- erwannmillon 3y ago
  You can do this with spatial palette t2i or controlnet. Give a super lores spatial palette as conditioning like this: https://camo.githubusercontent.com/8e488996fd309165fb065b0cdef17e55d68b1a2e9ea9ecfaee91ebb7ee38ca58/68747470733a2f2f68756767696e67666163652e636f2f54656e63656e744152432f5432492d416461707465722f7265736f6c76652f6d61696e2f6173736574732f636f6c6f722e706e67 https://camo.githubusercontent.com/8e488996fd309165fb065b0cd... https://github.com/TencentARC/T2I-Adapter https://github.com/TencentARC/T2I-Adapter
geon 3y ago
How was anything destroyed? the original grayscale is still there.
- bemusedthrow75 3y ago
  Colourised images absolutely replace mono images in image searches, unfortunately; I've seen this again and again. It gets more difficult to find originals. But also you have to consider that bias is being introduced in the colour rendition. That causes damage. For example, you could see a photograph of an African American woman in the 20s or 30s, and your AI would say, this is an African American woman and colour her skin in some way. But a lighter-skinned-looking African American woman in a pre/early-post-war photo is a challenge. She may have had darker skin -- been unable to "pass" -- and the film simply didn't get that across because of its colour sensitivity. Or she may actually have been light-skinned and able to "pass" (or wearing makeup that helped). Automatically colouring that image introduces risks to the reading of history; you can read that woman's entire life completely wrong. It's also common with photos of men from that era who worked outdoors. Many of them will come across much darker-skinned in photos than they actually would have appeared in real life, because not-readily-visible sun damage can look odd in mono. But if you colourise all those sun-baked people the same way, what happens to those of mixed heritage among them? (A thing that is already rather "airbrushed out" of history.) Without knowing about the lighting, the material, the processing and the source of the positive (is it a negative scan? was it a good one? or is it a scan of a print?) you cannot make accurate impressions of skin tone. And given the power and importance of photography in the history of the USA in particular -- photography coincides with and actually helps define the modern unified US self-image -- this is not something to blaze through without care. This is a far less tricky problem in more homogeneous societies, obviously. But even then, there is this perception from photographs that British women in the 1920s were all deathly pale; colourisation preserves that illusion that actually comes in part from photographic style.
mrkeen 3y ago
Nice, I'll have to try smashing vintage glass windows. Thanks for the tip!
pkoiralap 3y ago
Making music without actually knowing anything about it is the banal apotheosis application of Generative AI. - Music nerd in me Creating art without actually knowing anything about it is the banal apotheosis application of Diffusion AI. - Artist in me Using ChatGPT to write essays that are better than anyone could have ever written is the banal apotheosis application of LLMs - Teacher in me It is already here. Better use, appreciate, and try to understand how it works rather than complaining about it doing a better job. In this instance, for example, the model can be made to generate multiple outputs or even better, generate output based on precise user input.
- bemusedthrow75 3y ago
  I'm actually concerned it is doing a worse job, in important ethical ways, than a hand colourist. But I've explained elsewhere. Colourisation cannot be done accurately from a black and white image without context that is almost always lacking. Hand colouring is less dishonest.
- erwannmillon 3y ago
  I work at krea.ai. We are for sure making art extremely accessible, but we consider it enhancing creativity rather than replacing. I fully agree that being able to generate an aesthetically pleasing image with an AI that has been optimized to do exactly that is a banal application of creativity. I do think that AI has incredible potential to make (and become art). The best AI artists don't just throw art into midjourney, they experiment, create their own secret sauce. Training models has become an art form in and of itself: ai artists curate incredible datasets and devise recipes for training stunning models. Their workflows span multiple companies / tools / models. AI just means that the goalposts for creativity are shifting. Boring people will use AI to make boring art, artists will find completely unexpected ways to use the tools we build to create art forms we've never imagined before.
dragonwriter 3y ago
> But since you do not have access to colour originals of historical photos in almost every instance, you cannot possibly train the network to have any instinct for the colour sensitivity of the medium, can you? Plenty of people say that about colorization period, which, while I disagree, seems more sensible than your position to me, which just seems to be fetishizing suffering.

crazygringo 3y ago

If you're going to write code, have the decency to do it on punch cards. If possible by hand punching, rather than using a keypunch machine.

bemusedthrow75 3y ago
This isn't the point I am making. The point I am making is that colourisation is subjective art, and that alone. Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways. Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation. Automating it is banal at best and dangerous at worst; colourised images risk distorting history.
- coldtea 3y ago
  >Automating it is banal at best and dangerous at worst; colourised images risk distorting history Well, faces still have a certain tint, the sky is mostly blue, the grass green, water is blue, mud pools are brown, the ground too, a lot of historical fabrics are certain inherent colors, known flowers have known colors, brownstones have red/brown color. A lot of it, is just not that subjective. Besides different color film stock (or camera sensor "color science") can already result in dozens of widely different colorings of the same exactly scene.
  - bemusedthrow75 3y ago
    > Well, faces still have a certain tint Do they? A certain tint? You cannot accurately colourise skin from photographic film without an _enormous_ amount of knowledge of the taking and processing of the film, and of the lighting and subject. An AI can't do it any better than a painter. You can't take a scan of a print or a negative and get skin tones right. Think about how weird the skin tones are from scans of wet-plate photography plates compared to the same process used in antiquity with the aim of producing a carbon print.
    - coldtea 3y ago
      >Do they? A certain tint? Yes. There's just not a single one across all faces - but I wasn't meaning that. What I mean is, we know the kind of tints a face will have. A face is not suddenly going to be blue or green or poppy red. And by how light a black and white face appears, we can tell quite well if it's a darker one (oilish to brown) or lighter (pinkish towards more pale). If we get it wrong within a range it's no big deal. Color film stocks would also vary it widely. Hell, even actual people who met the person we colourise in real life will remember (or even experience in real time) their face's hue somewhat differently each.
      - bemusedthrow75 3y ago
        But how brown? How pink? How light? How dark? This is an enormously important issue. Black and white films of different technologies and manufacturers and eras actually lighten or darken skin tones. Really very significantly. And it's not going to be obvious from the final positive, unless there's _extensive_ data with those images about how the photography was done. And there never is. Editing because I can no longer reply: the question of whether a skin tone is a dark one or a light one has had severe real life impacts on people whose lives are now only represented in photographs. You can't write this off as micromanagement; it's about the ethics of representation.
        coldtea 3y ago
        >But how brown? How pink? How light? How dark? This is an enormously important issue Is it? If 2 colour film stocks took the same image of them, it would show their hue a little (or a lot) different. Even if two different people actually met the same person, they will probably describe their face as slightly different tones from memory. (And let's not even get into different types of color-blindness they could have had). Hell, a person's hue will even look different to the same person looking at them, in real time, depending on the changes in lighting and the shade at the scene as they talk (e.g. sun behind clouds vs directly sun vs shade vs bulbs). It's not really "enormously important" to micromanage the (non-existent) exact right brown or right pink.
- crazygringo 3y ago
  > Colourisation cannot fail to enforce contemporary biases based on poor understanding of the materials. It will darken or lighten skin inappropriately, and mislead in any number of ways. If anything, an AI trained on a large and diverse dataset is probably going to wind up being much more accurate with regards to skin color than a human colorist would be in most cases. The problem here isn't whether colorization is done by man or machine; it's just ensuring that colorized photos are identified as such. Which they usually are -- that's not a new problem to be solved.
  - bemusedthrow75 3y ago
    No it's not, not really. A diverse data set of black and white images doesn't have any kind of knowledge of the colour sensitivity of the medium in that moment. What film was it? How was it processed? Is it a scan of a negative or a print? What was the colour of the lighting? Was a particular colour tint filter used on the lens? Was the subject wearing makeup optimised for black and white photography? The black and white image, standing alone, cannot tell you this, I think. Sure, it might get a bit better at, say, identifying a 1950s TV show. But what is the "correct" accurate colour representation of that scene, when televisual makeup was wildly unnatural in colour?
    - crazygringo 3y ago
      But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. And the dataset an AI is going to train on should be using original color photos that are then converted to B&W across a wide variety of color curves. So it should be fairly robust to all sorts of film types. So again, I repeat that it's probably going to wind up being more accurate with regard to skin tone than a human (with their aesthetic biases) usually would.
      - bemusedthrow75 3y ago
        > But do people have any of that knowledge either? Most of the time, I don't think so -- they colorize stuff in a way that just "looks right" or "looks natural" or "looks nice" to their eye, that's all. No, indeed. Which is why doing it by hand is more respectful of the notion that it is subjective. Automatic colourisation is and will be viewed differently, as more "scientific", when it's still absolutely beholden to the same biases and maybe misconceptions that we can't unpick because they come from poor training data. Finally: "original colour photos" are also a problem. Not only for the part of the history where they don't exist. But also for the part of history (until the early 1960s) when the colour rendition of those photos was false or incomplete. You can get a little closer to understanding what that colour looked like, but it's important to understand that colour emulsions vary in the way they work: it's not black and white film with extra colour sensitivity. So at best you will be colourising the black and white film to look like the colour film, which is not reality. And there are well-understood problems with correct representation of skin tones with colour film until the mid-eighties. I can see your point; I just think there's a bigger picture here (pun not intended) that you're not seeing.
- geon 3y ago
  How can it affect the lightness channel when it is locked?
  - bemusedthrow75 3y ago
    The point is that the source black and white image is not truthful about skin colour. The film locks in a level of lightness but that lightness may be very wrong (depending on the red and blue sensitivity of the film, the colour of the light, the time of day, the print, whether a filter was being sued). So if you colourise an image of someone who appears to be a light-skinned 1930s African-American with colours that appear to conform to our contemporary understanding of light-skinned Black people of our era, you might be getting it right, of course. But you might be getting it quite, quite wrong, in a way that matters.
  - erwannmillon 3y ago
    So we're using a color space that has two channels dedicated entirely to color, which is the only thing the model needs to learn. The model doesn't need to touch the lightness channel at all, only predict the noised added to the color channels at train time. At inference time, we start with a real lightness channel (b/w image), and initialize the color channels to random noise. The model iteratively denoises the color channels while keeping the lightness channel locked.
- dragonwriter 3y ago
  > Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation. No, doing it by hand doesn't acknowledge that your interpretation is a fallible interpretation shaped by bias, just like translating a written work (e.g., the Bible, for a noted example where this has been done often without any such acknowledgement being conveyed) by human effort doesn’t do that. Acknowledging bias in translation of either kind is an entirely separate action, orthogonal to the method of the translation itself.
- erwannmillon 3y ago
  Fair enough. Honestly this was just a fun side project. I actually coded this up last october when I was doing a deep dive to learn about diffusion models, and saw that no one had ever applied them to colorization. This was just a fun opportunity to build a project that no one had done before
- PartiallyTyped 3y ago
  > Automating it is banal at best and dangerous at worst; colourised images risk distorting history. There's a lot of irony in acknowledging this but not acknowledging that each and everyone of us has their own biases inherent to our perception and experiences. Like the blue and white dress; we all perceive things differently even on identical images, monitors, screens, etc.
  - bemusedthrow75 3y ago
    My post you are replying to contains the sentence "Doing it by hand (in photoshop or on a print) acknowledges the inherent bias that is involved in colourisation".
    - PartiallyTyped 3y ago
      That's the point; you acknowledge the inherent bias in constructing, but not the bias in observing.
      - deleted 3y ago
        [deleted]
        deleted 3y ago
        [deleted]
      - bemusedthrow75 3y ago
        So because I didn't list out my argument with completeness, my argument is defective and thus it is laden with "huge" irony. You're imagining this irony to suit your personal requirement that I am wrong or foolish. But if you look at what I am talking about elsewhere in my comments on this topic, it is human bias that concerns me. One of the things automated colourisation cannot get right is historical depictions of human skin. In a way that really matters. Human biases will creep into automatic colourisation because they can't _not_ creep in: the test data cannot fully describe the subject matter so contemporary bias will take over. One of the areas where this really matters is historical depictions of Black people. Automatically colourising a black and white photo of a Black person's skin will almost certainly get their skin tone wrong in a way that might well very significantly misrepresent their history. The same is true in mixed cultures all around the world; colourism is as much an issue as racism. People had different lives based on how their skin tone was perceived. Automatic colourisation will not (cannot!) automatically produce a colour image that fits that experience. Because dark skin can appear light (and light appear dark) depending on complexities of reproduction. This stuff matters. Hence my position: if you wish to ethically colourise an image, first consider not doing it at all. Second, consider doing it by hand based on real knowledge of the subject (their lived history etc.). I am fully cognisant of the bias issues here (well, as fully cognisant as a white amateur student of photographic history can be)

erwannmillon 3y ago

touché, nevertheless, colors go brrrrrrrr

bemusedthrow75 3y ago
Don't get me wrong. It's impressive technology. I'm amazed at what it can do. Also horrified.

coldtea 3y ago

When did colorizing images become an "art"? What if the "effort" way is less accurate?

vorpalhex 3y ago
There is a community of people who carefully recolor historical photos by hand. It's really beautiful time consuming work and often they invest heavily to get the colors to be correct.
bemusedthrow75 3y ago
The effort is obviously going to be less accurate. But it reflects the fact that an accurate colourisation of a black and white image without access to every possible detail about the scene and processing from the photographer's perspective is impossible. Black and white film is substantially more complex and varied than people understand. Its sensitivities are complex and vary from processing run to processing run, and people at the time knew of the weaknesses of black and white and often used false colour to get an acceptable rendition. Colourisation is a form of expression, not a form of recovery.
- coldtea 3y ago
  >But it reflects the fact that an accurate colourisation of a black and white image without access to every possible detail about the scene and processing from the photographer's perspective is impossible. Accurate colourisation is impossible even in a color photograph. There is no "canonical" film stock that accurately represents all actual real-life colors. The expectation from colourisation is not an accurate representation of the original colors, but a good application of color based on our knowledge (whether from historical facts a human colorist knows or from training with similar objects and materials a NN did) that matches a realistic representation of the scene. If a human colourist draws a dress and doesn't know the color of it, nor have they any historical information about what the person depicted wore that day, they're going to take a guess. That's kind of what the NN will do as well.
bemusedthrow75 3y ago
Side note: colourising images became an art in about 1860. Hasn’t ever gone away.

vorpalhex 3y ago

I think a lot of it depends on what you are doing and why. Yes, recolors can be inaccurate but they can make historical moments feel more alive and connected. At the same time one can imagine the issues of a recolor that is inaccurate and that is troubling with historical photographs. At the same time I have a bunch of old family photos I'd love to recolorize. Maybe the colors won't be quite right but that's an OK failure mode for family photos! I'd love to see a version where you can drop just a spot or two of the correct color and let the AI fill it out. My grandmother had stark red hair but most algorithms will color her as a blond. It'd be nice to fix that, using one of the color photos we do have.

erwannmillon 3y ago
You can do this with spatial palette t2i or controlnet. Give a super lores spatial palette as conditioning like this: https://camo.githubusercontent.com/8e488996fd309165fb065b0cdef17e55d68b1a2e9ea9ecfaee91ebb7ee38ca58/68747470733a2f2f68756767696e67666163652e636f2f54656e63656e744152432f5432492d416461707465722f7265736f6c76652f6d61696e2f6173736574732f636f6c6f722e706e67 https://camo.githubusercontent.com/8e488996fd309165fb065b0cd... https://github.com/TencentARC/T2I-Adapter https://github.com/TencentARC/T2I-Adapter

geon 3y ago

How was anything destroyed? the original grayscale is still there.

bemusedthrow75 3y ago
Colourised images absolutely replace mono images in image searches, unfortunately; I've seen this again and again. It gets more difficult to find originals. But also you have to consider that bias is being introduced in the colour rendition. That causes damage. For example, you could see a photograph of an African American woman in the 20s or 30s, and your AI would say, this is an African American woman and colour her skin in some way. But a lighter-skinned-looking African American woman in a pre/early-post-war photo is a challenge. She may have had darker skin -- been unable to "pass" -- and the film simply didn't get that across because of its colour sensitivity. Or she may actually have been light-skinned and able to "pass" (or wearing makeup that helped). Automatically colouring that image introduces risks to the reading of history; you can read that woman's entire life completely wrong. It's also common with photos of men from that era who worked outdoors. Many of them will come across much darker-skinned in photos than they actually would have appeared in real life, because not-readily-visible sun damage can look odd in mono. But if you colourise all those sun-baked people the same way, what happens to those of mixed heritage among them? (A thing that is already rather "airbrushed out" of history.) Without knowing about the lighting, the material, the processing and the source of the positive (is it a negative scan? was it a good one? or is it a scan of a print?) you cannot make accurate impressions of skin tone. And given the power and importance of photography in the history of the USA in particular -- photography coincides with and actually helps define the modern unified US self-image -- this is not something to blaze through without care. This is a far less tricky problem in more homogeneous societies, obviously. But even then, there is this perception from photographs that British women in the 1920s were all deathly pale; colourisation preserves that illusion that actually comes in part from photographic style.

mrkeen 3y ago

Nice, I'll have to try smashing vintage glass windows. Thanks for the tip!

pkoiralap 3y ago

Making music without actually knowing anything about it is the banal apotheosis application of Generative AI. - Music nerd in me Creating art without actually knowing anything about it is the banal apotheosis application of Diffusion AI. - Artist in me Using ChatGPT to write essays that are better than anyone could have ever written is the banal apotheosis application of LLMs - Teacher in me It is already here. Better use, appreciate, and try to understand how it works rather than complaining about it doing a better job. In this instance, for example, the model can be made to generate multiple outputs or even better, generate output based on precise user input.

bemusedthrow75 3y ago
I'm actually concerned it is doing a worse job, in important ethical ways, than a hand colourist. But I've explained elsewhere. Colourisation cannot be done accurately from a black and white image without context that is almost always lacking. Hand colouring is less dishonest.
erwannmillon 3y ago
I work at krea.ai. We are for sure making art extremely accessible, but we consider it enhancing creativity rather than replacing. I fully agree that being able to generate an aesthetically pleasing image with an AI that has been optimized to do exactly that is a banal application of creativity. I do think that AI has incredible potential to make (and become art). The best AI artists don't just throw art into midjourney, they experiment, create their own secret sauce. Training models has become an art form in and of itself: ai artists curate incredible datasets and devise recipes for training stunning models. Their workflows span multiple companies / tools / models. AI just means that the goalposts for creativity are shifting. Boring people will use AI to make boring art, artists will find completely unexpected ways to use the tools we build to create art forms we've never imagined before.

dragonwriter 3y ago

> But since you do not have access to colour originals of historical photos in almost every instance, you cannot possibly train the network to have any instinct for the colour sensitivity of the medium, can you? Plenty of people say that about colorization period, which, while I disagree, seems more sensible than your position to me, which just seems to be fetishizing suffering.