7 ms·

Restoring these photographs is a perfect project for machine learning. Does anyone know a grad student or researcher who would be willing to take on a restorati

Restoring these photographs is a perfect project for machine learning. Does anyone know a grad student or researcher who would be willing to take on a restoration project for these damaged photographs?

gruez 5y ago
If you're talking about "restoring" in the sense of colorizing it: Do we really need yet another grad student to work on a colorizing-with-ML project? There's dozens of such projects already. Posting colorized black and white photos was a popular trend a few years ago, but nowadays it's died down. If you're talking about "restoring" in the sense of fixing the holes: I doubt ML can do anything when a huge section of the picture is missing. Any detail you put in would be hallucinated and not representative of reality.
- Tempest1981 5y ago
  > Any detail you put in would be hallucinated and not representative of reality. Sure, but Photoshop (for example) has a feature that does just that: Content-Aware Fill. https://helpx.adobe.com/photoshop/using/content-aware-fill.html https://helpx.adobe.com/photoshop/using/content-aware-fill.h...
- nomel 5y ago
  > Any detail you put in would be hallucinated and not representative of reality. A good algorithm will absolutely represent reality, otherwise you would notice that it's not working. It may not reflect the actual reality behind the black hole, but it would remove the whole, making it indistinguishable from what you would think should be behind the hole. With claims of "the hole is part of the history" aside, it would make these photos much easier to look at, allowing one to focus on the content of the photo, rather than the hole.
  - VWWHFSfQ 5y ago
    > it would remove the whole, making it indistinguishable from what you would think should be behind the hole There is no algorithm in existence or will ever be discovered in our lifetimes that could cover those black holes with enough accuracy, or creativity, to make it indistinguishable from the real thing.
    - random314 5y ago
      The training data for such a neural network is extremely easy to generate. Combine it with a GAN, and Demis Hassabis will get it working in 5 years - if you can get him interested. https://www.whichfaceisreal.com/methods.html https://www.whichfaceisreal.com/methods.html
    - Ma8ee 5y ago
      I'd say that Lightroom already today does quite impressively, and I'm certain that there are specialised software that does even better.
    - oneeyedpigeon 5y ago
      Not even if it's supplied other images from the same location, potentially at the exact same time period?
      - gruez 5y ago
        That will get you a "best guess", not necessarily what was actually captured. It'll be the same if you told a historian + artist to draw in the blanks.
    - GaggiX 5y ago
      LaMa give you already pretty robust result. https://github.com/saic-mdal/lama https://github.com/saic-mdal/lama Certainly neural networks capable enough to perform this task in our lifetime will be developed.
  - Ansil849 5y ago
    > It may not reflect the actual reality behind the black hole And therein lies the problem. I'd rather see literally nothing, a black hole, than some AI-constructed representation of what an algorithm determines 'should' be there.
    - pvaldes 5y ago
      A trade-off solution could be to turn the hole color from black to a gray that match the average tone from the perimeter area surrounding it. The hole would be less noticeable at the first sight and would allow people to focus in the rest of the photo.
      - Ansil849 5y ago
        My impression was that that is precisely the whole point of this project: to draw attention to the gaping hole in the photos, to highlight the crass censorship which led to these photos being suppressed for years. The hole _is_ the focus of the photo, hence why the title is 'The black hole photographs', to highlight that these photos were chucked into a black hole.
        pvaldes 5y ago
        This is not censorship, is editorial choices
        Ansil849 5y ago
        Try reading the article. In this instance the editors were the censors, as they were government workers. And regardless of whether or not you want to call it censorship or some obfuscated term like "editorial choices", the point remains that the black holes draw attention to these "editorial choices", which seems to very clearly be the point of this project. Removing them, or making them unobtrusive is counter to the aims of the project.
        nomel 5y ago
  - watwut 5y ago
    > A good algorithm will absolutely represent reality, otherwise you would notice that it's not working. It may not reflect the actual reality behind the black hole If you define "represent reality" as "lie in a convincing way", then maybe. But, it has zero with reality.
  - ben_w 5y ago
    Filling the hole in a way that nobody can find where the hole was without the original is easy. But that is likely to mislead people into thinking that what filled the hole is what was really there at the time. Even small errors can radically change an image, as this (super-resolution rather than content-aware-fill) example shows: https://www.theverge.com/21298762/face-depixelizer-ai-machine-learning-tool-pulse-stylegan-obama-bias https://www.theverge.com/21298762/face-depixelizer-ai-machin...
  - npteljes 5y ago
    >you would notice I don't think this is a good metric. For fun and entertainment, it works, but lots of contexts need more than that. Take an example of lossy audio, like MP3. People are generally fine with 128k mp3s, but I doubt audio engineers would want to store masters in that format and bitrate. In case of colorization, the issue is that the color really is lost on the film. Multiple shades of different colors map to the same shade of grey on black and white media - my favorite example is that B&W TV make-up was bright green. So them no matter what you do, it'll be just a conjecture of what the original color might have been.
l0b0 5y ago
We don't know (and in a sense can't know, unless we have separate first hand records) what was behind the black spots. Cigarettes? Disfigurement? Nudity? Famous people's faces?
- ars 5y ago
  These photos are mostly duplicates and poor photos. You can often find the missing info in the published versions. Don't buy the "censored" garbage the article is peddling, it's not true.
  - the_af 5y ago
    Yes, the implication that it was censorship or a cover up is unwarranted. It's still sad that the censor felt it was necessary to completely ruin the photos in order to mark them as "not fit for purpose", why simply not use them? He didn't burn them up, so they still take up space, he just decided to permanently damage them. Some of the photos are obviously flawed or uninteresting, but others look good and even touching. Why ruin them?
    - ars 5y ago
      He was working off of negatives, so he marked the negatives he didn't want to publish. The negatives come as a long strip, so you can't just remove the bad ones. Presumably if there was an entire strip of bad ones, it would be tossed, but most of the time it was a mix, and it was a way to mark them. Not a great way, since someone else might like the photo, but I guess he saw zero value in the poor photos (basically the same as pressing delete on a digital camera - do you keep every photo you take, even bad ones?) Don't forget - he wasn't working off of "historical" photos, to him, these were present day photos, and easily replaceable. To us these are irreplaceable parts of history, but it wasn't like that to him. > It's still sad that the censor Editor, not censor - he was preparing interesting photos for publication.
      - BrandoElFollito 5y ago
        > do you keep every photo you take, even bad ones? Unfortunately yes. This is the problem with digital photos, at least on a phone: the effort to remove the bad ones is not worth it for someone like me who takes pictures occasionally and never looked at them anyway (this is why I do not take a lot either) I wonder if people who are not into photography actually manage their photos (sort, tag, prune)?
KaiserPro 5y ago
Most of them dont need machine learning. https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-FSA-black-hole-photos-9.jpg https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-... for example is a simple fix. Its something I can do now with, even with my skills. https://flashbak.com/wp-content/uploads/2016/04/killed-photograph-FSA-ohio.jpeg https://flashbak.com/wp-content/uploads/2016/04/killed-photo... That however would require someone skilled with photoshop. You'd need to find and blend in a cat's face. Its not impossible, but its hard. But then we get to: https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-FSA-black-hole-photos-20.jpg https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-... Thats not very hard to restore, but what text are you going to put in? unless you're careful its going to be misleading. which kinda leads to the greater point, when you "restore" something, colourise or up-res, you end up putting in anachronisms, something that should be there. Perhaps its better to have these photos in context, rather than "healed" with misleading data.
- netsharc 5y ago
  Won't putting in some random cat's face be worse? Meanwhile the hole in the last picture seems to be just over some part of the building, the hole takes away a part of the building's number, but a sign above it says "1233". I suppose there could've been another signage (like the one between the building number and the "Wines & Liquors" sign).
shirman 5y ago
We are doing exactly this at neural.love, I will check what we could do about it too
Yenrabbit 5y ago
https://cleanup.pictures/ https://cleanup.pictures/ (not affiliated) worked really well on a number of them - especially those where the hole is simply an area of sky or grass. For many I can't tell where the hole was, despite it being almost a one-click solution.

gruez 5y ago

If you're talking about "restoring" in the sense of colorizing it: Do we really need yet another grad student to work on a colorizing-with-ML project? There's dozens of such projects already. Posting colorized black and white photos was a popular trend a few years ago, but nowadays it's died down. If you're talking about "restoring" in the sense of fixing the holes: I doubt ML can do anything when a huge section of the picture is missing. Any detail you put in would be hallucinated and not representative of reality.

Tempest1981 5y ago
> Any detail you put in would be hallucinated and not representative of reality. Sure, but Photoshop (for example) has a feature that does just that: Content-Aware Fill. https://helpx.adobe.com/photoshop/using/content-aware-fill.html https://helpx.adobe.com/photoshop/using/content-aware-fill.h...
nomel 5y ago
> Any detail you put in would be hallucinated and not representative of reality. A good algorithm will absolutely represent reality, otherwise you would notice that it's not working. It may not reflect the actual reality behind the black hole, but it would remove the whole, making it indistinguishable from what you would think should be behind the hole. With claims of "the hole is part of the history" aside, it would make these photos much easier to look at, allowing one to focus on the content of the photo, rather than the hole.
- VWWHFSfQ 5y ago
  > it would remove the whole, making it indistinguishable from what you would think should be behind the hole There is no algorithm in existence or will ever be discovered in our lifetimes that could cover those black holes with enough accuracy, or creativity, to make it indistinguishable from the real thing.
  - random314 5y ago
    The training data for such a neural network is extremely easy to generate. Combine it with a GAN, and Demis Hassabis will get it working in 5 years - if you can get him interested. https://www.whichfaceisreal.com/methods.html https://www.whichfaceisreal.com/methods.html
  - Ma8ee 5y ago
    I'd say that Lightroom already today does quite impressively, and I'm certain that there are specialised software that does even better.
  - oneeyedpigeon 5y ago
    Not even if it's supplied other images from the same location, potentially at the exact same time period?
    - gruez 5y ago
      That will get you a "best guess", not necessarily what was actually captured. It'll be the same if you told a historian + artist to draw in the blanks.
  - GaggiX 5y ago
    LaMa give you already pretty robust result. https://github.com/saic-mdal/lama https://github.com/saic-mdal/lama Certainly neural networks capable enough to perform this task in our lifetime will be developed.
- Ansil849 5y ago
  > It may not reflect the actual reality behind the black hole And therein lies the problem. I'd rather see literally nothing, a black hole, than some AI-constructed representation of what an algorithm determines 'should' be there.
  - pvaldes 5y ago
    A trade-off solution could be to turn the hole color from black to a gray that match the average tone from the perimeter area surrounding it. The hole would be less noticeable at the first sight and would allow people to focus in the rest of the photo.
    - Ansil849 5y ago
      My impression was that that is precisely the whole point of this project: to draw attention to the gaping hole in the photos, to highlight the crass censorship which led to these photos being suppressed for years. The hole _is_ the focus of the photo, hence why the title is 'The black hole photographs', to highlight that these photos were chucked into a black hole.
      - pvaldes 5y ago
        This is not censorship, is editorial choices
        Ansil849 5y ago
        Try reading the article. In this instance the editors were the censors, as they were government workers. And regardless of whether or not you want to call it censorship or some obfuscated term like "editorial choices", the point remains that the black holes draw attention to these "editorial choices", which seems to very clearly be the point of this project. Removing them, or making them unobtrusive is counter to the aims of the project.
        nomel 5y ago
- watwut 5y ago
  > A good algorithm will absolutely represent reality, otherwise you would notice that it's not working. It may not reflect the actual reality behind the black hole If you define "represent reality" as "lie in a convincing way", then maybe. But, it has zero with reality.
- ben_w 5y ago
  Filling the hole in a way that nobody can find where the hole was without the original is easy. But that is likely to mislead people into thinking that what filled the hole is what was really there at the time. Even small errors can radically change an image, as this (super-resolution rather than content-aware-fill) example shows: https://www.theverge.com/21298762/face-depixelizer-ai-machine-learning-tool-pulse-stylegan-obama-bias https://www.theverge.com/21298762/face-depixelizer-ai-machin...
- npteljes 5y ago
  >you would notice I don't think this is a good metric. For fun and entertainment, it works, but lots of contexts need more than that. Take an example of lossy audio, like MP3. People are generally fine with 128k mp3s, but I doubt audio engineers would want to store masters in that format and bitrate. In case of colorization, the issue is that the color really is lost on the film. Multiple shades of different colors map to the same shade of grey on black and white media - my favorite example is that B&W TV make-up was bright green. So them no matter what you do, it'll be just a conjecture of what the original color might have been.

l0b0 5y ago

We don't know (and in a sense can't know, unless we have separate first hand records) what was behind the black spots. Cigarettes? Disfigurement? Nudity? Famous people's faces?

ars 5y ago
These photos are mostly duplicates and poor photos. You can often find the missing info in the published versions. Don't buy the "censored" garbage the article is peddling, it's not true.
- the_af 5y ago
  Yes, the implication that it was censorship or a cover up is unwarranted. It's still sad that the censor felt it was necessary to completely ruin the photos in order to mark them as "not fit for purpose", why simply not use them? He didn't burn them up, so they still take up space, he just decided to permanently damage them. Some of the photos are obviously flawed or uninteresting, but others look good and even touching. Why ruin them?
  - ars 5y ago
    He was working off of negatives, so he marked the negatives he didn't want to publish. The negatives come as a long strip, so you can't just remove the bad ones. Presumably if there was an entire strip of bad ones, it would be tossed, but most of the time it was a mix, and it was a way to mark them. Not a great way, since someone else might like the photo, but I guess he saw zero value in the poor photos (basically the same as pressing delete on a digital camera - do you keep every photo you take, even bad ones?) Don't forget - he wasn't working off of "historical" photos, to him, these were present day photos, and easily replaceable. To us these are irreplaceable parts of history, but it wasn't like that to him. > It's still sad that the censor Editor, not censor - he was preparing interesting photos for publication.
    - BrandoElFollito 5y ago
      > do you keep every photo you take, even bad ones? Unfortunately yes. This is the problem with digital photos, at least on a phone: the effort to remove the bad ones is not worth it for someone like me who takes pictures occasionally and never looked at them anyway (this is why I do not take a lot either) I wonder if people who are not into photography actually manage their photos (sort, tag, prune)?

KaiserPro 5y ago

Most of them dont need machine learning. https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-FSA-black-hole-photos-9.jpg https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-... for example is a simple fix. Its something I can do now with, even with my skills. https://flashbak.com/wp-content/uploads/2016/04/killed-photograph-FSA-ohio.jpeg https://flashbak.com/wp-content/uploads/2016/04/killed-photo... That however would require someone skilled with photoshop. You'd need to find and blend in a cat's face. Its not impossible, but its hard. But then we get to: https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-FSA-black-hole-photos-20.jpg https://flashbak.com/wp-content/uploads/2016/04/Roy-Stryker-... Thats not very hard to restore, but what text are you going to put in? unless you're careful its going to be misleading. which kinda leads to the greater point, when you "restore" something, colourise or up-res, you end up putting in anachronisms, something that should be there. Perhaps its better to have these photos in context, rather than "healed" with misleading data.

netsharc 5y ago
Won't putting in some random cat's face be worse? Meanwhile the hole in the last picture seems to be just over some part of the building, the hole takes away a part of the building's number, but a sign above it says "1233". I suppose there could've been another signage (like the one between the building number and the "Wines & Liquors" sign).

shirman 5y ago

We are doing exactly this at neural.love, I will check what we could do about it too

Yenrabbit 5y ago

https://cleanup.pictures/ https://cleanup.pictures/ (not affiliated) worked really well on a number of them - especially those where the hole is simply an area of sky or grass. For many I can't tell where the hole was, despite it being almost a one-click solution.