5 ms·

It's still got all the noise and shadows frame-to-frame. It wouldn't be historically accurate, but it strikes me that an ML algo could probably smooth the noise

by ip26 3y ago

It's still got all the noise and shadows frame-to-frame. It wouldn't be historically accurate, but it strikes me that an ML algo could probably smooth the noise and shadows, colorize, and even add frames into a more immersive video to bring Twain even closer to the viewer. There's always the ethical and artistic arguments around colorizing and so forth, but if one of the values of old video reels is to connect our conscience to the past, it's my opinion that smoothing, colorizing, de-noising and so forth can be benefits in that goal, even if the frames are no longer "authentic". (It's not like the real Twain was choppy and black-and-white)

JasonFruit 3y ago
But if that ML tool is trained on modern video, depending on its capabilities it may subtly modernize aspects of the video in ways that lessen the "past is a foreign country" effect, leading us to believe that the past was more like the present than it in truth was. It's a trade-off between removing technological barriers between us and the person, so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve.
- WalterBright 3y ago
  > leading us to believe that the past was more like the present than it in truth was So what? I just don't get this. Can't we just enjoy a much improved film experience? > so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve. Nobody is suggesting destroying the originals.
  - kortilla 3y ago
    But what’s the point then? If you’re going to make fake videos might as well just generate a completely new one with actors.
    - WalterBright 3y ago
      Would you rather see a colorized picture of your grandma or a color picture of another woman?
      - kortilla 3y ago
        If the colors are just someone guessing I would honestly not want to see it. I find those kind of wishful fantasies harmful. Not much different than when favorite book is converted to a movie and actors stomp out your imagined characters. If you hadn’t had a picture of your grandma at all, would you want AI to generate one based on a description of her and tell you it’s a picture of your grandma?
        WalterBright 3y ago
        There is a short movie on Google which is a film of a train rolling into a station, and then the same one sharpened by AI. The second one is marvelous to watch. There's another of a colorized ancient movie of a trolley ride in Europe. It's very fun to watch. Ya know, when people restore old cars, if they want to drive them, they'll usually do a few upgrades to make the car safer and more driveable. And there's nothing wrong with that. I'm curious. Do you ever watch and enjoy movies from the 20's and 30's? For the silent flicks, do you think that the music played during the screening by some random starving artist plinking on a crummy piano or squeaking away on a violin is part of the "vision" of the director? Lots of people today simply turn away from any movie that is black and white.
        bredren 3y ago
        I’d like to see rolling data on how people are turning away from lower quality media. Or what premium is being placed on 720p+ video versus standard. There’s a reason video sites offer a filter for HD and YouTube videos get keyword stuffed for 8k even when they aren’t. I am very curious how Apple’s spatial video lands. From reviewer descriptions, the experience is such a major change from standard HD video I am wondering if depthscaling “old” video will become a thing too.
  - kouru225 3y ago
    What they’re saying is that it’s your opinion that that would be an improved experience, and that’s not an objective opinion that other people share.
  - iancmceachern 3y ago
    Because it's not real, red pill man
- hnlmorg 3y ago
  The past was more like the present than we like to believe. The past was no more sepia than it was black and white. Just like the 80s wasn’t as bold and neon as pop culture portrays. In media we use colour to exaggerate the differences of the past, to make it feel different and foreign to our senses, rather than represent them accurately. So if you want a more accurate representation then you absolutely should be training your models on modern video.
hutzlibu 3y ago
Here is a colorized version .. but with all the noise: https://m.youtube.com/watch?v=EIeNnDee-Ok https://m.youtube.com/watch?v=EIeNnDee-Ok
WalterBright 3y ago
I'd like to see ML colorize, sharpen, etc., older B+W movies. Silent movies would be greatly improved with the addition of a soundtrack (music[2], foley effects[1], and replacing the dialog cards with voice actors). 1. Yes, they already do add some foley effects. But very little. 2. Yes, they do add a music soundtrack. But it's almost always awful period music. > the ethical and artistic arguments Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music. There's a reddit newsgroup where people colorize old B+W photos. They really do an amazing job. By colorizing key frames in a movie, ML can take care of the rest. BTW, I've seen many bad attempts at colorizing old footage, like those "WW2 In Color" documentaries. You can always tell because the teeth & gums of the people are grey :-/ and the faces are a uniform brown. ML ought to be able to do a lot better.
- shiroiuma 3y ago
  >> the ethical and artistic arguments >Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music. I think there's 2 separate issues actually: there's the "artist's intent", and then there's refusal to utilize any modern technology. The latter I agree is baloney: if you can use modern technology to clean up and improve old stuff (such as taking an original Star Wars film print and removing the scratches in the film, improving flaws in the FX, etc.), I think it's ridiculous to argue against that kind of thing. Then there's the artist's intent: if the original artist intended that the movie have "bad piano music" as you call it, should you be changing that? It's like changing the dialog in Lord of the Rings so the characters all talk like modern American hipsters: how is this helpful? These movies are a product of their time, and the music is too. If you could bring the original artists back from the dead, and ask them if they'd like to use modern technology to clean up their old silent movies to sharpen them, remove the noise, etc., I'm sure they'd agree: why wouldn't they want the print to look as good as possible? If you ask them about colorizing, there might be some debate; some might like it, others might not. It's not like they had a choice back then, but even today some people like B&W photos for artistic reasons. If you ask them about adding modern music, I don't think they'd like that much at all.
dclowd9901 3y ago
It’s not destroying the old footage to make it more viewable, so long as it’s preserved in its originality in some way.
proc0 3y ago
AI could do a 1080p with sharp details. Someone just needs to process the frames, and use something like DALL-E or stable diffusion with the right settings.
smcleod 3y ago
I'll do this and let you know how it goes, I don't think I can easily add colour but I can work some other magic. ETA is likely 2-3 hours + whatever it takes to upload (assuming I get back to it tonight - otherwise I’ll upload tomorrow morning).
- sgt 3y ago
  Looking forward to this. You can probably submit it to HN separately and it'll get heavily upvoted!
  - smcleod 3y ago
    I haven’t tried enhancing anything this old and crusty before so don’t hold your breath!
    - sgt 3y ago
      Done yet?
      - smcleod 3y ago
        Yeah I shared yesterday - https://news.ycombinator.com/item?id=37898748 https://news.ycombinator.com/item?id=37898748 It’s quite hard with such low quality footage. I think could be done is use some sort of algorithm to stop that sort of wavy chemical look that might be beyond me anyway that video did clean up a bit!
        sgt 3y ago
        Very cool, it definitely gives an increased feeling of "being there".
- smcleod 3y ago
  First attempt has processed at 1080p 60fps, uploaded to YouTube - it will take some time for YouTube to finish processing it in 1080p - https://youtu.be/hlhn9HSH4OI https://youtu.be/hlhn9HSH4OI I think I can do better though, so will try another method in the morning.

by ip26 3y ago

JasonFruit 3y ago

But if that ML tool is trained on modern video, depending on its capabilities it may subtly modernize aspects of the video in ways that lessen the "past is a foreign country" effect, leading us to believe that the past was more like the present than it in truth was. It's a trade-off between removing technological barriers between us and the person, so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve.

WalterBright 3y ago
> leading us to believe that the past was more like the present than it in truth was So what? I just don't get this. Can't we just enjoy a much improved film experience? > so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve. Nobody is suggesting destroying the originals.
- kortilla 3y ago
  But what’s the point then? If you’re going to make fake videos might as well just generate a completely new one with actors.
  - WalterBright 3y ago
    Would you rather see a colorized picture of your grandma or a color picture of another woman?
    - kortilla 3y ago
      If the colors are just someone guessing I would honestly not want to see it. I find those kind of wishful fantasies harmful. Not much different than when favorite book is converted to a movie and actors stomp out your imagined characters. If you hadn’t had a picture of your grandma at all, would you want AI to generate one based on a description of her and tell you it’s a picture of your grandma?
      - WalterBright 3y ago
        There is a short movie on Google which is a film of a train rolling into a station, and then the same one sharpened by AI. The second one is marvelous to watch. There's another of a colorized ancient movie of a trolley ride in Europe. It's very fun to watch. Ya know, when people restore old cars, if they want to drive them, they'll usually do a few upgrades to make the car safer and more driveable. And there's nothing wrong with that. I'm curious. Do you ever watch and enjoy movies from the 20's and 30's? For the silent flicks, do you think that the music played during the screening by some random starving artist plinking on a crummy piano or squeaking away on a violin is part of the "vision" of the director? Lots of people today simply turn away from any movie that is black and white.
        bredren 3y ago
        I’d like to see rolling data on how people are turning away from lower quality media. Or what premium is being placed on 720p+ video versus standard. There’s a reason video sites offer a filter for HD and YouTube videos get keyword stuffed for 8k even when they aren’t. I am very curious how Apple’s spatial video lands. From reviewer descriptions, the experience is such a major change from standard HD video I am wondering if depthscaling “old” video will become a thing too.
- kouru225 3y ago
  What they’re saying is that it’s your opinion that that would be an improved experience, and that’s not an objective opinion that other people share.
- iancmceachern 3y ago
  Because it's not real, red pill man
hnlmorg 3y ago
The past was more like the present than we like to believe. The past was no more sepia than it was black and white. Just like the 80s wasn’t as bold and neon as pop culture portrays. In media we use colour to exaggerate the differences of the past, to make it feel different and foreign to our senses, rather than represent them accurately. So if you want a more accurate representation then you absolutely should be training your models on modern video.

hutzlibu 3y ago

Here is a colorized version .. but with all the noise: https://m.youtube.com/watch?v=EIeNnDee-Ok https://m.youtube.com/watch?v=EIeNnDee-Ok

WalterBright 3y ago

I'd like to see ML colorize, sharpen, etc., older B+W movies. Silent movies would be greatly improved with the addition of a soundtrack (music[2], foley effects[1], and replacing the dialog cards with voice actors). 1. Yes, they already do add some foley effects. But very little. 2. Yes, they do add a music soundtrack. But it's almost always awful period music. > the ethical and artistic arguments Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music. There's a reddit newsgroup where people colorize old B+W photos. They really do an amazing job. By colorizing key frames in a movie, ML can take care of the rest. BTW, I've seen many bad attempts at colorizing old footage, like those "WW2 In Color" documentaries. You can always tell because the teeth & gums of the people are grey :-/ and the faces are a uniform brown. ML ought to be able to do a lot better.

shiroiuma 3y ago
>> the ethical and artistic arguments >Those are all baloney. Doing what I suggest does not destroy the originals in any way. It's simply making the movies more enjoyable. Apparently, a lot of purists want people to suffer watching them: suffer with grainy, blotchy film, difficult to make out flickering black & white, and bad piano music. I think there's 2 separate issues actually: there's the "artist's intent", and then there's refusal to utilize any modern technology. The latter I agree is baloney: if you can use modern technology to clean up and improve old stuff (such as taking an original Star Wars film print and removing the scratches in the film, improving flaws in the FX, etc.), I think it's ridiculous to argue against that kind of thing. Then there's the artist's intent: if the original artist intended that the movie have "bad piano music" as you call it, should you be changing that? It's like changing the dialog in Lord of the Rings so the characters all talk like modern American hipsters: how is this helpful? These movies are a product of their time, and the music is too. If you could bring the original artists back from the dead, and ask them if they'd like to use modern technology to clean up their old silent movies to sharpen them, remove the noise, etc., I'm sure they'd agree: why wouldn't they want the print to look as good as possible? If you ask them about colorizing, there might be some debate; some might like it, others might not. It's not like they had a choice back then, but even today some people like B&W photos for artistic reasons. If you ask them about adding modern music, I don't think they'd like that much at all.

dclowd9901 3y ago

It’s not destroying the old footage to make it more viewable, so long as it’s preserved in its originality in some way.

proc0 3y ago

AI could do a 1080p with sharp details. Someone just needs to process the frames, and use something like DALL-E or stable diffusion with the right settings.

smcleod 3y ago

I'll do this and let you know how it goes, I don't think I can easily add colour but I can work some other magic. ETA is likely 2-3 hours + whatever it takes to upload (assuming I get back to it tonight - otherwise I’ll upload tomorrow morning).

sgt 3y ago
Looking forward to this. You can probably submit it to HN separately and it'll get heavily upvoted!
- smcleod 3y ago
  I haven’t tried enhancing anything this old and crusty before so don’t hold your breath!
  - sgt 3y ago
    Done yet?
    - smcleod 3y ago
      Yeah I shared yesterday - https://news.ycombinator.com/item?id=37898748 https://news.ycombinator.com/item?id=37898748 It’s quite hard with such low quality footage. I think could be done is use some sort of algorithm to stop that sort of wavy chemical look that might be beyond me anyway that video did clean up a bit!
      - sgt 3y ago
        Very cool, it definitely gives an increased feeling of "being there".
smcleod 3y ago
First attempt has processed at 1080p 60fps, uploaded to YouTube - it will take some time for YouTube to finish processing it in 1080p - https://youtu.be/hlhn9HSH4OI https://youtu.be/hlhn9HSH4OI I think I can do better though, so will try another method in the morning.