5 ms·

Watch this improved version. It's the same footage, corrected for speed and light consistency. https://www.youtube.com/watch?v=mqaSOw1WhjI https://www.youtube.c

Watch this improved version. It's the same footage, corrected for speed and light consistency. https://www.youtube.com/watch?v=mqaSOw1WhjI https://www.youtube.com/watch?v=mqaSOw1WhjI

chrisco255 3y ago
Didn't realize how windy it was in that scene with his daughters until watching this version.
sramam 3y ago
interesting that the digital version is LR flipped (is there a phrase for that?). wondering if there is a reason for that to happen.
- wahnfrieden 3y ago
  The image has been flipped left to right to correct the camera-to-subject orientation.
  - stavros 3y ago
    What do you mean? What's that?
    - alibosworth 3y ago
      That's just a quote from the youtube description. My take is that it is a fancy way of saying that they corrected it to match reality.
      - alibosworth 3y ago
        Interestingly... on looking closer at the "restored" version, he is now sipping his tea with his right hand, as are his daughters, but he is smoking with his left hand.
        wahnfrieden 3y ago
        He was right-handed, arthritic in the right, so he learned to write with his left; it doesn't seem reliable to know what hand he'd use for what activity
ip26 3y ago
It's still got all the noise and shadows frame-to-frame. It wouldn't be historically accurate, but it strikes me that an ML algo could probably smooth the noise and shadows, colorize, and even add frames into a more immersive video to bring Twain even closer to the viewer. There's always the ethical and artistic arguments around colorizing and so forth, but if one of the values of old video reels is to connect our conscience to the past, it's my opinion that smoothing, colorizing, de-noising and so forth can be benefits in that goal, even if the frames are no longer "authentic". (It's not like the real Twain was choppy and black-and-white)
- JasonFruit 3y ago
  But if that ML tool is trained on modern video, depending on its capabilities it may subtly modernize aspects of the video in ways that lessen the "past is a foreign country" effect, leading us to believe that the past was more like the present than it in truth was. It's a trade-off between removing technological barriers between us and the person, so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve.
  - WalterBright 3y ago
    > leading us to believe that the past was more like the present than it in truth was So what? I just don't get this. Can't we just enjoy a much improved film experience? > so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve. Nobody is suggesting destroying the originals.
    - kortilla 3y ago
      But what’s the point then? If you’re going to make fake videos might as well just generate a completely new one with actors.
      - WalterBright 3y ago
        Would you rather see a colorized picture of your grandma or a color picture of another woman?
keithnz 3y ago
and colorized... https://www.youtube.com/watch?v=EIeNnDee-Ok https://www.youtube.com/watch?v=EIeNnDee-Ok
- zamadatix 3y ago
  At what point does it stop being film of Mark Twain and start being what someone/a computer imagines film of Mark Twain to be like? Most of the data in this, from the color to the frames themselves, did not actually exist and, while looking reasonable, very likely does not agree with what the actual data would have been. One other downside of this kind of approach is it makes high quality versions of the source significantly harder to find (I doubt the 240p encoding with blocks bigger than Mark Twain's head is the best copy of the source available).
  - proc0 3y ago
    Arguably the tech at the time did not allow Mark Twain to be properly captured at all, so even the original contains imperfections that distort the image. A high res AI version that restores it based on statistical training is still better and would show an "educated guess" of what he looked like.
  - imiric 3y ago
    To be fair, all photography is an approximation of reality. Whether that approximation is improved by color data added at time of capture or a century later, or if the image is cleaned up by AI instead of a human, doesn't change that fact. Most images taken with a smartphone today don't capture reality reliably either, and you're allowed to alter them to look whichever way is most pleasing. Even our eyes perceive reality slightly differently, and photography is not much different. If anything, looking at 2D reproductions of reality adds another layer of subjective perception.
  - bloak 3y ago
    For me there's a big difference between applying a simple algorithm to the data, a formula which someone could scribble down on the back of an envelope, on the one hand, and, on the other hand, some neural network thing with millions of weights that might be hallucinating. Even compression algorithms can hallucinate: https://www.theregister.com/2013/08/06/xerox_copier_flaw_means_dodgy_numbers_and_dangerous_designs/ https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea... I want my historical records to be processed only with simple algorithms that help me see what is really there.
trackofalljades 3y ago
Thank you, good heavens...the other video is awful by comparison, besides just needing cleanup and being terribly compressed it was horizontally backwards and played at the wrong framerate (so common with silent film).
dclowd9901 3y ago
It still looks off speed wise (0.75x this speed looks perfect), and I bet AI could fix the burns.
two_in_one 3y ago
There is a point after which 'restoration' should be called 'deep fake'. Like intentionally modifying (mirroring in this case) the original. It may look better, but...

Watch this improved version. It's the same footage, corrected for speed and light consistency. https://www.youtube.com/watch?v=mqaSOw1WhjI https://www.youtube.com/watch?v=mqaSOw1WhjI

chrisco255 3y ago

Didn't realize how windy it was in that scene with his daughters until watching this version.

sramam 3y ago

interesting that the digital version is LR flipped (is there a phrase for that?). wondering if there is a reason for that to happen.

wahnfrieden 3y ago
The image has been flipped left to right to correct the camera-to-subject orientation.
- stavros 3y ago
  What do you mean? What's that?
  - alibosworth 3y ago
    That's just a quote from the youtube description. My take is that it is a fancy way of saying that they corrected it to match reality.
    - alibosworth 3y ago
      Interestingly... on looking closer at the "restored" version, he is now sipping his tea with his right hand, as are his daughters, but he is smoking with his left hand.
      - wahnfrieden 3y ago
        He was right-handed, arthritic in the right, so he learned to write with his left; it doesn't seem reliable to know what hand he'd use for what activity

ip26 3y ago

It's still got all the noise and shadows frame-to-frame. It wouldn't be historically accurate, but it strikes me that an ML algo could probably smooth the noise and shadows, colorize, and even add frames into a more immersive video to bring Twain even closer to the viewer. There's always the ethical and artistic arguments around colorizing and so forth, but if one of the values of old video reels is to connect our conscience to the past, it's my opinion that smoothing, colorizing, de-noising and so forth can be benefits in that goal, even if the frames are no longer "authentic". (It's not like the real Twain was choppy and black-and-white)

JasonFruit 3y ago
But if that ML tool is trained on modern video, depending on its capabilities it may subtly modernize aspects of the video in ways that lessen the "past is a foreign country" effect, leading us to believe that the past was more like the present than it in truth was. It's a trade-off between removing technological barriers between us and the person, so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve.
- WalterBright 3y ago
  > leading us to believe that the past was more like the present than it in truth was So what? I just don't get this. Can't we just enjoy a much improved film experience? > so we can connect to their recorded legacy, and removing sociological barriers, which we probably want to preserve. Nobody is suggesting destroying the originals.
  - kortilla 3y ago
    But what’s the point then? If you’re going to make fake videos might as well just generate a completely new one with actors.
    - WalterBright 3y ago
      Would you rather see a colorized picture of your grandma or a color picture of another woman?

keithnz 3y ago

and colorized... https://www.youtube.com/watch?v=EIeNnDee-Ok https://www.youtube.com/watch?v=EIeNnDee-Ok

zamadatix 3y ago
At what point does it stop being film of Mark Twain and start being what someone/a computer imagines film of Mark Twain to be like? Most of the data in this, from the color to the frames themselves, did not actually exist and, while looking reasonable, very likely does not agree with what the actual data would have been. One other downside of this kind of approach is it makes high quality versions of the source significantly harder to find (I doubt the 240p encoding with blocks bigger than Mark Twain's head is the best copy of the source available).
- proc0 3y ago
  Arguably the tech at the time did not allow Mark Twain to be properly captured at all, so even the original contains imperfections that distort the image. A high res AI version that restores it based on statistical training is still better and would show an "educated guess" of what he looked like.
- imiric 3y ago
  To be fair, all photography is an approximation of reality. Whether that approximation is improved by color data added at time of capture or a century later, or if the image is cleaned up by AI instead of a human, doesn't change that fact. Most images taken with a smartphone today don't capture reality reliably either, and you're allowed to alter them to look whichever way is most pleasing. Even our eyes perceive reality slightly differently, and photography is not much different. If anything, looking at 2D reproductions of reality adds another layer of subjective perception.
- bloak 3y ago
  For me there's a big difference between applying a simple algorithm to the data, a formula which someone could scribble down on the back of an envelope, on the one hand, and, on the other hand, some neural network thing with millions of weights that might be hallucinating. Even compression algorithms can hallucinate: https://www.theregister.com/2013/08/06/xerox_copier_flaw_means_dodgy_numbers_and_dangerous_designs/ https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea... I want my historical records to be processed only with simple algorithms that help me see what is really there.

trackofalljades 3y ago

Thank you, good heavens...the other video is awful by comparison, besides just needing cleanup and being terribly compressed it was horizontally backwards and played at the wrong framerate (so common with silent film).

dclowd9901 3y ago

It still looks off speed wise (0.75x this speed looks perfect), and I bet AI could fix the burns.

two_in_one 3y ago

There is a point after which 'restoration' should be called 'deep fake'. Like intentionally modifying (mirroring in this case) the original. It may look better, but...