6 ms·

Smoother sailing: Studying audio imperfections in Steamboat Willie

176 points by firloop 3y ago

derbOac 3y ago
I've often wondered, if there are multiple different prints of an older film available, they could be averaged or something using modern ML or computational statistics. I get the sense that "unsupervised" methods are underutilized a bit in film and audio restoration but I know next to nothing about this area.
- morsch 3y ago
  Yeah, there are tools that do this. I know nothing about this stuff, but a recent Tech Tangents video had him combining multiple degraded prints of Contact. It's the final segment and the tool is called Id-disc-stacker. You need a lot of time, space and processing power. https://youtu.be/RduEpIiG0O0?si=uurvcfzVoOkf3zSW https://youtu.be/RduEpIiG0O0?si=uurvcfzVoOkf3zSW
  - saberdancer 3y ago
    I really liked the bit about first rule of government spending.
- enriquto 3y ago
  I've seen this done for vinyl discs. You play several copies of the same old disc. The noise defects appear in different places, and can be easily removed by merging the aligned the tracks. Even with just two discs, the output result is spectacular.
  - Solvency 3y ago
    What software?
    - enriquto 3y ago
      It was a project by some signal processing students in my lab. As far as I know, they didn't get to publish anything besides writing a technical report; and the code was a matlab thing that just ran on the computer of one of them... But their presentation was excellent! They brought their vinyls, the turntable, and did a live demo of the whole thing running.
      - Solvency 3y ago
        Weird, why not sell it, or at least open source it? I imagine thousands of people would use it.
        enriquto 3y ago
        I don't know, releasing a sellable product from an algorithm is a huge undertaking, probably harder and definitely less interesting than developing the algorithm itself. This was just a three-month work project by some smart students, who afterwards got lucrative jobs working in industry. In more practical terms, people who actually listen to vinyls get to appreciate the "cracks", and they wouldn't really see the point in such a product. Major studios have surely developed similar tools internally. The method is probably a scientific curiosity, mostly of educative interest, like observing the helium in the sun atmosphere with a hand-made spectometer built with an old CD and a cereal box.
modeless 3y ago
I've noticed a lot of jitter in the video, even in "stabilized" versions. It seems like there was jitter in how the cels were placed on the backgrounds, in addition to jitter of the whole frame. It's not so easy to stabilize it out. It would be cool to clean that up too with modern techniques. More speculatively it seems to me like AI is approaching the level of image understanding where it could draw new in between frames in a much, much smarter way than typical terrible frame interpolation. That would be a great project and potentially commercially valuable. Steamboat Willie is animated at 24 FPS but it's common for animated TV shows to be done at 8 FPS and they could really benefit.
- actionfromafar 3y ago
  It doesn't look to me like Steamboat Willie is animated at anything close to 24fps. I couldn't step frames, but just look at the video, the animation is jerky. (Not worse than animation usually is, but it's not 24fps.)
jlarcombe 3y ago
very interesting indeed, reminds me of some of the work done by Jamie Howarth and co with the 'Plangent Process' which uses the bias tone recorded by analogue tape machines to correct flutter on high-sample-rate transfers from tape...
h2odragon 3y ago
also: https://hackaday.com/2024/01/24/steamboat-willie-never-sounded-better/ https://hackaday.com/2024/01/24/steamboat-willie-never-sound...
NoZebra120vClip 3y ago
The film has been played on endless repeat, with "original sound", since the 1970s, at Disneyland's Main Street Cinema. https://en.wikipedia.org/wiki/Main_Street_Cinema https://en.wikipedia.org/wiki/Main_Street_Cinema It would be interesting to see if anyone has made remarks on the sound quality and whether the flutter has been noticeable over the past 50+ years.
- djmips 3y ago
  It certainly has been noticeable. It just gets mentally filed under 'old timey' sound.
  - weinzierl 3y ago
    "filed under 'old timey' sound" A little flutter was an unavoidable part of our audio for a very long time. I think producers sometimes add a little flutter even to modern music productions to make them sound less digital. At least that is my assumption from the number plugins that are available for that purpose and from the obsession producers and mixing engineers have with vintage tape machines.
  - m463 3y ago
    and people just must have been more frenetic in old-time movies, moving faster and more jerkily not a hand-cranked/fps mismatch I kind of wonder if the cameramen cranked slower sometimes to save film
    - madaxe_again 3y ago
      Undercranking was generally used for dramatic effect, but I definitely have read about it being used to allow longer takes with a fixed film reel length - if you undercrank by 10%, you get 10% more time before you have to stop and reload.
      - deredede 3y ago
        It's even better due to the way percentages work - if you undercrank by 10% you get a bit over 11% more time (after 100% of the original time, you still have 10% of the reel left, but you keep undercranking, so by the end of the 10% extra time, you still have 1% of the reel left, etc.).
Onavo 3y ago
Neural upscaling would make it even better
- bagful 3y ago
  In what sense? Higher resolution formats provide more true-to-life fidelity, but that's only more true-to-nonsense when the additional information is produced from extrapolation.
thsksbd 3y ago
"This is an experiment [...] soundtrack was being read in the same way as the picture is – stopped 24 times per second? Would this be the ultimate flutter distortion?" That bothered me like crazy - how did they scan the audio whilst the frame was physically stopped? Then I realized how it was done: the soundtrack of frame A is stored in frame B, where frame B is far enough away that it has constant velocity.
- WalterBright 3y ago
  > Then I realized how it was done Oh, I love ingenious solutions like that!
  - thsksbd 3y ago
    Aint it neat?
- stavros 3y ago
  How does it have constant velocity? Is there a physical "buffer" for the stopped frame (ie slack in the reel)? Otherwise you can't stop one part of the reel while the other is still traveling.
  - HPsquared 3y ago
    There must be some "slack", the entire reel isn't going to stop and start 24 times a second.
  - thsksbd 3y ago
    Short: There is a buffer! Long: So the frame being projected has to stop. Otherwise you'll get blurring unless your shutter is very fast, but then you'd burn out the film from the intensity of the light. The film is stopped. But, of course, you cant start-stop a 15 lb spinning reel 24 times a second millions -> itll all fall apart. So the feed is a simple rotation at constant velocity (linear? Angular?). Then a piece of film ten to twenty (?) frames long gets put into a buffer (an empty box) and a mechanism akin to a sowing machine single steps through each individual frame. Now the stepping mechanism only has to start-stop ten frames worth of film, a negligible amount of mass compared to the mechanism itself and the projector can last a long time of heavy use with simple maintenance. Aint it neat?

176 points by firloop 3y ago

I've often wondered, if there are multiple different prints of an older film available, they could be averaged or something using modern ML or computational statistics. I get the sense that "unsupervised" methods are underutilized a bit in film and audio restoration but I know next to nothing about this area.

morsch 3y ago
Yeah, there are tools that do this. I know nothing about this stuff, but a recent Tech Tangents video had him combining multiple degraded prints of Contact. It's the final segment and the tool is called Id-disc-stacker. You need a lot of time, space and processing power. https://youtu.be/RduEpIiG0O0?si=uurvcfzVoOkf3zSW https://youtu.be/RduEpIiG0O0?si=uurvcfzVoOkf3zSW
- saberdancer 3y ago
  I really liked the bit about first rule of government spending.
enriquto 3y ago
I've seen this done for vinyl discs. You play several copies of the same old disc. The noise defects appear in different places, and can be easily removed by merging the aligned the tracks. Even with just two discs, the output result is spectacular.
- Solvency 3y ago
  What software?
  - enriquto 3y ago
    It was a project by some signal processing students in my lab. As far as I know, they didn't get to publish anything besides writing a technical report; and the code was a matlab thing that just ran on the computer of one of them... But their presentation was excellent! They brought their vinyls, the turntable, and did a live demo of the whole thing running.
    - Solvency 3y ago
      Weird, why not sell it, or at least open source it? I imagine thousands of people would use it.
      - enriquto 3y ago
        I don't know, releasing a sellable product from an algorithm is a huge undertaking, probably harder and definitely less interesting than developing the algorithm itself. This was just a three-month work project by some smart students, who afterwards got lucrative jobs working in industry. In more practical terms, people who actually listen to vinyls get to appreciate the "cracks", and they wouldn't really see the point in such a product. Major studios have surely developed similar tools internally. The method is probably a scientific curiosity, mostly of educative interest, like observing the helium in the sun atmosphere with a hand-made spectometer built with an old CD and a cereal box.

modeless 3y ago

I've noticed a lot of jitter in the video, even in "stabilized" versions. It seems like there was jitter in how the cels were placed on the backgrounds, in addition to jitter of the whole frame. It's not so easy to stabilize it out. It would be cool to clean that up too with modern techniques. More speculatively it seems to me like AI is approaching the level of image understanding where it could draw new in between frames in a much, much smarter way than typical terrible frame interpolation. That would be a great project and potentially commercially valuable. Steamboat Willie is animated at 24 FPS but it's common for animated TV shows to be done at 8 FPS and they could really benefit.

actionfromafar 3y ago
It doesn't look to me like Steamboat Willie is animated at anything close to 24fps. I couldn't step frames, but just look at the video, the animation is jerky. (Not worse than animation usually is, but it's not 24fps.)

jlarcombe 3y ago

very interesting indeed, reminds me of some of the work done by Jamie Howarth and co with the 'Plangent Process' which uses the bias tone recorded by analogue tape machines to correct flutter on high-sample-rate transfers from tape...

h2odragon 3y ago

also: https://hackaday.com/2024/01/24/steamboat-willie-never-sounded-better/ https://hackaday.com/2024/01/24/steamboat-willie-never-sound...

NoZebra120vClip 3y ago

The film has been played on endless repeat, with "original sound", since the 1970s, at Disneyland's Main Street Cinema. https://en.wikipedia.org/wiki/Main_Street_Cinema https://en.wikipedia.org/wiki/Main_Street_Cinema It would be interesting to see if anyone has made remarks on the sound quality and whether the flutter has been noticeable over the past 50+ years.

djmips 3y ago
It certainly has been noticeable. It just gets mentally filed under 'old timey' sound.
- weinzierl 3y ago
  "filed under 'old timey' sound" A little flutter was an unavoidable part of our audio for a very long time. I think producers sometimes add a little flutter even to modern music productions to make them sound less digital. At least that is my assumption from the number plugins that are available for that purpose and from the obsession producers and mixing engineers have with vintage tape machines.
- m463 3y ago
  and people just must have been more frenetic in old-time movies, moving faster and more jerkily not a hand-cranked/fps mismatch I kind of wonder if the cameramen cranked slower sometimes to save film
  - madaxe_again 3y ago
    Undercranking was generally used for dramatic effect, but I definitely have read about it being used to allow longer takes with a fixed film reel length - if you undercrank by 10%, you get 10% more time before you have to stop and reload.
    - deredede 3y ago
      It's even better due to the way percentages work - if you undercrank by 10% you get a bit over 11% more time (after 100% of the original time, you still have 10% of the reel left, but you keep undercranking, so by the end of the 10% extra time, you still have 1% of the reel left, etc.).

Onavo 3y ago

Neural upscaling would make it even better

bagful 3y ago
In what sense? Higher resolution formats provide more true-to-life fidelity, but that's only more true-to-nonsense when the additional information is produced from extrapolation.

thsksbd 3y ago

"This is an experiment [...] soundtrack was being read in the same way as the picture is – stopped 24 times per second? Would this be the ultimate flutter distortion?" That bothered me like crazy - how did they scan the audio whilst the frame was physically stopped? Then I realized how it was done: the soundtrack of frame A is stored in frame B, where frame B is far enough away that it has constant velocity.

WalterBright 3y ago
> Then I realized how it was done Oh, I love ingenious solutions like that!
- thsksbd 3y ago
  Aint it neat?
stavros 3y ago
How does it have constant velocity? Is there a physical "buffer" for the stopped frame (ie slack in the reel)? Otherwise you can't stop one part of the reel while the other is still traveling.
- HPsquared 3y ago
  There must be some "slack", the entire reel isn't going to stop and start 24 times a second.
- thsksbd 3y ago
  Short: There is a buffer! Long: So the frame being projected has to stop. Otherwise you'll get blurring unless your shutter is very fast, but then you'd burn out the film from the intensity of the light. The film is stopped. But, of course, you cant start-stop a 15 lb spinning reel 24 times a second millions -> itll all fall apart. So the feed is a simple rotation at constant velocity (linear? Angular?). Then a piece of film ten to twenty (?) frames long gets put into a buffer (an empty box) and a mechanism akin to a sowing machine single steps through each individual frame. Now the stepping mechanism only has to start-stop ten frames worth of film, a negligible amount of mass compared to the mechanism itself and the projector can last a long time of heavy use with simple maintenance. Aint it neat?