6 ms·
Author here. I spent much longer on this project than I expected to, and I learned a lot from the experience. I wrote this in hopes that it might be useful for
by mtlynch 6y ago
Author here.
I spent much longer on this project than I expected to, and I learned a lot from the experience. I wrote this in hopes that it might be useful for others who want to digitize their old photos and home videos.
I'm happy to answer any questions or take any feedback about this post. I'm by no means an expert on digitization or video processing, but I'll gladly share what I know.
- rendall 6y agoThis was a really lovely read. Thanks for sharing those details
- mtlynch 6y agoThanks for reading! I'm glad you enjoyed it.
- flerchin 6y agoYour insight to decouple editing from exporting along with using a csv as a data layer was really satisfying. Bravo.
- mtlynch 6y agoIt only took me about 7 years to realize it. : ) But, yes, definitely. That was the biggest breakthrough in this project. I'm glad it came to me, as it otherwise would have taken me hundreds more hours, or I'd have given up before finishing.
- Xavdidtheshadow 6y agoThanks for this detailed writeup! I've been staring down a similar problem and I'm really grateful you took the time and energy to do the legwork
- mtlynch 6y agoThanks for reading! I'm glad to hear that the post is helpful in planning your digitization project.
- cowmix 6y agoI'm on a 35+ year quest to digitize 350+ tapes... and the python lib you detailed is just the thing I need. thx!
- mtlynch 6y agoOh, awesome. Glad that it was helpful!
- dr_orpheus 6y agoThat was very insightful! As a Christmas present last year we paid someone to digitize all of my parents old tapes. It was awesome, but left off at the step of editing and I was disillusioned by the thought of trying to manually edit all of the clips. I might go back and try doing that now!
- mtlynch 6y agoThanks for reading! Once I realized I could script the editing, that became the least stressful part to me. Fixing mistakes was just a matter of tweaking some code and re-running the scripts. Capturing, to me, was harder because I constantly worried about missteps that would cause quality losses I'd only discover later.
- jeffbee 6y agoFor what it's worth, the timebase correction on a VHS deck has nothing to do with whether the audio is getting ahead of or behind the picture. If you think about the way VHS works, it's not possible for it to have audio-video sync drift. Reasons your audio was ahead/behind include that your capture device was recording the audio on a device with a free-running clock without reference to the video clock, or that the video capture device was dropping frames. Timebase correction just fixes up the sync pulses so they arrive in an orderly fashion, which improves the picture quality.
- mtlynch 6y agoOh, that's interesting. Thank you! In retrospect, I should have spent more time studying the technical side of of digitization. My process was to just Google problems as I encountered them so I could get unstuck. I never took a step back to understand the fundamentals, but I think that would have saved time overall.
- rhn_mk1 6y agoLast time I've seen time drift in a video file was from having the wrong frame rate in the video stream. A difference between 24 and 25 is rather obvious, but 23.97 and 24 fps might be hard to notice immediately.
- mikepurvis 6y agoVaguely related, anyone who hasn't seen it, definitely go and watch Matt Parker's legendary explanation of why NTSC runs at 29.97fps: https://www.youtube.com/watch?v=3GJUM6pCpew https://www.youtube.com/watch?v=3GJUM6pCpew
- rhn_mk1 6y agoThat was a great video. I wonder why the frequency got changed when the original reason for the choice was the power network frequency. Wouldn't that cause all kinds of interference problems?
- codazoda 6y agoI didn't read it all but I very much like the design, layout, and tone of the post. Nice job.
- silicon2401 6y agoCongrats, and thank you so much for sharing this. I've been planning out how to build out a personal archive/website to run at home, just for my family and disconnected from the internet, and mediagoblin looks like the perfect solution for hosting media in a more convenient way than just folders and files.
- mtlynch 6y agoThanks for reading! >mediagoblin looks like the perfect solution for hosting media in a more convenient way than just folders and files. I'll just warn you now that MediaGoblin is deceptively hard to work with. I chose it because I got it up and running quickly, but over time, I'd keep running into issues that should have been simple but required digging through their source and sometimes making my own patches. It also doesn't specify dependencies very precisely, so I constantly had issues installing it when its dependencies changed out from under it and broke everything. I tried hard for a while to get my patches merged upstream to prevent others from re-doing my work[0], but the project has effectively been unmaintained for the past 2+ years, save for a few brief spurts of work every few months. If I were starting from scratch, I'd use a static site generator like Hugo or Gatsby to just generate thumbnails and URLs for each clip. You'd have to write a little custom code, but I suspect it's less time overall than you'd spend getting MediaGoblin to work. [0] https://issues.mediagoblin.org/ticket/5574 https://issues.mediagoblin.org/ticket/5574 (their bugtracker is currently down, which should give you a sense of the state of the project)
- silicon2401 6y agooh, that's maybe even more helpful than the original post itself! Appreciate the heads up. I'll take your advice and start with Hugo or Gatsby in addition to checking out Mediagoblin.
- betamaxthetape 6y agoI'd be really interested to know what difference (if any) the S-VHS deck made compared to the regular one you were using. I've got a decent capture setup (semi-Professional gear that captures at up to 10-bit uncompressed 4:2:2 - over 100GB an hour!) that I obtained basically for free since it is fairly old (early 2000s). Since good VHS decks are getting harder (and more expensive) to find, I haven't got a good[1] one yet. I'd be interested to know if the difference between the two decks was noticeable! [1] I consider the recommendations on this thread: http://www.digitalfaq.com/forum/video-restore/1567-vcr-buying-guide.html http://www.digitalfaq.com/forum/video-restore/1567-vcr-buyin... to be fairly good at separating good VHS decks from the rest of the pack.
- mtlynch 6y agoHonestly, I couldn't tell the difference in quality between the two VCRs, but I'm also bad at assessing video quality. I thought the quality on my professionally digitized videos was good, but videophile readers have given me feedback that it looks like near-amateur work.
- jeffbee 6y agoHere's the most hackerly way to do this: rent a 10MHz, 10-bit or better data acquisition rig and hook its probe to the head amplifier of your VTR. Play all your tapes and capture the raw tape signal to your computer. This will only require 45 GB per hour, i.e. almost nothing. Process the signal after the fact, with perfect field/line sync correction, whatever audio compression/limiting/equalization you want, etc. VHS only has 3MHz bandwidth, let's don't pretend that a software-defined VTR is not practical. Answering your actual question: S-VHS vs VHS deck should not make any difference for playback of VHS cassettes. Cassettes recorded in S-VHS cannot be played on a VHS deck, so that would be an obvious difference.
- betamaxthetape 6y agoThere are in fact proof-of-concepts similar to what you describe[1] (this one is based off the Doomsday Duplicator[2], a hardware device used for Direct RF capturing off laserdisc players). My (admittedly basic) understanding of VHS vs S-VHS decks is that due to the stricter tolerances required by the S-VHS standard, S-VHS decks typically have better transports than regular VHS decks (lower wow/flutter, better tracking, etc...). And of course many of the high-end decks have a built in TBC. [1] https://github.com/happycube/ld-decode/issues/16 https://github.com/happycube/ld-decode/issues/16 [2] https://www.domesday86.com/?page_id=978 https://www.domesday86.com/?page_id=978
- 867-5309 6y agoGreat posts, thank you. It must be a joy to hold so many hours of footage. Even if 90% may seem boring to you, think of the legacy you have created for your descendents. I have a total of 45mins of family tapes, of which around 5mins includes footage of my late father - the only footage of him which will ever exist. It is a yearningly small amount. I was wondering if the digitisation company provided any kind of report. The sawtooth graph of your audio and video sync issue presents itself as being a hardware rather than software issue, envisioning ammonitic erosion of a plastic spindle. I'm just wondering whether this was fixed with the (despite your heroic attempts to source) right hardware, or if a software algorithm was involved (handcoded, ML?)
- mtlynch 6y agoThanks for reading! Yes, I feel very fortunate that my family recorded so much, and I do look forward to sharing it with my children. One of the interesting parts of revisiting this footage is just seeing how attitudes towards video changed over time, at least among my circles. When I was a child, home video recording seemed so novel, even to my parents, that there was an excitement and enthusiasm in a lot of the videos just based on the sheer fact that we were recording a home video. Nowadays, I don't think people are as excited to record these kind of "slice of life" videos, even my friends with children. The digitization company did provide a report, but it wasn't very detailed. They just listed which tapes seemed to have physical degradation or imperfections. I don't know what their methodology was, but I didn't get the impression that it was anything especially advanced. I imagine that they just trained some technicians to use their equipment and then run through a standard process for each tape.
- ooopsnevermind 6y agoDid going through this ever make you think about trying to record more from other family members, parents or grandparents, still alive? Curious because I ended up building an app as a tool for my own family but it was more for making new recordings vs digitizing old ones: https://trysaga.com https://trysaga.com
- mtlynch 6y agoIt actually didn't, but it probably should. I feel like social media has kind of tainted capturing video of personal events. Now, it kind of has the connotation that you're capturing something that can potentially go out to millions of people, whereas pre-2000, it felt like video was just a fun thing you'd record purely for the enjoyment of those close to you.
- soylentcola 6y agoThis is very familiar to me. When I was a kid, I thought home camcorders were amazing. In my high school/college years, people were still at the tail end of using disposable and pocket cameras so I have barely any photos of myself or friends from those years. Now when I see a camera out somewhere, all I can think is "oh jeez...are they gonna publish this on freaking Facebook?" and debate whether to ask (and risk coming off like the paranoid nutjob acting like a buzzkill).
- soylentcola 6y agoThis is very familiar to me. When I was a kid, I thought home camcorders were amazing. In my high school/college years, people were still at the tail end of using disposable and pocket cameras so I have barely any photos of myself or friends from those years. Now when I see a camera out somewhere, all I can think is "oh jeez...are they gonna publish this on freaking Facebook?" and debate whether to ask (and risk coming off like the paranoid nutjob being a buzzkill).
- bobsmooth 6y agoThat seems like a really cool idea. Looking forward to an Android app.
- neRok 6y agoThe "tearing problem" your video files have vs the professional is actually just the overscan area, see https://en.wikipedia.org/wiki/Overscan#Analog_to_digital_resolution_issues https://en.wikipedia.org/wiki/Overscan#Analog_to_digital_res... Simply put, the entirety of 1 "frame" has more data than required, but a CRT TV would hide those "error margins" behind the borders. Your digital capture however has grabbed all the available data, and the pro would have done the same, but they would have then cropped it down to the desired area/resolution. You can see this when you compare the videos side by side, as the baby is "longer" in the pro version.
- mtlynch 6y agoOh, interesting. I didn't realize that. Thanks for the explanation! >You can see this when you compare the videos side by side, as the baby is "longer" in the pro version. The baby in the video is actually me at five months old, although I'm quite a bit "longer" today.
- kodt 6y agoSo I actually had a very similar experience, purchasing the exact same TOTMC capture device from Amazon and a quality JVC S-VHS VCR on eBay. And later a higher quality capture card. I found any free video editing options out there difficult to learn and unintuitive. I ended up using Sony Vegas which was a big improvement. In my case I only needed to offset my audio track to get it to sync and it was fine. However some of my videos have that same tell-tale edge tearing, I didn’t bother trying to fix it.
- dredmorbius 6y agoKudos on an excellent writeup of a very worthwhile project. Your coverage of details from family and privacy to methods, process, tools, vendors, services, and frustrations is excellent. Glad this was reposted as I missed it first time 'round.
- djmips 6y agoCould you see a product being made from your experience?
- mtlynch 6y agoI thought about that, but I don't think I'd be able to offer something significantly better than what's already available. One of the difficult parts of the process that you can't really "productize" is cataloguing. If you receive a stranger's videotapes, you wouldn't be able to label a clip, "Grandma Alice Visits for Bobby's Kindergarten Graduation," because you don't know anyone's names or relationships or the context of the video. There might be a product in a tool that helps people edit and tag clips after they've digitized them, but I'm not passionate enough about that work to go after a product like that.