3 ms·

I've been thinking a lot about this problem myself! Making screencasts, I realized most of my editing time was spent cutting out silence. If I recorded with tha

I've been thinking a lot about this problem myself! Making screencasts, I realized most of my editing time was spent cutting out silence. If I recorded with that in mind (staying silent between re-takes), the editing became pretty straightforward but really tedious. I made my own automatic editor tool [0] recently, and the approach I started with is simply cutting out the silent parts. I looked at other options like jumpcutter.py and auto-editor.py before building it, and they helped prove out the idea, but I really wanted an interactive UI, so I built an app in Swift. I figured the automated approach is probably not gonna be perfect, so I built it to export XML/EDL/ScreenFlow files that can be imported to other editors for fine-tuning. It works pretty well and people seem happy with it so far. Someone else mentioned timestamps vs. timecode. Maaan, timing has been a real thorn in my side with this project. The "even" framerates aren't too bad (30fps/60fps/etc) but the uneven ones are a huge pain (29.97, 59.94). One fun problem recently was figuring out how to bin audio samples at 48khz into frames at 29.97. Because each frame holds an uneven number (1601.6), I had to alternate between assigning 1602/1601 to each frame, or else my idea of time would slowly but steadily skew out of sync. As someone who'd never worked with video before, this has been a fun adventure. Right now I'm working on adding more manual control over the cuts, and this kind of stuff is what I'd like to tackle next! Automatic scene detection, ability to leave markers during recording, more control over transitions, stuff like that would be really cool. Feels to me like automatic editing might get more popular as more people realize it's even possible. 0: https://getrecut.com https://getrecut.com

jjnoakes 6y ago
By "Automatic scene detection" do you mean somehow detecting in the video if interesting things are going on and avoiding cuts around those spots? Because a lot of videos I record have silence in the audio when important things might be happening in the video, and having to manually go find those places is a bit of a pain. Of course, automatically detecting which parts of the video are interesting or not is probably impossible, but it sure feels like an interesting problem to try to tackle.
- dceddia 6y ago
  Something like that. I could base it on frame-to-frame changes, or if my app was doing the screen recording, I could look at keyboard/mouse input as another signal of "non-silence". I'm pretty much looking at silence removal as a good starting point. My overall goal is to cut down on the manual editing required, so just looking for repetitive processes that I could add automations for.
clawoo 6y ago
Your software is such a great idea. The UI could use some work to be more attractive, but the core functionality is top notch. Here's a suggestion- using Speech.framework[0] you could probably quite easily transcribe the audio and identify filler words ("umm", "hmm", etc) and add an option to automatically exclude those as well. [0] https://developer.apple.com/documentation/speech https://developer.apple.com/documentation/speech
- dceddia 6y ago
  Thanks! I like the Speech framework idea. I've toyed with it a bit and the results not so great, especially offline, and the online one has some limits. I think if I want to properly add transcription I'll need to integrate with some SaaS solution, but I need to do a little more experimenting first. Do you have any specific suggestions or critiques for the UI? I definitely agree it could be more attractive but I've had trouble figuring out what to do besides "make it look like Final Cut" or whatever. (or maybe I should actually do just that!)
chrismorgan 6y ago
Ugh, those messy frame rates. I like my Sony α6100, but they’ve made such a mess of the frame rates. They’ve tied the 25/30 decision to a PAL/NTSC switch (and this fact is completely undocumented, of course), and so if I want to record at 30 or 60fps instead of 25 or 50, I have to go to NTSC, which makes it pop up an annoying “Running on NTSC” message every time I start the camera in video mode, and then to add insult to injury it’s not even 30fps but 29.97fps, which is a real nuisance in editing. (This is all when recording on the camera; I don’t know if it would be any different via HDMI capture.) But then I’ve been adding still more fun by recording audio on my laptop with a good USB microphone, and replacing the audio recorded on the camera. But the laptop’s clock runs a bit slower than the camera’s¹ so I have to stretch the computer-recorded audio by 0.003% (Audacity: Effect → Change Speed) to counter what would otherwise be about 110ms of drift per hour (3.3 frames at 30fps). Good times. — ¹ To measure rates absolutely rather than just one device to another, you need a canonical time source; internet time servers will do, so you compare what the audio device is sending with what internet time servers report over the course of a few hours to eliminate jitter. As for why you’d care, I cared so I could tune my piano more accurately. I measured the alleged 22050Hz as being about 22048.862Hz, which means it’s running at about 99.99484% of real speed. This difference amounts to −0.9 cents, which is utterly unimportant for a solo instrument, but hey, it’s the principle of the thing.

jjnoakes 6y ago

By "Automatic scene detection" do you mean somehow detecting in the video if interesting things are going on and avoiding cuts around those spots? Because a lot of videos I record have silence in the audio when important things might be happening in the video, and having to manually go find those places is a bit of a pain. Of course, automatically detecting which parts of the video are interesting or not is probably impossible, but it sure feels like an interesting problem to try to tackle.

dceddia 6y ago
Something like that. I could base it on frame-to-frame changes, or if my app was doing the screen recording, I could look at keyboard/mouse input as another signal of "non-silence". I'm pretty much looking at silence removal as a good starting point. My overall goal is to cut down on the manual editing required, so just looking for repetitive processes that I could add automations for.

clawoo 6y ago

Your software is such a great idea. The UI could use some work to be more attractive, but the core functionality is top notch. Here's a suggestion- using Speech.framework[0] you could probably quite easily transcribe the audio and identify filler words ("umm", "hmm", etc) and add an option to automatically exclude those as well. [0] https://developer.apple.com/documentation/speech https://developer.apple.com/documentation/speech

dceddia 6y ago
Thanks! I like the Speech framework idea. I've toyed with it a bit and the results not so great, especially offline, and the online one has some limits. I think if I want to properly add transcription I'll need to integrate with some SaaS solution, but I need to do a little more experimenting first. Do you have any specific suggestions or critiques for the UI? I definitely agree it could be more attractive but I've had trouble figuring out what to do besides "make it look like Final Cut" or whatever. (or maybe I should actually do just that!)

chrismorgan 6y ago

Ugh, those messy frame rates. I like my Sony α6100, but they’ve made such a mess of the frame rates. They’ve tied the 25/30 decision to a PAL/NTSC switch (and this fact is completely undocumented, of course), and so if I want to record at 30 or 60fps instead of 25 or 50, I have to go to NTSC, which makes it pop up an annoying “Running on NTSC” message every time I start the camera in video mode, and then to add insult to injury it’s not even 30fps but 29.97fps, which is a real nuisance in editing. (This is all when recording on the camera; I don’t know if it would be any different via HDMI capture.) But then I’ve been adding still more fun by recording audio on my laptop with a good USB microphone, and replacing the audio recorded on the camera. But the laptop’s clock runs a bit slower than the camera’s¹ so I have to stretch the computer-recorded audio by 0.003% (Audacity: Effect → Change Speed) to counter what would otherwise be about 110ms of drift per hour (3.3 frames at 30fps). Good times. — ¹ To measure rates absolutely rather than just one device to another, you need a canonical time source; internet time servers will do, so you compare what the audio device is sending with what internet time servers report over the course of a few hours to eliminate jitter. As for why you’d care, I cared so I could tune my piano more accurately. I measured the alleged 22050Hz as being about 22048.862Hz, which means it’s running at about 99.99484% of real speed. This difference amounts to −0.9 cents, which is utterly unimportant for a solo instrument, but hey, it’s the principle of the thing.