3 ms·
Don't sweat the CCTV or recordings: Reconstructing audio from video requires that the frequency of the video samples — the number of frames of video captured p
by maxerickson 8y ago
Don't sweat the CCTV or recordings:
Reconstructing audio from video requires that the frequency of the video samples — the number of frames of video captured per second — be higher than the frequency of the audio signal. In some of their experiments, the researchers used a high-speed camera that captured 2,000 to 6,000 frames per second. That’s much faster than the 60 frames per second possible with some smartphones, but well below the frame rates of the best commercial high-speed cameras, which can top 100,000 frames per second.
- zeta0134 8y agoRight, which is why the researchers were reliant on video artifacts caused by the rolling shutter effect. Even at NTSC resolution, that could get you 262.5 scanlines per frame at 60 FPS (525 at ~29.75 FPS, because of interlacing... NTSC is complicated), which could potentially get you ~15 KHz as a baseline sample rate, depending on how prominent the rolling shutter effect actually is for that image sensor, and how much of the frame your vibrating object consumes vertically. This isn't perfect and would require all sorts of calibration, but I could see it being used to recover windows of time at much higher sample rates, and make inferences about the inbetween data with some FFT analysis on what you recovered. Basically, it's more the rate of _scanlines_ that matters for this technique, and the quality of the image sensor used. The rate of full frames isn't the limiter.
- RugnirViking 8y agoThe article quite literally is about a technique to bypass that limitation and do it with a regular 60fps camera.
- maxerickson 8y agoIt sounds like that technique doesn't quite recover the audio though, just a portion of it: While this audio reconstruction wasn’t as faithful as that with the high-speed camera, it may still be good enough to identify the gender of a speaker in a room; the number of speakers; and even, given accurate enough information about the acoustic properties of speakers’ voices, their identities. Saying it may still be good enough pretty strongly implies that the quality is quite low.
- RugnirViking 8y agoThere is a video on the webpage that includes the recovered audio. It is low quality, but enough to understand the speaker most of the time, and definately enough to pose a security risk when analysed by a specialist
- maxerickson 8y agoThe youtube clip in the article? The voice reconstruction there is from high speed video. If you mean some other video, I'd appreciate a link.