6 ms·
Optical microphone can separate multiple instruments from afar
- IndySun 4y agoSince childhood I have been waiting for vinyl turntables that use a laser instead of a needle to 'play' records, without any wear and tear. This may be the breakthrough.
- anigbrowl 4y agoI've been doing pro audio stuff or 25 years and this is a landmark paper, biggest breakthrough I've seen in years. I'm astonished at the quality of extracted signals. Biggest thing I've seen since deconvolution became good enough for realtime or near-realtime adaptive noise reduction.
- de6u99er 4y agoI hope this can be used to objectively distinguish between real audiophile equipment and audiophile snake oil.
- capableweb 4y agoThen please read the submitted article, the press release (https://www.cs.cmu.edu/news/2022/optical-microphone https://www.cs.cmu.edu/news/2022/optical-microphone) or the paper itself (https://www.marksheinin.com/_files/ugd/a41a28_7d370603fafd419da387de85d8ecb5b4.pdf https://www.marksheinin.com/_files/ugd/a41a28_7d370603fafd41...) again, as it has nothing to do with audio quality.
- totetsu 4y agoThis is for espionage not hifi audio
- Dan_Sylveste 4y ago'Real' audiophile equipment is the stuff that sells to recording studios like powered studio monitors and rackmount solid state transports. It costs a lot less than the consumer audiophile stuff, too. If you're buying 'consumer' audiophile products like huge floorstanding speakers made of whatever wood of the day is in favour then you're buying status symbols, not audio reproduction quality.
- justinclift 4y ago> It costs a lot less than the consumer audiophile stuff, too. That's the first time I've heard anyone say that studio monitors cost _less_ than consumer audiophile stuff. Saying that as a person that buys and uses studio monitors, they are not cheap in any way. Even "affordable" entry level stuff (eg JBL Pro LSR 305P/310S) is going to set you back a good chunk of change compared to consumer gear.
- Dan_Sylveste 4y agoHmm from my perspective a pair of JBP LSR 308P MkII and a transport matches the price of entry-level consumer audiophile speakers and a cheap (Topping/SMSL) AMP/DAC pairing yet will outperform it any day.
- justinclift 4y agoHeh Heh Heh. You also need to add the matching 310S sub (https://jblpro.com/products/lsr310s https://jblpro.com/products/lsr310s) for things to sound "right". :) They're engineered to work together, and while you _can_ go without the sub for a while... it's a shit setup and only useful in the short term. Wouldn't want to be doing production stuff (or even end user setup) without the sub. ;)
- Dan_Sylveste 4y agoI thought that the sub was less necessary with the 8" monitors? Maybe I misread or misremember. One of the thing that's interesting to note with monitors is how 'biased flat' is very much a thing and that consumer audiophile speakers are very much not always biased flat, I think the varying companies have a sound profile which they aim to target.
- ackbar03 4y agoI thought something similar was done with the laser on window pane thing already
- TrainedMonkey 4y agoLaser stuff was a single point. The advantage here is capturing a set of 2d pixels which let you see how signal travels through space. This essentially means a large array of microphones which are next to each other and in perfect sync (which I hear is hard when you try to do this with multiple microphones).
- williamcotton 4y agoIt is hard due to phasing, that is, overlapping audio signals from the same source when summed during mixing can cancel out various parts of the signal. Channels on some mixers have a phase inverter and audio engineers will also move microphones around while looking at a phase monitor.
- cma 4y agoThere was this in 2014: https://newatlas.com/visual-microphone/33222/ https://newatlas.com/visual-microphone/33222/
- pishpash 4y agoInteresting interpretation of this as a high-density high-resolution array! But does it need to be isolated well from movement? Since sound wavelengths are long, so maybe not?
- rcxdude 4y agoThis isn't how the system works. It basically can give you a small number of contact microphones (you need to aim a laser at each point you want to record).
- infogulch 4y agoThis is better quality (see comparisions in the video demo) and it's done with mostly normal cameras, not specialized high framerate cameras.
- iguana_lawyer 4y agoCould an optical microphone be used to identify materials using resonance frequencies?
- macgyverismo 4y agoI love the creative use of the rolling shutter, instead of seeing it as a downside, they turned the line-by-line nature of the sensor into sample rate multiplier.
- okwubodu 4y agoThat blew me away before I even finished the paper.
- dougabug 4y agoThe use of rolling shutter to increase effective sampling rate was present in the original SIGGRAPH 2014 Davis et al paper from Bill Freeman’s group at MIT, “The Visual Microphone: Passive Recovery of Sound from Video” (https://dspace.mit.edu/handle/1721.1/100023 https://dspace.mit.edu/handle/1721.1/100023). The authors of the current paper cite this and other prior work. The key innovation is the use of both a rolling shutter and a global shutter reference.
- orbital-decay 4y agoIncredible separation, I don't think it's attainable by any other means. Should be super useful for speech in noisy environments. I have a question though, is capturing lateral movements of a single spot on the instrument enough to represent how it sounds for a human ear? I think it's equivalent to a polarizer filter as it doesn't seem to be capturing depth axis vibrations.
- infogulch 4y ago> it doesn't seem to be capturing depth axis vibrations Good point, the paper mentions x-axis and y-axis, but doesn't mention z-axis. Maybe depth vibrations could be resolved as changes to the interference pattern?
- pbhjpbhj 4y agoDo human ears resolve sound in more than one dimension? I've always considered that, per ear, we only get a sequence of compressions and rarefactions on the ear drum, that other aspects of hearing are through combination or 'cheating' (skin sensitivity and such). So, would it matter?
- infogulch 4y agoNo, but a thing vibrating in any dimension will still produce sound waves.
- taeric 4y agoYou can pinpoint with your two ears where sound is coming from, fairly well. Takes some training, of course. That said, I get the impression this is more complicated than the autocorrelation of multiple sensors?
- kimburgess 4y agoThere’s binaural localisation from phase and amplitude differences, but importantly also monoaural cues from frequency response. Your pinna act as a set of directional filters - which is part of why some binaural recordings can quite literally feel as though they’re in your head. What’s interesting with this is that while you can’t get those directional filters, you could use a system like this to provide 3 or more ‘sensors’ (eg the chip packet demo) within a scene and isolate signals the same way as an array mic.
- anfractuosity 4y agoOoh that's really cool that they're using the laser speckle pattern. I like the fact they exploit the rolling shutter too. Something which https://people.csail.mit.edu/mrub/VisualMic/ https://people.csail.mit.edu/mrub/VisualMic/ also does. There are devices which are called laser doppler vibrometers, which might also be able to do this by pointing at the strings/base of the guitar? There do seem to be videos of laser doppler vibrometers being used with guitars on youtube, but I'm not sure if the soundtrack that goes along with them is just from a normal mic. I had a little play with laser speckle patterns to detect keypresses, as they can help find very subtle changes to a surface - https://www.anfractuosity.com/projects/fun-with-speckle-patterns/ https://www.anfractuosity.com/projects/fun-with-speckle-patt... (by 'diffing' the patterns)
- ssfrr 4y agoLDVs are the gold standard for noncontact vibration measurement and are widely used in acoustics. The main problem is that they’re pretty expensive (I think on the order of a few 10s of k$)
- zbrozek 4y agoI came here to post about LDVs but was beaten to it. https://en.m.wikipedia.org/wiki/Laser_Doppler_vibrometer https://en.m.wikipedia.org/wiki/Laser_Doppler_vibrometer They're sensitive to very small vibrations. A friend of mine used them while working at a hard drive manufacturer to better understand head and platter vibrations.
- TheHideout 4y agoThis seems like an interesting optical solution to the "cocktail party problem", which can be solved using Independent Component Analysis [0] [0] https://en.wikipedia.org/wiki/Independent_component_analysis https://en.wikipedia.org/wiki/Independent_component_analysis
- 38_9503-94_5201 4y agocheck your PI mail, syn9; the game is on
- yodon 4y agoIs technique this significantly different from the laser microphones[0] that intelligence services have been using for many decades? [0]https://en.m.wikipedia.org/wiki/Laser_microphone https://en.m.wikipedia.org/wiki/Laser_microphone
- medler 4y agoIt builds on the technology behind laser microphones. According to the paper, visual vibrometry has historically required expensive cameras, and their method removes this need and appears to have other advantages over using a high-speed camera. They say they contribute “a novel method for sensing vibrations at high speeds (up to 63kHz), for multiple scene sources at once, using sensors rated for only 130Hz operation. Our method relies on simultaneously capturing the scene with two cameras equipped with rolling and global shutter sensors, respectively.“
- klabb3 4y agoCombined with sophisticated noise cancelation and other relatively mature tech, this could make intentional focus listening possible, analogous to looking at something, as well as closing your eyelids, but for hearing. Imagine being able to shut off specific ambient noises (and sometimes.. people) without losing spatial awareness. Or tune in a source you're paying attention to (the cocktail party problem). The issue with super-hearing would be to re-adjust expectations of who can reasonably hear us. Could be used for creepy things, obviously..
- wanderingstan 4y agoThis tech could be applicable to ticketing loud vehicles, as discussed here earlier this year: https://news.ycombinator.com/item?id=30364669 https://news.ycombinator.com/item?id=30364669
- thanatos519 4y agoI'm imagining Q delivering this technology to 007, except that it would shatter the audience's suspension of disbelief. Truly astounding sorcery, yet at the same time completely straightforward.
- 323 4y agoCan this be used for crowd audio surveillance? Lasers on light poles next to the already there high resolution cameras and you could record what everybody is speaking while walking on street.
- its_bbq 4y agoAn interesting thought experiment. Are there vibrations on our throats or is it all taking place hidden inside our bodies?
- yjftsjthsd-h 4y agoThroat mics exist, so probably? (If I'm understanding your question correctly)
- imchillyb 4y agoWhen you speak, even your chest vibrates with the sound. Our bones conduct the vibrations in our skulls. This system could eventually be tuned to read those vibrations.
- anewpersonality 4y agoDoes the laser have to track the instrument in space?
- knolan 4y agoThey discuss this in their video: https://youtu.be/_pq0d1oxtA0 https://youtu.be/_pq0d1oxtA0
- deleted 4y ago[deleted]
- pontifier 4y agoI was just wishing for something like this yesterday. I wanted to figure out how to detect some very very low infrasound reliably, and no conventional microphone technology seemed like it could do what I needed. This feels like it could form the basis of a new wave of scientific vibration measurement systems.
- tnjm 4y agoNot only is the tech here astonishing, but full credit to the authors for producing such a clear, concise video accessible to just about everyone explaining what they did, how they did it, and why it matters. Excellent storytelling.
- ben_w 4y agoOk, this is bizarre. A few days ago, and with no connection at all to any of this work, I had a similar idea. Not the same idea, just similar, and mine is just at the “hmm” stage (which also means mine may turn out to not work), but this still feels really weird. Anyway, my thought was: laser goes to semi-silvered mirror between camera CCD and camera lens, passes through lens to diverge outward into environment, reflects off environment in the same way as a normal laser microphone, separate return signal now exists for all pixels on CCD. Point this at a wall, do the right transformation (is a Fourier transform sufficient?) and the entire wall can be used as a computational phased array of microphones to listen to a specific (possibly moving) target. Possibly mix with the original laser light to get beat patterns due to red/blue shift, like radar speed guns, but that feels like a separate application entirely. I’m really impressed by this result in particular: > Combining 63-fps video from two cameras, one with a global shutter and one with a rolling shutter, allows the researchers to recover a sound signal at 63,000 Hz Because shutter speed was my first concern about possible limitations when I had my other idea.
- crispyambulance 4y agoI just watched the video and am a little confused about the laser illumination. Does the laser have to be "aimed" at each object of interest or is it just laser illumination of the whole scene? The graphics suggest there's a laser point on each object, but that means it has to be aimed and thus follow the subject, right? Moreover many objects, like guitars, have complex oscillation modes so if you are "listening" to just "one point" on the surface, you're not picking up the sound from the other parts of the guitar which are oscillating differently.
- rcxdude 4y agoThe laser needs to be aimed at each point of interest. The system can track multiple laser points simultaneously with one pair of cameras, but there's a tradeoff with quality (because essentially the camera's field of view is being divided into slices, one for each laser point). So you've got a small number of virtual contact microphones you can point at a surface, not a video where you can see the vibration at each point.
- kazinator 4y ago> a 63-fps limit on input data would seem to place a 63-Hz upper limit on the sound this device can "see." Obviously not because it's not taking one amplitude sample. It's analogous to taking 63 sliding FFT's per second, which may be based on on thousands of samples and capture high frequency content. This speckle pattern being sampled is some kind of FFT-like transform of the signal containing lots of information. But the 63 Hz sampling will have to show up as a limitation. I would expect it to be excellent for periodic signals, but to struggle with transients, like the attack of a percussion instrument such as a snare drum.