12 ms·
The Daredevil Camera
- visakanv 10y agoThis is insanely cool. I've often vaguely fantasized about stuff like this but this guy takes it all the way. He deserves a huge budget. Can we fund this somehow?
- randyrand 10y agoPatreon would be great for this.
- Artlav 10y agoEveryone is welcome to donate a bitcoin or two. :) The address is on my Orbital Designs site - http://orbides.org/ http://orbides.org/
- rkangel 10y agoI think the value of what he does is that it's driven by curiosity. A pile of money donated by people might put him under pressure to deliver things to his audience which might ruin that. A fund that hackers can apply to for building cool projects is an interesting idea though (to the extend that I'm sure one already exists).
- amerine 10y agoWow.
- visakanv 10y agoI wonder how many other cool-ass projects like this are going on that we have no idea about.
- DanBC 10y agoOr the pages that give tantalising hints about a project, but there doesn't seem to be a write-up anywhere. https://www.flickr.com/photos/fotoopa_hs/sets/72157611107153997/with/3636467040/ https://www.flickr.com/photos/fotoopa_hs/sets/72157611107153... https://www.flickr.com/photos/fotoopa_hs/4082464167/in/album-72157611107153997/ https://www.flickr.com/photos/fotoopa_hs/4082464167/in/album...
- skiningham 10y agoI think those Flickr albums are the write-up for how to make a Fotoopa.
- ISL 10y agoLots.
- visakanv 10y agoIt's a simultaneously awesome and saddening feeling. Awesome because you know that there are unexpected amazing things that are going to happen. Sad because a bunch of them might give up from lack of exposure, feedback, etc...
- Artlav 10y agoYeah. From the hacker's point of view many such projects seem trivial, not worth bothering to talk about. You really need a spotter to find out which of your projects are cool and which aren't. Then again, after trying to post a dozen times and getting nothing, one would just stop trying most of the time, and would work for the attic, so to speak.
- keville 10y agoExcellent narrative on top of an incredible project!
- rev_null 10y agoThe timing issue between the microphones doesn't surprise me. Multi channel soundcards require a single ADC that processes all of the channels in unison (same for the DAC). This is also why you can't just plug four stereo cards into a PC and expect it to make an 8-channel system.
- Artlav 10y agoIt's not between the microphones on a cell, but between the cells themselves - i screwed up on the sync logic somewhere.
- visakanv 10y ago> This is also why you can't just plug four stereo cards into a PC and expect it to make an 8-channel system. Huh, I always sorta wondered about this. Thanks for clearing it up!
- morsch 10y agoYou can't? I'm sure I tried multi-channel via separate sound cards and it sounded fine to me. Maybe I was lucky with the timing, or the audio subsystem somehow accounted for it?
- radiowave 10y agoDepending upon what you're using it for, you might well get away with it, but for some purposes the accumulated sample rate clock drift eventually gets too bad. You could make a long multitrack recording this way, and so long as you played it back from the start on the same hardware (i.e. so the soundcards would drift relative to one another more-or-less consistently with how they drifted during the recording), the playback might be ok. But if you begin playback half way through the recording, then the timing between channels might be out, as your software has no idea how much drift each channel ought to have accumulated by that point in the recording. Mind, it's been at least 15 years since I actually put any of this stuff to the test. Tolerances on modern soundcards might be really tight.
- radiowave 10y ago
- Bromskloss 10y agoYay, a sound-field camera! I've been wondering if such things are made. Should I take the existence of this project as an indication that they aren't? The use case I imagine is finding what part of a machine is emitting a noise, much like how you would use a thermal camera for finding what part of it is running hot.
- visakanv 10y agoThat's a very sensible use case. I was thinking more like, super-trippy live music visuals.
- joshu 10y agohttp://www.gizmag.com/seesv-s205-sound-camera-locates-noises/27447/ http://www.gizmag.com/seesv-s205-sound-camera-locates-noises...
- Bromskloss 10y agoYours is sensible too.
- Artlav 10y agoNot exactly unheard of, but quite rare. I found a company that made large arrays like that, but they seem to operate in near field, measuring only the sound volume. Imagine a wall of microphones next to a truck - you can tell which part of the wheel makes the sound. That sort of stuff. As for an actual "camera" that turns sound into an image - i've never heard of one.
- nxzero 10y agoIsn't this exactly what sonar does? SOund Navigation And Ranging: https://en.m.wikipedia.org/wiki/Sonar https://en.m.wikipedia.org/wiki/Sonar
- Artlav 10y agoNot exactly. A sonar maps a surface using sound, while my camera looks at the sound itself. Think laser rangefinder on a scanner platform vs a camera.
- ProAm 10y agoThis is the definition of what a 'hacker' is, great read.
- backprojection 10y agoreally cool. It looks pretty close to the ideas behind seismic imaging. https://en.wikipedia.org/wiki/Reflection_seismology https://en.wikipedia.org/wiki/Reflection_seismology
- jofer 10y agoJudging from your username, I'm going to guess you're an earthquake seismologist. Cheers from the reflection world!
- visakanv 10y agoRight on. Reminds me of Richard Feynman or Steve Wozniak, the way they just casually talk about taking things apart and putting things together while following a curiosity on an adventure. This is representative of the sort of ethos that drew everyone here in the first place, I think.
- kamaal 10y agoUnfortunately once you touch this sort of high, the normal day job starts to looking boring.
- based2 10y agohttps://www.reddit.com/r/dataisbeautiful/comments/2vuvhf/precise_3d_mapping_and_visualization_of_wifi/ https://www.reddit.com/r/dataisbeautiful/comments/2vuvhf/pre...
- rhema 10y ago"Daredevil" as in vision through sound. Very neat project! I wonder how much people can see in the dark through listening to sound sources by moving their heads for motion parallax.
- kuschku 10y ago> I wonder how much people can see in the dark through listening to sound sources by moving their heads for motion parallax. Listen to this with headphones: https://www.youtube.com/watch?v=IUDTlvagjJA https://www.youtube.com/watch?v=IUDTlvagjJA It’s amazing how much you can geolocate just in even a recording.
- nitwit005 10y agoExcellent project. I appreciate listing the insights along the way.
- bcherny 10y agoThis would be pretty powerful if you combine it with speech recognition. If you put this array in the middle of a crowded room, can you better tease apart different audio sources?
- Artlav 10y agoIronically, this thing can't record sound as we know it - it only have enough buffer space for a few hundreds of samples per mic, 30 times per second. They are to be FFT-ed rather than continuously recorded. While such a contraption should, in theory, be able to distinguish between dozens of voices in a room, it would take much, much heavier signal processing to achieve.
- jsprogrammer 10y agoIf you have more than one mic you can begin to triangulate sources (you might be able to use a single mic if you know the geometry of the room and/or could send out noises), so it would definitely be possible. Consider that most of us are carrying around multiple, networked mics. Some may already do it, but I doubt it will be too long before everyone can virtually tour real-life spaces and visualize the sounds there in real-time.
- lucb1e 10y agoNot this specific hardware, but in general, yes. Amazon Echo does pretty much this, except it focuses on one source instead of all possible sources (from what I know). PulseAudio also recently gained a feature to use beamforming (see a HN submission from either today or yesterday), but for now it needs the location of the object you're trying to record as an input. I guess you could kind of brute-force that and let it run many times over the same audio stream to find all possible audio sources.
- dharma1 10y agoYou can use the phase/timing information between different mics to separate sources, yes. Interestingly it's also doable with a single mic - previously with algorithms like NMF and ICA, more recently with neural networks. Our brains do it very well so there is probably more to discover. Google blind source separation or cocktail party problem if you want to find out more.
- soheil 10y agoI think the whole thing could be done pretty much all in software. Setup a bunch of phone numbers in Twilio, using a bunch of your friends iPhones, strategically located, call the numbers and process the data using Twilio API. Whatsapp or other real-time voice communication apps with an API may work too.
- jvdb 10y agoAwesome project! Nitpick; Duga 3 was never built. The one near Chernobyl is Duga 1.
- Artlav 10y agoFacepalm. Yep, somehow it got stuck in my memory as Duga-3. Something about the third prototype/iteration of the project? No idea by now...
- jvdb 10y agoOnly recently found out it isn't Duga 3 (as I also always thought it was), must be that wikipedia redirect. But I didn't mean to detract, that's really a very cool project that you did!
- wyager 10y agoOne of the coolest and most expository posts I have seen in recent memory. Well done.
- sbierwagen 10y agoHey Artlav, when did you start posting on ribbonfarm? I used to subscribe to it, but unsubscribed a couple years back after Rao wrote a dumb post on information security.
- Artlav 10y agoAbout a year ago. He invited me in after seeing the "How do you make batteries that work on Venus" article ( http://orbides.org/page.php?id=1005 http://orbides.org/page.php?id=1005 ) i wrote on my own semi-dead site.
- fudged71 10y agoArtlav, this is an incredible project! I have an idea for you. You mention that you want to use a webcam so that you can "overlay" this information on top of an image. What if you could send it directly to your brain? There is a technology called Brainport[0], which is effectively a 16x16 grid of electrodes small enough to place on your tongue. By feeding electrical impulses through these electrodes, you stimulate the nerves on your tongue. After two minutes of a "champagn bubbling" sensation, your brain rewires the senses from your tongue to your visual cortex and you can "sense" any input. This technology has been used to give sight to the blind, balance to those with inner ear problems, and sonar to NAVY seals. It would be amazing if you were the first person in the world to truly "see" sound. [0] https://en.wikipedia.org/wiki/Brainport https://en.wikipedia.org/wiki/Brainport
- therein 10y agoBrainPort sounds extremely cool. How does the brain decide to interpret them as visual input, though?
- fudged71 10y agoIt's a process called Sensory Substitution[0]. It's technically the perception of that sense. I understand that the rest of your brain 'trains' this pathway to occur. So because you know that there is a sound source and you can see it move in space, and you 'feel' this on your tongue as well, you are capable of correlating the two senses together and associating them with each other. [0] https://en.wikipedia.org/wiki/Sensory_substitution https://en.wikipedia.org/wiki/Sensory_substitution
- whatshisface 10y agoI'm pretty sure that would be mechanically impossible - the brain is very plastic, but touch-into-sight seems a little far for adult learning. Instead, you could expect this to feel more like acquiring a new sense - the experience reported by those who have used pager vibrators to give their body a compass[1]. [1]http://www.thinkgeek.com/product/f358/ http://www.thinkgeek.com/product/f358/ (sorry, I couldn't find a better source - but the product description explains it)
- rasz_pl 10y ago>Unfortunately, we are still talking about a total of 64 Mbps of data. That needs a USB 2.0 sampling board that would pull the data out of the cells over the bulk interface you mean a $10 EZ-USB FX2LP dongle? You might be interested in this https://sourceforge.net/p/manyears/wiki/Main_Page/ https://sourceforge.net/p/manyears/wiki/Main_Page/ Im sure you know about SeeSV-S205, its basically the same thing, but polished and packaged neatly for FAT commercial customers ($30K). posted on slashdot in 2013 about the same thing https://tech.slashdot.org/comments.pl?sid=3742015&cid=43716763 https://tech.slashdot.org/comments.pl?sid=3742015&cid=437167...
- Artlav 10y agoInteresting. Can it take 16 simultaneous SPI-like channels of data going at 40-ish MHz, and pack it into a USB uplink?
- rasz_pl 10y agonot 16, but 8 '40-ish MHz' should be no problem as long as you sync them. FX2LP gives you raw 8/16bit wide datapump in GPIF Mode mode, or a nice FPGA friendly fifo in Slave FIFO mode, you can sort your 'spi like' data streams out on the pc side later/in real time. and since modern PCs have at least 2 usb hosts you could use two FX2LPs for all 16 streams. Modern i7 cpu has surprising amount of flops and could handle this no problem (not to mention modern gpus). Consider modifying one of your boards, instead, or in addition to doing fft on fpga try pumping raw data over usb to a PC, 64 3MHz PDM streams should be 25MB/s? comfortably within usb 2.0 limits. FX2LP breaks no sweat while pumping ~40MB/s top ebay link $6 http://www.ebay.com/itm/CY7C68013A-56-EZ-USB-FX2LP-USB2-0-Develope-Board-Module-Logic-Analyzer-EEPROM-/200910097384 http://www.ebay.com/itm/CY7C68013A-56-EZ-USB-FX2LP-USB2-0-De... application: http://www.cypress.com/documentation/application-notes/an61345-designing-ez-usb-fx2lp-slave-fifo-interface http://www.cypress.com/documentation/application-notes/an613...
- Artlav 10y ago> 64 3MHz PDM That's "64 3MHz PDM" per cell, of which there should be 16. And, they have to be synced to a sub-microsecond precision.
- anotheryou 10y agoWhy is the resolution so much fixed to the number of mics? Shouldn't 3 mics be somewhat sufficient for intermediate values as well? More mics => more redundancy => clearer image of the waves, but why do the pixels/blobs "snap" to a microphone? edit: ah, he does not use timing (at 10fps obviously not) but needs continuous sound to see the wave pattern across the array... edit 2: so do these gunfire locators¹ only time stamp impulses? Or do they do continuous analysis? With natural, irregular sound, it should be possible to match and measure the delay between mics, no? edit 3: the gunfire locators would need to analyze for gunfire characteristics anyways, to weed out false positives, but that could be separated from the locating. edit x: oh cool, if you zoom out of the final picture so it's smaller on screen, you can even see some parallax effect happening: http://206hwf3fj4w52u3br03fi242.wpengine.netdna-cdn.com/wp-content/uploads/2016/06/4x4.gif http://206hwf3fj4w52u3br03fi242.wpengine.netdna-cdn.com/wp-c... ¹ http://wordpress.mrreid.org/wp-content/uploads/2013/06/boomerang-mounted.jpg http://wordpress.mrreid.org/wp-content/uploads/2013/06/boome...
- numinos1 10y agoThis reminds me of the "acoustic vector sensor" developed by a Dutch acoustics firm that can pinpoint any conversation in a crowd. https://www.newscientist.com/article/mg21929364-400-matchstick-sized-sensor-can-record-your-private-chats/ https://www.newscientist.com/article/mg21929364-400-matchsti...
- jparishy 10y agoAlmost, kinda related is this Invisbilia episode that's pretty interesting about a blind man that uses directed clicking in his mouth as a way of "seeing via echolocation". They talk about how it activates similar parts of the brain that vision does which I think is a cool parallel to what's been created here. http://www.npr.org/programs/invisibilia/378577902/how-to-become-batman http://www.npr.org/programs/invisibilia/378577902/how-to-bec...
- jd20 10y agoThis is pretty cool, and reminded me of the far-field microphone array in an Amazon Echo (which uses 7 spaced out microphones). Which also makes me wonder, taking something like this (which has way more than just 7 mics) and applying it to the problem of far field speech recognition, are there large gains to be had? Just thinking of what new applications become possible, when you have the ability to print hundreds or thousands of miniature mics on a single board that will fit in a small-ish device.
- anotheryou 10y agoI made a smaller and smoother version of the final gif: http://malea.lacerta.uberspace.de/up/128.gif http://malea.lacerta.uberspace.de/up/128.gif (downsampled back to 16x16, than upsampled with photoshops more sophisticated resize and doubled speed for more fluent movement)
- boxcardavin 10y agoThis speaks to my heart.
- zappo2938 10y agoWe can tell the location of a sound by timing of the wave hitting one ear than the other. Sound Localization.[0] [0] https://en.wikipedia.org/wiki/Sound_localization https://en.wikipedia.org/wiki/Sound_localization
- tlarkworthy 10y agoThe math is how underwater imaging sonars work. But there the sonar emits the sound too. http://www.blueview.com/videos/2d-imaging-sonar/ http://www.blueview.com/videos/2d-imaging-sonar/
- wbeaty 10y agoVery very cool! And it's about time someone did it! Yours is the first hobbyist project I've encountered since my own extremely crude ten-dollar version in 1981. No square array or processing, mine was a mechanical scanner-disk: eight LEDs glued in a spiral to an old vinyl record album, each with an electret microphone and and op amp. Spin the disk with a motor at about 6Hz. Wind noise, so add foam to the mics. It worked. A mechanical acoustic-phosphor raster sensor and display screen, where sound patterns caused light patterns. The best part was to hold some headphones near it while playing 10KHz sine wave. This produced interference stripes on the scanning disk! I hoped to build a huge version someday, but somebody beat me to it: Tom Zimmerman, an MIT student who patented the dataglove, then used the money to work at Exploratorium Museum for a year (as Artist in Residence), building a scanning-microphone LED disk about 6ft wide. Since then we have companies selling DIDSON full-blown video for underwater (with sonic illumination,) also a euro company with a 3D phase-array microphone imager positioned on a ring or a wire sphere. See: http://www.soundmetrics.com/Image-Gallery/Fisheries/Habitat-Observations/Divers-during-Aquarius-ocean-research-expedition http://www.soundmetrics.com/Image-Gallery/Fisheries/Habitat-... Heh, the divers appear to be skeletons wearing scuba tanks. http://www.acoustic-camera.com/en/products/microphone-arrays/ring-72-ac-pro.html http://www.acoustic-camera.com/en/products/microphone-arrays...
- shas3 10y agoOn the academic-industry/start up side, I know of one such device which does this on a sphere: http://visisonics.com/visisonics-audiovisual-camera/ http://visisonics.com/visisonics-audiovisual-camera/ It works quite well in my experience with it. The spherical geometry enables a 3D capture and visualization.
- Artlav 10y agoWow, that's one heck of a sonar. This is the sort of images i'm after.
- wbeaty 10y agoPut one on ROV for a trip to Loch Ness! That, or scavanging old outboard motors in muddy lakes. Don't miss multi-pages video gallery: http://www.soundmetrics.com/Image-Gallery http://www.soundmetrics.com/Image-Gallery
- acjohnson55 10y agoBrilliant project! I had a similar idea several years ago, when I was in a master's program in music technology at NYU. I had classes on acoustics and concert recording that really blew my mind as I started to understand how sound behaves within spaces. The idea of a soundfield camera occurred to me, and it seemed like it could potentially demystify a lot of phenomena sound workers have to deal with largely by intuition. Things like reverberation, acoustic modes, propagation patterns, wave effects, etc. I hope this project continues to mature!
- Artlav 10y agoSounds rather interesting. Sadly, there is no way anyone would let me into an opera house with this thing...
- humbertomn 10y agoGreat post, Artlav! Great to hear from you after Exosphere in Chile last year. Eager for the next part of the project and hopefully you get some funds for your always interesting ideas.
- chris_va 10y agoSo impressive If you get a x86 hooked up, doing some software cleanup might make the video look much better. Some ideas that you'll probably come up with on your anyway: - Assign colors based on frequency clustering, rather than a heatmap (so, red=first freq, green=second freq, and so on). - Persist colors if the frequency in a region shifts slowly enough (so every source stays mostly the same color). - Remove echoes by finding correlated pixels.
- oinsurance3 10y agoNice
- Animats 10y agoBeam-forming microphones are well known. Here's a fancy ceiling-mounted one for video conferencing.[1] Here's the theory in Matlab.[2] And, most impressive is the surveillance version, the Squarehead Audioscope Acoustic Camera.[3] This, a 1m disk with 384 microphones, can pick out a single conversation in a noisy arena. There's a version which records all the audio channels in real time, using a big disk array. This allows the phased-array processing to be done during playback, so the "aiming" can take place later. The company now has its catalog of surveillance products password-protected, but in the past, there's been more info about that application. The DEA has purchased two. There are also "acoustic cameras" for locating noise sources.[4] All this is a spinoff of submarine passive sonar technology from decades ago. [1] https://www.youtube.com/watch?v=Bepqav87D40 https://www.youtube.com/watch?v=Bepqav87D40 [2] http://www.mathworks.com/help/phased/examples/acoustic-beamforming-using-a-microphone-array.html http://www.mathworks.com/help/phased/examples/acoustic-beamf... [3] https://www.youtube.com/watch?v=bgz7Cx-qSFw https://www.youtube.com/watch?v=bgz7Cx-qSFw [4] http://www.acoustic-camera.com/ http://www.acoustic-camera.com/
- GIFtheory 10y agoIn fact, there is a good chance your phone does this when it is in speakerphone: https://developer.apple.com/library/ios/qa/qa1799/_index.html https://developer.apple.com/library/ios/qa/qa1799/_index.htm....
- votingprawn 10y agoI built a system a few years ago as part of my undergraduate degree that could localise aircraft using microphone arrays and acoustic beam forming. My primary sources for information were books on SONAR. The system worked in real time but also stored all the raw data so it could be processed again later if you wanted. I also had some fun trying to identify the aircraft based on frequency content and details on engine speeds / blade counts from Type Certificates.
- dingo_bat 10y agoMy laptop has a beamforming microphone array. The interesting part of the linked project is that you can literally see objects using homemade stuff. Well homemade except for the printed circuit board :)
- skrebbel 10y agoA Dutch startup called Sorama has been selling this commercially for a while. It's very cool actually, they sell mostly to companies who want to make their hardware less noisy. It turns out that noisy machines are often mostly noisy because a relatively small number of parts are too flexible, not screwed on tightly enough, and so on. Track that down and you know where your low-hanging fruit is. A daredevil cam is exactly what you need for tracking that down. Their site is a tad unimpressive sadly, but their tech is very cool: http://www.sorama.eu http://www.sorama.eu
- Artlav 10y agoAh, that might be the company i heard of. Thought they only did near-field. Yep, that looks very close to what i'm making.
- yread 10y agoPretty cool, here is a live demonstration of their 1024-microphone camera https://youtu.be/dJ4NKq7trTA?t=210 https://youtu.be/dJ4NKq7trTA?t=210
- Maxnuf 10y agoThere was a videoreport on this startup last year by Tweakers.net: https://tweakers.net/video/10249/een-camera-die-geluid-visualiseert-geluidsproblemen-opsporen-en-analyseren.html https://tweakers.net/video/10249/een-camera-die-geluid-visua... (it's in Dutch)
- SilasX 10y agoI love it! Would love to see it extended to cover a distributed network of emitter/receivers to image an area like in The Dark Knight.
- xerophyte12932 10y agoWhile the actual curiosity-following-adventure is amazing, i think one other aspect to all of this that deserves a lot of praise is his writing. He managed to take us on his adventure along with him. Not every blogger pulls that off successfully. Kudos to that!
- abcampbell 10y agoSomeone please give this guy $10m of funding to invent more cool stuff. Regardless of commercial application, this guy can do #hardtech
- AlexQ 10y agoAwesome. If I'd have the money, I'd fund every project of yours for the rest of your life.
- sinaa 10y agoExtremely cool! Now is the reverse (vision->sound) also possible? Perhaps using an array of laser sensors, so that we can reproduce the sounds in the environment. Also, is it possible to turn the images/video back into sound? Could be a fun experiment!
- Artlav 10y agoI've seen a phone app that does this - turns the camera's video into sound. With a bit of training, deaf people can use it to "see" the surrounding.
- wesleye 10y agoThis is exactly what http://www.sorama.eu/ http://www.sorama.eu/ does. They have created a large array of cheap microphones to do an otherwise expensive setup. They work, for example with Coolermaster to make their cases more silent: https://www.youtube.com/watch?v=JLx9DsNcM6o https://www.youtube.com/watch?v=JLx9DsNcM6o
- dubmax123 10y agoThis is awesome! I'm sensing some Machine Learning applications here.