7 ms·
Guessing the pressed keyboard keys by analyzing the audio from the microphone
- daenz 8y agoNow determine the keypresses by filming the vibrations of a potato chip bag[0] [0]http://news.mit.edu/2014/algorithm-recovers-speech-from-vibrations-0804 http://news.mit.edu/2014/algorithm-recovers-speech-from-vibr...
- hashmap 8y agoThey manage to do it (just sound, not keypresses) with a 60fps DSLR camera at the end by examining the rows of the video - does this mean that for any videos already in existence, sound can be decoded from the images?
- singularity2001 8y agoFor any old 60fps homemovie of your potato chips: maybe.
- aaaaaaaaaab 8y agoNot for handheld videos.
- pbhjpbhj 8y agoWouldn't you be able to filter out the camera motion, as long as the movement due to sound was fixed relative to something in frame?
- aaaaaaaaaab 8y agoI would say the motion blur and the rolling shutter effect would overpower the subtle vibrations due to audio.
- kibibu 8y agoRolling shutter is exactly why this works. Its not a complicated paper, give it a go.
- rapnie 8y agoPrimary use case is more tracking and surveillance, I presume?
- IIAOPSW 8y agoPrimary usecase is hacking. A keylogger without any software or trace on the target computer.
- spockz 8y agoThis is another reason why you want 2FA.
- _Nat_ 8y agoFor the technology itself, sure. For this particular open-source hobbyist project, not likely; those who're most likely to attempt to use such a tactic maliciously are likely to keep it to themselves rather than demonstrate that the security concern exists.
- scarlac 8y agoAttempted to train it by typing for ~2 minutes. Basically typed everything on the GitHub page and tried getting predictions. Results were disappointing. I didn't see any accurate predictions. Even with the default p/q program I see mostly random results. I tested on a 15" Macbook Pro 2018 (latest version of keyboard that is softer to type of / less noisy)
- justAnotherNET 8y agowell if anyone has the resources to train NN's for evrry keyboard out there... it's government.
- deleted 8y ago[deleted]
- ggerganov 8y agoThe p/q test fails on my Macbook Pro 2013 as well. Plugging an external mechanical keyboard makes it work, so it's definitely due to the soft type of keyboard and not the mic.
- carbocation 8y agoSounds like the author has the same experience: if the keyboard is not mechanical, then this library doesn’t work: https://github.com/ggerganov/kbd-audio/issues/3 https://github.com/ggerganov/kbd-audio/issues/3
- ggerganov 8y agoCorrect. I just made a short video to demonstrate what a working setup looks/sounds like: https://www.youtube.com/watch?v=2OjzI9m7W10 https://www.youtube.com/watch?v=2OjzI9m7W10
- kalmi10 8y agoMacbooks usually have multiple mics for noise filtering. That could also be an issue.
- amelius 8y ago
- 0db532a0 8y agoCould you maybe do away with training beforehand on a single source and instead use multiple, very sensitive microphones and triangulate the locations of the keys being pressed? The estimations might not be accurate, but you could put the results through a smartphone typo correction algorithm.
- switz 8y agoI've always thought Twitch streamers were opening up an attack vector through this exact method. Cool to see someone follow through with it. Any streamers out there should figure out a way of avoiding keypress bleed or muting their mics when typing sensitive info (e.g. passwords).
- nmstoker 8y agoYes, countermeasures, such as playing other sounds around those frequencies and/or filtering the microphone at those frequencies would be interesting to explore. Filtering the mic seems the less annoying from a user perspective!
- drankula3 8y agoMost streamers, myself included, use a filter called a noise gate, which requires the volume of the mic reach a certain threshold before being broadcast. This filters out the majority of background noise on the stream.
- TazeTSchnitzel 8y agoDon’t Skype & Type! Acoustic Eavesdropping in Voice-Over-IP https://www.math.unipd.it/~dlain/papers/2017-skype.pdf https://www.math.unipd.it/~dlain/papers/2017-skype.pdf
- mad_hominem 8y agoPerhaps accidentally, a yubikey or something similar would avoid this attack.
- pbhjpbhj 8y agoIf someone hunts-and-pecks how close can you get with gaze tracking?
- syntaxing 8y agoThis is super cool. Very similar to timing attacks. I wonder if there is a way to tune the model. Essentially cater the probability of each key to a person with maybe a sentence or something.
- sdenton4 8y agoYeah, this is exactly the sort of thing where augmenting the base model with a language model will have some good returns...
- ggerganov 8y agoIn keytap2 I'm trying to make use of the statistical distribution of n-grams in the language. The idea is to first group the unknown keys into clusters based on how similar they sound. The prediction then is performed by breaking the obtained substitution cypher (assuming each cluster corresponds to a letter).
- sentrysapper 8y agoCan you combine this with or does it use relative positioning of the microphone? Seems like a good way to map where keys are (measure lower decibels which would be keys further from the mic).
- danschumann 8y agoSomeone has seen the movie sneakers! The blind guy did this in that movie.
- hnmonkey 8y agoIn what scene? I don't think he actually did... I'm pretty sure when they're trying to crack the guy's password they're using video to record and zoom in to watch him and he's blocking it.
- frakr 8y agodefinitely keep buying mechanical keyboards....
- ppod 8y agoThere is more research on this than what you have cited here, unfortunately the literature goes under the catchy keyword "acoustic keyboard emanations", try that in Google scholar.