5 ms·

If anyone would like to see this in action, try this out on your laptop https://quiet.github.io/quiet-js/ https://quiet.github.io/quiet-js/ </plug>

tripzilch 10y ago
Cool! I always wondered what the accuracy/success rate of such a sidechannel would be, nice that your tool allows me to test this easily :) Interestingly enough, after some fiddling I could get it from my speakers to phone mic, and phone speakers to laptop mic, but only in the audible spectrum, not in the ultrasonic. Only response I got was one time, an error--probably one packet that failed the CRC check, but the rest of the time nothing. I didn't have time to mess with the code and try different modulation frequencies yet. But it's definitely a cool toy! :D
- brian-armstrong 10y ago
  Would you mind sharing details about your setup? I assume you have an Android phone since in mobile Safari you can't access the mic, full stop. I'm also curious to hear what the error was. Do you mean an error occurred in the JS console? That isn't supposed to happen in any event. If you want to more thoroughly test, try https://quiet.github.io/quiet-profile-lab https://quiet.github.io/quiet-profile-lab although it's worth noting that this tool can only work on one device, transmitting to itself Finally, it's intended to be a production library, not just a toy ;) edit: it's worth noting that if your receiver is Firefox, you'll never get anything from ultrasonic. That's because firefox resamples to 32kHz, cutting off any frequencies above 16kHz
  - tripzilch 10y ago
    My apologies for calling it a toy!! ;-) Still fun to play with though ;-) I used Chromium Linux on the laptop and and Chrome for Android on the phone end. I normally use Firefox as my main browser but I opted for Chrome because I assumed it was a PoC and you'd need a lot of uninteresting work to make it smoothly cross-browser and most people here develop for Chrome first--I'd save that effort for the weaponised exploit ;) Or production library, in your case ;) (what is the use case there btw?) Thanks for the heads-up on Firefox's resampling, so I know not to try that ;) Wonder if they chose that for security reasons? Seems a solid protection exactly against this. If so, then comes the question how good their lowpass (resampling) filter is, if nothing leaks through and you can't secretly grab ultrasonics anyhow via longer statistical methods trying to infer which freqs are aliased and which aren't :) So, I tried the audible signal first, to be sure it was making sound (I have my Firefoxes locked down rather tightly, JACK and PulseAudio often fight, so unless it's a mediaplayer or audio-production tool, it doesn't make sound usually). But Chromium did (after re-enabling Pulse). I got decent results for both ways (sometimes partial text, I didn't try images), so recording worked too and the multimedia JS code seems to work fine. One idea on that end; And I never wrote JS audio (I plan to, though) so I don't know if this is hard, but maybe next to the "listen" buttons, you could add a simple VU-meter bar for the mic, so I can clap my hands and check if the mic is indeed receiving (maybe your profile-lab already does this, havent checked). The error I saw looked like an intentional popup-message you wrote in the code. It was a modal DIV popup, styled like the site, more like a debug message perhaps. I didn't quite understand it, so I don't remember it exactly: that it received 1/100 packets, and failed checksum ... sorry I can't be more specific, I only saw it once. It seemed to indicate it detected one packet but it was an incorrect one, or something similar. Given the config mentions CRC checks, I assumed it was that. I didn't check the JS console for messages, but the behaviour of the site didn't indicate to me that there were actual JS errors. Extra info: The sound from Chromium Linux was played through high quality large speakers, the microphone is built-in (Eee 1215B netbook, rather old thingy) so probably not very good. On the phone side, it's running Cyanogenmod on a Samsung S4. I believe the mic on the phone is better than on the laptop because it's rather old and quite noisy. Is there a reason the profile-lab tool only works for one device transmitting to itself? Cause that's part of the fun (yeah yeah no toy :p), transmitting data between two devices that have no connection to each other. If I was to be thorough, I'd set it to airplane mode just to prove the fact :) I'll have another play with quiet.js and also the quit-profile-lab. If I see the error message again or find anything I didn't describe above, I'll report back. If you got any specific things you'd like me to test, let me know.
    - tripzilch 10y ago
      Ok, followup: First, I played again with quiet.js This time it seemed harder to get it to detect even audible signals. I'm pretty sure I don't have more background noise today than yesterday, but my laptop fan can be pretty loud (and is less than a foot from the mic), so maybe it's spinning louder today. I have to hold my phone (mic) real close to the speakers to get it to do anything (yesterday as well, btw). The other way around, I also have to hold my phone (speakers) real close to the laptop mic. But IMHO that might be more expected since the phone speakers are so tiny and tinny, and the laptop mic is so crappy. I also got the error messages again! (and I misremembered, they're not modal). Seems it's just the message you get when it loses some packets. Yesterday I just happened to get either 100% loss or perfect transmission, so it wasn't immediately clear what ".. set the volume to 50%. Packet Loss: 1/1 (100%)" meant. For a production library, I'd definitely take note, for usability. I'd like to have a clear indication that the mic is actually receiving (can be very minimal, even just a little circle that fades from black to red depending on the level). Also "set volume to 50%" is (for me at least) quite ambiguous :) I have ALSA, Jack and Pulse applying their dB levels, through my laptop jack to my stereo amp which has again a big volume button, I won't even ask what is 50% here, because there is no answer :) I presume this is to prevent distortion artifacts, which may be present on phone and laptop speakers (but not my big ones ;) Are there maybe any modulation schemes that are (somewhat) robust against (overdrive/amp) distortion? Here's screenshots of the messages: http://imgur.com/a/4x3Tu http://imgur.com/a/4x3Tu . First one is mobile (receiving), also IIRC that's the exact message I got yesterday on the laptop. The second one is from the laptop today (also receiving), only after seeing that message I understood what the earlier really meant because the numbers were more arbitrary ;) (1/15 7%) I didn't try the ultrasonic frequencies because I couldn't get the audible ones to work reliably (I'll leave the no-fun debugging of something you can't hear to the professionals ;) ) Then I tried the quiet-profile-lab. I can't get it to work. The mic frequency spectrum visualiser works well, showing frequencies as expected, also for other sounds like clapping my hands. But "Frames Received" stays at 0. Screenshot: http://i.imgur.com/Jj0UWwx.png http://i.imgur.com/Jj0UWwx.png -- I set the centre frequency to 8kHz for this one, because it has a lot of noise on the low and the high ends. The low noise is probably the crappy mic, but the high-freq noise is somewhat surprising to me. I'm not sure at what freq my laptop-fan is whooshing, but it sounds like a low-mid tone plus mid/high-mids noise to me (based on the sound, not visualiser). Also tried other centre-frequencies, no luck. Oh WOW, trying quiet-profile-lab on my phone, the mic is so much clearer, it's not even funny, check the screenshots: http://i.imgur.com/iUpWpOW.jpg http://i.imgur.com/iUpWpOW.jpg http://i.imgur.com/MtrEYrb.jpg http://i.imgur.com/MtrEYrb.jpg loud and clean signal at 4kHz (centre freq) and a much quieter harmonic at 8kHz (and you can even see a tiny 3rd hardmonic at 12kHz). And also in the ultrasound, look at this nice clean spectrum: http://i.imgur.com/9DUO7V7.jpg http://i.imgur.com/9DUO7V7.jpg So, I guess if you guys want to improve accuracy on crappy systems like my laptop, you may get much better results after a noise-removal filter? From experience a lot of my laptop mic's noise is quite stationary. A tip: the Audacity wiki on its noise-removal filter is quite extensive, technical and informative. Also the (C++) source code for that filter is surprisingly readable (bit long, but most of it is comments). Reading it, I gained a lot of respect for that feature and believe it's one of the better ones out there (definitely in FOSS). Sorry I'm not going to play with the quiet-profile-lab on my phone much further, because it's not a toy and I've already spent 1.5 hour on my saturday evening ;-) [what can I say, I love DSP, and I love side-channels] Friendly advice on the quiet-profile-lab UI (even though it's not for production): You got the settings left and the output right. Occasionally, a change in settings wasn't always applied right away, so I figure it would be nice if the output column would display a short summary of the parameters on the left, that it is currently playing. I'm not that familiar with various modulation / operating modes, but I assume the flow goes like: listen to sound > detect a packet > decode+checksum the packet. Maybe it'd be nice to visualise this onto a slow-moving (cyclical buffer) waveform of the mic input: black=just sound, blue=possible packet, red=decoded bad checksum, green=decoded correct checksum. So! I hope this extensive feedback is useful to you! Sorry if it's a bit rambling but it's weekend so I'm not going to edit (the thread is old enough I'm sure no one will mind this mammoth post). If you make got a cool new feature or improvement in your library, I'd love to hear about it and you can hit me via email (see my profile). afterthought: it's kind of disturbing the difference in perceived level between 14.5 and 15.5 kHz, I can hear it but need to turn it up quite a bit, I am 35 years old. Even higher and it disappears further but replaced by a cleaner mid-frequency "windy" noise (that isn't quite there at lower centre freqs). quite a difference from when I tested my hearing range using pure sine tones generated in audacity (when you do this, tip: always add a fade-in otherwise the click makes it even harder to tell if you're hearing something or not). also I really feel for the person who has to test this setup every day ;-) given it's a serieous project, do you ever invite kids into your office? I work with kids in a creative science-lab / hackerspace type of centre, so we did the hearing tests a couple of times, as I said for the 15kHz I need to turn it up to be able to hear it clearly, when I did that with kids in the room (ages 9-13) they were all like "AAAAAAAAAAAA TURN THAT OFF!!" at the level when I felt, "I can comfortably detect this tone". And kids can easily hear a few kHz's higher than that even. But for me, the perceived level sharply plummets after a certain frequency. So maybe interesting to check, those sounds you perceive as barely audible might be distressingly loud beeps for other people. (fun fact: on the low end, 20-30Hz, seems we're all pretty much equal regardless of age--you need quality headphones to test those freqs though or the sound you think you hear is in fact harmonics from distortion).

tripzilch 10y ago

Cool! I always wondered what the accuracy/success rate of such a sidechannel would be, nice that your tool allows me to test this easily :) Interestingly enough, after some fiddling I could get it from my speakers to phone mic, and phone speakers to laptop mic, but only in the audible spectrum, not in the ultrasonic. Only response I got was one time, an error--probably one packet that failed the CRC check, but the rest of the time nothing. I didn't have time to mess with the code and try different modulation frequencies yet. But it's definitely a cool toy! :D

brian-armstrong 10y ago
Would you mind sharing details about your setup? I assume you have an Android phone since in mobile Safari you can't access the mic, full stop. I'm also curious to hear what the error was. Do you mean an error occurred in the JS console? That isn't supposed to happen in any event. If you want to more thoroughly test, try https://quiet.github.io/quiet-profile-lab https://quiet.github.io/quiet-profile-lab although it's worth noting that this tool can only work on one device, transmitting to itself Finally, it's intended to be a production library, not just a toy ;) edit: it's worth noting that if your receiver is Firefox, you'll never get anything from ultrasonic. That's because firefox resamples to 32kHz, cutting off any frequencies above 16kHz
- tripzilch 10y ago
  My apologies for calling it a toy!! ;-) Still fun to play with though ;-) I used Chromium Linux on the laptop and and Chrome for Android on the phone end. I normally use Firefox as my main browser but I opted for Chrome because I assumed it was a PoC and you'd need a lot of uninteresting work to make it smoothly cross-browser and most people here develop for Chrome first--I'd save that effort for the weaponised exploit ;) Or production library, in your case ;) (what is the use case there btw?) Thanks for the heads-up on Firefox's resampling, so I know not to try that ;) Wonder if they chose that for security reasons? Seems a solid protection exactly against this. If so, then comes the question how good their lowpass (resampling) filter is, if nothing leaks through and you can't secretly grab ultrasonics anyhow via longer statistical methods trying to infer which freqs are aliased and which aren't :) So, I tried the audible signal first, to be sure it was making sound (I have my Firefoxes locked down rather tightly, JACK and PulseAudio often fight, so unless it's a mediaplayer or audio-production tool, it doesn't make sound usually). But Chromium did (after re-enabling Pulse). I got decent results for both ways (sometimes partial text, I didn't try images), so recording worked too and the multimedia JS code seems to work fine. One idea on that end; And I never wrote JS audio (I plan to, though) so I don't know if this is hard, but maybe next to the "listen" buttons, you could add a simple VU-meter bar for the mic, so I can clap my hands and check if the mic is indeed receiving (maybe your profile-lab already does this, havent checked). The error I saw looked like an intentional popup-message you wrote in the code. It was a modal DIV popup, styled like the site, more like a debug message perhaps. I didn't quite understand it, so I don't remember it exactly: that it received 1/100 packets, and failed checksum ... sorry I can't be more specific, I only saw it once. It seemed to indicate it detected one packet but it was an incorrect one, or something similar. Given the config mentions CRC checks, I assumed it was that. I didn't check the JS console for messages, but the behaviour of the site didn't indicate to me that there were actual JS errors. Extra info: The sound from Chromium Linux was played through high quality large speakers, the microphone is built-in (Eee 1215B netbook, rather old thingy) so probably not very good. On the phone side, it's running Cyanogenmod on a Samsung S4. I believe the mic on the phone is better than on the laptop because it's rather old and quite noisy. Is there a reason the profile-lab tool only works for one device transmitting to itself? Cause that's part of the fun (yeah yeah no toy :p), transmitting data between two devices that have no connection to each other. If I was to be thorough, I'd set it to airplane mode just to prove the fact :) I'll have another play with quiet.js and also the quit-profile-lab. If I see the error message again or find anything I didn't describe above, I'll report back. If you got any specific things you'd like me to test, let me know.
  - tripzilch 10y ago
    Ok, followup: First, I played again with quiet.js This time it seemed harder to get it to detect even audible signals. I'm pretty sure I don't have more background noise today than yesterday, but my laptop fan can be pretty loud (and is less than a foot from the mic), so maybe it's spinning louder today. I have to hold my phone (mic) real close to the speakers to get it to do anything (yesterday as well, btw). The other way around, I also have to hold my phone (speakers) real close to the laptop mic. But IMHO that might be more expected since the phone speakers are so tiny and tinny, and the laptop mic is so crappy. I also got the error messages again! (and I misremembered, they're not modal). Seems it's just the message you get when it loses some packets. Yesterday I just happened to get either 100% loss or perfect transmission, so it wasn't immediately clear what ".. set the volume to 50%. Packet Loss: 1/1 (100%)" meant. For a production library, I'd definitely take note, for usability. I'd like to have a clear indication that the mic is actually receiving (can be very minimal, even just a little circle that fades from black to red depending on the level). Also "set volume to 50%" is (for me at least) quite ambiguous :) I have ALSA, Jack and Pulse applying their dB levels, through my laptop jack to my stereo amp which has again a big volume button, I won't even ask what is 50% here, because there is no answer :) I presume this is to prevent distortion artifacts, which may be present on phone and laptop speakers (but not my big ones ;) Are there maybe any modulation schemes that are (somewhat) robust against (overdrive/amp) distortion? Here's screenshots of the messages: http://imgur.com/a/4x3Tu http://imgur.com/a/4x3Tu . First one is mobile (receiving), also IIRC that's the exact message I got yesterday on the laptop. The second one is from the laptop today (also receiving), only after seeing that message I understood what the earlier really meant because the numbers were more arbitrary ;) (1/15 7%) I didn't try the ultrasonic frequencies because I couldn't get the audible ones to work reliably (I'll leave the no-fun debugging of something you can't hear to the professionals ;) ) Then I tried the quiet-profile-lab. I can't get it to work. The mic frequency spectrum visualiser works well, showing frequencies as expected, also for other sounds like clapping my hands. But "Frames Received" stays at 0. Screenshot: http://i.imgur.com/Jj0UWwx.png http://i.imgur.com/Jj0UWwx.png -- I set the centre frequency to 8kHz for this one, because it has a lot of noise on the low and the high ends. The low noise is probably the crappy mic, but the high-freq noise is somewhat surprising to me. I'm not sure at what freq my laptop-fan is whooshing, but it sounds like a low-mid tone plus mid/high-mids noise to me (based on the sound, not visualiser). Also tried other centre-frequencies, no luck. Oh WOW, trying quiet-profile-lab on my phone, the mic is so much clearer, it's not even funny, check the screenshots: http://i.imgur.com/iUpWpOW.jpg http://i.imgur.com/iUpWpOW.jpg http://i.imgur.com/MtrEYrb.jpg http://i.imgur.com/MtrEYrb.jpg loud and clean signal at 4kHz (centre freq) and a much quieter harmonic at 8kHz (and you can even see a tiny 3rd hardmonic at 12kHz). And also in the ultrasound, look at this nice clean spectrum: http://i.imgur.com/9DUO7V7.jpg http://i.imgur.com/9DUO7V7.jpg So, I guess if you guys want to improve accuracy on crappy systems like my laptop, you may get much better results after a noise-removal filter? From experience a lot of my laptop mic's noise is quite stationary. A tip: the Audacity wiki on its noise-removal filter is quite extensive, technical and informative. Also the (C++) source code for that filter is surprisingly readable (bit long, but most of it is comments). Reading it, I gained a lot of respect for that feature and believe it's one of the better ones out there (definitely in FOSS). Sorry I'm not going to play with the quiet-profile-lab on my phone much further, because it's not a toy and I've already spent 1.5 hour on my saturday evening ;-) [what can I say, I love DSP, and I love side-channels] Friendly advice on the quiet-profile-lab UI (even though it's not for production): You got the settings left and the output right. Occasionally, a change in settings wasn't always applied right away, so I figure it would be nice if the output column would display a short summary of the parameters on the left, that it is currently playing. I'm not that familiar with various modulation / operating modes, but I assume the flow goes like: listen to sound > detect a packet > decode+checksum the packet. Maybe it'd be nice to visualise this onto a slow-moving (cyclical buffer) waveform of the mic input: black=just sound, blue=possible packet, red=decoded bad checksum, green=decoded correct checksum. So! I hope this extensive feedback is useful to you! Sorry if it's a bit rambling but it's weekend so I'm not going to edit (the thread is old enough I'm sure no one will mind this mammoth post). If you make got a cool new feature or improvement in your library, I'd love to hear about it and you can hit me via email (see my profile). afterthought: it's kind of disturbing the difference in perceived level between 14.5 and 15.5 kHz, I can hear it but need to turn it up quite a bit, I am 35 years old. Even higher and it disappears further but replaced by a cleaner mid-frequency "windy" noise (that isn't quite there at lower centre freqs). quite a difference from when I tested my hearing range using pure sine tones generated in audacity (when you do this, tip: always add a fade-in otherwise the click makes it even harder to tell if you're hearing something or not). also I really feel for the person who has to test this setup every day ;-) given it's a serieous project, do you ever invite kids into your office? I work with kids in a creative science-lab / hackerspace type of centre, so we did the hearing tests a couple of times, as I said for the 15kHz I need to turn it up to be able to hear it clearly, when I did that with kids in the room (ages 9-13) they were all like "AAAAAAAAAAAA TURN THAT OFF!!" at the level when I felt, "I can comfortably detect this tone". And kids can easily hear a few kHz's higher than that even. But for me, the perceived level sharply plummets after a certain frequency. So maybe interesting to check, those sounds you perceive as barely audible might be distressingly loud beeps for other people. (fun fact: on the low end, 20-30Hz, seems we're all pretty much equal regardless of age--you need quality headphones to test those freqs though or the sound you think you hear is in fact harmonics from distortion).